29

Is it a bad sign if users submit bug reports for things that are by design?

Does it typically mean that the application is confusing or unclear, or should I just chalk it up to a one-off user mistake unless specifically stated?

(I don't actually have any such reports. This is a purely hypothetical question about whether or not the existence of by-design "bugs" is a bad thing.)

Maxpm
  • 3,146
  • 1
  • 25
  • 34
  • 19
    Perhaps some of the programers at Lotus Notes would care to comment, since there standard reply is "its not a bug its a feature you don't understand". – James Anderson Sep 01 '11 at 06:29
  • 5
    It suggests to me that the user requirements as given to the developers do not match what the users think they asked for. – temptar Sep 01 '11 at 09:42
  • 7
    @temptar "It's what I asked for, but it's not what I wanted." – StuperUser Sep 01 '11 at 10:37
  • 5
    I was assigned a "bug" work item recently for behavior that was exactly, specifically, what was explicitly spelled out in the specification (and which had been discussed during the specification phase). While the behavior may very well have been unexpected from a user's point of view (a particular setting, which was paid attention to elsewhere, was being ignored), if the *spec* literally all but says "let's ignore that for now to save time", then to me it's not a bug, but rather a change request. Certainly, a good case could be made for wanting that changed, but *don't call it a bug*. – user Sep 01 '11 at 12:43
  • @Michael: If the spec says something that the developer knows is a problem and the developer chooses not to clarify the issue, then it is most certainly a bug. Also, issues like that would question my confidence in the developer. "All but says" is not the same as "says". – Dunk Sep 01 '11 at 13:09
  • 7
    @Dunk: I've implemented a system that assumed 24 hours in every day, because we failed to convince the customer of the existance of Daylight Savings Time. They just wouldn't believe that there are days with 25 hours. Is that a bug in the program? – MSalters Sep 01 '11 at 13:17
  • 1
    ever wondered what those WONTFIX tag in many popular bug tracking softwares are for? – Lie Ryan Sep 01 '11 at 14:05
  • -1 I have another question who wants free beer? A question like this allows people to rant and soapbox but is not really constructive. – SoylentGray Sep 01 '11 at 18:24
  • @Chad I disagree. It's a clear question about whether or not by-design "bugs" are signs of deeper troubles. How does that open the door to ranting? – Maxpm Sep 01 '11 at 18:29
  • @Maxpm - It is a question of good or bad. It is not actually seeking an answer. You did not seek to address the problem or help determining the cause. You just want to be able to [Appeal to authority](http://www.nizkor.org/features/fallacies/appeal-to-authority.html) or an [Appeal to Popularity](http://www.nizkor.org/features/fallacies/appeal-to-popularity.html) in some arguement down the road. – SoylentGray Sep 01 '11 at 18:35
  • @Dunk, in this case, the issue *was* raised during the specification phase, and it was **decided** to leave the behavior as stated rather than change it to take the setting in question into account. And if the spec says `behavior X is controlled by A and B`, the question about `C` *is* raised and it is decided that due to time constraints, X should be controlled by A and B, and not C, then how is it a *bug* (rather than a change request) that X is not influenced by C? I'm not arguing against the change per se - that one was needed - but let's please call an apple an apple, not an orange. – user Sep 02 '11 at 07:59
  • In addition, the behavior in question in this *particular* case was controlled by a long list of criteria (not just one or two), one of which is mutually exclusive with setting `C` in my generalization above (if setting C had been adhered to, then one of the other criteria would have been an error). So it can very well be argued that the actual writing in the spec even precluded the possibility of adhering to `C`. – user Sep 02 '11 at 08:06
  • 1
    When you actually do have such bug reports, write them down at the [User Experience StackExchange](http://ux.stackexchange.com/). That might help you to sort out if it can be solved. – Lode Sep 03 '11 at 09:53

10 Answers10

54

Is it a bad sign? I think it's a warning that's worth looking into, but I also think it's bound to happen.

When people submit any kind of feedback to me, I try to filter it into three buckets:

  • Bugs
  • Feature Requests
  • Mis-communication

Bugs

Bugs are when something obviously doesn't work the way you would expect, nor the way the user would expect. Like, it asked me for my name, I entered "Scott", hit enter, and it said, "Hi Joe!"

Feature Requests

This is like "I know we never talked about this, but can the program infer from my mouse gestures that I'm left-handed and move the OK button to the left side of the screen?" This is when the current behaviour matches both your and the user's expectations, but they want to change the expectation.

Mis-communication

This is when you would expect one outcome from a scenario, but the user expects a different outcome. Sometimes this becomes a feature request, if they just haven't communicated their expectations, but they thought they did. Sometimes this becomes a bug if your expectation is proven to be wrong.

However, many times you have knowledge that the user doesn't have. What if they said, "On this screen, I can add a record for myself twice with the same first and last name! That's obviously a bug!" Your response might be, "There are lots of people in the world with the same first and last name, so we don't require that combination to be unique. We have a cleanup task that runs at night and emails a Possible Duplicates Report to customer service when it thinks it detects a duplicate with a similar name and address, and asks them to check it manually."

So you should read every bug report, but most complex systems are going to have bug reports that are really just feature requests, or possibly a mis-communication of the requirements. Not understanding the underlying complexity of the real world is probably the biggest source of these issues.

Scott Whitlock
  • 21,874
  • 5
  • 60
  • 88
  • 3
    Mis-communication also includes mis-remembering (or plain forgetting) the expectations which were communicated and agreed. – Peter Taylor Sep 01 '11 at 09:02
  • 1
    Great distinctions, but I still think that the application should attempt to explain the mis-communication if it can. With your multiple people with the same name the app might say, "We already have 3 John Smiths are you sure it isn't one of these" with a 'No I'm sure this is a new John Smith' option. – Alex Andronov Sep 01 '11 at 14:10
  • @Alex Andronov - a miscommunication doesn't mean there isn't an action to take: either change the documentation, tooltips, etc., updating the training material, or possibly just a brief explanation and moving on. – Scott Whitlock Sep 01 '11 at 15:11
7

This wasn't touched on in previous answers thus far so I will add that it could also be a sign of an ignorant user base. I don't use the word "ignorant" in a derogatory or condescending way, merely as a way to express that they are without proper knowledge or education in their domain or of the complexities of the software itself.

Most users are not aware of just how complicated software has to be to meet certain requirements. Something to the effect of 80% of effort goes into only 20% of functionality (fringe and exception cases).

They sometimes don't understand why the software inherently needs to behave in a certain way, many times to prevent a multitude of defects or corruption of data, etc...

This isn't worrying as this becomes better with clear and concise documentation and communication.

maple_shaft
  • 26,401
  • 11
  • 57
  • 131
  • 2
    +1 but please look up the difference between complex and complicated... Software doesn't need to be complicated at all, but it is often times very complex. – Marjan Venema Sep 01 '11 at 06:24
  • This is very true. The most common case I run across is users not realizing the difference between many-to-one vs. many-to-many relationships. A common request I get is, "can you make the report show the order number in the header?" and I have to explain that while in about 95% of the cases there is only one order number on the stuff they work on, there are many cases where the query might include data across multiple orders. Then I have to ask, do you want me to list all the order numbers in the header separated by commas or do you want the order number on each detail line in the report? – Scott Whitlock May 21 '13 at 11:53
  • @ScottWhitlock Yes it is the same thing. A good developer or business analyst then shouldn't just do what the customer asks but try to uncover the core problem that they are experiencing for why they made such a request in the first place. Once you identify their problem you can identify a proper solution and write appropriate requirements or user stories. – maple_shaft May 21 '13 at 16:47
5

If you have a user that is an expert in their field, you could have a major problem. Imagine an accountant that finds your software, by design, isn't following general accounting procedures or an engineer who discovers you have an incorrect formula. This shouldn't be too hard to research to see if they are correct. Redesign and fix it if needed-quick.

One opinion on a particular UI feature or a field they think should have a currency symbol or some other formating, should be considered, at least until you get more feedback. Researching this could be a little more difficult.

JeffO
  • 36,816
  • 2
  • 57
  • 124
  • 1
    There's no single answer, then, I suppose. It should be assessed on a case-by-case basis. I figured as much. – Maxpm Sep 01 '11 at 02:18
5

By-design bugs in my experience means that your use cases don't fit your real users. Try reading about using real users for your use cases ( just giving them names and a thumbnail description does wonders for the quality of use cases.)

When prominent OS vendors say "this behaviour is by design" they generally had ease of implementation or commercial advantage in mind. If that's not you, try to find out your users real skill-set and relationship to your software. Do they use it all day ? For fun ? Once a week because it replaced the TPS forms ?

Tim Williscroft
  • 3,563
  • 1
  • 21
  • 26
5

The UI should follow The Principle of Least Astonishment - repeated bug reports on a feature that works as designed are an indication that this principle has not been adhered to correctly.

Joris Timmermans
  • 8,998
  • 2
  • 36
  • 60
  • 3
    Or, that there is a divided user group. For example, let's say your webapp mimmicks a Desktop. Do you expect that closing a window closes the programm? Yes for Windows users, No for Mac-users. – keppla Sep 01 '11 at 07:33
  • 1
    @Keppla - you make a good point. You cannot please everyone, so in the case of "bugs" like the ones mentioned here some investigation is needed to make sure you won't be getting more bug reports after the "fix" than before. – Joris Timmermans Sep 01 '11 at 07:54
  • 1
    maybe, but it's a lot more powerful to cite the data of "hundreds of people have filed bugs on this!" than it is to have a few rabid users telling you that you have violated their *personal* interpretation of the Magical Principle of Least Astonishment and *they* are astonished. I'm a bit weary of the latter, since one man's "astonishment" is another man's "straightforward". – Jeff Atwood Sep 01 '11 at 09:51
  • @Jeff Atwood - as the saying goes "one swallow does not make a summer", "one bug report does not imply a design error". A single bug report on something that is a feature is no cause for a redesign, and even several reports on a common feature do not necessarily mean that the majority of your users would not be happier without a "fix". Especially if your original release is already popular and people are used to using it "that way". – Joris Timmermans Sep 01 '11 at 13:28
2

Not necessarily. It could be that the reported bug is in a use of the software which is just outside of its originally defined scope. Consider software which was designed to do A, B, and C (for a trivial example, draw lines, triangles, and rectangles). If D is a logical next step (e.g. pentagons), the user may assume it should do that as well, and that not doing so is a bug. But if this is outside of the original scope, it's not a bug. It could instead be an oversight (bug) in design, or a gray area in the specifications, or a different set of assumptions made by the developer and the user.

(Edit - added my comment to the answer per @Marjan Venema's suggestion.

James McLeod
  • 7,613
  • 4
  • 21
  • 34
  • I don't really see that being marked as a bug report. More like a suggestion or feature request. (Although, with some bug-tracking systems, it might count as one anyway.) – Maxpm Sep 01 '11 at 02:23
  • 1
    I agree, most of the time, but it can mean an oversight (bug) in design, or a gray area in the specifications, or a different set of assumptions made by the developer and the user. – James McLeod Sep 01 '11 at 02:30
  • 1
    James, if you add that comment to your answer, the whole answer becomes better. – Marjan Venema Sep 01 '11 at 06:25
2

There are two kinds of bugs: the functionality doesn't match the programmer's intentions, or the functionality doesn't match the requirements. Programmers have a tendency to focus on the former to the detriment of the latter. To put it simply, if your users are reporting a lot of "by-design bugs," your requirements aren't what you think they are.

Karl Bielefeldt
  • 146,727
  • 38
  • 279
  • 479
1

I believe its a bad sign mainly due to the fact that the design is not generic and users requirement is not fully understood/analyzed.

I see two catogories of design, - Design by accident. - Design by intention.

Design by accident leads to such issues frequently and cannot sustain.

18bytes
  • 181
  • 1
  • 5
1

I'd like to add to maple_shaft's answer about users' "ignorance". You also have to keep in mind that 90% of the users will only care about their own experience and way to use the system. They don't care about other users, why should they? It's our job as designers/developers to take in input from all the different kinds of users and make something that fits everyone as good as possible. Most of the times you can't make a solution optimal for everyone.

But of course you need to read and evaluate feedback from your users! They are, after all, those who use your creation!

AndSoYouCode
  • 134
  • 3
0

The users submitting the bug requests were generally not consulted on the design so it is not surprising that they see things as bugs that were deliberate design decisions. To a user anything that doesn't work as expected is a bug.

I have found the problem is often a sign that the BA or PM only got requirements from managers not actual users. What managers expect of the system is often vastly different from what the people entering the data actually need. I was taught to collect data directly from the actual users, but most BAs I have run into lately only talk to managers (and generally relatively senior managers at that).

HLGEM
  • 28,709
  • 4
  • 67
  • 116