19

I've gotten a bug report from one of my users in a section of the software. The scenario is basically a databinding scenario where the user enters info, and that info is printed to pdf.

The problem is, that the functionality:

  • Is used frequently (about 40 times a week)
  • Hasn't been updated/modified in months
  • The area of code is relatively simple to walk through
  • The validation appears fine (ie, if the information wasn't filled out in the app, validation fires indicating it with a msgbox before the pdf is generated)

But this one user claims that in the past 2 weeks it's happened about 3 times out of 50 and I just can't reproduce it.

So what do you do in the case of a heisenbug?

Steven Evers
  • 28,200
  • 10
  • 75
  • 159
  • 1
    similar question to http://programmers.stackexchange.com/questions/1376/what-to-do-with-bugs-that-do-not-repro – Bill Oct 26 '10 at 22:44
  • 2
    try to add some log statements. If you debug, the GUI might be repainted if you switch windows from your application to your IDE. – keuleJ Feb 19 '13 at 12:14

6 Answers6

22

Add some logging to this users code.

kasterma
  • 432
  • 3
  • 8
  • 5
    What if deployment is a bit of a heavyweight process? (it's not, and I'm going to take your suggestion, but for the sake of argument...) – Steven Evers Oct 26 '10 at 22:42
  • give the user a specific debug version for them only perhaps? – Bill Oct 26 '10 at 22:46
  • 3
    @SnOrfus: in that case you could perhaps try to arrange some sort of screen sharing session with the user. During that session he would only try to reproduce the issue. Depending on how long it actually takes for a single usage of the functionality it may take a while but would give you a chance of seeing the bug "live in action". Also, if the bug reports are missing some details you could ask for more detailed reports - perhaps the user is doing something he thinks has nothing to do with the bug but that is actually important. – Baelnorn Oct 26 '10 at 22:52
  • @SnOrfus: we try to include some logging in our application with the capability to add more details. Perhaps using MS' Enterprise Library for logging; one can configure what gets logged by level, so by changing a number in the config file, one can turn on/off debug code (off for deployment and general shipping code, with tech support telling the customer what entry to change to turn on the level of logging needed). – Tangurena Oct 27 '10 at 18:51
  • Definitely. I've only had one heisenbug in my life that didn't fall to logging code and that one never got proper scrutiny as it was too disruptive to hunt. (It would only show on the production server and only when it was actually in use and when it showed it put all the work in the factory on hold.) – Loren Pechtel Mar 02 '11 at 03:14
13

I've seen things like this on an embedded system take 6 months to find. Really frustrating.

However in desktop land, it's amazing what happens if you go an actually watch what the user does. They may be doing things in some order / manner that had not been expected and this in turn causes the trouble.

quickly_now
  • 14,822
  • 1
  • 35
  • 48
  • 1
    This. Sitting down beside the user and having him walk through what he does. Often things are not as straight forward as the users think they are. – Vetle Oct 27 '10 at 08:08
  • If you can't get to sit beside them, desktop sharing is a good alternative so you can watch them work and see what happens. Users often do things that seem totally unlikely to us as developers. – glenatron Oct 27 '10 at 10:48
  • If you use desktop sharing, make sure to be on the phone to them as well so you can listen to what they are saying! – quickly_now Oct 27 '10 at 23:44
12

depending on your situation you might have success with:

  • Monitor the user's machine (perfmon, eventlog, etc)
  • Monitor the user (sit with them until they have the issue again)
  • Replace the user's machine temporarily (get them on another desktop to see if it is a hardware/os thing)

kasterma's suggestion of logging is still good, give them a debug build or use injectable logging if the full deploy is too troublesome.

Bill
  • 8,330
  • 24
  • 52
7

This is often caused by concurrent processes (not OS-level processes, just... general things that happen in your application: events, threads, input/output etc.) which both affect the rendering in some way. This leads to different behaviour depending on the order in which they happen, and debugging and breaking often interferes with that.

One good strategy is to replace stepping through the debugger with more logging - this takes much less time and therefore is much more likely to leave things as they are while still giving you more information.

Ultimately, though, nothing beats understanding what the system actually does. Is there one component, and one only, which should be responsible for maintaining the state of the UI? (Usually there should.) If so, why is it getting inconsistent commands in the first place? Obviously, logging can often help answering these questions as well.

Kilian Foth
  • 107,706
  • 45
  • 295
  • 310
5

The best thing to do is to add logging and try to catch it in the act. If that is impractical, then the only thing left is to do a very thorough code review. Going through the change log would be a reasonable way to begin.

Dima
  • 11,822
  • 3
  • 46
  • 49
4

Check the hardware.

Run a memory test on the machine showing the problem. Run a heavy CPU load and verify it. Something like Prime95.

Hardware isn't perfect and if the hardware is bad a programmer can waste a lot of time looking for bugs that just don't exist.

Zan Lynx
  • 1,300
  • 11
  • 14