How do you respond to: "Ever since the update..." questions from clients?

Question

Ever since the update, people keep calling and saying "Ever since the update X, Y and Z are slow, bad and crashing"

This has happened ever since the dawn of updates.

What do people expect? Gamma comes after beta, and gamma testing always turns our users into The Incredible Hulks...

Perhaps you've never heard this from a client, perhaps you're in college or a FLOSS Dev who can spread the blame around more than 5 or 6 guys, perhaps you unit test your code, perhaps you're not in that interesting situation where customers actually call you requesting the exact time of day you'll be releasing today's patch (I'd love to do that to Microsoft) or perhaps you're a sorry son-of-a-biscuit like me who just shipped a new update and went home and is dreading going back to work tomorrow.

Anyway, ya'll are smarter than me anyway. How do you field criticism framed in "You must be a bad programmer because you're making your software worse"?

Answer 1

If this happens to you every time you deploy, there could be a serious flaw in your development process. I would suspect a couple of things that are causing the problems.

1. Do you develop against a database that is the same size (roughly) as production? If not then you will always have these problems because queries that are fine with small data sets are often dogs with bigs ones.
2. Do you load test in QA? What works fine with one user testing is very different from how things will respond with 1000 users trying to do things at the same time.
3. Do you assume the user's perception is wrong and treat them like they are stupid for complaining? If so, then your attitude is causing more complaints not lessening them.
4. Are you doing a good job of testing? Do you regression test features not changed to see if they are affected by the change? Do you even care how long things take until they hit prod?
5. Do you pay attention to when is a good time for the users when you deploy or do you merrily deploy a change to the accounting system the day payroll is run and wonder why users are angry at the slow down?
6. Do you have environmental differences between dev and prod. Sometimes those pesky differences in operating systems or database versions will cause issues like this as well. It is often a good idea to have a staging enviromnet that is exactly like prod, some equipment same operating system, same database with dat that is as close to prod data as possible. This is used to test your deployment. Run it first on this server and tehn have some users or testers go to it and run through some tests.
7. How good is your deployment process, do you often miss steps? Can it be run by someone other than the developer because it is clear what code goes in the branch you are deploying? We got a lot better at deploying code when we got a configuration management team and nobody had the rights to sit with prod and babysit it making "oopsie" changes. Automating your build can help tremendously. No one should be guessing what needs to go to prod as it should have gone to QA and staging first and any deplyment issues worked out. Scripting database changes is key too. They shoud be in scripts and in source control, so the build process can pick them up without someone having to remember, oh yeah, we need to change the length on Column B to 241 from 50. I find that people often forget to treat database changes as code preferring to use the GUI which is virtually always a bad idea on prod.

Answer 2

How do you field criticism framed in "You must be a bad programmer because you're making your software worse"?

But such criticism is mostly justified. A new release should not be worse than the previous, but as we know, in reality it often it is, and it is our fault because we made it.

Making mistakes is human, and it doesn't make anybody a "bad programmer", so don't take the criticism personally (I would never take any professional criticism from a non-colleague seriously anyway). Just thank the customer for reporting the issue, and fix it as soon as you can. It's your job as a good programmer.

Answer 3

Well, at work we don't interact directly with the clients much, so I'll have to answer this one from personal project work. I'm writing a game engine that people can use to build their own games. It's still in pre-alpha, but I've got a few interested users, and sometimes I get bugs.

When I get a report like this from a user, I try to use the personal touch. I didn't mean to introduce bugs, and I want them to have a good experience with my engine, so I need to make them believe that. First, get an IM handle so we can talk. I'll ask the user about their project and try to get a copy of it. This makes reproduction a lot easier. Ask them what they were doing when the glitch occurred. Meanwhile, I've got the engine open in the debugger and I'm homing in on the issue while we're talking.

If it's an exception, it's usually pretty simple. Reproduce the issue and the debugger grabs it and takes you straight to the error location with a full stack trace, and it's obvious what's going on. If it's slow performance or incorrect behavior, it might take a bit longer. But in most cases I can have a fix ready within the first 20 minutes, tops. I zip it and send it to them to test. "OK, I think I got it. See if this works at your end?"

The response is almost universally some mixture of amazement and gratitude, because most developers (read: software companies) just don't fix bugs and rerelease that fast. And then, if it's actually fixed, I've turned a potential critic into a fan. It's a really good technique; I just wish more developers would adopt it.

Answer 4

I personally take the problem positively. I do interact all the time with a lot of customeers, and I still code too.

When they download a new release and tell me something like that, I always say something like this:

Thanks for reporting me that bug. Maybe it has been introduced together will all the new features we added. We'll fix it asap.

In fact, the customer is your real boss. If the experience with you is bad, it's bad for you too.

Even if he is not right, you as a part of the company, you should take this opportunity to:

• learn from him, by collecting potential improvements you could make to the product.
• convert an unhappy customer into an happy one, so happy that he will talk about you around (including your boss)
• be proud of what you are doing

Answer 5

Details, Details, Details. I ask what they were doing and when, be specific. It could be something or it could just be that the Justin Beaber video just released on youtube. Log files are your friend in both cases.

I also ask for dates when they noticed it. Sometimes especially in enterprise shops users don't know when a release comes out, they just know that something takes a long time to complete and they are just now complaining about it.

Job Shadow. Can't emphasize this enough. If you are lucky enough to have users nearby, just watch them work from time time. I often find they ignore glaring issues and never report them. They will often only complain when they know something is new or they initially notice an issue.

Answer 6

Step 1 is that you must come from a mindset that this (the update breaks other stuff) is not normal. Your update should not break or slow other parts of the app. It's not ok, it's not to be expected, and it's not the user's fault when they complain about it. You should be doing as much testing as you can to try to prevent it. When it happens you have a problem, and an urgent one.

Step 2 is that you must know what you did. Your source control system might be able to help you, or some kind of work tracking system, but you must be able to say the minute you get one of these complaints "ok, I added a column to this table, changed this grid to calculate the new taxes, added those two new reports ..." and so on.

Step 3 is that you must have experience finding perf issues and crashes quickly, so you know what sorts of things are likely to cause them, and can get to the problem right away. This thing has gone live and you must find the problem quickly and get a patch out. Changing a report can't slow down a part of the app that doesn't use the report. You are in emergency mode now and have to figure out where the mistake is and what to do about it - without breaking another part of the app in the process.

Step 4 is for each one of these miseries, you should learn a lesson that you will test for next time. You will become "that guy" who objects to certain constructs because "that will be horrible when there are 10,000 records".

A little more on the "this is normal" front. I run (among all the other things we have going on) an agile project for an external customer. We've been doing releases roughly every 6 weeks for two or three years now. And yes, the release is scheduled to the minute. We just did one at 8am yesterday. And roughly every 4th or 5th release (once or twice a year, in other words) something gets broken live, and we leap into action and make it right as quick as we can. Even though we test and test and test before the release. Then we tell them what happened. "There was a little bug in the June deploy that let this field be blank, but we never noticed because we weren't using the value at that time. Then in this deploy when we started using the field, the ones that were blank caused that error message you saw. We've fixed the bug so they can't be blank, put values in the bad records, and confirmed that it doesn't blow up any more. Our apologies." Or "That emergency change you begged for, just two days before the release, had consequences we didn't think of and didn't test for. Remember why we resist emergency changes?" I may not be a bad programmer for making it worse with the update, but I surely did do a bad thing. And I need to make it right.

Answer 7

Just to cover another aspect:

We keep a list of customers who claim this when it turned out not so. While are software is buggy, often very buggy, many of our customers will claim "started with the update" to get prompt attention, not realizing this ends up wasting everybody's time as we will walk the deltas for the indicated feature looking for the problem. If the customer is telling the truth this tends to get it found fast. If the customer is on the false list too many times we don't bother as we don't like wasting time.

I cannot imagine what manner of mindset is required to think telling us a lie would speed up the process. These people are or work with doctors and should know full well what happens when people lie to doctors. Same principle applies.