What are your advice, methods, or practices to take out the most from a day on-site at a customer?

Question

We just deployed a large software that affects the way the user work-day looks like in many aspects. It changes a lot of things in the way they interact between eachothers.

The developers of the team are taking rounds and spending one day at the customer's site to understand better what is a typical day for our user, and the learning process they go through.

In this context, what How would you approach that day, what kind of questions do you ask, how do you observe the user. Are there defined practices that exist?

Also what wouldn't you do? Did you have some bad experience out of this?

Answer 1

What not to do:

• Forget to tell those you are observing who you are and why you are there. They will assume you are there for some bad reason like determining if they should get fired if you don't explain.
• Tell jokes. You don't know their corporate culture and what is appropriate in the IT world is often vastly different thtn the rest of the world. At best they won't think you are funny, at worst you could have a sexual harrassment suit on your hands.
• Be careful with scheduling, don't pick a day when they won't have time to talk to you (such as when payroll is done if you are observing the financial department.)
• Don't get defensive if they criticize your application! There is a good chance it will be very humbling to hear how much they hate your product. Remember what is important to them often bears little resemblance to what is important to you.

What to do:

• Take notes.

• Ask if you can observe before observing someone. Not everyone feels comfortable being observed. Ask the management to point you to a relatively experienced person and a relatively new person and people at different levels of the organzation or who have differnt functions to observe. You want to see a broad group of users actually using the product. Never under any circumstances talk only to managers unless only managers use your system. Managers do NOT know how users use the system but they think they do.

• Ask them what parts of the application that they find awkward or difficult to use

• Ask them what they like
• Ask them where they typically experience performance problems
• If they enter data from paper forms, get blank copies if at all possible - it helps if the user interface is designed to follow the flow of the form.
• Ask them if there is something they still have to do manually (by this I mean using something other than your application) because the application doesn't do it. Also look for them doing such things while you observe. Particularly note if you see them doing something manually that the applciation actually should do. That may mean a pain point - a place where it is easier to use an Excel spreadsheet than use your application.
• Observe for awhile before you start asking them questions. Note questions as you observe, you will probably find other things you want to ask.
• If there appear to be performance problems you were unaware of, find out the stats on their systems (then you can build a test system that is a similarly slow machine using the same operating system) and time some of the tasks that seem to be taking so long. In fact do this even if you don't have performance problems right now. It helps to see exactly what operating sytems, internet browser they use as well as how fast in general their machines are.
• Be unobtrusive while observing - they should know you are there and why, but then stay out of their way. You will get better data about what they do after they forget you are there.
• Ask for more detail after they tell you something. Often the first answer is far from the real answer. People forget to mention details that are critical to you. Don't forget to ask about edge cases. If they tell you they then send this to be approved, find out what should happen when it isn't not just when it is.
• When they describe a problem, ask if they can show you. Often they will miss parts of the process or will not understand how something is done. This can help with the user documentation and training.
• If you get time, show them what you need from them when they report a bug. This can improve your bug reports greatly.

(Can you tell I spent almost 15 years collecting this kind of data on processes?)

Answer 2

I would shadow the user throughout their day. Sitting with the user and observing how they use the system and what their workflow really is. Ask questions about any aspect of their process that you don't understand by observing. Its a good way to learn what users are really doing.