Think Outside the Screen When Conducting Research
As we entered the credit union in a quiet East Texas town, everything in the lobby, office cubicles and workrooms appeared clean and immaculate. We were there to observe an account manager using some aging banking software.
Upon entering her office, I noticed the decor was sparse and corporate. Promotional materials and a customer-facing monitor took up the front part of her desk. She answered all of our questions, repeatedly mentioning that she had no complaints with her job nor with the software she was using: “This is just the way it is; I love my job.”
My mantra going into these engagements: “Focus on what people do, not what they say they do.”
After our initial interview, we asked to sit with her behind her desk. From this view, an entirely different dynamic appeared. Her monitor and surrounding area were littered with Post-it notes, handwritten numbers, codes, file folders, quotes and affirmations, pens, and unfinished paperwork. Why was all of this stuff here? How did it help her?
When conducting contextual inquiries, you must go beyond observation of the application itself. Everything surrounding it must be taken into context.
“What’s the purpose of these sticky notes?” we asked. She told us she needed to remember several different passwords for the systems she used (again, this is bank software).
She indicated the system required complex formulas to complete specific actions, so a key to these formulas was carefully handwritten and laminated. She had also created a printed checklist she would review before submitting each new account application.
These were just three hurdles she used to overcome the deficiencies of the software and we hadn’t yet seen a screen of it.
Here’s a partial list of things to observe to truly place the “context” in Contextual Inquiries outside of the software in question:
- Sticky notes: Do they contain reminders of dates, phone numbers, passwords, contacts, codes or formulas? Do they look old and worn? Probe to understand the purpose and see if there is a reason the software they use can’t facilitate these (e.g., single sign-on, notifications, reminders). Even if the user has committed bits of information to memory and may not use them anymore, the fact that they had to memorize them in the first place is telling.
- Printed paper: Do legacy processes exist that require printing out, faxing (yes, a lot of businesses still use a fax machine), filing of forms, or documentation that could be digitized?
- Desk organization: Are they neat freaks or messy? What is the level of entrenchment? Does it appear they ‘moved in’ years ago? Do they have pictures of family members? Do they maintain plants? Do these accessories help them cope with the stress of their jobs?
- Reference books and materials: Are they there for show or does the employee regularly reference them? Does the employee require constant re-education to use new or ancillary areas of the system?
- Operating system’s desktop: Do they have a screen filled with file shortcuts or is it neatly organized with a few folder icons? The use of more file icons points to the employee’s need to operate outside of the software to complete a task.
- General environment: What is the temperature of the work area? Do employees have coats on their chairs in the middle of the summer? What is the lighting like: bright fluorescents, natural light, or dim light? Do these affect their ability to perform optimally?
- Ergonomics: Do they use hotkeys to reduce mouse click stress? Do the positions of their chairs or desks appear awkward or improperly adjusted?
- Monitor size and resolution: How many monitors do they use to do their job? Are they constantly having to maximize, minimize or move windows?
- Interruptions and noise level: Do phone calls or coworkers interrupt their flow? One user constantly had to log in multiple times a day as the system logged the employee out after five minutes of idle time.
We have interviewed a myriad of employees and customers in numerous verticals. More often than not, they will initially state, “My job is not stressful, I’m happy here, everything is fine,” partially because they consider themselves under the microscope rather than the ancient or broken software they are using. After an hour or more of strict observation, what they said was “easy” is clearly not.
People learn to overcome deficiencies of a system after years of trial and error, manual processes and other analog methods. To them, it is just “the way it is.” They even feel a sense of pride that “I am an important asset to this business.” perhaps not because of their business knowledge, but their ability to understand and deal with the limitations of the software they use.
You can get a much clearer picture of reality when you observe the external supports to a piece of software.