Agile Usability Design & Testing
Apr
28
When I first found out that our company was going to switch over to the SCRUM agile development methodology, I was extremely skeptical. After being with the company for 5 years as a web designer, I had already been a part of two major application overhauls. Both of which took months and months to design. Now, we were planning on implementing a methodology that crammed a set of tasks to be done by a team of designers, developers, QA engineers, and analysts into 30 days worth of work. Since day one, I wasn't sure how we'd be able to incorporate design and usability into this type of process. There were a number of questions I had:
- How many days out of the 30 can be dedicated to design and usability testing?
- How can you possibly design a usable interface in such a short time frame?
- How can you execute usability tests in such a short time frame?
- How can you make modifications based on the UAT results in such a short time frame?
As you can see, a pattern was developing here: "How can (fill in the blank) be accomplished in such a short timeframe?" When I approached anyone with these questions, the responses were not good. The reason: usability design wasn't necessarily thought about when this methodology was created.
Agile’s iterative development cycle is one of the method’s strengths, but it also makes for some tight deadlines. As the now infamous interview between Alan Cooper and Kent Beck [2] shows, the timeline is perhaps the most controversial aspect of agile methods. In such a high-speed development cycle, do we have time to fully understand users’ needs? The short answer is no. The long answer: If we’re defining users’ needs during development—even in an agile development process—something has gone horribly wrong. -- Clash of the Titans: Agile and UCD
If at this moment, you are feeling a bit uncertain, never fear, if you look closely, you can see a shiny silver lining around this dark and dreary cloud. Never fear, there is a way to incorporate usability design (UX Design) into the agile development process. I'm pleased to report that we have been able to execute a very successful sprint cycle that involved iterative design processes and usability tests to create a well received deliverable.
Start Small
I cannot stress enough just how important starting small is. In our first attempt at a design based sprint, we failed. And we failed miserably. Why? The reason was because we tried to tackle a task that was way too big. Being the awesome designers that we knew we were, we decided to try and fix the system wide navigation. We knew there were some workflow issues, and we were going to fix them. Ummmm. BAD IDEA!. This task was way too large to try and take on in 30 days. Needless to say, the deliverable for that sprint was a rather useless Visio document showing everyone everything they already knew.
So, in the next sprint planning meeting, we realized that we had our priorities a bit out of whack. We decided to take a look at the product backlog a bit more closely, and found one task that was rather high up in priority on the list that jumped out at us: "Display more information on this page." So, we took a closer look at the page, and realized that the story itself "Display more information on this page", wasn't exactly a desirable solution. In actuality what was needed for this page, was a complete redesign. So instead of adding more results to this page, we decided to make this page much more user friendly.
The key here, is to try and understand the reasoning behind a requirement. Don't hesitate to challenge the original story/requirement. It often times helps to ask questions about each requirement, so that you end up getting the entire story. Once you have this, you can begin to understand what is truly desired, and you can begin adding tasks based on this information.
Stay A Sprint Ahead
In order to create a usable interface, you'll need at least 30 days. If you want to deliver a great product that will impress your users and upper management, you will need enough time to brainstorm, plan, design, and prototype. So, give yourself a 30 day head start. Then, when your 30 days are up, you can hand off your prototype to the engineers, who can in turn add the task of completing the code to roll out to production.
It's okay to be 2 or 3 sprints ahead, however, any longer, and your prototype sits on the shelf for too long, and by that point will more than likely be considered obsolete.
Get A Room
When you are ready to get started, I highly, highly recommend commandeering a room. In this room you will need:
- a whiteboard
- dry erase markers (not having these will render the whiteboard useless)
- post-it notes
- writing utensils
Brainstorm and Whiteboard
At this point, you should know the problems and issues that are the reason that have you pent up in this room. Take these problems, and flesh them out. Brainstorm on what can be done to alleviate these problems. Write your answers on the whiteboard or on post-its, and leave them there. When you are doing this, be bold, be creative. Pretend as though there are no limitations, and any idea you come up with can be accomplished. You can dismiss these at a later date. This is the time where there is no such thing as a bad idea.
Prototype. Test. Repeat.
Once you've got your ideas squared away, the next thing to do is begin prototyping. So, start on the whiteboard. Draw out in rudimentary fashion the design ideas you have for your page. Once you feel like you got the idea down, start putting it down in digital form. We used Photoshop, and created a pretty extensive design. Now, you may hear from others that you do not want to make your designs / mockups too detailed because the people you are showing them to will begin picking apart things that you really don't want them focusing on. Let me say that I have had plenty of experience with this, and I agree - to an extent. The problem, however, is that with SCRUM or any agile methodology, you don't have that kind of luxury. By day 30, you want to be able to deliver something to your engineers that they can begin developing (i.e. you need to be done). That being said, create your mockups, and try and get the major details ironed out. Once you feel satisfied with the design, begin creating the prototype you'll be using for testing.
For our prototypes, we used PHP with mySQL, and created a quick and dirty test database with one table and the necessary columns that we were able to import a small portion of data into. Then we began work on the XHTML, Javascript, and CSS work. Once the development of this was done, we incorporated the PHP code. At this point, we had a working prototype that we could begin testing with actual users. All in all, it took about a week to get to this point, and I must say, we loved what we had. It was eons better than the page that currently existed in the application. We were ready to roll it out at this point. We couldn't imagine how our users could see much wrong with this new design. and that, my friends, is why you need to test with users.
So, in order to keep a consistent testing environment, we created a set of real life scenarios that a user would run into when using this page of the application. Then we ran each user through these scenarios. In our first round, we put three users to the test. Overall, the new design was extremely well received, however, there were quite a few of the smaller things we missed, that without testing, we would have never realized. When all was said and done we tested with 12 users, and based on our findings, changed the prototype seven different times. Prototype. Test. Repeat.
Involve Your Supporting Cast
When you feel like you have your prototype to a point where the changes you'll be making are minimal, bring in the engineers, QA, and the tech writers. Keep them updated as to what is on the horizon. Make sure your engineers know what you're planning, and that what you are proposing is possible. If you can, give QA a copy of your prototype so that they can begin to write automated test scripts. And finally, involve your tech writers. Give them the ability to begin writing the documentation in the early stages. Let them know that it isn't complete, but the meat of the design won't change.
Show It Off
When you're done, show off what you've got. Make sure you get buy in from everyone in the company, not just the product manager, and not just the users of the application, but you also want the developers to be happy with what you've created. You want everyone to anticipate what is coming up. With this project, the most common question we received from everyone was: "When will this be available?" Not a bad question to be asked.
To sum it all up in a couple of bullet points, here are the things to remember:
- You only have short timeframe, so start small, and get stuff done
- Make sure you have a workspace that accommodates for group collaboration
- Brainstorm and ignore any constraints
- Design quick, create working prototypes
- Test, take notes, modify prototype, test new prototype, take more notes, modify new prototype, etc. etc. etc.
- Involve the people that will be working to get this new design into the production application
- Show off what you've got, and get buy in company-wide
Next, I plan on explaining just how to get buy in from upper management to allow for these usability sprints. I also plan on writing about what was involved in our user acceptance tests, from setup, to scenarios, to post testing documentation.

- Tagged as: agile, Design, prototypes, SCRUM, usability, usability testing, UX
- Categorized under: Design, Usability, Web Development

Nice tips, guess they will really help me in adapting usability engineering practices into SCRUM methodology. Thanks!