Comments on Bringing User Centered Design to the Agile Environment

Anthony Colfelt — Wed, 03 Mar 2010 14:43:34 GMT

@ Isaac Loloely As you say, it’s easy to step on these mines. But, with experience, you can avoid them if you incorporate some good product development and UCD techniques. There are a few things you’ve said that I’d like to discuss.

(1) You’ve stated that scrum has a requirements gathering process. But I disagree. I think what you’ve described is a requirements DEFINITION process and there is a subtle difference. One is about working out what requirements to write down (gathering). The other is how you write them down (definition). What I advocate is that there is proper rigor applied to gathering via deep empathetic design research, some of Indy Young’s mental modelling technique or similar, then defining them through stories.

(2) XP describes a customer as your client. Frequently when you’re developing software for a client, that client actually has customers who will be using the product you’re building. I’d advocate that in this case, you need to be testing with that end customer, not just your client, who may have no better idea about what the end customer understands than you do.

In the case where the client is representative of the user of your software, having them on site allows them to get quite familiar with the difficulties you face and they sympathize with the trade offs you have to make. They get “Stockholm Syndrome” (i.e. identifying with their captors). They lose fresh eyes and become less critical as they become more and more familiar with the developing product. Great for the development team, not so great for the other users who will have to use the product on which your client has compromised the best UI. This is why we usually try not to user test with the same participant more than once. The more familiar they become with the interface, the less likely they are to help you see what is not intuitive about it. I know that there are some interfaces we design to be learnable before instantly usable, but still, these interfaces needs to be intuitive to learn too.

(3) You make a very good point. No process can guarantee that someone with the power to do so will call it “good enough” before time. However, by incorporating some UCD techniques such as user testing, we can implement some gating success criteria that go beyond just what the client (with an eye for saving money) would normally accept. Of course, we’d do this in the best interest of the client’s longer term development cost, since hopefully, they’d not need to come back and redo it again and again for what would have been cheaper to fix (in dev cost, customer acquisition or in brand damage) at the first time of development. The same goes for proper QA. The war story I told a the beginning of the article suffered greatly from Mine 4 at the expense of user testing AND rigorous QA. Why? Because we were slavishly following an Agile process rather than being pragmatic about what would ultimately be the best thing to do.

Isaac Loloey — Fri, 07 Jan 2011 20:38:27 GMT

I really enjoyed your article. It makes some very valid points, which I have made myself in the passed. I am a certified Scrum Master myself, and I often encounter these problems and abuses.

I wanted to add that there are some of your points that are well accommodated by Agile / Scrum / XP.

For instance:

(1) From the section “Mine 2: The requirements gathering process is not defined”.

Scrum does have a requirements gathering process. Requirements are documented in the form of User Stories, and managed in a product backlog.
Although it is true that one of the biggest abuses of Scrum/Agile is that people think that it eliminates documentation all together. This is a common misconception and untrue.

(2) In the section “Mine 3: Pressure to cut corners” the author implies that Agile does not consider the clients needs in the development cycle.

“This can and does lead to impulsive design. ... you’re not adhering to user centric principles which suggest you should test ideas with end users before committing them to code.”

In the eXtreme Programing (XP), a different flavor of Agile, practitioners advocate having an “On-Site Customer”. A concept emphasizing the importance of the client’s needs.

“With XP, we get extreme about customer involvement. we’ll mandate that the customer be on the project full-time for the duration and be located on-site with the team.” – Stewart Baird

Refer to the following links:

http://www.informit.com/articles/article.aspx?p=30187&amp…
http://www.extremeprogramming.org/rules/customer.html
http://www.agilemodeling.com/essays/activeStakeholderPart…
http://www.xpexchange.net/english/intro/onSiteCustomer.ht…

(3) The following comment from the section “Mine 4: The temptation to call it “good enough” – “Agile condones releasing whatever we have so long as it works. Sometimes, that means doing what we can get away with, not what is ultimately best for the user. Equally, if we do decide that a feature isn’t right yet, it’s amendments get fed back into the requirements backlog where temptation strikes again.”

The truth is, that in every process people are guilty of releasing a product just because it fulfills the requirements. This is nothing unique to Agile. Iterative development is beneficial to meeting deadlines, otherwise products would never be released.

There are several points here that I do agree with like:

“Agile is also good at avoiding feature bloat by encouraging developers to do only what is necessary to meet requirements.”

and

“Because Agile requires a lot of engagement with the client (i.e. at the end of every iteration, which can be as little as a week) it mitigates the risk of going too far toward creating something the client doesn’t want.”
We often see that different tasks do have dependencies on each other. Development being dependent on design, QA being dependent on developers. Creates a constraint for tasks to be done simultaneously.

aryan kirod — Fri, 05 Feb 2010 18:18:12 GMT

Good points Chris!

Rajeev Kumar — Thu, 04 Feb 2010 19:29:24 GMT

Good points, Chris. Very good suggestion to David’s point. I agree with you 100% that even the best can deliver “average” quality work if they work under bad process/culture/structure.

For figuring out what form of Agile works for the team, my approach so far has been to introduce the smart guys to Agile principles and let them help us figure out the practice details. As a change manager my job is to get them the culture and structure (and other org stuff that they need) to bring the best out of them. Not that i am successful in getting all they want. Agile philosophy doesnt really pays off if the “org and skill stuff” is not aligned. I tried to introduce to Agile at a very large company. The change did not take roots as i couldnt align the “org stuff” with Agile. Sometimes, a well defined gated/heavy process is what works for a company. Not that i want to be a part of that kind of company.

In my current organization, our product delivery cycle looks similar to what Anthony has described as Agile UCD. Thats how our practices evolved. For “good enough” and product backlog prioritization we let the team (include the CX/UX team members) decide what works for them. Our guidelines are to have customer experience/User Experience leads and product managers make the decision together . We run our “delivery” prioritization/planning meetings every other week that takes care of “Agile as we know”. Its not uncommon for us to take stories/epics out the delivey backlog and put into ideation part “research and concept” part aka I0/pre I0. “Dedicated” team can ask questions make suggestions but when it comes to the aspects of user experience, the decision is made by the CX/UX person. There are other reasons why a story can be thrown out of the delivery queue. This works well for us.

For us, Agile works both for new/big ticket items that go through good ideation process and for “lights on” work (that keep the team busy ). Ofcourse, we have longer time in I0 for big ticket items than for enhancments.

On a side note, I kind of understand what Anders says. I dont personally like house analogy when it comes to building software. Its funny that yesterday I had a product manager in my office using the same analogy to talk about what their product ideation cycle should be so much longer than the other products. Not that i knew the right answer on how long it should be..

Chris Baum — Wed, 03 Mar 2010 14:45:00 GMT

Due to a random technical issue, Anthony can’t post comments.

Here are Anthony’s responses to comments thus far:

@ David Travis I have never worked out a formula for estimating how long something will take to research and concept with relation its development time. I think that would be hard because different situations call for either more or less research time according to approach. Make sure you allow enough time to a) to clearly define what your user requirements were through whatever research techniques you have at your disposal and b) sketch out the ideas and validate them with users on paper then c) create yourself a kind of pattern library or interaction model that you can use as your reference point when going into the sprint cycles.

@ Anders Ramsay I respect what you’re saying about a totally new paradigm and competencies. But I think you dismiss the blueprint metaphor too hastily. The very fact that there are no “building codes” is the issue I’m trying to illuminate. When it comes to interface, it’s simply not good enough to have many ways to skin the cat. I personally don’t care what happens in code (though perhaps that’s just ignorant of me and I should) what matters to me is that the interface is consistent from one section of an application to the next. When you build one thing in isolation to the next without a “code” for how that should work at an interface level, you end up with different interaction styles for each item. Whether that’s a subtle difference or a huge one is down to the team’s ability to share and communicate with a shared frame of reference such as an interaction model. Maybe this is the Quick Start you talk about? What I don’t agree with, is that this all assumes requirements you’re building to are in fact the right ones to address the user’s needs and goals. I believe, that design has a huge role to play in defining requirements through deep empathetic design research into user needs and goals. That has to come before we start sketching anything – waterfall? It simply has to be if it is to mitigate the risk of wasted development cost. Too many companies treat Agile as some kind of substitute for good product development practice and that’s the minefield.

@ Rajeev Kumar - I accept your point about process vs approach. UCD is an approach too, but I think we can get caught up in semantics. Both have certain activities that get done in a particular ways. Your point about Apple, Google and Amazon and people is spot on. You really can’t expect a process/approach to give you good results. As the adage says “What you put in, is what you get out…” However, you can have talented people hamstrung by a poor process/approach… Who says “good enough” is the topic of more hot debate, and one that’s well worth discussing!

Julie Booth — Wed, 03 Feb 2010 17:13:42 GMT

Thank you for this article, Anthony. I think your conclusion touched on Rajeev’s points: “dogmatic attitudes about each of these [ucd /agile} approaches should be avoided if they are to be combined. ” Right, Agile is a philosophy and not a process… My experience has been that people who are ingrained in long time waterfall methodology and old-school practices cling to a process. It is really hard to get both parties to think philosophically :). Anthony points out the minefields there—for that I am grateful. Am bringing this article to a meeting today about “Institutionalizing UX” into our recently-converted-to-agile culture.

Rajeev Kumar — Tue, 02 Mar 2010 23:14:32 GMT

Overall, its a good article. It covers most common issues (in the context of user experience) that folks run into when they use Agile. However, based on my experience with Agile, many of the statements seem to be very generalized. Agile is a philosophy and not a “process”. Agile is supported by well thought practices and guidelines that are based on Lean principles. One can apply “Lean thinking” even on your UX design work as well. As a matter of fact my wife who manages finance uses Agile within her team and works well with planning, communication and collaboration. There are good adoption of this philosophy and there are bad adoptions. I liked this blog entry by Steve Yegge – http://steve-yegge.blogspot.com/2006/09/good-agile-bad-ag…. Here he talks about “good” agile and “bad” agile.

I have worked with companies where most of the “cycle time” was spent on defining the product vision and user experience. Even with people working on user research and design for months, there design and ideas were not even close to “Apple”, “Google”, “Amazon” or any other companies that seem to have a good handle on user experience and innovation. There is something to be said about the “people” and culture of those companies. More lead time and more process by themselves do not deliver good design and innovation.

Currently I am in the process of introducing Agile into my current organization. As we go through the Agile transformation (which includes people, process and tools components), I have been asked to define how the “ideation” happens in Agile world, where would user experience fit, how long should we should we be staying in ideation, when do we have good enough UI design to start the development and so on. My answer is that it depends. One thing is for sure that we need to think about overall product vision and overall user experience before burning development hours. How long will you have to stay in I0 or pre I0 depends what you are working on. If you do it right, you are very likely to strike a good balance where designers get enough time to research and design, and are empowered to say NO its not ready when its not good enough. When to say “good enough” is important and who says that its “good enough” is important as well.

Anders Ramsay — Tue, 02 Mar 2010 23:14:13 GMT

Hi Anthony – great article. At a high-level, I think you really do a great job of highlighting some of the shortcomings of Agile and the challenges of integrating User Experience Design into Agile. There is an overarching theme here of UX people trying to fit their work into an Agile model and getting really really frustrated.

I see two reasons why that is the case. First, Agile was not intended to solve the UX challenges; it was intended to solve developers challenges, which is why, as you say, there is an unclear role for design (tho’ “Design” has a different meaning in Agile.) Second, and this is a point I don’t think you make that is fundamental to this entire discussion: You talk about a typical Agile process. Let’s be very very clear: **There is no typical Agile Process** because Agile is not a process, it is a way of thinking about design, an attitude. More importantly, **Agile is a completely different paradigm compared to waterfall** – this is why UX folks keep banging their heads against the wall in frustration when trying to fit their roles into Agile. Trying to fit the role of Information Architect or Interaction Designer into an Agile team is like trying to figure out how to best put a steering wheel on a bicycle. Sure, it is possible to do it, but it will be awkward indeed, because a steering wheel was designed or an automobile paradigm and not a bicycle paradigm. This is something that is very very hard for traditional designers to get their head around. Agile replaces the idea of roles with the idea of competencies. Like a sports team, while you may generally play left field, you would not hesitate for a second to take over the center fielder role if the situation demanded it.

Re. your statement that “spending money on software development without a plan of what to build is like asking a construction crew to erect a tower with no blueprint” is a perfect example of old thinking being applied to a new paradigm and completely flies in the face of fundamental Agile thinking. The construction of a building is a *terrible* and misleading analogy for building software, which is nothing at all like physical engineering and construction. The very reason why we are stuck with a waterfall model is because the engineering model was misapplied to software. In contrast to building a physical building, in which you have standard communication notation (e.g. building codes), software has no such attributes. This is why software development plans in fact are no more than speculation. *That said* I absolutely agree with you that there is a need for some up-front planning, some big-picture design, something we at ThoughtWorks refer to as a QuickStart, with “Quick” being the operative term. In other words, it needs to be very brief and intense, and with the understanding that until you start building, you really won’t know what you’ve got. In other words, until you are building you are only speculating and proposing. It is not until you start building that you really are defining which is why I disagree 100% with the statement that Agile is good for refining, not defining.

Sorry to come off so negative. Great article overall.

Dave Rooney — Fri, 07 Jan 2011 20:37:03 GMT

Anthony,

This is a great article. I’ve been involved in the ‘Agile world’ for almost 10 years, and have seen some groups lose sight of the forest (the overall product) for the trees (the individual bits that make up the product). I’ve also seen IA & UX integrated very well into the process, with excellent results.

You should consider proposing this topic for the Agile 2010 Conference in Nashville in August. Submissions may be made until February 26th at http://agile2010.agilealliance.org/speaker.html .

Dave Rooney
Westboro Systems
http://www.westborosystems.com

David Travis — Tue, 02 Feb 2010 09:30:38 GMT

You’ve written an excellent article and helped me think through aspects of Agile that bothered me but that I couldn’t quite articulate. Thanks for that.

One question I’m often asked by developers when I emphasise that we need to do research in Iteration 0, is “How long?” I know the answer (“How long’s a piece of string”) but I wonder if you can put a more defined number on it. In your experience, in projects that get it right, what percentage of the time/budget is typically spent on Iteration 0?

Rick Butler — Tue, 02 Feb 2010 00:34:36 GMT

Great Article Anthony – I think you have clearly identified the obvious challenges to agile culture. I think you could take the very mines you describe and create a likert scale survey for stakeholders to measure the agile effect on an organization’s product development…

Doug Winter — Mon, 01 Feb 2010 19:34:05 GMT

We have been using a process very like the one you describe extremely successfully for the last few years, although only after a few bad starts and screw ups, of course.

We make a clear distinction between *what* and *how*. We want a well defined UX, with a complete wireframe stack *before* we plan a release. The release plan provides a good structure that we expect to conform to during the build – although Agile allows us in theory to change everything we’d consider that to be failure of planning.

What we do use the agile process for is to enable us to flex scope *instead* of either time or quality, and I think this is it’s greatest strength. We do therefore need to plan how the UX would work if a feature is omitted completely – but this is generally something you can do with a well designed UI. What we do NOT do is completely rearchitect the UX each iteration – UX is just too fragile and expensive to change.

In effect I think the wireframes are not part of the specification, they are part of the deliverable and our (often long) Iteration 0 reflects this, with the wireframes considered a key deliverable, to help the customer work out what they want and how it might work.

Clearly a lot of attention is still required to ensure we have happy customers – the overall promise of Agile that it naturally produces the best ROI is probably only possible in a situation of 100% trust, not something an agency will ever encounter. But it does mean we can flex as we need, rather than committing to a completely defined triangulation of scope, time and quality – which is what causes developers such pain.

Julia Debari — Mon, 01 Feb 2010 18:54:30 GMT

I currently just ended a very Agile project and ran into many of the minefields discussed in this article. There was a sprint 0 for planning and the first day of every sprint we had an inception meeting. Unfortunately, 99% of it was spent on the developers. The UX team did work a sprint ahead, which worked well enough.
I think one of the stickiest issues was that the business owners literally did not know what they wanted and went to the developers and asked them what could be done. The “what” was decided before the “why” which I think made for a much more complex situation.
Looking back on the project, I think certain aspects of the project would have worked better if the UX team sat with the developers and sketched out what they wanted in real time. While very difficult, the idea of seeing the “whole picture” needed to be let go and the UX team should have worked in a more component based model.

Tony Collen — Mon, 01 Feb 2010 18:56:06 GMT

I’m currently pushing for a UCD process at work, and we’ve drank the Scrum kool-aid already.

The points I’m having the largest difficulty resolving are mines 2 and 5.

For #2, because we tend to do 2- to 3-week sprints, on the development team we tend to not know what’s coming up until our sprint planning meeting. We tend to do a lot of responding to RFP’s, which is a minefield in itself. Often as developers, we get the standard spreadsheet of requirements directly from the customer, without much analysis into the problem.

Which leads into #5. Once we know what we need to do, the sprint timer has started, and we often don’t discuss designs and possible solutions until we’re in our sprint planning meeting. Our situation is made worse by the fact that the developers on this team are all remote to each other, so our meetings tend to take place over the phone.

I’ve been an advocate for getting more “look-ahead” in our process, and having a specifically defined role within the organization that works with project management and customers to come up with a design before dumping work on engineers.

Ray Sharma — Tue, 23 Aug 2011 09:07:14 GMT

Great article – In addition, I am intrigued to find out how much of this approach you are or have implemented and what were your SWOT outcomes – a follow up article I think. One of the biggest oxymorons is that Agile promotes leveraging those who know best how to implement technology/solution rather than those who define what should be implemented. Clients and other business units including UX, who are much more focused on the ‘what’, are being over shadowed today. As with many other evolved industries such as motoring, this will change where research, ideas and the market define what to build rather than technology alone. However, I suspect we need more UX business type people to help drive this, people who talk business, understand ROI and value and can sit with mangers, directors and CEO’s if not become them.