Scott Berkun

#23 – How to run a design critique

By Scott Berkun, January 2003

In the early and middle phases of a project, teams need a way to understand and explore the current direction of the design. The challenge is to create the openness needed for good ideas to surface, while simultaneously cultivating the feedback and criticism necessary to resolve open issues. Unlike a brainstorming meeting, where the exclusive goal is to come up with new ideas, a critique meeting is focused on evaluating a set of existing ideas, and possibly identify future directions or changes. Instead of hoping that hallway and email discussions will lead the team in a good direction, it’s generally worth investing time to set up critique meetings to drive the design forward.

Goals of a design critique

A design critique meeting usually involves a small group of 3-7 to discuss a set of design sketches or prototypes. For a website or software design, there are many different attributes or constraints that might be worth discussing. You could focus purely on branding elements, ease of use concepts, or even engineering feasibility. It’s really up to whoever is running the meeting and what the most pressing issues for the project currently are. However, what’s most important, is that the goals of the meeting are stated at its beginning. If there are 3 or 4 specific lines of thought you want to make sure get critiqued, define them. Without goals or a basic framework for the kinds of design questions you want to explore, everyone will work from different assumptions, making for a frustrating meeting. It’s also worth clarifying any kinds of issues or questions that you’re not ready to answer, and when you expect to have answers for them.

If you are early in a project, critique meetings should emphasize the higher level user, customer and business goals, and minimize the focus on specific engineering constraints. It will be worth flagging design ideas that engineers or business managers have large concerns about, but hold off on completely eliminating them from the discussion. There may be opportunities to ask for more resources or make other adjustments to a project, if a stellar design concept or idea is championed successfully  (e.g. perhaps a design idea exposes a new business plan that has more opportunities than the current one, and would justify a change in the project goals).

But as the project timeline progresses, and the end of the planning or design phase approaches, the tone of critique discussions should change. There should be increasing pressure to have definite answers or solutions to issues, and the bar for considering new ideas or directions should get continually higher. If managed well throughout the project timeline, the scope of critique discussions should peak during planning, and then continually decrease until specifications are written, and final decisions are made. (Shepherding the creative phase of a project is a significant challenge, and it’s rare to find a project manager than can manage it as well as the more production oriented implementation and release phases. Often there is a key leadership role for designers to play to fill this gap. Overall, the tone, content and quality of critique meetings is one indicator to how well the creative process is being managed).

Typical goals for critique meetings might include:

1. Obtain specific kinds of feedback from those in the room about a set of different design approaches for one feature or area of a website.
2. Compare how several different components of the same product are designed. (Are there elements that should be reused more? Do things that look similar behavior similarly? Etc.)
3. Discuss the user flow through a design, by examining each screen in the sequence that users would go through to complete a task. (Similar to a cognitive walkthrough).
4. Explore the designs of competing products, or designs of other products that have elements or qualities that you want to achieve.
5. Allow teammates with different job functions to provide feedback from their expertise. (QA might raising testing issues, development might ask feasibility questions, marketing might ask questions about advertising or partnerships, etc.)

These goals listed are mostly mutually exclusive. You might be able to manage two of these at the same time, if you’re a great meeting facilitator, but I wouldn’t recommend it.

Secondary goals often include:

1. Provide some structure to the creative process of a project.
2. Improve your team’s ability to think about and discuss design ideas.
3. Teach non-designers about the design critique technique, so they can apply it to other kinds of problem solving situations.

Independent of the specific critique goals: If there are questions from your teammates about your design that don’t fit your intent for the meeting, make sure you come up with some way to address them outside of the meeting. During the meeting, write them down on a whiteboard or notepad, and take them with you when you leave. The more inclusive your design thinking is, the more influence and authority you’ll have over how project decisions are made. Even if the issues you are confronted with arise from decisions out of your control (a demand from the marketing team, or a new constraint from engineering) you want your designs, and your design process, to work with these issues, not around them. (Unless you feel confident that your superior design and skills of persuasion will convince someone with authority to change their mind.)

Who is in the room

A critique should allow a small group of people to review and discuss many ideas quickly and informally. You can’t be informal and intimate about ideas with more than 5 or 6 people in the room. Instead, you must narrow down your invite list to the people most critical to the design process. Try to forget about job titles or hierarchy, and instead, focus on the people who are most likely to understand the creative process, and give useful and meaningful feedback, both positive and negative.

Depending on the personalities of your teammates, make adjustments as necessary. For anyone on your team that isn’t invited to the meeting, allow them to look at any handouts or pictures, and give you their feedback.  Or even better, make sure to forward them any of the notes you send out following the meeting. In most cases, they’ll see the quality of the dialog and kinds of discussions points that were raised, and ease up on their complaints about not being in the room. And even in the absolute worst case, make yourself available to listen to their feedback independent of the critique session. You can diffuse difficult teammates, appeasing them without derailing the critique meeting, and the creative momentum of the team.

One alternative for designers in larger organizations: you might be able to do design critiques with the other designers in your organization, even if each of you works on different projects. This can be a great way to build a sense of design community in your organization, and give you the benefit of other well trained design eyes, that are fresh to the problems your trying to solve. The downside to this is that you miss on the opportunity to build better design relationships with the non-designers on your team. In the best possible world, you might have time to do both kinds of critiques, at different times in your project.

Materials and rooms

Depending on the kinds of designs your working with, and the goals you have, you might arrange the room differently, and bring different kinds of materials.

In the simplest kind of critique, where you have several alternatives to the same design problem, make it easy for everyone to see each design approach. There are several ways to do this:

1. Print a stapled handout of the 5 or 6 pictures and give each person a copy. This works fine, unless you have prototypes for each design approach – the printouts won’t capture that :) It might be fine to have rough hand drawn sketches, if the folks in the room are capable of working with rough representations, or you might need to have more complete visual presentations of the design ideas. (I once made the mistake of showing some high powered marketing folks some hand drawn sketches: it was a disaster. Unfamiliar with design work, or design process, they naturally confused the low fidelity of the sketch, with the quality of the ideas. Learn from this :)
2. Use wall space in the room to display each of the designs. This is by far the best way to examine branding or compare/contrast different areas of the same website. If you’re new to a project, and want to illustrate how inconsistent certain design elements are, there is nothing better than putting them all up on the wall together, and asking everyone in the critique to walk the room and examine them. If you’ve never done this before, I guarantee you’ll hear a few people gasp.
3. If the room has a television or monitor, use your laptop to show each of the sketches or designs. If you’ve made prototypes, demo them. Personally, I find this is the most convenient way to work. It usually requires little preparation beyond the prototypes themselves, and if I’m facilitating the meeting, it gives me additional control over what we discuss and how the discussion goes. Typically, what’s on the monitor is what we’re going to talk about.
4. If you work in a large organization, you might have your choice of rooms to use. I recommend small conference rooms that would accommodate 4-8 people. You want a room with lots of whiteboard space for new ideas, or for taping up printouts. Ideal is to also have a television or monitor so that everyone can easily look at the same designs from a laptop or computer.

If you are working on a long project, there is value in reusing the same room for critiques. You may be able to leave certain screenshots up on the walls ,or in the hallway outside. Plus you have the psychological advantage of identifying a single physical place with the kind of thinking and dialog you want for a critique.

The Rules of order for good critiques

Without some basics set of rules or guidelines, discussions about ideas can go in any direction. Many creative people (writers, filmmakers, artists, etc.) recognize this, and have certain shared guidelines or assumptions for how critiques should be run. Instead of starting with opinions and points of view, participants in the critique work to clarify the creators intent with the work, and only then, respond to how well the work achieves or does not achieve that intention. (e.g. – If the film director wanted you to feel angry when watching the opening scene, and you don’t feel angry when you look at it, this is useful. But telling the filmmaker you don’t like movies that make you feel angry, might not be as useful.) So when it comes to software or web design, it’s important to clarify assumptions before offering a criticism or challenging an assumption.

The general rules of order are:

1. Start with clarifying questions. Clarify any assumptions about what the presented design is intended to do, or what kind of experience it is intended to create. Hopefully, this intent is derived from the overall project goals, which is already agreed upon.
2. Listen before speaking. Many times in work environments, we confuse conversations, which should be exchanges of ideas, with opportunities to inflict our opinions on others. If you take a moment to listen and understand before voicing an opinion, you’re open to hear something new that might challenge your old thinking. So don’t just wait for other people to finish, actively try to understand what’s being said, and reflect it back to the speaker.
3. Lead into explorations of alternatives. Ask questions that surface other choices the designer might not have recognized. Postpone judgments, unless there are obvious gaps between the designers intent, and the designs you are critiquing.
4. If it fits with the goals of the critique, point out situations, sequences, or elements within the design that may be problematic given what you know about your customers, the scenarios involved, or the project goals.
5. Avoid statements that refer to absolutes. Instead, make points referent to the goals of the design. Example:

   bad: “This sucks and it’s ugly”
   good: “Well, if the goal is to make this feel friendly, black and flaming red doesn’t convey that to me.”

   bad: “How could anyone figure that out?”
   good: “I think there’s something missing between step 3 and 4. It’s not clear to me what the sequence of operations is.  How do you expect people to know where to click?”

6. Speak in context of your point of view. It’s fine to have a personal opinion, expressing your own preferences. But don’t confuse this with your perception of what your customers need or want. So make sure to specify which kind of opinion you’re offering. Hopefully there is data and research to help everyone agree on the likely customer perspective on different ideas.

Running the meeting

Someone should be responsible for leading and driving the meeting. This is more about facilitation than dictation. For a critique to work, everyone has to feel open about voicing their opinions and discussing ideas. It requires a different style of leadership than a status or accounting type meeting. The meeting leader, or facilitator, has to be comfortable asking quiet people to speak up, or loud or obnoxious people to quiet down.

I recommend that the creator of the designs lead the meeting. They should be confident and mature enough with the creative process to lead other people through it.

The biggest challenge to this is their ego. If the designer is leading the meeting, and controlling the discussion, there is every opportunity to push for feedback that makes their pet ideas shine, and exclude everything else. It requires maturity on the part of the designer to walk in the room with the attitude that the value of the meeting is to hear new thoughts and opinions, rather than simply to defend the ideas they already have. On a healthy team, the designer should be rewarded for the quality of their output, regardless of who may have made what suggestion, or gave birth to what initial idea, so there is little real conflict of interest. However, in the end, who runs the meeting is less important than the quality of discussion, and the overall progress of the design effort.

When a critique meeting is going well, it’s fun. It should feel like an informal conversation between people with the same goals, all trying to explore and surface good thinking. The person running the meeting has the responsibility of setting the right tone for this, preferably by example, and doing everything in their power to maintain that attitude and spirit in the room throughout the meeting. On a good team, this responsibility should come easily.

Pre and post meeting work

You may want to do some up front work to ensure that the critique goes smoothly. If you can define the goals when you call the meeting, include them in the meeting announcement. Also attach any pictures of the designs or sketches, so interested folks can get a head start in thinking about the designs.

After the critique, there is some additional work you can do to close the loop, and set yourself up for your next critique, or follow up meeting. Make sure to take notes during the meeting of key questions that were raised, or new issues and ideas that came up that you hadn’t thought about before. Send out a mail after the meeting with these details, and what the next steps in the design process will be. How long will it take before you have new designs to show? Is there a usability study that is the next milestone? How will this effort integrate with the project managers plans? If you don’t provide answers to these questions, someone else will, and you’ll default to yielding some control over the design process to them.

There are some organizations or project teams create critique forms, listing the standard questions or criteria that should be considered in a critique discussion. These are handed out during the meeting, used by participants to take notes, and then might be collected at the end. This might be more process than your team might want – so it’s up to you to figure out how formal or informal the critique discussions should be. I might recommend something like this for the first time you do it to help define this kind of meeting, but probably not as a general practice.

How often to run critiques

It depends heavily on the project and the team. If you have easy access to the people you are working with on the project, you might not need to have defined meetings for critiques. If the team is healthy, critique like conversations are probably happening in the hallway all the time. As long as you are in most of them, and people see you, the designer, as the driving force for the design effort, things might be fine. On the other hand, If you as the designer feel that most of the design conversations happen without you, intentionally or not, creating a weekly critique discussion can help put you back in the middle of the creative process.

(Note: A separate type of design meeting is a brainstorming discussion, where the dominant goal is to generate new ideas and explore as many different ideas as possible. This should not be confused with a design critique.)

Possible discussion points / questions to use

Here’s a sample list of design questions that might be of us to help guide the discussion. Again, depending on your goals for the critique, you might focus on, or avoid, some of these.

• What are the user scenarios the site is designed for? Walkthrough how each design would enable those scenarios.
• What known usability / design / business issues are these sketches trying to solve?
• What is the intended style of the design, and is it appropriate for the target audience?
• What is the intention of the style, and does it achieve the desired effect?
• Are there standard brand elements that should be used, and are they used appropriately?
• Are there similar software products or features that these designs should relate to?
• What usability heuristics does each design support well? (or not?)
• Where in the design are the most likely places for users to have trouble? and why?
• Are there reasonable design changes that might avoid these problem points?
• Does each design idea take advantage of things the user might already have learned?
• What are the pros and cons of each design idea, relative to each other?
• Are there any hybrid design ideas that are worth exploring, based on the designs in the room?
• What open issues might best be resolved by a usability study or other research?

Other references for design critiques

The art of web design critiques - mostly about one on one discussions about designs.
Teamwork and Critique - A nice set of slide decks on engineering processes, including critiques.

17 Responses

• 
• rachel - January 9, 2009 at 7:55 am #

• this is extremely helpful! thank you so much for posting it. do you know of any books on this subject?

  
  

• 
• Scott - January 9, 2009 at 10:26 am #

• I don’t know of entire books on design critiques, but I can ask around and find out.

  
  

• 
• Scott - January 10, 2009 at 5:01 pm #

• Found this good list recently:

  http://risdcollegiateteaching.wordpress.com/2009/01/10/critiques/

  
  

• 
• jj - March 1, 2009 at 2:29 pm #

• Using the “Six Thinking Hats” method might be useful. It also lets everybody focus on different aspects at the same time. So when it comes the time to focus on defects, it’s not “personal”, but part of the task.

  
  

• 
• Frédérick Brault - November 20, 2009 at 4:29 am #

• Great post! Comprehensive and smart. Glad I stumbled upon this! Will explore your work. Thanks!

  
  

• 
• Brian Wigham - March 5, 2010 at 10:04 am #

• Thankyou for this information, I believe Airbus could learn from this.

  
  

Links to this article

• Business to buttons « MW - June 16, 2007 at 9:09 am

• [...] Jens talked about the nuts and bolts of his work at Yahoo! It was a nice change of pace from the usual high level stuff. Too bad he had so much to say that we never got to the Design Critique part… I’ll see if I can get some more information on the topic from him some how. In the meantime, check out Scott Berkun’s Essay #23 – How to run a Design Critique. [...]

  
  

• Infovore » links for 2007-08-14 - August 15, 2007 at 9:50 am

• [...] scottberkun.com » #23 – How to run a design critique “The challenge is to create the openness needed for good ideas to surface, while simultaneously cultivating the feedback and criticism necessary to resolve open issues.” Tell me about it. (tags: design review product management development) [...]

  
  

• Tipple of Choice - April 21, 2008 at 7:44 am

• [...] this quote from a great post about how to run a design critique. Thought it’d be useful to distil the recommendations and publish to clients – but also to [...]

  
  

• ADGA Intro to Web Design » Blog Archive » Class 5: Critique - October 8, 2009 at 2:25 pm

• [...] How to run a design critique By Scott Berkun, January 2003 [...]

  
  

• Design Critiques: | - November 15, 2009 at 1:17 pm

• [...] Resources I have gathered: How to run a design critique Critique Your [...]

  
  

• Fragmented Communities and the Chapter/SIG Web Site Problem | I'd Rather Be Writing - Tom Johnson - February 1, 2010 at 8:35 am

• [...] more on running a design review, see Scott Oberkun’s How to Run a Design Critique and Makiko Itho’s The Delicate Art of (Web) Design [...]

  
  

• User Experience Critique Guidelines « Obviously Easy - April 27, 2010 at 8:09 am

• [...] of the following steps are s drawn from a Scott Berkun Essay on How to Run a Design Critique , and are used with the authors expressed written [...]

  
  

• UX for the masses | Tips for bringing UX to the Agile party - May 22, 2010 at 3:16 am

• [...] quickly and collaboratively generated and discussed using workshops, such as the design studio and design critiques, or simply by bringing designs out in to the open so that everyone can collaborate and critique [...]

  
  

• » Beer and learning this week at iQ - iQ Blog - May 31, 2010 at 2:41 am

• [...] templates for all major design programs Stefan Klocek’s blog on conducting a design critique Scott Berkun’s blog on how to run a design [...]

  
  

• UX for the masses | The joy of sketch(ing) - July 15, 2010 at 1:55 am

• [...] Having sketched out a bunch of designs you now want to workout which work best. Ideally you want to whittle your stack of designs down to one or two that show the most promise. Of course this doesn’t necessarily need to be a particular design – it could be combination of designs. If you’re in a project team then you might want to run a design critique to evaluate the various design sketches (checkout Scott Berknun’s excellent article about how to run a design critique). [...]

  
  

• Futurechat | Home - August 9, 2010 at 5:19 am

• [...] Digital, Life Design Conference – DLD How to Run a Design Critique – Scott Berkun [...]