Line Judging

Line judging involves co-designers voting on a graduated scale on ideas, prototypes, or technologies. It can be particularly helpful in evaluating many similar items quickly.

Line Judging is good for use when your team wants to rapidly get feedback on numerous ideas to help to determine design directions. For example, you might be creating a website for teaching children a specific math concept and want to get the feedback of the team quickly on many other websites that teach that concept. Or, you might have already mocked up a few math websites. Line judging can be used either for existing technologies or for ideas for potential future technologies.

• Supplies: Masking or similar tape

• Preparation: Use masking or other tape to make a line on the ground. Mark one end of the line as positive and one as negative, such as with a smiley face and a frowny face, or with a “+” and “-”. The line should be long enough for each member of the co-design team to stand along.

• One designer describes or quickly demonstrates a technology or idea. Each member of the design team moves their body to a space on the line to indicate how they feel about the technology – more positive, negative, or neutral. This allows for a more nuanced form of feedback than a simple “like” or “do not like” vote.
• A discussion is then had as to why each designer moved to where they did on the line. If it happens that all co-designers are tending to move to the same space each time, guide the conversation to determine if this is truly how co-designers feel, or if they are moving to vote with each other. Adult co-designers should also be moving along the line to indicate their thoughts on the various design prompts, which should help with all co-designers feeling comfortable to indicate their preferences.
• During the discussion, designers might find that they wish to move along the line to another space.
• It can be helpful to both take notes as well as photos of the end results of the line judging for each technology or idea.

• If working online, co-designers can participate in a modified version of Line Judging by finding three objects of three colors (for example, red, yellow and green) at their homes. Then they can hold up an object to react to each prompt, for example, red for I don’t like it, yellow for neutral, and green for I like it. Similarly, you can ask co-designers to hold their hands high for I like it, middle for neutral, and low for I don’t like it. This does lose some nuance and the immediate ability to judge where each co-designer falls in line with each other and thus some possible conversation; however, it does preserve some of the initial intent of the technique.

Lee, K.,J., Roldan, W., Zhu, T. Q., Saluja, H.K., Na, S., Chin, B., Zeng, Y., Lee, J. H., Yip, J. (2021). The show must go on: A conceptual model of conducting synchronous participatory design with children online. In CHI ’21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1 – 16, https://doi.org/10.1145/3411764.3445715.

Walsh, G., Foss, E., Yip, J., and Druin, A. (2013). FACIT PD: A framework for analysis and creation of intergenerational techniques for participatory design. In Proceedings of CHI 2013, 2893 – 2901.