Judging to deliver a winner

Back to Resources


Vital to any competition is the judging process - it is at the core of choosing a successful winning design.  There are some key aspects of the judging process that should be considered.

 

Setting up the judging panel

This takes time so is best started as soon as the decision to run a competition has been made.  There is no set number of judges that need to be on the judging panel but keeping the numbers to a manageable level while also encouraging a robust process is the aim.  Between 3 and 7 judges may be ideal.  Judges can be selected from all walks of life - from youth and community representatives through to eminent designers and experts.  Selection will depend to some extent on the goals of the competition, but usually a mixture of opinions and perspectives is sought.  High profile members of the judging panel can often encourage more participation in the competition.

Nominations for judges can come from professional institutes and stakeholder contacts.  The selection of the judges should be agreed by the core project team and communicated to project partners and sponsors to ensure that everyone is happy with the panel choices.  Potential judges should be approached informally to see if they would be interested, and then followed up with a formal letter of invitation.

 

Terms of reference

A written set of guidelines for judges should be prepared as their terms of reference.  This should include any specific aspects of the judging process and make clear the judges’ roles and responsibilities.  When you have a multi-disciplinary group, it is important for all judges to see where the professional skills and personal experience needed for the panel to complete its task lie.  This background helps judges who don’t know each other understand the full scope of the job and where their experience is supported by others.

 

The judging panel

From the outset, it is important that judges are clearly briefed on the vision and goals of the competition as well as the process and timeline to be followed.  Being a judge is a prestigious position but one that carries responsibility and needs commitment to see the project through.  More often than not this is an unpaid position, but travel and accommodation costs should be covered, as well as a koha, if appropriate.   Given judges are unpaid, careful management of their time commitment is important (busy people need long lead-in times to secure a day or two in their calendars).  Take time and care that this is well managed so competition admin doesn’t become a burden to your free judges!

 

Pre-materials

To help the actual judging day, it is useful to provide the judges with a pack of the entries and some form of pre-assessment against competition criteria.  Work with your judges on when and how they want to receive this material.  Architectural competition entries are usually submitted in A1 format; you have to carefully consider how and if you print entries for each judge and, if providing digitally, how you provide entrants and pre-assessment in a structure that is easy to navigate.  The pre-assessment helps eliminate entrants that don’t qualify (avoids wasting judges’ time), and provides the judging panel with a first cut of how the project interprets competition criteria against qualifying entries.  Naturally, the panel is free to accept or reject this pre-assessment, but it provides a competition-specific foundation from which judges from different disciplines can begin their deliberations.

 

Judgement day

Decision making is a complex dynamic process; you are asking a group of people to come together for a short intense time, consider your competition criteria, and decide which entrant, in their view, meets project outcomes.  Your panel will probably have a mix of life experience, gender, ages, communications style and previous judging experience as well as professional and personal skills.  Your chairperson is critical to holding this fluid process together (select wisely!).   Another option is to engage a facilitator to support the judging process; this enables the Chair to participate in judging rather than managing the process.  A facilitator is an independent person who acts as a neutral servant of the judging panel, who encourages participation, maintains group processes, resolves conflict and makes agreed tasks happen.

There are a range of decision making tools which can support the panel coming to a consensus.  Different tools are appropriate to different stages of the decision-making. At the beginning, a simple joint triage of entrants can help whittle down the pile and start conversations about what each judge values or doesn’t against each entry.  In short, the panel takes each entrant in turn and puts it in one of three piles:

  • We all agree this entry is great and should proceed into our detailed decision-making.
  • Some of us think this is great, others aren’t sure or disagree, needs more consideration.
  • We all agree this doesn’t make the grade (important that the secretariat records the underpinning panel justification for this decision).

 

Bag of options

The way in which the entries are dealt with will depend on the previous skill and experience of your panel and the dynamic among them.  It is possible the secretariat (or facilitator) could have a ‘bag of options’ to support decision making if the panel seem stuck. An example is provided below (Breathe Stage 2 Judges’ Scoring Matrix) of a highly structured scorecard, which can actively structure discussion around judging criteria, but you could also opt for more participatory methods.   Options include:

  1. “Bag of beans”: where each judge is given equal number of counters, stickers or stones and use these to show their support for different entries. Judges may spread their counters across several entries or simply empty their entire bag on one entrant. This may quickly identify the “no-goes”. While this allows a bit of anonymity for judges to honestly signal their preferences, it is a means to encourage discussion.
  2. Positive / Negative / Interesting: Another quick way to capture key issues is ask judges to identify top positive, negative and interesting points for each entry. This is, then, the basis for group discussion.
  3. Pros and cons: a simple table can summarise the group’s perspective of the pros and cons of each entry
  4. Swap sort: this works when fewer entries are under consideration. Agree up to three criteria you will use to sort.  Make a card for each entry and shuffle them. Draw a line and place the first two cards side by side, put the priority one to left or (or above) the second. Complete the swap in pair-wise manner, then go to top priority (left or top) and sweep along/down the line repeating until you have an agreed priority order.
  5. Positive / Intriguing / Negative / Concerning filter: This works well with a short list of entrants, less than six, ideally two or three. The judging panel fill in a PINC filter for each entrant in this format (use big paper that allows the panel to stand back and jointly review each PINC.

Entrant:

 

Positives

Things that ADD value

Negatives

Things that REMOVE value

Intriguing

Curious things that COULD be of value

Concerning

Worrying things that could REMOVE value

 

It is important to note that these methods can be a “rubbish in - rubbish out” process.  Judging should be underpinned with good criteria and accurate scoring and a constructive approach across the panel.  The chairperson and/or facilitator wants to avoid individual judges ‘gaming’ the process if they don’t feel heard or they aren’t listening!

 


  • 16-Jul-2013 (Publication )

    Breathe Stage 2 Judges Scoring Matrix Template (Excel 44KB)

    The spreadsheet template of the judging matrix used in Stage 2 (Detailed design) of the Breathe competition.  Scores each entrant against the criteria for the competition.


Breathe logo

Breathe logo

Breathe finalist

Breathe finalist: Walker Architecture & Design Ltd + Ceres NZ

“It is exciting to see a design which attempts to craft a new vernacular of building that is unique to this part of the world and through which - thanks to a clear passion for narrative - local distinctiveness, architectural idiom and a sense of place and history are all placed at the fore.“
Judges

 


Breathe highly commended

Breathe Highly Commended: Ganellen, Tonkin Zulaika Greer Architects

“The scheme uses interesting shapes and forms in the northwest corner and considers distinctive architectural design throughout.
Judges

 


Breathe highly commended

Breathe Highly Commended: Westbrooke Capital Partners, Boffa Miskell Ltd + Sheppard & Rout Architects

“Good to see some public realm design here…and the considerations for social sustainability through the sharing of resources.“
Judges

 


We invite you to submit feedback to improve this toolkit in the future:

Contact Beacon