Your Job as a Juror

You've been invited to judge awards for a festival. What next?

Written by George Reese

Last published at: April 19th, 2023

Thank you for supporting your festival with your time and expertise.

Your festival is using a tool called the Sparq Festival Platform (SparqFest) to manage the juried voting process. SparqFest enables you to focus your time on what matters instead of messing with spreadsheets or physical media.

Your primary job as a juror is to review all finalists for each award you are judging. Once you have reviewed each finalist, you deliberate with your fellow jurors and pick a winner according to the rules established for your award.

Once deliberations are complete, you finalize your deliberations through a formal “finalization” process.

Finalization

We'll start a discussion of the process with the last step: finalization.

We start here because the single most common user-error among judges is finalizing your vote before you have discussed your vote with your fellow jurors or before you have reviewed all of the selections.

ONCE YOU FINALIZE YOUR VOTE, YOU CANNOT MAKE ANY CHANGES.

We apologize for the bold, all-caps above and for repeating this mantra over and over again in the SparqFest Knowledge Base. Finalization is your way of telling the festival directors that you are done and no one can change your mind.

Prior to finalization, you can do whatever you want. We encourage you to rate selections and place preliminary votes as you review selections. You can change your ratings and votes as many times as you like until either your vote is finalized or the judging deadline has past.

The Rules

A festival has the option of picking from a number of different voting rules for each award. Different rules may apply to different awards in the system. You should therefore always be aware of the rules that apply to the award you are currently working with.

Vote-based Rules

The most commonly used rules by festivals are “vote-based”. Under a vote-based system, you pick your favorite nominee for a given award. Once all judges have picked their favorite nominees, SparqFest tallies the votes and crowns a winner.

How SparqFest crowns a winner depends on the details behind the specific vote-based rules. An award may require unanimity, a plurality of votes, or a simply majority:

  • Unanimous - all judges must agree
  • Plurality - 50% of the judges + 1 judge must agree
  • Majority - anyone achieving the most votes

If no nominee achieves enough votes to win or if two nominees are tied under the majority rules, the award will enter a tie breaking scenario.

Rating-based Rules

An alternative to voting is rating selections. Note that you may rate nominees under a vote-based system, but those ratings are solely for your personal use. The system does not assign any meaning to those ratings.

Under rating-based rules, however, your ratings form the basis of selecting a winner. Instead of picking your favorite, you rate every single finalist according to the judging criteria for the category. The rating is on a scale of 1-10, with the values having the following meaning:

  1. You don't think this selection should have even been nominated for this award
  2. You don't think the selection should be considered for this award
  3. This selection is a weak finalist and barely worthy of runner-up status
  4. This selection is slightly below average for the finalists for this award
  5. The selection is average among finalists and thus not worthy of the award
  6. The selection is slightly above the average among the finalists, but probably not worthy of the award
  7. This is a really good selection and you would be happy if they won, but they aren't your choice
  8. You believe this is the best finalist, but you can see how others might disagree
  9. You would be disappointed if this finalists does not win the award
  10. This finalist is among the best works you have ever seen as it relates to this category

Given the above criteria, only one of the finalists should receive a rating of 8 or above. You should generally have exactly one finalist ranked 8 or above unless you believe that the entire slate of finalists is sub-par.

You are required to rate every finalist before you can finalize your vote. Once all judges have finalized their votes, SparqFest will crown a winner either by taking the average of all jury ratings or calculating the totals.

Finalists vs Runners-Up

The nominees for each award are divided into finalists and runners-up. All are promoted as nominees in public communications and are otherwise treated as equals. SparqFest does not make public any distinctions between finalists and runners-up and, as a juror, you should never disclose which nominees were finalists and which were runners-up.

A runner-up is a selection that the initial screening process has determined to be worthy of a nomination, but not worthy of an award when compared to the finalists. Practically speaking, you can ignore runners-up with one caveat: If your festival accepts nomination recommendations, you may be asked to review runners-up for the sake of making recommendations. You will know that recommendations are requested because you will see a “My Recommendations” section on the nominee review dialog.

In short, you vote on the finalists but refer both finalists and runners-up for recommendations to other awards categories.

Making Recommendations

As noted above, a festival can be configured to allow you to refer a nominee in a category you are judging to the festival directors for a possible nomination into another category. A practical example of this process is the way Minnesota WebFest executes its juried awards.

Minnesota WebFest has a two-phased voting process:

  1. Genre awards
  2. Skill and other awards

In phase 1, the juries reviewing the awards for different genres are tasked with recommending the nominees they watch for skill and other awards. Thus, a member of the jury for “Best Drama” can recommend one or more of their nominees to receive a nomination for one of the skill awards such as “Best Director”.

There is no correlation between your deliberations for your award and whom you may recommend for other awards. In other words, just because you gave a work a “2” for Best Drama doesn't mean you cannot recommend it for a nomination to Best Production Design. If you think the work was weak as a drama, but had great production design, you can recommend it for a Best Production Design award.

You may also recommend runners-up for other awards. In fact, we strongly encourage you to review the runners-up for potential recommendations.

You may recommend a single nominee for any number of awards and you may nominate multiple works for the same award.

Taking Notes

SparqFest provides a tool to enable you to record secure notes for your own personal deliberations. When you enter notes, the notes are stored in an encrypted format in the SparqFest database. No one can read these notes except you. Even if the SparqFest database were to be stolen, the notes will remain secure and private thanks to the encryption. These notes are never provides to SparqFest staff, festival staff, other jurors, or anyone else.