Rangel

Judgify

A web app that decides which professor judges which research poster, then ranks the posters fairly.

Read the code Open the app
Built for
ECS Challenge, Syracuse University
When
February 2025
Team
Rangel Koli, Suved Ganduri, Saad Shah
Stack
Next.js, TypeScript, Supabase, Flask, PuLP, SPECTER, CrewAI
Problem
Matching 40 judges to posters by hand takes hours, and it is easy to send a professor to their own student's poster.
Solution
An optimizer that reads each judge's research and each poster's abstract, then builds the best schedule that follows the event's rules.
Result
Judges sign in with a code, score on their phone, and the final ranking corrects for tough and generous graders.

The problem

At Syracuse University's ECS poster challenge, students present research posters and faculty judge them. Someone has to decide who judges what. Doing that by hand means juggling four rules at once:

  1. 2

    judges on every poster

    Never one opinion, never three.

  2. ≤ 6

    posters per judge

    Enough to compare, few enough to finish inside the hour.

  3. 1 or 2

    the hour a judge is free

    Odd posters are shown in hour one, even posters in hour two. A judge only gets posters from the hours they marked free.

  4. 0

    posters from a judge's own student

    Matched on advisor name, so no one grades their own lab.

And within those rules, every poster should go to the judges who know the most about its topic.

The assignment

This is the real result from the sample data in the repo. Each poster on the left is linked to its two judges on the right. Hover over, or tab to, any row to follow its lines.

  1. 01Flexible tactile sensorsEECS
  2. 02Parallel sensing in metaversesEECS
  3. 03Smart-city IoT securityCEE
  4. 04Free-space quantum commsEECS
  5. 05Smart drug-delivery materialsBMCE
  6. 06Smart fibers for energyMAE
  7. 07Lead-free double perovskitesCEE
  8. 08Semiconductor process dataBMCE
  1. Nadeem Ghani
  2. Chikukuri Monhan
  3. Qinru Qiu
  4. Senem Velipasalar
  5. Elizabeth Carter
  6. Svetoslava Todorova
  7. C.Y. Roger Chen
  8. Jason Pollack
  9. Shikha Nangia
  10. Yaoying Wu
  11. Bing Dong
  12. Wanliang Shan
  13. Ruth Chen
  14. Mary Beth Monroe✕ 05
  15. Ashok Sangani
The solver's real draw for the 8-poster sample in the repo: 40 judges available, 15 used. The dashed line is the pairing the rules forbid: Mary Beth Monroe advises poster 05's author, so she judges 08 instead.

How matching works

Three steps, run once before the event.

  1. Read up on every judge

    A CrewAI agent on Gemini 1.5 Flash searches Google Scholar through Serper for each of the 40 faculty judges and writes a short profile: core areas, key papers, current work.

  2. Score every pairing

    Judge profiles and poster abstracts go through AllenAI SPECTER, a SciBERT model trained on scientific papers, to measure how close they are. Each poster's program is also mapped to a department (MAE, BMCE, CEE or EECS), and the two signals count half each.

  3. Solve it in one go

    PuLP turns every judge–poster pair into a yes/no choice, and the CBC solver picks the set with the highest total score that still obeys all four rules. The result is saved as spreadsheets and pushed to Supabase for the web app.

What the solver maximises

maximise Σ c(j,p) · x(j,p)

where c = 0.5 · cos(judge, abstract) + 0.5 · same department

and x(j,p) = 1 if judge j gets poster p, else 0

On the day

The Next.js app has two sides: one for the organiser running the event, one for the judges walking the room.

The organiser

  • Uploads posters and judges as spreadsheets and runs the matching.
  • Checks the result judge by judge.
  • Emails every judge a five-character access code.
  • Watches scores arrive, exports them, and runs the final ranking.

The judge

  • Signs in with the code. No account, no password.
  • Sees only the posters they were given.
  • Scores each one from 1 to 10, with notes, and can edit until judging closes.
  • Can only read and change their own scores, enforced by row-level security in Supabase.

Fair ranking

Some judges grade hard and some grade easy, so raw averages aren't fair. Judgify compares each score with the rest of that judge's scores instead:

  1. Extreme scores are clipped to the judge's 5th and 95th percentile.
  2. Each score becomes a z-score: how far above or below that judge's own average it is. A minimum spread of 0.5 stops a judge who gives everyone a 7 from distorting the result.
  3. Each poster's z-scores are added up and scaled to 0–10.
  4. A score of 0 means “didn't review” and is left out, not counted as bad.
Why a 6 can beat an 8

Raw score, 1–10

Judge A (tough) gives 6

Judge B (generous) gives 8

14710

Compared with the judge's own scores

Judge A (tough): z +1.55

Judge B (generous): z −0.35

−2−10+1+2

Judge A's 6 is the best score A gave all day. Judge B's 8 is about average for B. After normalising, A's 6 counts for more. Example scores; the method is the one in the repo.