When Compiler Engineers Act as Judges, What Can Possibly Go Wrong?