Skip to main content
We split out judges and classifiers into two different primitives. Why? In many ways, a judge is just a classifier (typically multiclass), but it’s a special case.
  • They typically operate over model outputs as input data
  • Their primary purpose is to provide verifiable measurements to otherwise unverifiable model outputs
  • They are specifically is trying to follow the judgement rubric of a human expert, therefore requiring autoregressive reasoning capabilities (can’t really be built as a traditional ML classifier)
  • Judges may be composed of multiple classifiers (composing multi-dimensional rubrics), rather than single field outputs
Judge design, purpose, and application areas often differ from other types of AI classifiers. While not fully dissimilar, we have broken them out into two distinct primitives for the purposes of this guide.