Data study

Do AI models rate faces like humans? Inter-rater agreement, shared vs private taste and determinism

How much two human raters agree with each other, how geometry-plus-vision models compare against human panel consensus and why unanchored chat models drift across identical photos.

Published
Last reviewed
Version
1.0
Reading time
9 min
Author
LooksMeter Research

Key findings

  1. 01Variance-component studies show that human attractiveness ratings split roughly evenly between shared taste (consensus across raters) and private taste (individual observer preference), capping the correlation between any two single humans well below 1.0.1
  2. 02When individual noise is averaged across panels of 15 to 40 raters, group mean ratings correlate strongly within cultures (around 0.90) and across cultures (around 0.85 to 0.94).2, 3
  3. 03Pure neoclassical canon formulas explain only roughly a quarter of human panel variance, whereas combining measured facial ratios with learned visual appearance cues reaches correlations comparable to human-to-panel agreement.4, 5
  4. 04Even with a deterministic scorer, changing camera distance or lighting across two photos of the same person produces real physical differences in depicted ratios and rated appeal.6, 7

Section 1

The claims

People testing automated face raters usually ask one of two opposite questions: whether beauty is completely subjective (so no score can ever mean anything) or whether an algorithm can output one objective truth that every stranger on earth will agree with. Psychometric research on human raters and computer-vision benchmarks establish clear numbers between both extremes.

Table 1Psychometric and computational findings on human vs automated face ratings.
ClaimWhat the studies reportEvidence
Facial attractiveness is purely in the eye of the beholder with near-zero rater agreementRefuted. Meta-analyses of hundreds of rating panels find high reliability for group averages both within and across cultures.2, 3Not supported
Any two individual human observers give virtually identical scores to the same faceRefuted. Private taste accounts for roughly half of reliable rating variance when individual raters are decomposed from group consensus.1Not supported
Hybrid models combining landmark geometry and image appearance track human panel means better than classical canons aloneSupported. Adding pixel/texture representation to landmark ratios raises correlation with human panel means substantially over classical canon equations.4, 5Consistent

Consistent: several independent studies point the same way. Mixed: studies point in different directions. Limited: one or two studies, not yet repeated. Not supported: tested, with no effect found or one too small to matter.

Section 2

How much two human raters actually agree

Before asking how closely a machine matches humans, psychometrics asks how closely one human matches another human. Hönekopp (2006) had judges rate facial photographs twice so repeatable personal preference could be separated from random momentary noise.1 Across three experiments, shared taste and private taste contributed almost equally to reliable attractiveness judgements.

Table 2Typical correlation benchmarks in facial attractiveness psychometrics.
ComparisonTypical correlation (r) / reliabilitySource & interpretation
One human rater vs one other human raterr ≈ 0.45 - 0.55Reflects ~50% shared taste plus ~50% private taste and scale noise (Hönekopp 2006)
Human panel mean vs second panel (same culture)Effective r ≈ 0.90Individual idiosyncrasies cancel across 15+ raters (Langlois et al. 2000)
Human panel mean across different culturesr ≈ 0.85 - 0.94Cross-cultural consensus on shared structural and expressive cues (Cunningham et al. 1995, Langlois et al. 2000)
Neoclassical canon ratios alone vs human panel meanR² ≈ 0.24 (r ≈ 0.49)Geometry alone misses skin, hair, expression and non-canon balance (Schmid et al. 2008)
Landmark ratios + visual appearance model vs human panelr ≈ 0.60 - 0.65+Outperforms a single random human rater in predicting the panel mean (Eisenthal et al. 2006)

Section 3

Shared consensus across cultures

Langlois and colleagues (2000) computed meta-analytic inter-rater reliabilities across adult and child faces: within-culture effective reliability reached 0.90, cross-ethnic reliability reached 0.88 and cross-cultural reliability reached 0.94.2 Cunningham and colleagues (1995) found correlations of 0.85 to 0.93 across Asian, Hispanic, Black and White rater groups evaluating portrait sets.3

What an automated rating system models, therefore, is never every individual stranger's private taste. At best, a calibrated model estimates the shared-taste consensus - the baseline average a large unfamiliar panel would converge toward on that exact photograph.1, 2

Section 4

Geometric canons vs hybrid vision models

Early computational studies tested whether textbook Renaissance proportions could reproduce human panel scores by themselves. Schmid, Marx and Samal (2008) combined neoclassical canons, bilateral symmetry and golden-ratio distances on frontal portraits: geometric terms were statistically significant yet accounted for under 25 percent of rating variance because two faces with identical ratios can differ dramatically in eye contour, skin clarity, hair framing and expression.5

Eisenthal, Dror and Ruppin (2006) demonstrated on frontal portrait datasets that combining explicit point-to-point geometric ratios and symmetry with holistic image-level appearance features substantially outperformed either representation on its own, reaching correlations with human panel ratings around 0.60 to 0.65 - higher than the typical correlation between one single human judge and the panel.4

Section 5

Why general-purpose chat models drift on face ratings

  • Politeness and flattery bias: Conversational assistants are post-trained to keep users happy, compressing almost every adult selfie into a narrow 7-to-8.5 out of 10 band instead of a true 50th-percentile bell curve.
  • Sampling randomness: Non-zero decoding temperature causes the same uploaded photo to receive different numbers and contradictory feature critiques on back-to-back runs.
  • No explicit 3D coordinate anchor: Guessing angles and ratios purely from pixels without roll-correcting head tilt or measuring landmark coordinates directly makes chat responses sensitive to slight framing shifts.

Even when a dedicated tool locks repeatability on identical files, photo physics still applies across different photos: Jenkins et al. (2011) showed huge natural score spread across 20 candid photos of one person while Ward et al. (2018) proved that moving a phone from 1.5 metres to 30 cm inflates apparent nasal width by roughly 30 percent.6, 7

Section 6

How to read a LooksMeter score responsibly

Section 7

Limits of this review

Published machine-learning benchmarks evaluate static frontal portraits scored by student or online panels, so correlations reflect consensus on still photographs rather than face-to-face interaction. Automated measurements are informational estimates and are never clinically validated assessments.

References

  1. 1Hönekopp J. Once more: is beauty in the eye of the beholder? Relative contributions of private and shared taste to judgments of facial attractiveness. Journal of Experimental Psychology: Human Perception and Performance. 2006;32(2):199-209. doi:10.1037/0096-1523.32.2.199 · PubMed
  2. 2Langlois JH, Kalakanis L, Rubenstein AJ, Larson A, Hallam M, Smoot M. Maxims or myths of beauty? A meta-analytic and theoretical review. Psychological Bulletin. 2000;126(3):390-423. doi:10.1037/0033-2909.126.3.390 · PubMed
  3. 3Cunningham MR, Roberts AR, Barbee AP, Druen PB, Wu CH. Their ideas of beauty are, on the whole, the same as ours: consistency and variability in the cross-cultural perception of female physical attractiveness. Journal of Personality and Social Psychology. 1995;68(2):261-279. doi:10.1037/0022-3514.68.2.261
  4. 4Eisenthal Y, Dror G, Ruppin E. Facial attractiveness: beauty and the machine. Neural Computation. 2006;18(1):119-142. doi:10.1162/089976606774841602 · PubMed
  5. 5Schmid K, Marx D, Samal A. Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios. Pattern Recognition. 2008;41(8):2710-2717. doi:10.1016/j.patcog.2007.11.022
  6. 6Jenkins R, White D, Van Montfort X, Burton AM. Variability in photos of the same face. Cognition. 2011;121(3):313-323. doi:10.1016/j.cognition.2011.08.001 · PubMed
  7. 7Ward B, Ward M, Fried O, Paskhover B. Nasal distortion in short-distance photographs: the selfie effect. JAMA Facial Plastic Surgery. 2018;20(4):333-335. doi:10.1001/jamafacial.2018.0009 · PubMed

Every reference was checked against Crossref before publication. Those indexed in PubMed were checked there as well. Findings are summarised in our own words. Read the original before relying on a number.

How to cite

LooksMeter Research. Do AI models rate faces like humans? Inter-rater agreement, shared vs private taste and determinism. Version 1.0. 19 November 2026. https://www.looksmeter.com/research/do-ai-models-rate-faces-like-humans

About this document

LooksMeter is a consumer appearance analysis product. This document is written by the LooksMeter team for general information. It has not been peer reviewed or reviewed by a medical professional. The measurements it describes are not clinically validated and are not a diagnosis or a medical device. See a qualified clinician about any health or surgical decision.

Corrections are welcome at [email protected]. Material changes are recorded by raising the version number above.

Continue