Stanford Study: AI Wins 75% of Head-to-Head Matchups Against Law Professors in Blind Contract Law Evaluation — NotebookLM Tops Every Human Instructor
Summary
Stanford Law Professor Julian Nyarko published a study June 2 showing AI outperformed peer-written professor answers in 75% of nearly 3,000 blind contract-law comparisons evaluated by 16 law professors across U.S. law schools. Google's Gemini 2.5 Pro achieved a 75.92% win rate while NotebookLM outperformed every human instructor; professors flagged AI responses as pedagogically harmful in only 3.5% of cases versus 12% for human-written answers. The study targeted law specifically because it demands judgment, nuanced reasoning, and ambiguity navigation — areas where LLMs were not previously expected to dominate human experts.
Originally reported by forbes.com
Read the original article →Original headline: Stanford Study: AI Wins 75% of Head-to-Head Matchups Against Law Professors in Blind Contract Law Evaluation — NotebookLM Tops Every Human Instructor