forbes.com via Reddit

Stanford Study: AI Wins 75% of Head-to-Head Matchups Against Law Professors in Blind Contract Law Evaluation — NotebookLM Tops Every Human Instructor

research education google notebooklm research legal-ai

Summary

Stanford Law Professor Julian Nyarko published a study June 2 showing AI outperformed peer-written professor answers in 75% of nearly 3,000 blind contract-law comparisons evaluated by 16 law professors across U.S. law schools. Google's Gemini 2.5 Pro achieved a 75.92% win rate while NotebookLM outperformed every human instructor; professors flagged AI responses as pedagogically harmful in only 3.5% of cases versus 12% for human-written answers. The study targeted law specifically because it demands judgment, nuanced reasoning, and ambiguity navigation — areas where LLMs were not previously expected to dominate human experts.