angela zhou

assistant prof at USC Data Sciences and Operations and Computer Science; phd Cornell ORIE. data-driven decision-making, operations research/management, causal inference, algorithmic fairness/equity bureaucratic justice warrior angelamzhou.github.io

Articles & links

angela zhou reposted
Fernando Diaz @841io.bsky.social

Please try to look deeper than success rate for agent evaluation. Thank you. arxiv.org/abs/2606.17541

Offline Preference-Based Trajectory Evaluation arxiv.org
AI Weekly's analysis
  • Fernando Diaz argues success-only metrics tie agent comparisons on roughly 75% of instances, gutting statistical power.
  • His preference-based trajectory evaluation compares progress and time-to-return profiles, cutting ties to roughly 35%.
  • The paper suggests apparent benchmark saturation may reflect the evaluation measure, not exhausted data or problems.
Read full analysis →
View on Bluesky →

Recent commentary

i don't think we should waste real expert reviewer's time reviewing mediocre AI-written papers and having real expert reviewer provide valuable feedback that, presumably, will just get sent to LLMs to complete. at that point the reviewers might as well be authoring along with the authors

View on Bluesky · ♥ 38 ↻ 0 ↩ 3 · 31d ago

need an ai agent to pm me on my multiple projects (or keep me from adding too many projects to my stack)

View on Bluesky · ♥ 14 ↻ 0 ↩ 5 · 43d ago

In angela zhou's orbit

Center = angela zhou. Left = members they follow (green edges). Right = members who follow them (blue edges). Top = mutual follows (orange edges, slightly larger). Drag any node to reposition; click to open that profile.