Sculley paper: ML's quick wins hide 'massive' system costs
TL;DR
- A 2014 Google paper applies the technical-debt framework to machine-learning systems, arguing the 'quick wins' of ML are not free.
- It names seven categories of ML-specific risk, from boundary erosion and entanglement to hidden feedback loops and undeclared consumers.
- D. Sculley and seven co-authors presented the piece at the SE4ML workshop at NIPS 2014.
"Machine learning offers a fantastically powerful toolkit for building complex systems quickly," the paper opens. "This paper argues that it is dangerous to think of these quick wins as coming for free."
Authored by D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary and Michael Young, the piece appeared at the SE4ML: Software Engineering for Machine Learning workshop at NIPS 2014. Applying the technical-debt framework to ML systems, the authors note "it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning."
The paper enumerates the hazards by name: "boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, changes in the external world, and a variety of system-level anti-patterns." The abstract lists them; the full paper expands each.
Two of the researchers we track resurfaced the link, more than a decade after its original workshop appearance.
Shared on Bluesky by 2 AI experts
-
It's not sexy to address tech debt. It's not an area where you can argue for headcount. And people at Google were writing over a decade ago about how machine learning systems incur significant debt. research.google/pub…
View on Bluesky →
Originally reported by research.google
Read the original article →Original headline: Machine Learning: The High Interest Credit Card of Technical Debt