simonwillison.net web signal

Willison tests GPT-6 Astra route planning, flags compaction gap

TL;DR

  • Simon Willison asked ChatGPT Work with GPT-6 Astra (Max) to plan 5K and 10K loops from his address using OpenStreetMap data.
  • The run took 27 minutes and produced GPX, GeoJSON and an HTML visualization, including a 5.1 km 'El Granada harbor loop.'
  • Willison could not retrieve the Python the model executed because the thread had been compacted, which he calls an 'anti-feature.'

Simon Willison handed ChatGPT Work, running on GPT-6 Astra (Max), a one-line brief: "I live at <my address>. Figure out 5K and 10K running routes from me that loop from my house. Use OSM data." Twenty-seven minutes later, he writes on his blog, the agent returned an embedded map, downloadable GPX and GeoJSON files, and a 5.1 km loop it named the "El Granada harbor loop."

Asked how it did the work, the system said it "used Nominatim to locate the address and Overpass to download local OpenStreetMap roads and trails," then calculated the loops locally. The HTML view was rendered with D3 loaded from a CDN.

Willison's issue is with what he could not see. "Frustratingly, the actual code it ran and exact details of what it did weren't visible to me in the ChatGPT UI," he writes. "I see this lack of transparency is an anti-feature." When he went back to ask for the Python, the thread had been compacted and the original code was gone. His prescription is blunt: "any LLM system that uses compaction needs to both preserve the pre-compacted text and make that text available via agent tool calls." The post was passed around by two of the practitioners we track in our Who's Who directory the same day.

Shared on Bluesky by 2 AI experts