It is now possible to ingest OSM data and query it over HTTP using Overpass QL from a pure Go service. I built overpass-go over about a dozen evening couch sessions, vibe coding.
This was a hugely valuable learning experience, and I got to really test out the pi harness along the way. What I built works end to end, and I am happy with the overall result.
It is licensed under the AGPL and derived from the original C++ Overpass implementation. This was not a clean-room rewrite. I literally pointed the agents at both and told them to go ham.
It was a good way to learn Overpass QL, the OSM data model, and how the query, storage and execution engines actually work.
New models and harnesses need to be tested, and this was a good fit for what I'm thinking about. Through the build I used Claude Code, Codex, and a custom pi harness.
I also wanted a codebase that was easier to extend. The C++ project is hard to approach. A fresh Go implementation with high test coverage and purpose-built agent context seemed like a good place to start.
This version hosts its own HTTP server using the Go standard library. I want a minimal Docker image that I can run anywhere, and a standalone Go service is a good fit for that.
This project started with just porting the query language parser. I thought that would be the end. It grew to include a minimal set of queries executed in memory, getting the whole thing working end to end.
This happened over several evening sessions on the couch. I would poke Claude to continue working on the plan, look at the tests, and tell it to keep going. Progress seemed slow, but we were getting there.
Eventually I wanted to make this a more realistic test. I built tooling to query an existing Overpass instance, parse the XML into memory, and run queries against that data instead of relying on synthetic test data.
I built a storage index next, an HTTP server after that, and then a process to ingest Geofabrik protobuf files. At this point it was becoming something I could actually host.
The storage layer uses bbolt, a fast key-value database written in pure Go. I was being tactical about bringing in a dependency here. bbolt is 'complete' software.
The C++ storage implementation was clever, but also optimised for memory maps. That isn't something that translates easily to Go. Putting bbolt behind the common storage interface seemed like a sensible way to go.
Eventually I could see this being replaced with a remote key-value database if that fits. For now, we store files locally.
The first spatial index used a 0.1-degree grid, purely to make queries work end to end. The C++ implementation uses an ingenious quad tile approach, so I looked at that next.
I went down a rabbit hole comparing quad tiles with an S2 index. Ultimately I decided to implement quad tiles first because it could be done in about 300 lines of code. S2 is still something I want to look at for a full-world dataset.
At the moment I only query the NZ dataset: tens of millions of points and about 9 GB of memory. A full scan is not the end of the world, but I would like to make this work for whole-earth datasets eventually.
On the note of making it scale, I added an on-disk merge sort so ingestion can handle more data than we have memory for. It sorts chunks of about 512 MB at a time.
The HTTP server was another place where the port grew its own implementation. I wanted the behaviour to stay as close as possible to the C++ version, but it made little sense to build a CGI app in Go.
The standard library already has great support for concurrency and HTTP servers. So we have a worker pool that executes queries and a very standard-looking Go HTTP handler.
One of the biggest differences was error handling. The C++ project throws errors and catches them at the top level. In Go, errors are values returned from functions.
I didn't translate that to the Go idiom along the way, which means I currently have 25 occurrences of the panic keyword!
I want to get out of this eventually, but it would be a big lift, and I wanted to prove the overall approach first. I think I've done that now. We can start focusing on changes that make the project easier to operate and maintain.
I started with a very plain Claude Code + grill-me setup. Later I moved to a custom pi extension built on the wayfinder skill, using herdr for sub-agent pane management.
To complete the swap to pi and really give it a test, I needed to swap my AI bill of choice.
As the codebase grew into the tens of thousands of lines, I started thinking about quality metrics and deterministic checks. I settled on code coverage, Go linters, and static analysis tools.
This is all wrapped up in a convenient ci.sh that agents can call to see what needs to be improved.
I also built CONTEXT.md, a glossary of domain terms for the project, to help keep the language and terminology consistent as the codebase grew.
Overpass is something we depend on at myWorksites. I want a lightweight OSM service that we can host and easily extend.
I also want to gradually match the rest of Overpass's capabilities. There are big things still missing from the query language, indexing, and storage layer. We can't ingest data diffs yet, and we don't support history queries.
Beyond that, I want to explore vector tiles, cached layers, and new storage interfaces.
I want this to become a general-purpose write-once, read-many server where you can combine OSM data with your own: filter on ingestion, merge in your datasets, and query the result.