TOON, or Token-Oriented Object Notation, is a new format designed to reduce token usage when passing information to large language models (LLMs), thereby reducing cost.
Passing data to LLMs is essential to provide the context needed so that it understands what is being asked of it. That data might be a simple chat message, like when you use ChatGPT, but for more sophisticated AI use cases you’ll often need to pass tables of data or even nested hierarchies of objects.
Typically, you will find information to be passed to LLMs being encoded as JSON – JavaScript Object Notation, like this:
{
"users": [
{ "id": 1, "name": "Alice", "role": "admin" },
{ "id": 2, "name": "Bob", "role": "user" }
]
}But you will notice there is a lot of duplication in this structure – for example, the repetition of “id”, “name”, and “role” properties. And when using hosted LLMs, each character counts towards how much you pay to use it, so duplication is undesirable.
The TOON Format
TOON removes this duplication, providing a “header” containing the type / name of the objects in the list, and the headings, followed only by the data without the property names (headings).
So the above JSON would look like this when encoded as TOON:
users[2]{id,name,role}:
1,Alice,admin
2,Bob,userThe TOON format is said to save 40-60% of the size of encoded data, working most optimally for “uniform tables of information”, which in reality is a very common use case for passing data to LLMs.
Note that the data itself is indented. This borrows from YAML (Yet Another Markup Language) where indentation signifies relationships. The same is true with TOON.
No Silver Bullet
TOON is a promising innovation, solving a real, widespread problem with a simple solution. However, as with everything, there are trade-offs.
The TOON format has some limitations. Where data is a complex, nested structure, the “headings + data” structure loses its benefits and JSON may be a most efficient option.
There is also risk around any encoding/decoding or serialisation/deserialisation processes which can lead to security problems (example). Encoders/decoders need to be designed to be resistant to carefully crafted data that could risk system integrity.
There’s also the challenge of accuracy. Passing data to LLMs is not guaranteed to give you the result you want. LLMs are “non-deterministic”, meaning even if you pass the same information twice you could get different answers – even more likely with today’s reasoning models that retain memory of previous requests.
Accuracy vs. Token Cost
The smart folks over at ImprovingAgents.com have tested TOON along with a number of other common data formats such as JSON, YAML, CSV and XML to compare how the data format affects the quality of responses from LLMs.
They have a great set of charts and analysis here that are worth your time to read: https://www.improvingagents.com/blog/is-toon-good-for-table-data
To summarise their findings, TOON is not a silver bullet, but for cases where token cost is a priority, data is uniform and you’re happy to take a 15-20% hit on accuracy, then TOON is in that sweet spot.
According to their comparison method:
- XML is around 4x the size of an equivalent TOON payload but offers 56% accuracy.
- JSON is more than 3x the size of the TOON data, offering 52.3% accuracy.
- The TOON baseline data demonstrated 47.5% accuracy.
- And for comparison, CSV was marginally smaller than the TOON data, but with 44.3% accuracy.
Echoes of the Past
I’ve got a strong feeling of deja vu.
In my early career I built a lot of XML SOAP web services, marvelling at the incredibly dense XML payloads that would criss-cross the networks of large enterprises.
And then I discovered JSON and XML felt obscenely verbose. XML’s “angle bracket tax” resulted in enormous payloads while JSON was simple and much more human-readable.
I have the same feeling now as I look at TOON. I don’t think this will be the only token-efficient language we’ll see, especially as TOON is designed as a transport format rather than a static format for working with.
We could see a protobuf-style binary format emerge that even further reduces the size of the payload going to the LLM. In fact, we already kind of have…
A Picture paints a thousand words
Deepseek recently released OCR, a next-generation Optical Character Recognition (OCR) tool.
If OCR sounds familiar, it was popularised in the late 1990s and early 2000s with the rise of Windows, the Internet, and….. printer/scanner devices. I remember scanning documents on my scanner in around 2002 to pull text from them.
Today, OCR is used as part of a Data Pipeline to feed information into LLMs or “vector databases” that can be searched by LLMs using AI agents or MCP servers.
But Deepseek OCR has a special superpower – it offers token-efficient upload of images for OCR processing, and some analysts have found that by uploading an image of a lot of text, rather than the text itself, they can save a significant percentage of the token cost.
Images are transferred to LLMs as base64-encoded strings of text, making them quite efficient as a transport format. So this makes sense.
In Conclusion
To summarise, TOON offers a token-efficient way to send data to LLMs, but has certain operating parameters within which it works best.
The good news is, those operating parameters are some of the most common use cases for passing data to LLMs.
But TOON is not the answer to everything, JSON, YAML, CSV and even XML (shudder) still have their place, it just depends what you’re optimising for.
If the AI revolution has taught us anything, it’s that we don’t know what the next shocking innovation will look like – and I don’t think we’ve seen the last optimisation at the data format level.