When JSON Is Too Big and Protobuf Is Too Heavy: A Field Guide to CBOR

When JSON Is Too Big and Protobuf Is Too Heavy: A Field Guide to CBOR

A closing thought on when CBOR is the right tool

Reach for CBOR when the payload is small, the network is slow, the schema is stable, and the reader is not a browser. Reach for JSON when a human will read it, when the schema evolves weekly, or when every layer of your stack already speaks JSON natively. CBOR is not a replacement for JSON — it is a sharper tool for a narrower job. Use it deliberately and the size savings will pay for themselves for years.

The format most engineers have never seen

CBOR stands for Concise Binary Object Representation, defined by RFC 8949 since 2013. It does for JSON roughly what MessagePack does, but with a standards track, deterministic encoding options, and tag-based extensions that handle dates, big numbers, and byte strings without ambiguity. Most engineers have never met it because the world they work in — REST APIs, configuration files, webhooks — is dominated by JSON, and JSON is good enough for almost all of it.

The cases where JSON stops being good enough share three traits: the payload is small, the channel is narrow, and the count of payloads is large. That profile describes LoRaWAN firmware updates, satellite telemetry, NFC tags, embedded device boot manifests, and any mobile API that bills by the byte. In each of those settings, the JSON envelope of {"name":"value","count":42,"active":true} — keys duplicated on every record, white space and quotes adding 30–40% overhead — is a tax the system pays on every message. CBOR removes the tax. The same record becomes a 12-byte blob that the decoder can read without parsing, schema-less.

This article is a practical field guide to using CBOR with confidence: what it is, when to choose it, what the encoding looks like, and how to convert between JSON and CBOR without losing your data. If you want a tool that already handles the encode/decode cycle and prints the result as hex or base64, the CBOR Converter is the fastest path to a working example.

Why CBOR exists, in one sentence

CBOR is a binary self-describing format that maps every JSON value to a smaller byte sequence while keeping the schema-less flexibility that made JSON popular in the first place.

The mapping is direct. JSON numbers become CBOR integers or floats. JSON strings become UTF-8 byte strings. JSON arrays become CBOR arrays. JSON objects become CBOR maps. JSON booleans and null become CBOR simple values. There is no schema to maintain, no IDL to compile, no codegen step. Drop a payload into the decoder and the same data structure comes out on the other side.

The size difference comes from two places. First, CBOR omits the syntax overhead — no quotes around keys, no colons, no commas, no whitespace. Second, CBOR uses one byte for the type tag and length prefix combined for small values, instead of paying for each character of "true" or 42 separately. A record that weighs 80 bytes in JSON typically lands between 20 and 35 bytes in CBOR, depending on how long the keys are.

When CBOR beats JSON, and when it loses

The honest answer: CBOR wins on the wire and loses everywhere else.

The third column is the rule of thumb. CBOR wins when bytes cost money, time, or battery. CBOR loses when a person, a debugger, or a third-party integration has to read the payload directly. If you find yourself needing to “just decode it real quick” in production, that is a signal you chose the wrong format for that boundary.

A concrete encoding, end to end

Take this JSON record:

{"sensor":"temp-7","value":21.5,"unit":"C","ts":1700000000}

In JSON, the wire form is 60 bytes including the braces, quotes, colon, and comma. In CBOR, the same record is 30 bytes as hex:

a4 65 73 65 6e 73 6f 72  64 74 65 6d 70 2d 37 65
76 61 6c 75 65 fb 41 35
66 36 36 36 36 36 36 66
36 36 64 75 6e 69 74 61
43 01 74 73 1a 65 6e 69

The first byte a4 is the map header: a means “map” and 4 means four entries. Then the decoder reads four key-value pairs in order: a 5-byte UTF-8 string key (sensor), a 6-byte UTF-8 string value (temp-7), a 5-byte UTF-8 string key (value), a 64-bit float (fb 41 56 66 66 66 66 66 66 — IEEE 754 doubles for 21.5), and so on. The reader does not need the schema to walk the structure because every value carries its own type and length.

If you want to see this in your own browser without installing anything, paste the JSON into the CBOR Converter and switch the output format to hex. The tool will show you both the hex and the base64 representations side by side, which is useful when you are debugging a payload that you have intercepted from a device and cannot easily print.

The three subtleties that bite first

CBOR has three behaviors that JSON does not have, and the first time you meet them in production they will each feel like a bug.

First, keys must be sorted for deterministic encoding. RFC 8949 defines a “deterministic encoding” mode where maps must be sorted by key length, then lexicographically. Some libraries do this by default; others do not. If two devices encode the same record differently because one sorted and one did not, their byte representations will not match and any hash or signature you took over the payload will not verify. This is not a CBOR bug. It is the correct behavior, and it is the price of being able to canonicalize a payload.

Second, integers do not distinguish from floats at the wire level. CBOR has separate major types for integers and floats, but a value like 21.5 is unambiguously a float, while 21 is unambiguously an integer. Decoders must preserve that distinction. If your language’s JSON parser auto-converts integers to floats on the way in (JavaScript, looking at you), you may find that a value you encoded as an integer round-trips as a float. This is rarely a correctness problem, but it shows up when you measure payload sizes and wonder why the count is one byte longer than the spec says it should be.

Third, indefinite-length items exist but most libraries discourage them. CBOR allows a map or array to be written without a length prefix, terminated by a break byte. This is useful for streaming encoders that do not know the size in advance. It is a footgun for canonical encoding, and most tooling defaults to definite-length. Use definite-length unless you have a concrete reason not to.

How to convert between JSON and CBOR safely

The conversion is bidirectional, but each direction has its own small hazard.

JSON to CBOR. Parse the JSON, walk the resulting tree, encode each value to its CBOR equivalent. The hazard is JSON’s tolerance for null, undefined, and duplicate keys. A JSON null becomes a CBOR simple value, which is fine. Undefined does not exist in JSON. Duplicate keys are technically not allowed but parsers accept them; the encoder will pick one. Before encoding, normalize the input: round-trip through your parser to drop duplicates, replace any non-standard values with their canonical form, and confirm the resulting tree is what you expected.

CBOR to JSON. Decode the bytes back to a tree, then serialize. The hazard is CBOR’s extra types: byte strings, tags, big numbers, and simple values that JSON cannot represent. A common practice is to encode byte strings as base64 strings on the JSON side. A CBOR tag for a date-time (tag 0 or tag 1) can become an ISO 8601 string. Big integers outside JavaScript’s safe range need a string representation because JSON cannot hold them as numbers. Decide these conventions before the wire format goes live, not when the first decoding fails.

The CBOR Converter handles both directions in a single paste, with hex and base64 output formats, which is enough for most ad hoc inspection work. For production code, you want a tested library on both ends — cbor-x for Node.js, cbor2 for Python, cbor for Go — and the same set of conventions documented somewhere the next engineer can find.

When to stop and choose something else

CBOR is not the lightest binary format available. If your payload is a fixed schema and your code is already in C, Protocol Buffers or FlatBuffers will out-compress it by another 30–50%. CBOR’s advantage over those formats is the schema-less flexibility — you can add a field on the encoder side and the older decoder will keep working. That property is valuable exactly as long as your format keeps evolving without coordinated deployments. The day you find yourself versioning the payload because old decoders cannot read new fields, you have crossed the line where a schema’d format would be cheaper overall.

For everything in between — REST backends, mobile telemetry, embedded manifests, browser-to-server compression candidates — CBOR is the format that gets out of your way and lets you ship.

Explore more format-conversion tools at elysiatools.com.