Streaming-first
Serialize and deserialize in arbitrarily small chunks — you never need the whole message in memory. No length-prefixed envelope, so encoders emit data before the final size is even known.
SofaBuffers
Structured Objects For Anyone
… so optimized, feels amazing.
A compact, streamable binary format for structured data — tightly packed, schema-driven, and tiny enough for the smallest microcontroller yet fast enough for the cloud.
Core libraries
One format, ten languages
Each core library implements the same streamable wire format and shares a common suite of conformance test vectors — and some languages ship more than one build, each tuned for a different target.
CC / C++Heap-free C99 / C++20 for embedded MCUs
C++C++Pure C++20, no C backend, speed-tuned
RsRustHigh-speed std build for big machines
RsRust#![no_std], heap-free, for microcontrollers
GoGoStreaming, dependency-free runtime core
PyPythonPure Python 3.9+, zero dependencies
TsTypeScriptRuns on Node, browsers, Deno & Bun
JvJavaAllocation-light, runs anywhere a JVM does
C#C#Allocation-light .NET, cloud-ready
DtDartPure Dart, streaming, zero dependencies
ZigZigAllocation-free, zero-copy, max throughput
What makes it cool
Small on the wire, simple in your code
Every byte and every design decision is tuned for moving structured data fast — across slow links and onto the tiniest devices — without complicating the code you write.
Tightly packed
Varint (LEB128) encoding shrinks small integers to a single byte, and zig-zag mapping keeps negative numbers compact. Low overhead is reserved for the field types you actually use most.
Self-describing TLV
Type information lives in the stream itself: each field header packs (id << 3) | type. Decoders can read, descend, or skip any field — even ones they don't recognize.
Nested sequences
Delimiter-based sequences create fresh ID namespaces for deeply nested, hierarchical structures — without ever pre-computing a size or buffering a sub-tree.
Schema → code
Write a definition once; the generator emits dead-simple, language-native objects with .serialize() / .deserialize() plus streaming variants.
Runs everywhere
A tiny footprint with no heavy runtime makes it equally at home on resource-constrained MCUs and high-throughput cloud services. Little-endian on the wire, identical on every architecture.
Why SofaBuffers?
It fills the gap the others left open
Born out of real IoT and embedded communication needs, where every existing format made an awkward trade-off.
JSONHuman-readable, but carries significant byte overhead.
MsgPack / BSON / CBORMore efficient, but lack first-class message definitions.
Protocol BuffersPowerful, but generate unnecessarily heavy code.
FlatBuffersRandom-access, but not tightly packed for slow interfaces.
"Because JSON is too fat,
and Protobuf is too Google."
Under the hood
A wire format you can reason about
Simple, predictable building blocks — varints, a 3-bit type tag, and delimiter-based scopes.
One header, eight wire types
Each field starts with a single varint header. The low three bits select the wire type; everything above is the field ID (0 … 2,147,483,647).
- Unsigned & signed integers (varint / zig-zag)
- Fixed-length values: float32, float64, UTF-8 strings, binary blobs
- Arrays of integers and of fixed-length values
- Sequence start / end markers for nested structures
- No top-level envelope — a message is just a flat stream of fields
wire-format.txt
# Field header = (id << 3) | type 0x0 unsigned int (varint) 0x1 signed int (zig-zag varint) 0x2 fixlen value (f32/f64/str/blob) 0x3 unsigned int array 0x4 signed int array 0x5 fixlen value array 0x6 sequence start (nested scope) 0x7 sequence end # Varints are little-endian, 7 bits each: 300 -> 0xAC 0x02 # Zig-zag keeps small negatives tiny: -1 -> 1 -2 -> 3
example.py
# Generated objects are dead simple person = Person() person.name = "Ada" person.id = 42 # One-shot convenience data = person.serialize() person2 = Person.deserialize(data) # ...or stream it in arbitrarily small chunks, # never holding the whole message in memory.
Two ways to use it
Power users can drive the raw encoder and decoder by hand for maximum control. Everyone else writes a schema and lets the generator produce language-native classes.
- Generated objects with intuitive field access
- Convenience serialize() / deserialize() helpers
- Streaming variants for constrained memory
- Shared conformance test vectors across every language
Quality & performance
Battle-tested, and benchmarked
Every core library is held to the same bar — proven interoperable and kept fast — by two dedicated harnesses that run across all of them.
Proven correct, everywhere
Our Crucible differential-fuzzing harness feeds identical bytes to every language at once and treats any disagreement as a bug. The result: implementations that stay byte-for-byte interoperable, round-trip clean, and hardened against malformed input.
Benchmarked to stay fast
The Arena suite measures every library head-to-head against Protocol Buffers — from raw encode/decode throughput on server CPUs to firmware footprint on bare-metal ARM and RISC-V. Fair, reproducible, and always watching for regressions.
Ready to slim down your wire format?
Browse the core libraries, read the format specification, and start serializing structured data the lightweight way.