Meta, the company behind Facebook, has introduced OpenZL, a new open source data compression framework — basically, a software system that reduces the size of data files so they take up less space and move faster across networks, all without losing any information.
What makes OpenZL stand out is that it’s designed for structured data, such as databases, tables, and other well-organized information. Instead of treating every file as just a long string of bytes, OpenZL looks at the data’s internal structure — for example, columns, numbers, or repeating patterns — and uses that knowledge to squeeze the data much more efficiently.
The result is lossless compression, meaning the original data can be perfectly restored, with performance similar to format-specific compressors but using only one executable. In other words, you don’t need different tools for each file type. Everything can be handled by a single universal decompressor, simplifying maintenance and improving security.
Meta says OpenZL is its answer to the limitations of existing compressors, such as Zstandard. Zstandard already delivers great speed and solid ratios, but it’s still a general-purpose tool that doesn’t fully exploit the structure of modern datasets. OpenZL, by contrast, combines that speed with deeper awareness of data formats — a balance that’s been hard to achieve until now.
Developers can describe their data’s structure using a small schema, or through the Simple Data Description Language (SDDL), and OpenZL’s built-in “trainer” automatically learns the best compression plan. That plan can then be reused across similar datasets, saving time and ensuring consistency.
Early benchmarks show impressive results. When compressing structured data like the “sao” file from the Silesia Compression Corpus, OpenZL reached a 2.06x compression ratio, outperforming xz (1.64x) and zstd (1.31x), while also running faster — 340 MB/s for compression and 1.2 GB/s for decompression on an Apple M1 chip.
OpenZL can also adapt over time. It supports automatic re-training and runtime adjustments, so compression strategies can evolve as data changes — without breaking compatibility. This flexibility builds on Meta’s internal Managed Compression system, initially used with Zstandard across Facebook’s infrastructure and now brought to the open source community.
Developers and researchers can try OpenZL today. The source code, documentation, and Quick Start guide are available on GitHub, along with a whitepaper detailing the design and performance results.
For more information, see the announcement.
