F3 is a research columnar file format. The SIGMOD 2025 paper argues for a different split than Parquet: a stable way to find bytes in the file, and the option to ship a decoder with an unfamiliar encoding. Upstream says not to use it in production. The Elixir library in github.com/thanos/f3 is an experiment on top of one pinned upstream commit.
Library: github.com/thanos/f3 · Upstream: future-file-format/F3 · Paper: doi:10.1145/3749163
Parts 1–2 explain the idea and the byte layout. Parts 3–5 open, stream, and write files from Elixir when the native decoder is compiled. Part 6 is Wasm (opt-in only). Part 7 places F3 next to Arrow and Parquet as jobs, not as a speed contest. Parts 8–9 are the safety rules and a recorded bench run that refuses to invent decode numbers.
Hex has not published the package. Wire version 0.1 only. Wasm stays off unless a read passes wasm: :sandbox. If an article and the repository disagree, trust the repository.
Articles in this series
- F3: Why Another File Format? — What the SIGMOD 2025 paper actually argues — and what this Elixir library refuses to claim.
- F3: Anatomy of an F3 File — The 32-byte postscript, then the footer — walking a real fixture from the magic backward.
- F3: Your First F3 File in Elixir — Open a fixture, print the schema, and read three rows when the native decoder is loaded.
- F3: Streaming and Column Projection — Batches, projection, and when not to call to_rows.
- F3: Writing F3 from Elixir — Schema, compression, and a round trip through the native writer helper.
- F3: Taking the Decoder with the Data — Wasm, trust, and what stays off unless you pass wasm: :sandbox.
- F3, Apache Arrow, and Parquet — Three different jobs — not a speed contest.
- F3: Building a Safe Binary-File Reader on the BEAM — Bounds, dirty NIFs, and Wasm that stays off by default.