← Back to DevBytes
Developer Tutorial July 09, 2026 15 min read

FlatBuffers Format: A Complete Reference Guide

What is FlatBuffers?

FlatBuffers is an efficient cross-platform serialization library and wire format developed by Google. It allows you to store and access structured data without any parsing or unpacking overhead — you can read data directly from the serialized byte buffer as if it were a native data structure in memory. The format is designed for zero-copy deserialization, meaning the data on disk or over the network can be mapped directly into memory and accessed without first transforming it into a separate object tree.

Unlike JSON or Protocol Buffers, where you typically need to parse the entire message before accessing any field, FlatBuffers lets you navigate to any field in O(1) time by following offsets stored inline in the binary layout. This makes it exceptionally fast for scenarios where you only need a subset of the data, such as reading a specific attribute from a large game state or configuration blob.

Why FlatBuffers Matters

🚀 Deploy your AI agent in 10 minutes

Managed Hermes hosting. Zero DevOps. 100M tokens/mo included.

Try it free →

The Zero-Copy Advantage

Traditional serialization formats like JSON, XML, and even Protocol Buffers require allocating a separate in-memory representation and copying data from the wire format into that representation. FlatBuffers eliminates this step entirely. The wire format itself is the in-memory format — you simply get a pointer to the buffer and start reading fields directly.

Performance Characteristics

Use Cases

Core Concepts

The FlatBuffer Binary Layout

A FlatBuffer is built around a few key structures:

Alignment and Endianness

FlatBuffers stores data in little-endian format by default and aligns fields according to their natural alignment requirements. Scalars are aligned to their size (e.g., int32 on 4-byte boundaries, double on 8-byte boundaries). Vectors and tables are aligned to 4 bytes. This alignment ensures that direct pointer dereferencing works correctly on platforms that require aligned access.

Defining a Schema

FlatBuffers uses its own schema definition language (IDL) with a file extension .fbs. The schema describes all types that can be serialized. Here is a complete example schema file:

// monster.fbs — a complete schema for a game monster entity

namespace game;

// An enum for monster categories
enum Category : byte {
  None = 0,
  Melee,
  Ranged,
  Magic,
  Boss
}

// A fixed-layout struct for 2D position
struct Vec3 {
  x: float;
  y: float;
  z: float;
}

// A struct for RGB color (no vtable, always present)
struct Color {
  r: ubyte;
  g: ubyte;
  b: ubyte;
  a: ubyte;
}

// A table for weapon data
table Weapon {
  name: string;
  damage: short;
  range: float;
  enchantments: [string];   // vector of strings
}

// The main Monster table
table Monster {
  pos: Vec3;                // struct field (inline)
  mana: int = 150;          // default value
  hp: int = 100;
  name: string;
  friendly: bool = false;
  color: Color;             // inline struct
  category: Category = None;
  inventory: [ubyte];       // vector of bytes
  weapons: [Weapon];        // vector of tables
  equipped_weapon: Weapon;  // single table reference
  path: [Vec3];             // vector of structs
}

// The root type declaration
root_type Monster;

Schema Syntax Reference

Field Types

Compiling the Schema

The FlatBuffers compiler flatc takes .fbs schema files and generates code for your target language. Install it via your package manager or build from source:

# On macOS
brew install flatbuffers

# On Ubuntu/Debian
apt-get install flatbuffers-compiler

# Or build from source
git clone https://github.com/google/flatbuffers.git
cd flatbuffers
cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release
make -j$(nproc)
sudo make install

Generate code from your schema:

# Generate C++ headers
flatc --cpp -o ./output ./monster.fbs

# Generate Python code
flatc --python -o ./output ./monster.fbs

# Generate multiple languages at once
flatc --cpp --python --rust -o ./output ./monster.fbs

# Generate binary schema reflection data
flatc --binary --schema -o ./output ./monster.fbs

The compiler outputs language-specific files that contain both the type definitions and the builder/accessor APIs for constructing and reading FlatBuffers in that language.

Using FlatBuffers: Creating and Reading Buffers

C++ Example: Building a Monster

In C++, FlatBuffers uses a builder pattern. You create a FlatBufferBuilder instance and use it to serialize data sequentially, starting from the leaves of the object graph and working up to the root. This reverse-order construction is necessary because offsets must be known before they can be referenced.

// build_monster.cpp — Complete example of creating a FlatBuffer in C++

#include "monster_generated.h"  // generated by flatc
#include <flatbuffers/flatbuffers.h>
#include <iostream>
#include <fstream>
#include <vector>

int main() {
  // Step 1: Create a FlatBufferBuilder with initial capacity
  flatbuffers::FlatBufferBuilder builder(1024);

  // Step 2: Build from the leaves inward.
  // Start with strings (they must be created first since offsets
  // are required for tables that reference them).

  // Create weapon name strings
  auto sword_name = builder.CreateString("Excalibur");
  auto bow_name   = builder.CreateString("Longbow of Doom");

  // Create the Weapon tables
  // Note: Weapon fields are set using the generated WeaponT struct
  // or inline builder methods.

  // Build sword weapon
  game::WeaponBuilder sword_builder(builder);
  sword_builder.add_name(sword_name);
  sword_builder.add_damage(45);
  sword_builder.add_range(1.5f);
  // enchantments vector — create a vector of string offsets
  auto ench1 = builder.CreateString("Fire");
  auto ench2 = builder.CreateString("Lightning");
  std::vector<flatbuffers::Offset<flatbuffers::String>> sword_ench = {
    ench1, ench2
  };
  auto sword_ench_vec = builder.CreateVector(sword_ench);
  sword_builder.add_enchantments(sword_ench_vec);
  auto sword = sword_builder.Finish();

  // Build bow weapon
  game::WeaponBuilder bow_builder(builder);
  bow_builder.add_name(bow_name);
  bow_builder.add_damage(25);
  bow_builder.add_range(100.0f);
  auto ench3 = builder.CreateString("Piercing");
  std::vector<flatbuffers::Offset<flatbuffers::String>> bow_ench = { ench3 };
  auto bow_ench_vec = builder.CreateVector(bow_ench);
  bow_builder.add_enchantments(bow_ench_vec);
  auto bow = bow_builder.Finish();

  // Create a vector of weapon offsets (for the weapons field)
  std::vector<flatbuffers::Offset<game::Weapon>> weapon_offsets = {
    sword, bow
  };
  auto weapons_vec = builder.CreateVector(weapon_offsets);

  // Create a vector of Vec3 structs (for the path field)
  // Structs are created inline using CreateVectorOfStructs
  game::Vec3 path_points[] = {
    game::Vec3(0.0f, 0.0f, 0.0f),
    game::Vec3(10.0f, 5.0f, 0.0f),
    game::Vec3(20.0f, 10.0f, 5.0f)
  };
  auto path_vec = builder.CreateVectorOfStructs(
      path_points, sizeof(path_points) / sizeof(game::Vec3));

  // Create inventory bytes
  unsigned char inventory_data[] = { 0x01, 0x02, 0x03, 0xFF };
  auto inventory_vec = builder.CreateVector(
      inventory_data, sizeof(inventory_data));

  // Create the Monster name string
  auto name_str = builder.CreateString("Aragorn the Destroyer");

  // Now build the Monster table
  game::MonsterBuilder monster_builder(builder);
  monster_builder.add_pos(game::Vec3(15.0f, 20.0f, 3.0f));
  monster_builder.add_mana(200);
  monster_builder.add_hp(350);
  monster_builder.add_name(name_str);
  monster_builder.add_friendly(false);
  monster_builder.add_color(game::Color(255, 128, 0, 255));
  monster_builder.add_category(game::Category_Boss);
  monster_builder.add_inventory(inventory_vec);
  monster_builder.add_weapons(weapons_vec);
  monster_builder.add_equipped_weapon(sword);  // single table reference
  monster_builder.add_path(path_vec);
  auto monster = monster_builder.Finish();

  // Mark the buffer as complete and set the root type
  builder.Finish(monster);

  // The buffer is now ready. You can access the raw bytes:
  uint8_t* buf = builder.GetBufferPointer();
  size_t size = builder.GetSize();

  // Write to file
  std::ofstream out("monster.bin", std::ios::binary);
  out.write(reinterpret_cast<const char*>(buf), size);
  out.close();

  std::cout << "Monster buffer written: " << size << " bytes" << std::endl;

  return 0;
}

C++ Example: Reading a Monster with Zero-Copy Access

// read_monster.cpp — Zero-copy reading of a FlatBuffer

#include "monster_generated.h"
#include <flatbuffers/flatbuffers.h>
#include <iostream>
#include <fstream>
#include <vector>

int main() {
  // Read the entire file into a buffer
  std::ifstream infile("monster.bin", std::ios::binary | std::ios::ate);
  std::streamsize size = infile.tellg();
  infile.seekg(0, std::ios::beg);
  std::vector<char> buffer_data(size);
  infile.read(buffer_data.data(), size);
  infile.close();

  // Verify the buffer is valid
  flatbuffers::Verifier verifier(
      reinterpret_cast<const uint8_t*>(buffer_data.data()),
      buffer_data.size());
  if (!game::VerifyMonsterBuffer(verifier)) {
    std::cerr << "Buffer verification failed!" << std::endl;
    return 1;
  }

  // Zero-copy: get a pointer to the root Monster
  const game::Monster* monster = game::GetMonster(
      buffer_data.data());

  // Access fields directly — no parsing step!
  std::cout << "Name: " << monster->name()->c_str() << std::endl;
  std::cout << "HP: " << monster->hp() << std::endl;
  std::cout << "Mana: " << monster->mana() << std::endl;
  std::cout << "Position: (" << monster->pos()->x()
            << ", " << monster->pos()->y()
            << ", " << monster->pos()->z() << ")" << std::endl;

  // Access the color struct (inline, always present)
  std::cout << "Color: rgba("
            << static_cast<int>(monster->color()->r()) << ", "
            << static_cast<int>(monster->color()->g()) << ", "
            << static_cast<int>(monster->color()->b()) << ", "
            << static_cast<int>(monster->color()->a()) << ")"
            << std::endl;

  // Iterate over weapons vector
  if (monster->weapons()) {
    std::cout << "Weapons:" << std::endl;
    for (const game::Weapon* weapon : *monster->weapons()) {
      std::cout << "  - " << weapon->name()->c_str()
                << " (damage: " << weapon->damage()
                << ", range: " << weapon->range() << ")" << std::endl;
      // Enchantments sub-vector
      if (weapon->enchantments()) {
        for (const auto ench : *weapon->enchantments()) {
          std::cout << "      enchant: " << ench->c_str() << std::endl;
        }
      }
    }
  }

  // Access equipped weapon (single table reference)
  if (monster->equipped_weapon()) {
    std::cout << "Equipped: " << monster->equipped_weapon()->name()->c_str()
              << std::endl;
  }

  // Iterate over path vector of structs
  if (monster->path()) {
    std::cout << "Path:" << std::endl;
    for (const game::Vec3* point : *monster->path()) {
      std::cout << "  (" << point->x() << ", " << point->y()
                << ", " << point->z() << ")" << std::endl;
    }
  }

  // Category enum
  std::cout << "Category: " << static_cast<int>(monster->category())
            << std::endl;

  return 0;
}

Python Example: Building and Reading

Python bindings provide a similar builder API. Note that FlatBuffers for Python requires the flatbuffers package (pip install flatbuffers).

# monster_example.py — Complete build and read in Python

import flatbuffers
from MyGame import Monster, Weapon, Vec3, Color, Category

def build_monster():
    builder = flatbuffers.Builder(1024)

    # Create enchantment strings for sword
    ench1 = builder.CreateString("Fire")
    ench2 = builder.CreateString("Lightning")

    # Create sword weapon name
    sword_name = builder.CreateString("Excalibur")

    # Create enchantments vector for sword
    Weapon.WeaponStartEnchantmentsVector(builder, 2)
    builder.PrependUOffsetTRelative(ench2)
    builder.PrependUOffsetTRelative(ench1)
    sword_ench_vec = builder.EndVector(2)

    # Build sword Weapon table
    Weapon.WeaponStart(builder)
    Weapon.WeaponAddName(builder, sword_name)
    Weapon.WeaponAddDamage(builder, 45)
    Weapon.WeaponAddRange(builder, 1.5)
    Weapon.WeaponAddEnchantments(builder, sword_ench_vec)
    sword = Weapon.WeaponEnd(builder)

    # Build bow weapon
    bow_name = builder.CreateString("Longbow of Doom")
    ench3 = builder.CreateString("Piercing")
    Weapon.WeaponStartEnchantmentsVector(builder, 1)
    builder.PrependUOffsetTRelative(ench3)
    bow_ench_vec = builder.EndVector(1)

    Weapon.WeaponStart(builder)
    Weapon.WeaponAddName(builder, bow_name)
    Weapon.WeaponAddDamage(builder, 25)
    Weapon.WeaponAddRange(builder, 100.0)
    Weapon.WeaponAddEnchantments(builder, bow_ench_vec)
    bow = Weapon.WeaponEnd(builder)

    # Create weapons vector
    Monster.MonsterStartWeaponsVector(builder, 2)
    builder.PrependUOffsetTRelative(bow)
    builder.PrependUOffsetTRelative(sword)
    weapons_vec = builder.EndVector(2)

    # Create path vector of structs
    # Structs are prepended directly in reverse order
    Monster.MonsterStartPathVector(builder, 3)
    builder.PrependStruct(Vec3.CreateVec3(20.0, 10.0, 5.0))
    builder.PrependStruct(Vec3.CreateVec3(10.0, 5.0, 0.0))
    builder.PrependStruct(Vec3.CreateVec3(0.0, 0.0, 0.0))
    path_vec = builder.EndVector(3)

    # Create inventory vector
    Monster.MonsterStartInventoryVector(builder, 4)
    builder.PrependUByte(0xFF)
    builder.PrependUByte(0x03)
    builder.PrependUByte(0x02)
    builder.PrependUByte(0x01)
    inventory_vec = builder.EndVector(4)

    # Create name string
    name_str = builder.CreateString("Aragorn the Destroyer")

    # Build Monster table
    Monster.MonsterStart(builder)
    Monster.MonsterAddPos(builder, Vec3.CreateVec3(15.0, 20.0, 3.0))
    Monster.MonsterAddMana(builder, 200)
    Monster.MonsterAddHp(builder, 350)
    Monster.MonsterAddName(builder, name_str)
    Monster.MonsterAddFriendly(builder, False)
    Monster.MonsterAddColor(builder, Color.CreateColor(255, 128, 0, 255))
    Monster.MonsterAddCategory(builder, Category.Boss)
    Monster.MonsterAddInventory(builder, inventory_vec)
    Monster.MonsterAddWeapons(builder, weapons_vec)
    Monster.MonsterAddEquippedWeapon(builder, sword)
    Monster.MonsterAddPath(builder, path_vec)
    monster = Monster.MonsterEnd(builder)

    builder.Finish(monster)
    return builder.Output()


def read_monster(buf):
    # Zero-copy: get root monster directly from bytes
    monster = Monster.Monster.GetRootAsMonster(buf, 0)

    print(f"Name: {monster.Name().decode('utf-8')}")
    print(f"HP: {monster.Hp()}")
    print(f"Mana: {monster.Mana()}")
    print(f"Position: ({monster.Pos().X()}, {monster.Pos().Y()}, {monster.Pos().Z()})")

    # Access color
    color = monster.Color()
    print(f"Color: rgba({color.R()}, {color.G()}, {color.B()}, {color.A()})")

    # Iterate weapons
    print("Weapons:")
    for i in range(monster.WeaponsLength()):
        weapon = monster.Weapons(i)
        print(f"  - {weapon.Name().decode('utf-8')} "
              f"(damage: {weapon.Damage()}, range: {weapon.Range()})")
        # Enchantments
        if weapon.EnchantmentsLength() > 0:
            for j in range(weapon.EnchantmentsLength()):
                ench = weapon.Enchantments(j)
                print(f"      enchant: {ench.decode('utf-8')}")

    # Equipped weapon
    if monster.EquippedWeapon():
        eq = monster.EquippedWeapon()
        print(f"Equipped: {eq.Name().decode('utf-8')}")

    # Path
    print("Path:")
    for i in range(monster.PathLength()):
        pt = monster.Path(i)
        print(f"  ({pt.X()}, {pt.Y()}, {pt.Z()})")


if __name__ == "__main__":
    buf = build_monster()
    print(f"Buffer size: {len(buf)} bytes")
    read_monster(buf)

Advanced Features

Unions (Tagged Variants)

Unions allow a field to hold one of several table types, discriminated by an enum tag. This is useful for polymorphic message types like "a UI element can be a button, slider, or dropdown."

// unions_example.fbs
namespace ui;

table Button {
  label: string;
  width: int;
  height: int;
}

table Slider {
  label: string;
  min_value: float;
  max_value: float;
  current_value: float;
}

table Dropdown {
  label: string;
  options: [string];
  selected_index: int;
}

// The union combines all widget types
union Widget {
  Button,
  Slider,
  Dropdown
}

// An enum to tag which type is active
enum WidgetType : byte {
  None = 0,
  Button = 1,
  Slider = 2,
  Dropdown = 3
}

table UIElement {
  widget_type: WidgetType;
  widget: Widget;          // union field — must be paired with the type enum
}

root_type UIElement;

When accessing a union, you check the tag enum and then cast the union to the appropriate type:

// Accessing a union in C++
const ui::UIElement* element = ui::GetUIElement(buf);
switch (element->widget_type()) {
  case ui::WidgetType_Button: {
    const ui::Button* btn = element->widget_as_Button();
    std::cout << "Button: " << btn->label()->c_str() << std::endl;
    break;
  }
  case ui::WidgetType_Slider: {
    const ui::Slider* slider = element->widget_as_Slider();
    std::cout << "Slider value: " << slider->current_value() << std::endl;
    break;
  }
  case ui::WidgetType_Dropdown: {
    const ui::Dropdown* dd = element->widget_as_Dropdown();
    std::cout << "Dropdown selected: " << dd->selected_index() << std::endl;
    break;
  }
  default: break;
}

Nested FlatBuffers

You can embed an entire FlatBuffer inside another as a raw byte vector. This is useful for bundling independent data blobs together without merging schemas:

table Bundle {
  metadata: string;
  payload: [ubyte];   // contains another complete FlatBuffer
}

To read the nested buffer, you simply call GetRootAs on the byte vector's data pointer.

Schema Evolution and Deprecation

FlatBuffers supports adding and deprecating fields while maintaining compatibility:

table Monster {
  pos: Vec3;
  mana: int = 150;
  hp: int = 100;
  name: string;
  friendly: bool = false;
  // New field added later — old readers see the default
  level: int = 1;
  // Deprecated field — still accessible but marked for removal
  deprecated_old_stat: int (deprecated);
}

The (deprecated) attribute generates a compiler warning when the field is accessed, encouraging migration to newer fields.

FlexBuffers: Schema-Less FlatBuffers

FlexBuffers is a variant of FlatBuffers that doesn't require a predefined schema. It's similar in spirit to JSON but encoded in the FlatBuffers binary format. Data is self-describing, allowing dynamic access without code generation:

// C++ FlexBuffers example
flexbuffers::Builder fbuilder;
fbuilder.Map([&]() {
  fbuilder.Key("name", "Aragorn");
  fbuilder.Key("hp", 350);
  fbuilder.Key("weapons", [&]() {
    fbuilder.Vector([&]() {
      fbuilder.Map([&]() {
        fbuilder.Key("name", "Excalibur");
        fbuilder.Key("damage", 45);
      });
      fbuilder.Map([&]() {
        fbuilder.Key("name", "Longbow");
        fbuilder.Key("damage", 25);
      });
    });
  });
});
fbuilder.Finish();

FlexBuffers trades some performance for flexibility — field access requires parsing the type descriptor, making it slower than schema-based FlatBuffers but still faster than JSON parsing.

Best Practices

1. Design Schemas for Forward Compatibility

Always provide default values for fields that might not be present in older buffers. This ensures that new readers can safely read old data without crashes or logic errors.

table Config {
  timeout_ms: int = 5000;       // default prevents null access
  retry_count: int = 3;
  endpoint: string = "localhost";
}

2. Prefer Structs for Dense, Always-Present Data

If a group of fields is always present and relatively small, use a struct instead of a table. Structs are inline and require no vtable lookup, saving both space and access time. Ideal candidates include vectors, matrices, colors, timestamps, and coordinate tuples.

3. Use Vectors of Structs for Arrays of Fixed-Size Records

When you have an array of homogeneous records (e.g., animation keyframes, data points), use [MyStruct] — a vector of structs. This stores them contiguously without per-element vtables, making iteration extremely cache-efficient.

4. Validate Buffers from Untrusted Sources

Always run the generated Verify*Buffer function before accessing data from network or file sources. This prevents crashes from malformed or malicious buffers that could cause out-of-bounds reads.

flatbuffers::Verifier verifier(data_ptr, data_size);
if (!VerifyMonsterBuffer(verifier)) {
    // Reject the buffer — it may be corrupt or malicious
    return ERROR_INVALID_BUFFER;
}
// Safe to access now

5. Reuse FlatBufferBuilder When Possible

The FlatBufferBuilder can be reset and reused to avoid repeated heap allocations in hot loops. Call builder.Reset() or builder.Clear() between serialization rounds.

6. Size the Builder Appropriately

Pre-allocate sufficient capacity in the builder constructor to avoid internal reallocations during serialization. If you know your message will be roughly 4KB, pass 4096 or slightly more. The builder doubles capacity when full, but pre-sizing avoids the first few reallocs.

7. Don't Nest Too Deeply

Each table reference requires a pointer indirection. While FlatBuffers handles deep nesting gracefully, extremely deep object graphs (hundreds of levels) can still incur cumulative pointer-chasing costs. Flatten where practical.

8. Use file_identifier for Format Detection

Adding a 4-byte file identifier to your schema helps detect the buffer type at runtime:

file_identifier "MONS";
root_type Monster;

You can then check MonsterBufferHasIdentifier(buf) to confirm the buffer matches before processing.

9. Benchmark Your Access Patterns

The zero-copy promise is real, but it depends on how you use the data. Random access to scattered fields benefits enormously; linear full-message scans benefit less. Profile your specific workload to understand where FlatBuffers shines in your context.

10. Consider FlatBuffers for gRPC Too

FlatBuffers integrates with gRPC as an alternative serialization format. If your gRPC service is latency-sensitive, using FlatBuffers instead of Protocol Buffers can eliminate deserialization overhead on both client and server.

Language Support Matrix

FlatBuffers officially supports code generation for the following languages:

🚀 Need a reliable AI agent for your project?

Deploy Hermes Agent in 10 minutes. Managed hosting, zero DevOps.

Get Started — $23.99/mo
← Back to all articles