MareArts ANPR is a license-plate recognition app for iOS and Android,
with a web viewer and a desktop viewer on the same account.
Plates are read on the phone. Cloud MMC (make, model, colour, type, side, nation),
team sync, rules, maps, and stats sit on
marearts.com.
Scan — live box, crop, confidenceDetections — mosaicked plates plus MMC
Scan: on-device ANPR and cloud
Hold the phone at the lane. The app finds the plate, reads the characters,
and stores time plus GPS. No extra camera box and no PC at the gate.
Single — one capture.
Continuous — keep scanning; a duplicate window (default 5 seconds) skips the same plate.
Cloud — send the frame for OCR plus MMC vehicle info in one step.
On-device mode keeps detection and OCR on the phone after the models are installed.
Cloud mode is optional when you want vehicle identity with the plate.
Camera: 1080p, 720p recommended, or 480p. Preview 60 or 30 fps.
Lower resolution is faster and uses less storage. Zoom 1× / 2×, flash, front or rear camera.
Landscape scanning keeps boxes and controls upright.
Trial is 10 scans per day without login. A license on
the product page
removes that cap.
MMC here means MareArts cloud vehicle identity, not a third-party brand.
For each detection the API can return:
Make and model (with confidence)
Colour (with a colour chip)
Type (car, SUV, van, …)
Side — front or rear
Nation — country of the plate
The phone list shows a short line such as “Volvo XC60 · black”.
The detail card shows all six fields. Cloud scan writes MMC immediately;
on-device scans pick it up on sync. The
web viewer
filters the same fields (make, model, colour, type).
MMC uses the MareArts cloud API and is included with the subscription.
Daily vehicle-info usage is shown in Settings.
List — Volvo XC60 · black, and the restCard — make, model, colour, type, side, nation
Detections, map, and stats
List — grouped by day; All / Latest; swipe to set whitelist, blacklist, or unknown.
Detail — full frame, plate crop, confidence, first/last seen, GPS, history of that plate.
Map — pins and clusters, satellite or road, optional plate labels, search.
Stats — today / week / month / year / custom range; unique vehicles; filter by rule status.
CSV export — plate, time, GPS, confidences, rule note, plus MMC columns.
Map — clusters, plate labels, GPS
Whitelist and blacklist
Rules mark a plate as allowed (green) or blocked (red) on the next scan,
with sound and vibration. Notes can sit on the detection card.
Search, letter groups, swipe to delete, CSV import and export.
Upload a large CSV on
marearts.com,
then pull the package on the phone (replace or merge).
Team project
One team leader creates a team and shares a password.
Members join and contribute detections and rules.
Cloud sync is two-way. Auto-sync can run when the app goes to the background.
Leader sees every member’s data on the web viewer and can switch member in the dashboard.
Desktop viewer for a leader can pull the same team detections.
Leave or break team returns a user to solo mode.
Typical use: several guards at a gate, one supervisor on
the web viewer
without standing at the lane.
Web viewer
After login, My ANPR Data
is the browser console for the same account: detections, MMC filters,
rule packages, team member switch, export.
The phone captures; the browser reviews.
Web viewer — detections with MMC columnsWeb viewer — filter by make, model, colour, type
Desktop viewer
MareArts ANPR Desktop
is a free companion for Windows, macOS, and Linux.
Same login. It is a read-only viewer: it downloads cloud data
and does not edit or delete it. Capture and rule changes stay on the phone.
Detections with MMC and location
Map with clustering
Rules (view)
Statistics
Local image drop — run a still through the MareArts ANPR cloud API
One-click sync, optional auto-sync
Team leader: all members’ detections, same as the web viewer
Desktop viewer — detections and MMC on a large screen
Webhook, plate regions, SDK
Webhook — JSON to Discord, Slack, Zapier, Make, or your own URL (plate, time, GPS, confidences, bbox, rule, optional image).
Plate region — Universal, Europe+, Korea, North America, China.
Thresholds — detection and OCR 60–95% (90% recommended); max plates per frame 1–10.
Python ANPR SDK and Road Objects SDK — same license family; offline after activation. See
the license page.
How to start
Install MareArts ANPR (search “marearts anpr” on the App Store or Google Play).
Open Scan and use the daily trial.
Subscribe on
www.marearts.com/products/anpr.
The serial key is emailed to the PayPal address and also appears after login.
Two acronyms one letter apart, describing promises made at two different moments. An API is what
you agree to when you write the call. An ABI is what your compiled bytes agree to when they meet
somebody else's compiled bytes — and unlike the API, breaking it usually produces no error at all.
The clearest way in is to put them side by side. Only the middle word differs:
Programming or Binary.
Acronym
Level
A promise that sounds like
API Application Programming Interface
source code
"there is a function resize(img, width, height) and it returns an int"
ABI Application Binary Interface
machine code and memory
"width is at byte 0 and is 4 bytes; the first argument arrives in register rdi"
An API is a contract between a programmer and a library. An ABI is a contract between two piles of
already-compiled machine code. That is why two languages can call each other at all: Python has no
idea what a C header looks like, but if it lays the bytes out the way the ABI says and puts the
arguments where the ABI says, the call works.
This post is in five parts. First the distinction and what an ABI actually specifies. Then the three
pieces you can see and measure — struct layout, symbol names, argument registers — each with a
program you can run. Then the same boundary crossed from Python, including two mistakes that produce
wrong answers in total silence. Finally, what it takes to keep an ABI stable once other people
depend on it.
Everything was compiled and run before publishing: gcc 11.4.0 on x86-64 Linux, CPython 3.10.12, GNU
binutils 2.38. Every output block is copied from a real run.
Contents
Two contracts— the distinction, and why it costs you a rebuild
Take an ordinary header. It declares a struct and a function:
the API, as written
struct config {
int width;
int height;
double scale;
};
int config_area(const struct config *c);
Everything visible there is API: the names, the types, the order of the parameters, the fact that
the function takes a pointer rather than a value. If any of it changes, code that calls it stops
compiling — loudly, with a file and a line number.
The ABI is everything the compiler decided underneath that text, and none of it appears in
the header: that width begins at byte 0, that scale begins at byte 8 and
not byte 12, that the whole struct occupies 16 bytes, that the pointer argument arrives in the
register rdi, that the int comes back in eax, and that the
symbol is spelled config_area rather than _Z11config_areaPK6config.
Those decisions are just as much a contract, and both sides are bound by them. The difference is
what happens when one side changes its mind.
Recompile, relink, neither
The practical consequence of the distinction is what a change costs the people who use your library.
Change
API
ABI
What users must do
Fix a bug inside a function body
intact
intact
Nothing. Drop in the new .so.
Add a brand-new function
intact
intact
Nothing, unless they want the new one.
Rename a function
broken
broken
Edit their source, rebuild. The compiler tells them.
Reorder two struct fields
intact
broken
Rebuild — and nothing tells them.
Change a field from int to long
mostly intact
broken
Rebuild. Again, silently.
The last two rows are the whole reason this topic matters. A broken API is a compiler error; a
broken ABI is a program that runs and gives you the wrong number. The rest of this post is mostly
about making that second row concrete enough that you can spot it.
What an ABI decides
For a plain C interface on one machine, the list is short and worth memorising:
Struct layout — which byte each field starts at, and how big the whole thing is. Part 2.
Symbol naming — what the function is called in the object file. Part 3.
The calling convention — which register or stack slot each argument travels in, and who cleans up. Part 3.
Return values — which register the result comes back in, which depends on its type.
Sizes and alignments of the basic types sit underneath all four. C only guarantees relationships —
that a long is at least as wide as an int — and the ABI picks the actual
numbers, which is why long is 8 bytes on Linux and 4 bytes on 64-bit Windows.
Naming
The convention used throughout this post is the x86-64 System V ABI, which is what Linux and macOS
use. Windows has a different one, as do 32-bit x86 and the various ARM targets. The concepts
transfer exactly; the specific registers do not.
Part 2 · Struct layout
Where the fields actually land
You do not have to guess at any of this. offsetof and sizeof report what
the compiler decided, and _Alignof reports the rule it was following.
layout.c — same fields, two orders
struct badly_ordered {
char tag;
double value;
int id;
};
struct well_ordered {
double value;
int id;
char tag;
};
#define SHOW(S, M) \
printf(" %-6s offset %2zu size %2zu align %2zu\n", #M, \
offsetof(S, M), sizeof(((S *)0)->M), _Alignof(((S *)0)->M))
./layout
struct badly_ordered sizeof = 24 align 8
tag offset 0 size 1 align 1
value offset 8 size 8 align 8
id offset 16 size 4 align 4
struct well_ordered sizeof = 16 align 8
value offset 0 size 8 align 8
id offset 8 size 4 align 4
tag offset 12 size 1 align 1
field bytes actually used: 13
padding in badly_ordered : 11
padding in well_ordered : 3
Alignment and padding
Two rules produce every number in that output.
Each field starts at a multiple of its own alignment. A double has
alignment 8, so it may only begin at byte 0, 8, 16 and so on. In badly_ordered,
tag occupies byte 0, and value cannot start at byte 1 — the compiler skips
to byte 8 and leaves seven bytes of nothing behind.
The struct's own size is rounded up to its largest field alignment. That is what
makes arrays work: if sizeof were not a multiple of 8, the second element of an array
would put value on a misaligned address. In badly_ordered,
id ends at byte 20 and the size rounds up to 24.
The two structs contain identical data — a double, an int and a char, 13 bytes of payload — and one
is 50% larger than the other. Sorting fields from widest to narrowest, as well_ordered
does, is a reliable way to get most of that back.
Note
Padding bytes have no defined value. This is why memcmp on two structs can report a
difference when every field is equal, and why writing a struct straight to a file can leak
whatever happened to be on the stack.
Reordering is an ABI change with no API change
Now put the two structs next to each other again and ask what a caller sees. Both have the
same three fields with the same names and the same types. Any code that says
c.value = 1.5 compiles unchanged against either one. The API is identical.
The bytes are not. Field value lives at offset 8 in one and offset 0 in the other. A
library compiled against the first and a caller compiled against the second will agree on every
name and disagree on every address — and no tool in the build will say a word, because each half is
internally consistent.
Trap
sizeof matching is not evidence that two layouts agree. Two different orderings can
easily produce the same total size — you will see exactly that in
part 4, where a reordered struct is 16 bytes on both sides and still reads
the wrong bytes.
Part 3 · Symbols and registers
Name mangling
A linker does not match calls to functions by reading your source. It matches strings. What string a
function gets is part of the ABI, and C and C++ answer that question very differently.
mangling.cpp
int add(int a, int b) { return a + b; }
double add(double a, double b) { return a + b; }
namespace geometry {
int add(int a, int b) { return a + b; }
} // namespace geometry
extern "C" int add_c(int a, int b) { return a + b; }
0000000000000018 T _Z3adddd
0000000000000000 T _Z3addii
0000000000000040 T _ZN8geometry3addEii
0000000000000058 T add_c
Three functions are all called add in the source and none of them is called
add in the object file. C++ encodes the namespace and the parameter types into the
symbol — that encoding is what makes overloading possible, since the linker needs a different string
for each version. _Z3addii is add taking two ints;
_Z3adddd is the double one.
nm -C undoes it, as does piping any symbol through c++filt:
nm -C mangling.o
0000000000000018 T add(double, double)
0000000000000000 T add(int, int)
0000000000000040 T geometry::add(int, int)
0000000000000058 T add_c
What extern "C" is for
The fourth symbol is the odd one out: add_c, spelled exactly as written. That is the
entire effect of extern "C" — it tells a C++ compiler to use the C naming rule (the
plain name) and the C calling rule for this function, so that anything outside C++ can find it.
It is not a performance hint and it does not change the function body. It is a promise about the
string in the symbol table, which is why every library meant to be called from another language has
a wall of it around its public header. And it is why C++ features that need mangling — overloads,
templates, namespaces — cannot appear on such a boundary: there is only one add_c
available, so there can only be one function of that name.
Look at the exported symbols of a C library and you get the flat list you would expect:
nm -D --defined-only libconfig.so
0000000000001119 T config_area
0000000000001174 T config_describe
00000000000011de T config_offset_scale
0000000000001137 T config_scaled_area
00000000000011cf T config_sizeof
That list, plus the layout of struct config, is the library's entire ABI surface. The
T means the symbol is defined here and visible to others; U would mean
undefined and needed from somewhere else.
The calling convention
The last piece is where arguments physically go. Three tiny functions and a disassembler answer it
directly.
callconv.c
int sum4(int a, int b, int c, int d) { return a + b + c + d; }
double product2(double x, double y) { return x * y; }
int mixed(int a, double x, int b) { return a + b + (int)x; }
There is no prologue, no stack, no memory traffic at all. In sum4 the four arguments
are already sitting in edi, esi, edx and ecx
when the function begins, and the result is left in eax. That order is not something
gcc invented; it is written down in the ABI, and every compiler targeting this platform obeys it.
Kind
Arguments 1–6
Beyond that
Return
Integers and pointers
rdi rsi rdx rcx r8 r9
pushed on the stack
rax
Floats and doubles
xmm0 … xmm7
pushed on the stack
xmm0
product2 confirms the second row: mulsd %xmm1,%xmm0 multiplies the two
arguments in place and the result is already in the return register.
The detail that surprises people
Look again at mixed(int a, double x, int b). The instruction is
add %esi,%edi — so a is in edi and b is in
esi, even though b is the third parameter.
The two register sequences are counted independently. Integers take rdi,
rsi, … in order among themselves; floats take xmm0, xmm1, …
in order among themselves. The double in the middle consumes xmm0 and does
not consume an integer slot, so b gets the next integer register rather than the third
one.
Why you care
Because it means you cannot work out where an argument goes by counting parameters. Any hand-written
declaration that gets a parameter's type wrong — say, declaring a double
parameter as int in an FFI binding — does not merely mistranslate that one value. It
shifts every argument after it into the wrong register.
Part 4 · Crossing over from Python
The library
Everything so far has been observation. This part is where a mismatch actually costs you, and the
most convenient way to build one is a C library and a Python client, because Python has no access
to the header and has to be told the layout by hand.
libconfig.c
struct config {
int width;
int height;
double scale;
};
int config_area(const struct config *c) {
return c->width * c->height;
}
const char *config_describe(const struct config *c) {
static char buf[64];
snprintf(buf, sizeof buf, "%dx%d @ %.2fx", c->width, c->height, c->scale);
return buf;
}
/* Self-description, so a caller can verify its mirror instead of trusting it. */
size_t config_sizeof(void) { return sizeof(struct config); }
size_t config_offset_scale(void) { return offsetof(struct config, scale); }
Those last two functions are worth a moment. They export nothing useful to a program — but they let
a caller check its assumptions instead of hoping. A library that describes its own layout
turns a silent class of bug into an assertion, for the price of two lines.
A mirror that matches
On the Python side, ctypes needs the layout restated. This declaration is a hand-copy of
the C struct, and the whole exercise depends on it being right.
== 1. matching mirror ==
sizeof : python 16 | c 16
offset(scale) : python 8 | c 8
area : 2073600
describe : 1920x1080 @ 1.50x
Python computed offset 8 for scale by applying the same alignment rules the C compiler
applied, independently, from the field types alone. That is not a coincidence — ctypes
implements the platform ABI. It is also the reason the next section works the way it does.
A mirror that doesn't
Now the realistic accident: somebody writes the same three fields in a different order. Perhaps they
copied from documentation that listed them alphabetically. Every name is right, every type is right.
== 2. reordered mirror ==
sizeof : python 16 | c 16 -> agrees, and proves nothing
offset(scale) : python 0 | c 8 -> the actual disagreement
area : 0
describe : 0x1073217536 @ 0.00x
no exception, no warning, no crash
A 1920×1080 image now has an area of zero, and a height of 1073217536. Those numbers are not random:
C read width from bytes 0–3, which in this layout hold the bottom half of the double
1.5, and that half is all zeros. It read height from bytes 4–7, the top
half of the same double, which is 0x3FF80000 — the number printed.
Three things about that output deserve to be uncomfortable. There is no exception. There is no
warning. And sizeof agrees perfectly on both sides, because reordering these particular
fields happens to produce the same total. A size check would have passed this straight through; only
the offset check catches it.
Trap
ctypes type-checks the Python side thoroughly and the C side not at all. It will
refuse a ConfigWrong where you declared POINTER(Config) — but it has no
way to know what the library thinks either of them looks like. Every guarantee stops at the
boundary.
The restype trap
The other everyday mistake is forgetting to declare what comes back. ctypes assumes a
C int unless told otherwise, and part 3 of the calling convention says an
int comes back in rax while a double comes back in
xmm0. Two different registers.
The first call reads an integer register that the function never wrote to. It printed 0
here and it will print whatever is left over there on your machine; the point is only that it has
nothing to do with the answer.
The same failure with a pointer return is worse. Left as the default int, a returned
pointer is truncated to its low 32 bits, and dereferencing the result is a segmentation fault that
appears to come from inside the C library. Setting argtypes and restype on
every function you import is not ceremony — it is the only place the ABI gets written down on the
Python side.
What the Python examples actually demonstrate
An FFI binding is a hand-copy of an ABI. Nothing checks it against the library, so it drifts the moment the library changes.
A matching sizeof proves very little. Compare offsets, field by field, or compare nothing.
Have the library describe itself. Two exported functions turned an invisible failure into an assert that fires on line one.
Part 5 · Keeping an ABI stable
What breaks which
Once other people ship binaries built against your library, every change falls into one of these
boxes. The dangerous column is the one where the API survives and the ABI does not, because that is
the change nobody notices.
Change to a public struct or function
API
ABI
Add a function
ok
ok
Change a function body
ok
ok
Rename a parameter in the header
ok
ok
Reorder struct fields
ok
broken
Insert a field in the middle
ok
broken
Widen a field, int → long
ok
broken
Add a field at the end
ok
broken if callers allocate it
Change a parameter's type
broken
broken
Remove or rename a function
broken
broken
Change a #define constant
behaviour changes
ok
"Add a field at the end" is the interesting one. If callers only ever receive a pointer your library
allocated, growing the struct is harmless — they never compute its size. If callers declare one on
their own stack, they will allocate the old, smaller size, and your library will write past the end
of it.
How libraries stay compatible
Hide the struct entirely
The strongest option: callers never see the fields, only a pointer to something they cannot inspect.
opaque handle
/* in the public header */
typedef struct config config_t; /* declared, never defined */
config_t *config_create(void);
void config_set_scale(config_t *c, double scale);
int config_area(const config_t *c);
void config_destroy(config_t *c);
The layout is now a private implementation detail — it can change in any release, because no caller
ever computed an offset into it. The cost is a function call per field and an allocator round trip.
This is how most C libraries with long support windows do it.
Put the size in the struct
When callers must allocate, have them declare how big their version is, and check.
versioned struct
struct config {
size_t struct_size; /* caller sets this to sizeof(struct config) */
int width;
int height;
double scale;
/* new fields may only be appended below */
};
int config_area(const struct config *c) {
if (c->struct_size < offsetof(struct config, scale)) return -1;
...
}
The library can now tell an old caller from a new one at runtime and read only the fields that
caller actually has. The rule is absolute, though: fields may only ever be appended, and existing
ones may never move or change width.
Reserve space up front
A cruder version of the same idea — char reserved[32]; at the end of the struct, spent
later on new fields without changing the size. It works, it is ugly, and it is everywhere.
Version the symbol, not the struct
When a function genuinely has to change shape, keep the old one and add a new name:
config_area stays, config_area2 appears. Old binaries keep finding the old
symbol. The C standard library has done this for decades, which is why symbols with version suffixes
turn up in nm output on any mature library.
soname, and how the loader picks a file
Shared libraries carry a name inside them, and it is that name — not the filename — that gets
recorded in every program linked against them.
Dynamic section at offset 0x2e10 contains 25 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x000000000000000e (SONAME) Library soname: [libconfig.so.1]
0x000000000000000c (INIT) 0x1000
The convention that falls out of this is the familiar three-level filename. The file is
libconfig.so.1.0.0; the soname is libconfig.so.1, and that is what programs
record; the bare libconfig.so is a symlink that only the linker uses at build time.
The middle number is the promise. Bump the patch level for a bug fix, bump the minor for added
functions, and bump the soname to .so.2 when the ABI breaks. Because
the soname is part of the filename, version 1 and version 2 can sit in the same directory and old
programs keep loading the one they were built against. That is the whole mechanism by which a Linux
system runs binaries compiled years apart.
Part 6 · Reference
Tools worth knowing
nm -D --defined-only lib.so
Exported symbols — the callable ABI surface
nm -C, c++filt
Demangle C++ symbol names
readelf -d lib.so
soname and the list of libraries it needs
readelf -Ws lib.so
Full symbol table with sizes and visibility
objdump -d --no-show-raw-insn f.o
Disassembly — see the calling convention directly
ldd ./program
Which libraries actually get loaded, and from where
gcc -fdump-lang-raw, pahole
Struct layout and padding, reported by the toolchain
offsetof, _Alignof, _Static_assert
Assert the layout from inside the program itself
Cheat sheet
The distinction
API
Source-level contract. Broken → compiler error, with a line number.
ABI
Binary-level contract. Broken → wrong answers, usually with no message at all.
x86-64 System V, in four lines
Integer / pointer arguments
rdi rsi rdx rcx r8 r9, then the stack
Float / double arguments
xmm0–xmm7, counted separately
Return value
rax, or xmm0 for floating point
Struct field offsets
Each field aligned to its own width; total rounded up to the largest
Writing an FFI binding
Set argtypes and restype
On every single function. The defaults are wrong for pointers and doubles.
Assert sizeofand offsets
Sizes can match while layouts differ.
Export a self-description
Two functions returning sizeof and an offsetof pay for themselves.
Prefer opaque pointers
A layout nobody can see is a layout nobody can get wrong.
Wrapping up
What to take away
The mnemonic in the first table is worth keeping — Programming versus Binary, one
contract for people writing code and one for code that has already been written. But the reason to
care is narrower than that.
The short list
An API break is loud; an ABI break is silent. The compiler is on your side for
one of them and completely absent for the other.
Field order is part of your public interface. It does not look like it in the
header, and swapping two lines is a breaking change.
Every hand-written binding is a copy of a layout, so make it checkable. Export
sizeof and a couple of offsets, assert them at import time, and the entire category
of bug in part 4 turns into a failed assertion.
The stability techniques in part 5 only matter once someone else ships a binary against your code.
Until then the ABI is a private matter and you can rearrange whatever you like. The day that stops
being true is the day the opaque pointer starts looking cheap.
Further reading
The x86-64 psABI — the actual specification; chapter 3 is the calling convention and the struct classification rules
ctypes — the standard-library FFI used throughout part 4
Five short files produced every output on this page. They are reproduced in full below, so copying
the snippets into one directory is enough to reproduce all of it.
Verified on gcc 11.4.0 and GNU binutils 2.38 (Ubuntu 22.04, x86-64) with CPython 3.10.12. The one
value that is not reproducible is the integer read in part 3 of client.py, which is
whatever the return register happened to contain.
/* Where do arguments actually go? Build and look:
gcc -O1 -c callconv.c -o callconv.o
objdump -d --no-show-raw-insn callconv.o */
int sum4(int a, int b, int c, int d) { return a + b + c + d; }
double product2(double x, double y) { return x * y; }
int mixed(int a, double x, int b) { return a + b + (int)x; }
mangling.cpp 12 lines
mangling.cpp
/* What a linker actually sees. Build with:
g++ -c mangling.cpp -o mangling.o && nm mangling.o */
int add(int a, int b) { return a + b; }
double add(double a, double b) { return a + b; }
namespace geometry {
int add(int a, int b) { return a + b; }
} // namespace geometry
extern "C" int add_c(int a, int b) { return a + b; }
libconfig.c 29 lines
libconfig.c
#include <stddef.h>
#include <stdio.h>
/* Everything a caller in another language has to agree with lives here:
the layout of struct config, and the exported symbols below. */
struct config {
int width;
int height;
double scale;
};
int config_area(const struct config *c) {
return c->width * c->height;
}
double config_scaled_area(const struct config *c) {
return c->width * c->height * c->scale;
}
const char *config_describe(const struct config *c) {
static char buf[64];
snprintf(buf, sizeof buf, "%dx%d @ %.2fx", c->width, c->height, c->scale);
return buf;
}
/* Self-description, so a caller can verify its mirror instead of trusting it. */
size_t config_sizeof(void) { return sizeof(struct config); }
size_t config_offset_scale(void) { return offsetof(struct config, scale); }
client.py 76 lines
client.py
"""Three ways to talk to libconfig.so: one right, two wrong."""
import ctypes
lib = ctypes.CDLL("./libconfig.so")
lib.config_sizeof.restype = ctypes.c_size_t
lib.config_offset_scale.restype = ctypes.c_size_t
# ---- 1. a mirror that matches the C struct -------------------------------
class Config(ctypes.Structure):
_fields_ = [
("width", ctypes.c_int),
("height", ctypes.c_int),
("scale", ctypes.c_double),
]
lib.config_area.argtypes = [ctypes.POINTER(Config)]
lib.config_area.restype = ctypes.c_int
lib.config_describe.argtypes = [ctypes.POINTER(Config)]
lib.config_describe.restype = ctypes.c_char_p
print("== 1. matching mirror ==")
print("sizeof : python", ctypes.sizeof(Config), "| c", lib.config_sizeof())
print("offset(scale) : python", Config.scale.offset, "| c", lib.config_offset_scale())
assert ctypes.sizeof(Config) == lib.config_sizeof()
assert Config.scale.offset == lib.config_offset_scale()
cfg = Config(width=1920, height=1080, scale=1.5)
print("area :", lib.config_area(ctypes.byref(cfg)))
print("describe :", lib.config_describe(ctypes.byref(cfg)).decode())
# ---- 2. the same fields, declared in a different order -------------------
# A second handle, as if a different program had written its own mirror.
class ConfigWrong(ctypes.Structure):
_fields_ = [
("scale", ctypes.c_double),
("width", ctypes.c_int),
("height", ctypes.c_int),
]
other = ctypes.CDLL("./libconfig.so")
other.config_area.argtypes = [ctypes.POINTER(ConfigWrong)]
other.config_area.restype = ctypes.c_int
other.config_describe.argtypes = [ctypes.POINTER(ConfigWrong)]
other.config_describe.restype = ctypes.c_char_p
other.config_offset_scale.restype = ctypes.c_size_t
print("\n== 2. reordered mirror ==")
print("sizeof : python", ctypes.sizeof(ConfigWrong), "| c", lib.config_sizeof(),
"-> agrees, and proves nothing")
print("offset(scale) : python", ConfigWrong.scale.offset, "| c",
other.config_offset_scale(), "-> the actual disagreement")
bad = ConfigWrong(scale=1.5, width=1920, height=1080)
print("area :", other.config_area(ctypes.byref(bad)))
print("describe :", other.config_describe(ctypes.byref(bad)).decode())
print("no exception, no warning, no crash")
# ---- 3. a function called without declaring its return type --------------
print("\n== 3. missing restype ==")
lib.config_scaled_area.argtypes = [ctypes.POINTER(Config)]
print("default (int) :", lib.config_scaled_area(ctypes.byref(cfg)))
lib.config_scaled_area.restype = ctypes.c_double
print("c_double :", lib.config_scaled_area(ctypes.byref(cfg)))
print("expected :", 1920 * 1080 * 1.5)
C forgets the name of every field the moment it finishes compiling. The X-macro is how a C
programmer gets those names back — by writing the list once and replaying it as an enum, a string
table, a descriptor table, and a set of registration calls that run before main.
Almost everyone meets #define as a way to name the number 100 and then never
thinks about it again. That is a shame, because the preprocessor is the only part of C that can
write C for you, and there is one pattern in particular that turns it from a curiosity into
something load-bearing.
The problem it solves is dull enough that it hides in plain sight: one piece of knowledge,
written down in two places. An enum and the array of names that goes with it. A struct and
the code that serialises it. A config table and the parser that reads it. Each pair is fine on the
day it is written and wrong three months later, when someone edits one half.
This is written as a path. Part 1 is the mechanics — what a macro actually does, and how a list
grows a hole in it. Part 2 builds the useful thing: a runtime table describing a struct, with
offsets the compiler computes. Part 3 makes that table reachable, first from elsewhere in the
program and then from Python. Part 4 is the part people learn the hard way.
Everything here was compiled and run before publishing — gcc 11.4.0 on x86-64, CPython 3.10 — and
every output block is copied from a real run, including the error message.
Contents
The preprocessor— what is really happening before the compiler starts
Most people meet #define as a way to name a constant, and stop there.
the half everyone knows
#define MAX_USERS 100
The other half is that a macro can take parameters, and that what it does is
substitute text before the compiler ever sees the file.
the other half
#define SQUARE(x) ((x) * (x))
int n = SQUARE(3 + 1); /* becomes ((3 + 1) * (3 + 1)), so 16 */
Nothing is evaluated. No types are checked. There is no function call, no scope, no
x anywhere at runtime. The preprocessor pastes characters and moves on, and the
compiler then reads the result as if you had typed it yourself.
Hold on to that sentence, because every clever thing below is just careful pasting. The parentheses
around (x) in that snippet are the classic example of why: without them,
SQUARE(3 + 1) pastes into 3 + 1 * 3 + 1, which is 7.
Mental model
A macro is not a function that runs early. It is a find-and-replace rule that runs on your source
text. When an expansion surprises you, the answer is always visible in the pasted text — see
debugging expansions.
Two lists that must agree
Say you have three colours. You need an enum for the code, and names for logging.
This is fine. It stays fine right up until someone adds COLOR_YELLOW to the enum and
does not touch the array. Now color_names[COLOR_YELLOW] reads past the end of the
array, and nothing warns you — as far as the compiler is concerned these are two unrelated
declarations that happen to sit next to each other. Keeping them in sync is a rule that lives only
in someone's memory.
The same shape shows up everywhere once you start looking for it: an enum and the switch
over it, a struct and its serialiser, a set of command-line flags and the help text. Two lists, one
fact, and a silent failure when they disagree.
The X-macro: give the list a parameter
The idea is to write the list exactly once, but leave a hole where the action goes.
COLOR_LIST is not a value and cannot be used on its own. It is a list that does not yet
know what to do with its items — X is the hole. By convention that parameter is named
X, which is the whole reason the pattern is called an X-macro; any name works.
Now define what a single item should become, and hand it in.
COLOR_LIST(AS_ENUM) pastes into COLOR_RED, COLOR_GREEN, COLOR_BLUE, and
COLOR_LIST(AS_STRING) pastes into "RED", "GREEN", "BLUE",. Two completely
different pieces of code out of one list, and the trailing comma is harmless in both because
COLOR_COUNT follows the first and C allows a trailing comma in an initialiser.
Add X(YELLOW) to the list and both the enum and the name array grow.
The synchronisation bug is not fixed by discipline here; it has been made impossible to write.
./demo
colors: 3
0 = RED
1 = GREEN
2 = BLUE
Note
Notice where the comma lives: inside AS_ENUM and AS_STRING, not in
COLOR_LIST. The list carries no punctuation at all, because different actions want
different separators — an enum wants commas, a sequence of statements wants none.
The two operators: # and ##
Those action macros used two operators that exist nowhere else in C.
Operator
Name
Given name = RED
#name
stringify
produces the string literal "RED"
A##B
token paste
COLOR_##name produces the single token COLOR_RED
The property that makes # worth the trouble is that the string comes from
the actual token you wrote. You cannot end up with a name string that disagrees
with the identifier beside it, because they are literally the same characters. Every hand-written
"id" next to a field called id is a copy waiting to go stale;
#M is not a copy.
## is what lets one list generate distinct identifiers — COLOR_RED,
sensor_reading_fields, FIELD_IDX_timestamp. It has one sharp edge, which
we will hit in the section on unique names: ## suppresses macro
expansion of its own operands.
Part 2 · Describing a struct
What C throws away
Colours are a warm-up. Here is the use that actually earns the pattern its place.
C discards field names at compile time. After the compiler is done, a struct is a size and a set of
byte offsets; the word timestamp exists in the debug info, if you asked for it, and
nowhere else. So the moment something outside the compiler needs to know that "byte 16 of this
struct is a double called timestamp" — a serialiser, a logger, a test
harness, another language — somebody has to write that down.
Writing it down by hand is the two-lists problem again, with worse consequences, because now the
second list contains numbers.
demo.c — the struct
struct sensor_reading {
int id;
int sample_count;
float temperature;
double timestamp;
char *label;
};
Same shape as before: write the list once, with a hole. Each item now carries three pieces of
information instead of one — the struct it belongs to, the member, and a type tag.
Two things in that action macro are worth stopping on.
offsetof(S, M) is a standard macro from <stddef.h>,
and the crucial word is standard: the compiler computes the byte offset. You never type a
number, so the table cannot drift from the real layout when a field is inserted, when the order
changes, or when padding rules differ between targets.
sizeof(((S *)0)->M) is the idiom for "the size of member
M of type S" when you have no instance to point at. It looks like a null
dereference and is not one: sizeof does not evaluate its operand, it only inspects the
operand's type, so nothing is ever read from address zero. C11 also offers
sizeof(((S){0}).M) if the cast makes you uncomfortable.
Here is what the preprocessor genuinely emits for the first entry, lifted out of
gcc -E -P demo.c:
…and what the compiler folds that into, before anything runs:
constant-folded
{"id", 0, 0, 4}
There is no runtime cost to any of this. The table is a static array of constants in
.rodata, exactly as if it had been typed out by hand — the difference is only in who
typed it.
The padding you would have got wrong
The full table, printed at runtime:
./demo
sizeof(struct sensor_reading) = 32
field count = 5
name type offset size
id 0 0 4
sample_count 0 4 4
temperature 1 8 4
timestamp 2 16 8
label 3 24 8
Look at temperature: it starts at byte 8 and is 4 bytes long, so it ends at 12. But
timestamp starts at 16, not 12. The compiler inserted four bytes of padding so the
double lands on an 8-byte boundary. The five field sizes add up to 28; the struct is 32.
A hand-written table would have said 12, and every read through it would have been off by four
bytes from that field onward — the kind of bug that produces plausible-looking garbage rather than a
crash. offsetof did not get it wrong, and cannot.
ABI
These numbers are for the x86-64 System V ABI. On a different target the offsets and the total
size may differ. That is an argument foroffsetof rather than a caveat about
it: the same source produces the right table on each target.
The same list, a different output
The list is now a reusable asset, and this is where the pattern starts paying compound interest. A
different action over the same list gives you named indices into the table:
demo.c
#define AS_INDEX(S, M, T) FIELD_IDX_##M,
enum { SENSOR_FIELDS(AS_INDEX) SENSOR_FIELD_COUNT };
./demo
FIELD_IDX_timestamp = 3
Note that AS_INDEX ignores two of its three parameters. That is normal and fine — the
list is a fixed shape, and each action takes what it needs. Adding one field to the struct and one
line to the list now updates the descriptor table, the index enum, and the count, in one edit.
Other actions people write over the same list, without changing it: a switch that
converts a field name string back to an index, a printf-style dumper with the right format specifier
per type tag, a JSON writer, a comparison function, a fuzzer that fills each field with the right
kind of noise.
Renaming a field breaks the build — on purpose
This is the property that makes the syntax worth tolerating, and it is easy to miss because nothing
visibly happens on a good day.
Rename id to reading_id in the struct and forget the list. The build stops:
gcc -std=c11 -Wall -Wextra -o demo demo.c
error: ‘struct sensor_reading’ has no member named ‘id’
50 | {#M, (T), (unsigned)offsetof(S, M), (unsigned)sizeof(((S *)0)->M)},
| ^~~~~~~~
43 | X(struct sensor_reading, id, FIELD_INT)
| ^ in expansion of macro ‘AS_FIELD_DESC’
Compare that with the hand-written table, where the string "id" would go on cheerfully
naming a field that no longer exists, and the offset next to it would now describe a different
field entirely. A stale hand-written mirror is silent; a stale X-macro list is a compile
error. The entire pattern can be justified on that one sentence.
Reading the error
The compiler points at line 50 — inside AS_FIELD_DESC, which is not where your
mistake is — and then adds an "in expansion of macro" note pointing at line 43, which is. Always
read down to the note.
Part 3 · Making it reachable
Registering yourself before main
A table is only useful if something can find it. The usual arrangement is a small registry that
types add themselves to:
You could call layout_register from main. But then every new type means
editing main, and a central list of every type in the program is exactly the drift
problem from part 1, moved up one level. Better to have the call happen by
itself:
That one invocation generates three things at once: a static table built from the field list, a
function that hands it to the registry, and the annotation that makes the function run on its own.
#NAME supplies the lookup key, and sizeof(arr)/sizeof(arr[0]) supplies the
count without anyone counting.
__attribute__((constructor)) marks a function to run before main — or, in
a shared library, at load time. It is a GCC and Clang extension rather than ISO C; MSVC has its own
mechanism via .CRT$XCU, and the portable fallback is an explicit init()
that you do have to remember to call. In C++ the same effect falls out of a global object's
constructor, which is why you see the trick spelled as a dummy static Registrar r{}
there.
Why this shape keeps showing up
The knowledge sits next to the thing it describes. The field list lives beside the struct, not in a registry file three directories away.
Adding a type is a local edit. No shared file to touch, so no merge conflict when two people add a type in the same week.
Nothing can be half-added. The table, the registration and the name all come from one macro invocation, so there is no state where one exists without the others.
Unique names when the macro is used twice
NAME##_fields works because the caller supplies a distinct NAME. Sometimes
a macro has to be usable twice in one file with nothing to distinguish the uses — two registrations
of the same struct under different keys, say. The standard-ish trick is __COUNTER__,
which expands to 0, 1, 2, … each time it is read.
The middle layer is not decoration. ## suppresses expansion of its operands, so
without UNIQUE_IMPL you get a variable literally named
tmp_var___COUNTER__ — and a redefinition error on the second use. The extra
indirection forces __COUNTER__ to expand to a number before it reaches the
paste.
__COUNTER__ is supported by GCC, Clang and MSVC but is not in the standard.
__LINE__ is the portable substitute and is fine whenever one use per line is an
acceptable restriction.
Reading the table from another language
Once the table lives in a shared library behind a plain C function, anything with an FFI can read
it. Python's ctypes is in the standard library, so this needs no dependencies:
The FieldDesc declaration on the Python side does have to match the C one by hand —
that is the one mirror you cannot escape. But it is a single four-field descriptor type that changes
approximately never, and it buys you the description of every other struct in the program.
Building the mirror at runtime
Because the table carries names, type tags and offsets, the caller can construct a matching
structure definition while the program runs, instead of hand-writing one per struct:
read_layout.py
TAGS = {0: ctypes.c_int, 1: ctypes.c_float, 2: ctypes.c_double, 3: ctypes.c_void_p}
fields = [(table[i].name.decode(), TAGS[table[i].type]) for i in range(count.value)]
Mirror = type("SensorReading", (ctypes.Structure,), {"_fields_": fields})
python3 read_layout.py
id tag=0 offset=0 size=4
sample_count tag=0 offset=4 size=4
temperature tag=1 offset=8 size=4
timestamp tag=2 offset=16 size=8
label tag=3 offset=24 size=8
built mirror : SensorReading
python sizeof: 32
c sizeof : 32
mirror.timestamp offset: 16
The two sizeof values agreeing is the check that matters, and it is worth asserting
rather than eyeballing. If they differ, the description is wrong somewhere and every subsequent
read through the mirror lands on the wrong bytes — silently, with plausible values.
This is the whole payoff, and it is worth stating plainly: the struct is defined once, in C,
and everybody else asks it what it looks like. No generated bindings to regenerate, no
Python file to update when a field moves, no version skew between a library and the script reading
it — because the description ships inside the library.
Part 4 · Practice
Pitfalls
The trailing backslash must be the last character on the line
One trailing space after it and the continuation silently ends, taking the rest of your list with
it. Some compilers warn; do not count on it. This is the single most common way to lose an hour to
this pattern, and it is invisible in a diff.
Put the punctuation in the action, not the list
Different actions need different separators. Keep the list free of commas and semicolons so it can
serve an enum, an initialiser and a sequence of statements without special cases.
Order matters when position matters
If consumers index the table by position, the list must follow the struct's declaration order.
Nothing enforces this — the build still succeeds, and only the offset column looks odd.
If you can, index by name instead.
Macros do not recurse
A list cannot include itself, directly or through another macro. Nested lists work only in a strict
hierarchy. If you find yourself wanting recursion, you have outgrown the pattern.
Keep the list next to the thing it describes
A field list in a different file from the struct can drift in the one way the pattern exists to
prevent: somebody adds a field, never sees the list, and the table is quietly incomplete. Note that
this failure is not caught by the compiler — the rename check only
fires for fields the list mentions.
The one gap
An X-macro list catches renamed and deleted fields. It does not catch added ones. If
completeness matters, add a static assertion that the field sizes plus known padding equal
sizeof the struct, or simply assert the struct size and let the build break when it
changes.
Debugging with the preprocessor, not by staring
Nearly every X-macro bug becomes obvious the moment you look at the pasted text. Do not reason about
it; print it.
see what the compiler sees
gcc -E -P demo.c # preprocess only, strip line markers
gcc -E -P demo.c | clang-format # ...and make it readable
gcc -E -P demo.c | sed -n '/sensor_layout/,/;/p'
-E stops after preprocessing and -P drops the #line markers
that otherwise dominate the output. Everything the compiler will see is in there, including the
expansion you were sure was correct.
When not to reach for it
When the list has exactly one consumer
If you only need the enum, write the enum. The pattern costs readability up front and repays it per
consumer; it breaks even at two and clearly pays at three.
When the list is large, or comes from somewhere else
Past a few dozen entries, or when the real source of truth is a schema file, a protobuf definition
or a database, generate a .c file from a script. Generated code can be read, diffed and
stepped through in a debugger; a wall of backslash continuations cannot.
When you want type dispatch, not list replay
C11's _Generic selects an expression based on a type. It solves a different problem and
reads far better for that one.
When the language has real reflection
This entire pattern is a workaround for C discarding names. In a language that keeps them, use what
the language gives you.
Cheat sheet
Preprocessor operators and predefined macros
#x
Stringify the argument as written
a##b
Paste two tokens into one; suppresses expansion of both operands
__COUNTER__
0, 1, 2, … per read — needs a two-step macro to paste with
__LINE__, __FILE__
Portable alternatives; one use per line
__VA_ARGS__
Variadic macro arguments, for actions of varying arity
Layout introspection
offsetof(S, M)
Byte offset of a member, from <stddef.h>
sizeof(((S *)0)->M)
Size of a member with no instance; operand is not evaluated
sizeof(a) / sizeof(a[0])
Element count of a real array — not of a pointer
_Static_assert(c, "msg")
Compile-time check; useful for pinning struct size
Running code before main
__attribute__((constructor))
GCC and Clang; also runs at dlopen time
Global object constructor
C++; the same effect, portably
.CRT$XCU section
MSVC equivalent
Explicit init()
Portable fallback, at the cost of a central call site
Commands
gcc -E -P f.c
Show the pasted text the compiler will read
gcc -Wall -Wextra
Catches trailing-comma and unused-macro-argument mistakes
pahole ./a.out
Print struct layout and padding, independently of your table
Wrapping up
What actually matters here
The X-macro has a reputation for being a clever trick, which undersells it. It is a way of making a
class of bug unrepresentable, and the syntax is the price.
The short list
The list with a hole is the whole idea.LIST(X) plus an action
macro; everything else in this document is a choice of action.
Let the compiler compute the numbers.offsetof and
sizeof mean the table cannot disagree with the struct, on any target, ever.
A stale hand-written mirror is silent; a stale list is a compile error. That
trade — a little syntax now for a loud failure later — is the reason to use it.
The registry and the FFI reader in part 3 are a step further than most code needs, but they are the
natural end of the road: once the description exists as data, the interesting question stops being
"how do I keep these in sync" and becomes "who else would like to read this". Usually the answer is
more people than you expected.
Further reading
The C Preprocessor — GCC's manual; the chapters on stringizing, concatenation and argument prescan are the relevant ones
offsetof — including exactly when it is undefined behaviour
ctypes — the FFI used in part 3, standard library, no build step
The Linux kernel — include/linux/kvm_host.h and the tracepoint headers are X-macros at industrial scale, for better and worse
Appendix · The complete programs
Building and running
Everything above came from three short files, reproduced in full below so this page stands on its
own — copy the three snippets into a directory and the commands work as written.
build and run everything
gcc -std=c11 -Wall -Wextra -o demo demo.c && ./demo
gcc -std=c11 -Wall -Wextra -shared -fPIC -o libreflect.so reflect.c
python3 read_layout.py
gcc -E -P demo.c | head -40 # see what the preprocessor produced
Verified on gcc 11.4.0 (Ubuntu 22.04, x86-64) and CPython 3.10.12. Every output block on this page
is copied from a real run, including the compiler error in the rename
section, which was produced by renaming id to reading_id in a copy of
demo.c.
demo.c 74 lines
The warm-up and the field table, in one standalone program. Nothing outside the C standard library.
field_desc is declared identically in both C files rather than shared through a
header, so that each one can be read on its own. In a real program it would live in a header, and
layout_register would be declared there too.
read_layout.py 37 lines
The other side of the wall: no C compiler involved, no generated bindings, just the library and the
standard library.
Setting argtypes and restype is not optional decoration. Without them
ctypes assumes every argument and the return value are int, which on
x86-64 truncates pointers to 32 bits and produces a segfault that looks like it came from the C
side.
