C forgets the name of every field the moment it finishes compiling. The X-macro is how a C
programmer gets those names back — by writing the list once and replaying it as an enum, a string
table, a descriptor table, and a set of registration calls that run before main.
Almost everyone meets #define as a way to name the number 100 and then never
thinks about it again. That is a shame, because the preprocessor is the only part of C that can
write C for you, and there is one pattern in particular that turns it from a curiosity into
something load-bearing.
The problem it solves is dull enough that it hides in plain sight: one piece of knowledge, written down in two places. An enum and the array of names that goes with it. A struct and the code that serialises it. A config table and the parser that reads it. Each pair is fine on the day it is written and wrong three months later, when someone edits one half.
This is written as a path. Part 1 is the mechanics — what a macro actually does, and how a list grows a hole in it. Part 2 builds the useful thing: a runtime table describing a struct, with offsets the compiler computes. Part 3 makes that table reachable, first from elsewhere in the program and then from Python. Part 4 is the part people learn the hard way.
Everything here was compiled and run before publishing — gcc 11.4.0 on x86-64, CPython 3.10 — and every output block is copied from a real run, including the error message.
Contents
- The preprocessor — what is really happening before the compiler starts
- Describing a struct — the payoff: a table the compiler fills in
- Making it reachable — from a table nobody can find to one everybody can
- Practice — traps, tooling, and knowing when to stop
- Appendix — the three programs in full, and how to build them
Part 1 · The preprocessor
A macro is text, not a value
Most people meet #define as a way to name a constant, and stop there.
#define MAX_USERS 100The other half is that a macro can take parameters, and that what it does is substitute text before the compiler ever sees the file.
#define SQUARE(x) ((x) * (x))
int n = SQUARE(3 + 1); /* becomes ((3 + 1) * (3 + 1)), so 16 */
Nothing is evaluated. No types are checked. There is no function call, no scope, no
x anywhere at runtime. The preprocessor pastes characters and moves on, and the
compiler then reads the result as if you had typed it yourself.
Hold on to that sentence, because every clever thing below is just careful pasting. The parentheses
around (x) in that snippet are the classic example of why: without them,
SQUARE(3 + 1) pastes into 3 + 1 * 3 + 1, which is 7.
A macro is not a function that runs early. It is a find-and-replace rule that runs on your source text. When an expansion surprises you, the answer is always visible in the pasted text — see debugging expansions.
Two lists that must agree
Say you have three colours. You need an enum for the code, and names for logging.
enum color { COLOR_RED, COLOR_GREEN, COLOR_BLUE, COLOR_COUNT };
static const char *color_names[] = { "RED", "GREEN", "BLUE" };
This is fine. It stays fine right up until someone adds COLOR_YELLOW to the enum and
does not touch the array. Now color_names[COLOR_YELLOW] reads past the end of the
array, and nothing warns you — as far as the compiler is concerned these are two unrelated
declarations that happen to sit next to each other. Keeping them in sync is a rule that lives only
in someone's memory.
The same shape shows up everywhere once you start looking for it: an enum and the switch
over it, a struct and its serialiser, a set of command-line flags and the help text. Two lists, one
fact, and a silent failure when they disagree.
The X-macro: give the list a parameter
The idea is to write the list exactly once, but leave a hole where the action goes.
#define COLOR_LIST(X) \
X(RED) \
X(GREEN) \
X(BLUE)
COLOR_LIST is not a value and cannot be used on its own. It is a list that does not yet
know what to do with its items — X is the hole. By convention that parameter is named
X, which is the whole reason the pattern is called an X-macro; any name works.
Now define what a single item should become, and hand it in.
#define AS_ENUM(name) COLOR_##name,
#define AS_STRING(name) #name,
enum color { COLOR_LIST(AS_ENUM) COLOR_COUNT };
static const char *color_names[] = { COLOR_LIST(AS_STRING) };
COLOR_LIST(AS_ENUM) pastes into COLOR_RED, COLOR_GREEN, COLOR_BLUE, and
COLOR_LIST(AS_STRING) pastes into "RED", "GREEN", "BLUE",. Two completely
different pieces of code out of one list, and the trailing comma is harmless in both because
COLOR_COUNT follows the first and C allows a trailing comma in an initialiser.
Add X(YELLOW) to the list and both the enum and the name array grow.
The synchronisation bug is not fixed by discipline here; it has been made impossible to write.
colors: 3
0 = RED
1 = GREEN
2 = BLUE
Notice where the comma lives: inside AS_ENUM and AS_STRING, not in
COLOR_LIST. The list carries no punctuation at all, because different actions want
different separators — an enum wants commas, a sequence of statements wants none.
The two operators: # and ##
Those action macros used two operators that exist nowhere else in C.
| Operator | Name | Given name = RED |
|---|---|---|
#name | stringify | produces the string literal "RED" |
A##B | token paste | COLOR_##name produces the single token COLOR_RED |
The property that makes # worth the trouble is that the string comes from
the actual token you wrote. You cannot end up with a name string that disagrees
with the identifier beside it, because they are literally the same characters. Every hand-written
"id" next to a field called id is a copy waiting to go stale;
#M is not a copy.
## is what lets one list generate distinct identifiers — COLOR_RED,
sensor_reading_fields, FIELD_IDX_timestamp. It has one sharp edge, which
we will hit in the section on unique names: ## suppresses macro
expansion of its own operands.
Part 2 · Describing a struct
What C throws away
Colours are a warm-up. Here is the use that actually earns the pattern its place.
C discards field names at compile time. After the compiler is done, a struct is a size and a set of
byte offsets; the word timestamp exists in the debug info, if you asked for it, and
nowhere else. So the moment something outside the compiler needs to know that "byte 16 of this
struct is a double called timestamp" — a serialiser, a logger, a test
harness, another language — somebody has to write that down.
Writing it down by hand is the two-lists problem again, with worse consequences, because now the second list contains numbers.
struct sensor_reading {
int id;
int sample_count;
float temperature;
double timestamp;
char *label;
};typedef enum { FIELD_INT, FIELD_FLOAT, FIELD_DOUBLE, FIELD_PTR } field_type;
typedef struct {
const char *name;
field_type type;
unsigned offset;
unsigned size;
} field_desc;Building the field table
Same shape as before: write the list once, with a hole. Each item now carries three pieces of information instead of one — the struct it belongs to, the member, and a type tag.
#define SENSOR_FIELDS(X) \
X(struct sensor_reading, id, FIELD_INT) \
X(struct sensor_reading, sample_count, FIELD_INT) \
X(struct sensor_reading, temperature, FIELD_FLOAT) \
X(struct sensor_reading, timestamp, FIELD_DOUBLE) \
X(struct sensor_reading, label, FIELD_PTR)#define AS_FIELD_DESC(S, M, T) \
{ #M, (T), (unsigned)offsetof(S, M), (unsigned)sizeof(((S *)0)->M) },
static const field_desc sensor_layout[] = { SENSOR_FIELDS(AS_FIELD_DESC) };Two things in that action macro are worth stopping on.
offsetof(S, M) is a standard macro from <stddef.h>,
and the crucial word is standard: the compiler computes the byte offset. You never type a
number, so the table cannot drift from the real layout when a field is inserted, when the order
changes, or when padding rules differ between targets.
sizeof(((S *)0)->M) is the idiom for "the size of member
M of type S" when you have no instance to point at. It looks like a null
dereference and is not one: sizeof does not evaluate its operand, it only inspects the
operand's type, so nothing is ever read from address zero. C11 also offers
sizeof(((S){0}).M) if the cast makes you uncomfortable.
Here is what the preprocessor genuinely emits for the first entry, lifted out of
gcc -E -P demo.c:
{"id", (FIELD_INT), (unsigned)__builtin_offsetof (struct sensor_reading, id),
(unsigned)sizeof(((struct sensor_reading *)0)->id)},…and what the compiler folds that into, before anything runs:
{"id", 0, 0, 4}
There is no runtime cost to any of this. The table is a static array of constants in
.rodata, exactly as if it had been typed out by hand — the difference is only in who
typed it.
The padding you would have got wrong
The full table, printed at runtime:
sizeof(struct sensor_reading) = 32
field count = 5
name type offset size
id 0 0 4
sample_count 0 4 4
temperature 1 8 4
timestamp 2 16 8
label 3 24 8
Look at temperature: it starts at byte 8 and is 4 bytes long, so it ends at 12. But
timestamp starts at 16, not 12. The compiler inserted four bytes of padding so the
double lands on an 8-byte boundary. The five field sizes add up to 28; the struct is 32.
A hand-written table would have said 12, and every read through it would have been off by four
bytes from that field onward — the kind of bug that produces plausible-looking garbage rather than a
crash. offsetof did not get it wrong, and cannot.
These numbers are for the x86-64 System V ABI. On a different target the offsets and the total
size may differ. That is an argument for offsetof rather than a caveat about
it: the same source produces the right table on each target.
The same list, a different output
The list is now a reusable asset, and this is where the pattern starts paying compound interest. A different action over the same list gives you named indices into the table:
#define AS_INDEX(S, M, T) FIELD_IDX_##M,
enum { SENSOR_FIELDS(AS_INDEX) SENSOR_FIELD_COUNT };FIELD_IDX_timestamp = 3
Note that AS_INDEX ignores two of its three parameters. That is normal and fine — the
list is a fixed shape, and each action takes what it needs. Adding one field to the struct and one
line to the list now updates the descriptor table, the index enum, and the count, in one edit.
Other actions people write over the same list, without changing it: a switch that
converts a field name string back to an index, a printf-style dumper with the right format specifier
per type tag, a JSON writer, a comparison function, a fuzzer that fills each field with the right
kind of noise.
Renaming a field breaks the build — on purpose
This is the property that makes the syntax worth tolerating, and it is easy to miss because nothing visibly happens on a good day.
Rename id to reading_id in the struct and forget the list. The build stops:
error: ‘struct sensor_reading’ has no member named ‘id’
50 | {#M, (T), (unsigned)offsetof(S, M), (unsigned)sizeof(((S *)0)->M)},
| ^~~~~~~~
43 | X(struct sensor_reading, id, FIELD_INT)
| ^ in expansion of macro ‘AS_FIELD_DESC’
Compare that with the hand-written table, where the string "id" would go on cheerfully
naming a field that no longer exists, and the offset next to it would now describe a different
field entirely. A stale hand-written mirror is silent; a stale X-macro list is a compile
error. The entire pattern can be justified on that one sentence.
The compiler points at line 50 — inside AS_FIELD_DESC, which is not where your
mistake is — and then adds an "in expansion of macro" note pointing at line 43, which is. Always
read down to the note.
Part 3 · Making it reachable
Registering yourself before main
A table is only useful if something can find it. The usual arrangement is a small registry that types add themselves to:
void layout_register(const char *type_name, const field_desc *fields,
size_t count, size_t struct_size);
const field_desc *layout_lookup(const char *type_name, size_t *count_out);
You could call layout_register from main. But then every new type means
editing main, and a central list of every type in the program is exactly the drift
problem from part 1, moved up one level. Better to have the call happen by
itself:
#define REGISTER_LAYOUT(NAME, STRUCT, FIELDS) \
static const field_desc NAME##_fields[] = { FIELDS(AS_FIELD_DESC) }; \
__attribute__((constructor)) static void NAME##_register_layout(void) { \
layout_register(#NAME, NAME##_fields, \
sizeof(NAME##_fields) / sizeof(NAME##_fields[0]), \
sizeof(STRUCT)); \
}
REGISTER_LAYOUT(sensor_reading, struct sensor_reading, SENSOR_FIELDS)
That one invocation generates three things at once: a static table built from the field list, a
function that hands it to the registry, and the annotation that makes the function run on its own.
#NAME supplies the lookup key, and sizeof(arr)/sizeof(arr[0]) supplies the
count without anyone counting.
__attribute__((constructor)) marks a function to run before main — or, in
a shared library, at load time. It is a GCC and Clang extension rather than ISO C; MSVC has its own
mechanism via .CRT$XCU, and the portable fallback is an explicit init()
that you do have to remember to call. In C++ the same effect falls out of a global object's
constructor, which is why you see the trick spelled as a dummy static Registrar r{}
there.
Why this shape keeps showing up
- The knowledge sits next to the thing it describes. The field list lives beside the struct, not in a registry file three directories away.
- Adding a type is a local edit. No shared file to touch, so no merge conflict when two people add a type in the same week.
- Nothing can be half-added. The table, the registration and the name all come from one macro invocation, so there is no state where one exists without the others.
Unique names when the macro is used twice
NAME##_fields works because the caller supplies a distinct NAME. Sometimes
a macro has to be usable twice in one file with nothing to distinguish the uses — two registrations
of the same struct under different keys, say. The standard-ish trick is __COUNTER__,
which expands to 0, 1, 2, … each time it is read.
#define UNIQUE_IMPL2(ctr) static int tmp_var_##ctr;
#define UNIQUE_IMPL(ctr) UNIQUE_IMPL2(ctr)
#define MAKE_UNIQUE() UNIQUE_IMPL(__COUNTER__)
The middle layer is not decoration. ## suppresses expansion of its operands, so
without UNIQUE_IMPL you get a variable literally named
tmp_var___COUNTER__ — and a redefinition error on the second use. The extra
indirection forces __COUNTER__ to expand to a number before it reaches the
paste.
__COUNTER__ is supported by GCC, Clang and MSVC but is not in the standard.
__LINE__ is the portable substitute and is fine whenever one use per line is an
acceptable restriction.
Reading the table from another language
Once the table lives in a shared library behind a plain C function, anything with an FFI can read
it. Python's ctypes is in the standard library, so this needs no dependencies:
class FieldDesc(ctypes.Structure):
_fields_ = [("name", ctypes.c_char_p), ("type", ctypes.c_int),
("offset", ctypes.c_uint), ("size", ctypes.c_uint)]
lib = ctypes.CDLL("./libreflect.so")
lib.layout_lookup.restype = ctypes.POINTER(FieldDesc)
count = ctypes.c_size_t(0)
table = lib.layout_lookup(b"sensor_reading", ctypes.byref(count))
The FieldDesc declaration on the Python side does have to match the C one by hand —
that is the one mirror you cannot escape. But it is a single four-field descriptor type that changes
approximately never, and it buys you the description of every other struct in the program.
Building the mirror at runtime
Because the table carries names, type tags and offsets, the caller can construct a matching structure definition while the program runs, instead of hand-writing one per struct:
TAGS = {0: ctypes.c_int, 1: ctypes.c_float, 2: ctypes.c_double, 3: ctypes.c_void_p}
fields = [(table[i].name.decode(), TAGS[table[i].type]) for i in range(count.value)]
Mirror = type("SensorReading", (ctypes.Structure,), {"_fields_": fields})id tag=0 offset=0 size=4
sample_count tag=0 offset=4 size=4
temperature tag=1 offset=8 size=4
timestamp tag=2 offset=16 size=8
label tag=3 offset=24 size=8
built mirror : SensorReading
python sizeof: 32
c sizeof : 32
mirror.timestamp offset: 16
The two sizeof values agreeing is the check that matters, and it is worth asserting
rather than eyeballing. If they differ, the description is wrong somewhere and every subsequent
read through the mirror lands on the wrong bytes — silently, with plausible values.
This is the whole payoff, and it is worth stating plainly: the struct is defined once, in C, and everybody else asks it what it looks like. No generated bindings to regenerate, no Python file to update when a field moves, no version skew between a library and the script reading it — because the description ships inside the library.
Part 4 · Practice
Pitfalls
The trailing backslash must be the last character on the line
One trailing space after it and the continuation silently ends, taking the rest of your list with it. Some compilers warn; do not count on it. This is the single most common way to lose an hour to this pattern, and it is invisible in a diff.
Put the punctuation in the action, not the list
Different actions need different separators. Keep the list free of commas and semicolons so it can serve an enum, an initialiser and a sequence of statements without special cases.
Order matters when position matters
If consumers index the table by position, the list must follow the struct's declaration order.
Nothing enforces this — the build still succeeds, and only the offset column looks odd.
If you can, index by name instead.
Macros do not recurse
A list cannot include itself, directly or through another macro. Nested lists work only in a strict hierarchy. If you find yourself wanting recursion, you have outgrown the pattern.
Keep the list next to the thing it describes
A field list in a different file from the struct can drift in the one way the pattern exists to prevent: somebody adds a field, never sees the list, and the table is quietly incomplete. Note that this failure is not caught by the compiler — the rename check only fires for fields the list mentions.
An X-macro list catches renamed and deleted fields. It does not catch added ones. If
completeness matters, add a static assertion that the field sizes plus known padding equal
sizeof the struct, or simply assert the struct size and let the build break when it
changes.
Debugging with the preprocessor, not by staring
Nearly every X-macro bug becomes obvious the moment you look at the pasted text. Do not reason about it; print it.
gcc -E -P demo.c # preprocess only, strip line markers
gcc -E -P demo.c | clang-format # ...and make it readable
gcc -E -P demo.c | sed -n '/sensor_layout/,/;/p'
-E stops after preprocessing and -P drops the #line markers
that otherwise dominate the output. Everything the compiler will see is in there, including the
expansion you were sure was correct.
When not to reach for it
When the list has exactly one consumer
If you only need the enum, write the enum. The pattern costs readability up front and repays it per consumer; it breaks even at two and clearly pays at three.
When the list is large, or comes from somewhere else
Past a few dozen entries, or when the real source of truth is a schema file, a protobuf definition
or a database, generate a .c file from a script. Generated code can be read, diffed and
stepped through in a debugger; a wall of backslash continuations cannot.
When you want type dispatch, not list replay
C11's _Generic selects an expression based on a type. It solves a different problem and
reads far better for that one.
When the language has real reflection
This entire pattern is a workaround for C discarding names. In a language that keeps them, use what the language gives you.
Cheat sheet
Preprocessor operators and predefined macros
#x | Stringify the argument as written |
a##b | Paste two tokens into one; suppresses expansion of both operands |
__COUNTER__ | 0, 1, 2, … per read — needs a two-step macro to paste with |
__LINE__, __FILE__ | Portable alternatives; one use per line |
__VA_ARGS__ | Variadic macro arguments, for actions of varying arity |
Layout introspection
offsetof(S, M) | Byte offset of a member, from <stddef.h> |
sizeof(((S *)0)->M) | Size of a member with no instance; operand is not evaluated |
sizeof(a) / sizeof(a[0]) | Element count of a real array — not of a pointer |
_Static_assert(c, "msg") | Compile-time check; useful for pinning struct size |
Running code before main
__attribute__((constructor)) | GCC and Clang; also runs at dlopen time |
| Global object constructor | C++; the same effect, portably |
.CRT$XCU section | MSVC equivalent |
Explicit init() | Portable fallback, at the cost of a central call site |
Commands
gcc -E -P f.c | Show the pasted text the compiler will read |
gcc -Wall -Wextra | Catches trailing-comma and unused-macro-argument mistakes |
pahole ./a.out | Print struct layout and padding, independently of your table |
Wrapping up
What actually matters here
The X-macro has a reputation for being a clever trick, which undersells it. It is a way of making a class of bug unrepresentable, and the syntax is the price.
The short list
-
The list with a hole is the whole idea.
LIST(X)plus an action macro; everything else in this document is a choice of action. -
Let the compiler compute the numbers.
offsetofandsizeofmean the table cannot disagree with the struct, on any target, ever. - A stale hand-written mirror is silent; a stale list is a compile error. That trade — a little syntax now for a loud failure later — is the reason to use it.
The registry and the FFI reader in part 3 are a step further than most code needs, but they are the natural end of the road: once the description exists as data, the interesting question stops being "how do I keep these in sync" and becomes "who else would like to read this". Usually the answer is more people than you expected.
Further reading
- The C Preprocessor — GCC's manual; the chapters on stringizing, concatenation and argument prescan are the relevant ones
- offsetof — including exactly when it is undefined behaviour
- ctypes — the FFI used in part 3, standard library, no build step
- The Linux kernel —
include/linux/kvm_host.hand the tracepoint headers are X-macros at industrial scale, for better and worse
Appendix · The complete programs
Building and running
Everything above came from three short files, reproduced in full below so this page stands on its own — copy the three snippets into a directory and the commands work as written.
gcc -std=c11 -Wall -Wextra -o demo demo.c && ./demo
gcc -std=c11 -Wall -Wextra -shared -fPIC -o libreflect.so reflect.c
python3 read_layout.py
gcc -E -P demo.c | head -40 # see what the preprocessor produced
Verified on gcc 11.4.0 (Ubuntu 22.04, x86-64) and CPython 3.10.12. Every output block on this page
is copied from a real run, including the compiler error in the rename
section, which was produced by renaming id to reading_id in a copy of
demo.c.
demo.c 74 lines
The warm-up and the field table, in one standalone program. Nothing outside the C standard library.
#include <stddef.h>
#include <stdio.h>
/* ---------- Warm-up: one list, two outputs ---------- */
#define COLOR_LIST(X) \
X(RED) \
X(GREEN) \
X(BLUE)
#define AS_ENUM(name) COLOR_##name,
#define AS_STRING(name) #name,
enum color { COLOR_LIST(AS_ENUM) COLOR_COUNT };
static const char *color_names[] = {COLOR_LIST(AS_STRING)};
/* ---------- Main: a runtime field table ---------- */
struct sensor_reading {
int id;
int sample_count;
float temperature;
double timestamp;
char *label;
};
typedef enum {
FIELD_INT = 0,
FIELD_FLOAT = 1,
FIELD_DOUBLE = 2,
FIELD_PTR = 3
} field_type;
typedef struct {
const char *name;
field_type type;
unsigned offset;
unsigned size;
} field_desc;
#define SENSOR_FIELDS(X) \
X(struct sensor_reading, id, FIELD_INT) \
X(struct sensor_reading, sample_count, FIELD_INT) \
X(struct sensor_reading, temperature, FIELD_FLOAT)\
X(struct sensor_reading, timestamp, FIELD_DOUBLE) \
X(struct sensor_reading, label, FIELD_PTR)
#define AS_FIELD_DESC(S, M, T) \
{#M, (T), (unsigned)offsetof(S, M), (unsigned)sizeof(((S *)0)->M)},
static const field_desc sensor_layout[] = {SENSOR_FIELDS(AS_FIELD_DESC)};
#define AS_INDEX(S, M, T) FIELD_IDX_##M,
enum { SENSOR_FIELDS(AS_INDEX) SENSOR_FIELD_COUNT };
int main(void) {
printf("colors: %d\n", (int)COLOR_COUNT);
for (int i = 0; i < COLOR_COUNT; i++)
printf(" %d = %s\n", i, color_names[i]);
printf("\nsizeof(struct sensor_reading) = %zu\n",
sizeof(struct sensor_reading));
printf("field count = %d\n", (int)SENSOR_FIELD_COUNT);
printf("%-14s %-5s %-7s %s\n", "name", "type", "offset", "size");
for (size_t i = 0; i < sizeof(sensor_layout) / sizeof(sensor_layout[0]); i++)
printf("%-14s %-5d %-7u %u\n", sensor_layout[i].name,
(int)sensor_layout[i].type, sensor_layout[i].offset,
sensor_layout[i].size);
printf("\nFIELD_IDX_timestamp = %d\n", (int)FIELD_IDX_timestamp);
return 0;
}reflect.c 83 lines
The registry plus a type that puts itself in it. Built as a shared library; it has no
main, and the only thing that runs is the constructor.
#include <stddef.h>
#include <string.h>
typedef struct {
const char *name;
int type;
unsigned offset;
unsigned size;
} field_desc;
typedef struct {
const char *type_name;
const field_desc *fields;
size_t count;
size_t struct_size;
} type_layout;
/* A tiny fixed-capacity registry. */
static type_layout g_registry[16];
static size_t g_registry_count;
void layout_register(const char *type_name, const field_desc *fields, size_t count,
size_t struct_size) {
if (g_registry_count >= 16) return;
g_registry[g_registry_count].type_name = type_name;
g_registry[g_registry_count].fields = fields;
g_registry[g_registry_count].count = count;
g_registry[g_registry_count].struct_size = struct_size;
g_registry_count++;
}
const field_desc *layout_lookup(const char *type_name, size_t *out_count) {
for (size_t i = 0; i < g_registry_count; i++) {
if (strcmp(type_name, g_registry[i].type_name) == 0) {
if (out_count) *out_count = g_registry[i].count;
return g_registry[i].fields;
}
}
if (out_count) *out_count = 0;
return NULL;
}
size_t layout_sizeof(const char *type_name) {
for (size_t i = 0; i < g_registry_count; i++)
if (strcmp(type_name, g_registry[i].type_name) == 0)
return g_registry[i].struct_size;
return 0;
}
/* ---- a type that registers itself ---- */
struct sensor_reading {
int id;
int sample_count;
float temperature;
double timestamp;
char *label;
};
#define FIELD_INT 0
#define FIELD_FLOAT 1
#define FIELD_DOUBLE 2
#define FIELD_PTR 3
#define SENSOR_FIELDS(X) \
X(struct sensor_reading, id, FIELD_INT) \
X(struct sensor_reading, sample_count, FIELD_INT) \
X(struct sensor_reading, temperature, FIELD_FLOAT) \
X(struct sensor_reading, timestamp, FIELD_DOUBLE) \
X(struct sensor_reading, label, FIELD_PTR)
#define AS_FIELD_DESC(S, M, T) \
{#M, (T), (unsigned)offsetof(S, M), (unsigned)sizeof(((S *)0)->M)},
#define REGISTER_LAYOUT(NAME, STRUCT, FIELDS) \
static const field_desc NAME##_fields[] = {FIELDS(AS_FIELD_DESC)}; \
__attribute__((constructor)) static void NAME##_register_layout(void) { \
layout_register(#NAME, NAME##_fields, \
sizeof(NAME##_fields) / sizeof(NAME##_fields[0]), \
sizeof(STRUCT)); \
}
REGISTER_LAYOUT(sensor_reading, struct sensor_reading, SENSOR_FIELDS)
field_desc is declared identically in both C files rather than shared through a
header, so that each one can be read on its own. In a real program it would live in a header, and
layout_register would be declared there too.
read_layout.py 37 lines
The other side of the wall: no C compiler involved, no generated bindings, just the library and the standard library.
import ctypes
class FieldDesc(ctypes.Structure):
_fields_ = [
("name", ctypes.c_char_p),
("type", ctypes.c_int),
("offset", ctypes.c_uint),
("size", ctypes.c_uint),
]
lib = ctypes.CDLL("./libreflect.so")
lib.layout_lookup.argtypes = [ctypes.c_char_p, ctypes.POINTER(ctypes.c_size_t)]
lib.layout_lookup.restype = ctypes.POINTER(FieldDesc)
lib.layout_sizeof.argtypes = [ctypes.c_char_p]
lib.layout_sizeof.restype = ctypes.c_size_t
TAGS = {0: ctypes.c_int, 1: ctypes.c_float, 2: ctypes.c_double, 3: ctypes.c_void_p}
count = ctypes.c_size_t(0)
table = lib.layout_lookup(b"sensor_reading", ctypes.byref(count))
fields = []
for i in range(count.value):
f = table[i]
print(f"{f.name.decode():14} tag={f.type} offset={f.offset:<3} size={f.size}")
fields.append((f.name.decode(), TAGS[f.type]))
Mirror = type("SensorReading", (ctypes.Structure,), {"_fields_": fields})
print()
print("built mirror :", Mirror.__name__)
print("python sizeof:", ctypes.sizeof(Mirror))
print("c sizeof :", lib.layout_sizeof(b"sensor_reading"))
print("mirror.timestamp offset:", Mirror.timestamp.offset)
Setting argtypes and restype is not optional decoration. Without them
ctypes assumes every argument and the return value are int, which on
x86-64 truncates pointers to 32 bits and produces a segfault that looks like it came from the C
side.
No comments:
Post a Comment