Reflection and attributes

A program can look at its own types while it compiles: list a struct’s fields, read the attributes written on them, and generate code for each one. This is how one function can turn any struct into JSON, declare a command-line option per field, or compare two values field by field.

The work happens in the compiler, in a static block, so it costs nothing at run time and needs no type information in the program. This is the model of Zig’s comptime and C++’s static reflection, not of Java or C# runtime reflection.

struct Point {
    var x: int;
    var y: int;
    var label: String;
}

# One function describes any struct: the `static` block runs in the
# compiler, once per type `describe` is called with.
describe<T>(value: T): String {
    var out = StringBuilder();
    static (out, value) {
        out.append(reflect.name<T>());
        out.append("(");
        for (var field in reflect.properties<T>()) {
            # `field.name` is a compile-time string; `field.get(value)` reads
            # the field. Both statements use `out`, a captured runtime
            # value, so they are emitted once per field.
            out.append($"{field.name} = {field.get(value)}");
            if (!field.is_last) {
                out.append(", ");
            }
        }
    }
    out.append(")");
    return out.to_string();
}

main(): int {
    var p = Point();
    p.x = 3;
    p.y = 4;
    p.label = "corner";
    assert(describe(p) == "Point(x = 3, y = 4, label = corner)", "one line per field, in declaration order");
    return 0;
}

static blocks

A static block is written inside a function. It names, in parentheses, the runtime values it may emit code against: its capture list.

static (out, value) {
    ...
}

static { ... } with no list is pure compile-time work, such as a check with reflect.assert.

Each statement in the block either runs in the compiler or is emitted into the function:

• A statement that uses a captured name, or a runtime local the block declares, is emitted. Its compile-time parts are filled in first: out.append(field.name) emits out.append("x").
• Everything else runs in the compiler: var declarations whose value uses no captured name, if on a compile-time condition (only the branch taken is kept), for over a compile-time list, which is unrolled, one copy of its body per element, and while on a compile-time condition, which emits its body once per iteration.
• An if whose condition uses a captured name is emitted as a runtime if; its branches can still hold compile-time steps.

A var with a captured name in its value is a runtime local, and using it counts as using a capture. Using a runtime value that is not captured is an error at that name:

main.pep:9:24: error: 'total' is a runtime value; add it to the static block's capture list

Each pass of an unrolled for or while, and the branch an if keeps, is its own block, like the body of a real loop.

A closure in an emitted statement is an ordinary closure: it captures runtime locals as any closure does. Its body cannot use a compile-time value such as field.name, since the closure outlives the block’s expansion; copy the value into a runtime local first, and the closure captures that local:

static (options, calls) {
    for (var field in reflect.fields<Options>()) {
        var text = $"{field.get(options)}";
        calls.append(() => text);
    }
}

Compile-time loops

break and continue in a compile-time for or while run in the compiler: continue drops the rest of the pass, and break ends the unrolling. They cannot sit in a runtime if, whose condition the compiler does not know:

main.pep:10:17: error: 'break' in a runtime 'if' cannot leave a compile-time loop; the condition is not known while compiling

A while that runs 100,000 times is taken to be endless and stops the compile at its while.

struct Row {
    var id: int;
    var _cache: int;
    var name: String;
    var score: int;
    var notes: String;
}

# The sum of the public `int` fields of `Row` before 'notes': `continue`
# and `break` steer the unrolled `for`.
sum_leading_ints(row: Row): int {
    var total = 0;
    static (total, row) {
        for (var field in reflect.fields<Row>()) {
            if (field.name.starts_with("_")) {
                continue;
            }
            if (field.name == "notes") {
                break;
            }
            if (reflect.is_same<field.Type, int>()) {
                total += field.get(row);
            }
        }
    }
    return total;
}

# A compile-time `while` emits its body once per iteration: here, the
# first four powers of three, as four additions.
powers_of_three(): int {
    var sum = 0;
    static (sum) {
        var power = 1;
        while (power < 30) {
            sum += power;
            power = power * 3;
        }
    }
    return sum;
}

main(): int {
    var row = Row();
    row.id = 2;
    row.score = 40;
    assert(sum_leading_ints(row) == 42, "id and score; _cache is skipped");
    assert(powers_of_three() == 1 + 3 + 9 + 27, "one addition per iteration");
    return 0;
}

In generic code

In a generic function, a static block runs once for each specialization, when its type arguments are known: describe<Point> and describe<Line> each get their own expansion. Code the block emits can call other generic functions, such as describe(field.get(value)) for a field that is itself a struct, and those are specialized in turn. A mistake in a block is reported when some specialization uses it, with the pass it happened in and, when a generic asked for that specialization, a note at each use that led to it ('dump<Point>' is used by 'main' here):

main.pep:14:13: error: 'print' cannot be called at compile time
main.pep:12:27: note: in pass 1 of 2 (member 'x')
note: a 'static' block can call the 'reflect' functions and the methods of what they return; calling other functions at compile time is not supported yet
main.pep:10:1: note: in the static block of 'dump<Point>'

At module scope

A static block outside any function runs once, while compiling. It has nothing to capture and emits no code; instead, each const declared at the top of the block becomes a module-scope constant holding what the block computed. Its other vars, and anything declared in its loops and branches, stay in the compiler. export static exports its constants, and attributes written above the block ([Target(...)]) apply to them too.

export struct Pixel {
    var red: u8;
    var green: u8;
    var blue: u8;
    var alpha: u8;
}

export enum Channel {
    Red,
    Green,
    Blue,
    Alpha
}

# A module-scope `static` block runs once, while compiling. Each `const` at
# its top becomes a module-scope constant holding what the block computed;
# `export static` exports them.
export static {
    var bytes = 0;
    for (var field in reflect.properties<Pixel>()) {
        reflect.assert(reflect.is_same<field.Type, u8>(), $"'{field.name}' must be one byte");
        bytes += 1;
    }
    const PIXEL_BYTES = bytes;
    const LAST_CHANNEL = reflect.cases<Channel>()[reflect.cases<Channel>().count - 1].get();
}

import "sizes";

main(): int {
    assert(PIXEL_BYTES == 4, "one byte per field of Pixel");
    assert(LAST_CHANNEL == Channel.Alpha, "the last case of Channel");
    return 0;
}

A constant takes a number, bool, character or enum case; a module-scope variable cannot be a String, since it must be thread-safe. At module scope, a case’s get() is the case itself, and a statement that would emit code is an error:

main.pep:7:9: error: a module-scope 'static' block cannot emit code; declare a 'const' with what it computes

What the compiler can evaluate

Compile-time values are integers, floating-point numbers, bool, strings, enum cases, and what the reflect functions return: members, cases, functions, lists and attributes. Expressions can use literals, the arithmetic, comparison and logical operators, + on strings, interpolated strings ($"{field.name}: "), ?:, assignment to a compile-time var (n += 1), and indexing a list. A string has length, is_empty, starts_with, ends_with and contains; a list has count and is_empty.

Emitted code can use a compile-time number, bool, string or enum case; the compiler writes it as a literal.

Seeing what a block does

A static block runs inside the compiler, so a print in it would be emitted into the program instead of showing anything while compiling. These show what the block computes:

• reflect.print(value) writes value as a note each time the compiler runs it, on every compile, including one that succeeds. In an unrolled loop each note says which pass printed it:

  main.pep:9:17: note: static print: "describing Point"
      9 |         reflect.print($"describing {reflect.name<T>()}");
        |                 ^~~~~
  main.pep:11:21: note: static print: "x"
     11 |             reflect.print(field.name);
        |                     ^~~~~
  main.pep:10:27: note: in pass 1 of 2 (member 'x')
     10 |         for (var field in reflect.properties<T>()) {
        |                           ^~~~~~~~~~~~~~~~~~~~~~~
  

• reflect.assert(condition, message) and reflect.error(message) stop the compile at that point with message, which can be an interpolated string holding the values that matter (reflect.assert(n < 8, $"{reflect.name<T>()} has {n} fields")). Like every error in a block, it carries the pass and, in generic code, the specialization it happened in (see In generic code).

• The emitted code runs like code written by hand. Each statement a block emits keeps the source line of the statement it came from, so a runtime print it emits shows values as the program runs, and a debugger breakpoint on that line (with -g) stops there, in every pass. Stepping goes through an unrolled loop pass by pass, stopping on the for line and then on the body, and the debugger shows the compile-time variables, such as the loop’s field (see Stepping through a static block).

• --emit-static writes the program’s source with each block replaced by what it emits (see the next section).

• --emit-llvm writes the program’s LLVM IR into the build directory (.pep/build/<name>.ll), where the function a block expanded in shows exactly what it emitted, one copy per specialization (describe<Point>). --emit-ast prints the program as parsed, before any block runs, so it does not show the expansion.

Seeing what a static block emits

pepper build main.pep --emit-static writes a copy of the program with each static block replaced by the statements it emits, the way a C compiler’s -E shows the preprocessed source. Nothing is compiled; the copy goes into main.static/ (or the folder -o names) and builds into the same program. For the describe example at the top of this page, the copy keeps describe<T> as written and adds what it is for Point:

# 'describe' specialized for T = Point, with its static blocks expanded
describe<Point>(value: Point): String {
    var out = StringBuilder();
    # static (out, value) expanded from reflection-basics.pep:11:5 for T = Point
    {
        out.append("Point");
        out.append("(");
        {
            out.append($"x = {value.x}");
            {
                out.append(", ");
            }
        }
        {
            out.append($"y = {value.y}");
            {
                out.append(", ");
            }
        }
        {
            out.append($"label = {value.label}");
        }
    }
    # end static
    out.append(")");
    return out.to_string();
}

• Each pass of an unrolled for or while is a block of its own, one after another; a compile-time if leaves only the branch it took; the compile-time vars and steps are gone.
• Compile-time values are written as literals: field.name becomes "x" (inside an interpolated string, its text), field.Type the type, field.get(value) value.x, case.matches(v) v is Shape.Circle, and function.call() a call by name. A case’s value, item.get(v), becomes a match that reads it.
• Everything else in the file is copied byte for byte, comments included. --emit-static=keep also keeps each block as # comments above its expansion.
• A block in a generic function expands differently for each type, so the copy adds a concrete override per specialization the program uses, as above. A block in a method of a generic struct, or in a generic method, stays as written, with each specialization’s expansion above it as comments.
• An error in the emitted code shows when the copy is built, at the line of the copy that has it.
• To debug the program in the expanded source, build it with -g --debug-static=expanded: the copy is written beside the program as main.static.pep, and the debugger steps through it (see Stepping through a static block). In a method of a generic struct or a generic method, where the copy shows the expansion only as comments, the debugger shows the block as written.

pep-build lists the details.

The reflect functions

reflect is built in: a static block uses it without an import. The reference has an entry for each function and for the members, cases and functions they return.

fields exists for code a struct’s author writes, such as a field-by-field ==; anywhere else it is an error, which points at properties:

main.pep:4:24: error: 'fields' of 'Account' is only available in the file that declares 'Account'; use 'properties'

A member (from properties or fields) has:

• name, index, is_last, is_const, is_property
• type_name: its type as a string; field.Type is the type itself, usable wherever a type is written: var copy: field.Type = ..., reflect.is_same<field.Type, String>(), parse<field.Type>(text). This works for a compile-time var holding a member too (var first = case.values[0]; gives first.Type).
• has_attribute<A>() and attribute<A>()
• get(value) and set(value, x), which emit value.name and value.name = x

A case (from cases) has:

• name, value (its integer value, or the tag of a case that can carry values), index, is_last, has_attribute<A>(), attribute<A>()
• values: the values the case carries, as members, in order; empty for a case that carries none. A value’s get(v) reads it from v, which must hold this case; set is an error, since a case’s values are fixed.
• matches(v), which emits a test of whether v holds this case
• get(values...), which emits the case made with those values: case.get() for a case that carries none, case.get(r) for Circle(radius: int)

A function (from functions_with) has name, has_attribute<A>(), attribute<A>(), and call(arguments...), which emits a direct call.

enum Level {
    Low,
    High = 10,
}

struct Settings {
    var name: String;
    var retries: int;
    var verbose: bool;
    var level: Level;
}

# A compile-time `if` keeps only the branch it takes, so each field is
# written by the code that fits its type.
render<T>(value: T): String {
    var out = StringBuilder();
    static (out, value) {
        for (var field in reflect.properties<T>()) {
            out.append($"{field.name}=");
            if (reflect.is_same<field.Type, String>()) {
                out.append($"\"{field.get(value)}\"");
            } else if (reflect.is_enum<field.Type>()) {
                out.append(level_name(field.get(value)));
            } else {
                out.append($"{field.get(value)}");
            }
            out.append(";");
        }
    }
    return out.to_string();
}

# `reflect.cases<E>()` lists an enum's cases; `case.get()` is the case
# itself in emitted code.
level_name(level: Level): String {
    var name = "";
    static (level, name) {
        for (var case in reflect.cases<Level>()) {
            if (level == case.get()) {
                name = case.name;
            }
        }
    }
    return name;
}

# `field.set(value, x)` assigns a field.
reset_counts<T>(value: T) {
    static (value) {
        for (var field in reflect.properties<T>()) {
            if (reflect.is_same<field.Type, int>()) {
                field.set(value, 0);
            }
        }
    }
}

main(): int {
    var settings = Settings();
    settings.name = "fast";
    settings.retries = 3;
    settings.level = Level.High;
    assert(
        render(settings) == "name=\"fast\";retries=3;verbose=false;level=High;",
        "each field is written by the code for its type"
    );
    reset_counts(settings);
    assert(settings.retries == 0, "set assigns the int fields");
    return 0;
}

struct Account {
    var owner: String;
    var _balance: int;

    construct(this.owner = "", this._balance = 0) {}

    # `reflect.fields<T>()` lists private fields too, so it is available
    # only in the file that declares `T`: code its author writes, like this
    # field-by-field `==`.
    ==(other: Account): bool {
        var same = true;
        static (same, other) {
            for (var field in reflect.fields<Account>()) {
                same = same && field.get(this) == field.get(other);
            }
        }
        return same;
    }
}

main(): int {
    assert(Account("ann", 5) == Account("ann", 5), "equal fields compare equal");
    assert(!(Account("ann", 5) == Account("ann", 6)), "a private field takes part");
    return 0;
}

enum Shape {
    Circle(radius: int),
    Rect(width: int, height: int),
    Point,
}

# `case.matches(value)` tests which case a value holds, and each of the
# case's `values` reads its value with `get`.
describe<E>(value: E): String {
    var out = "";
    static (out, value) {
        for (var case in reflect.cases<E>()) {
            if (case.matches(value)) {
                out = case.name;
                for (var item in case.values) {
                    out = out + $" {item.name}={item.get(value)}";
                }
            }
        }
    }
    return out;
}

# `case.get(values...)` makes a case; here, each value is `size`.
square<E>(name: String, size: int): E {
    var out: E;
    static (out, name, size) {
        for (var case in reflect.cases<E>()) {
            if (name == case.name) {
                if (case.values.count == 0) {
                    out = case.get();
                } else if (case.values.count == 1) {
                    out = case.get(size);
                } else if (case.values.count == 2) {
                    out = case.get(size, size);
                }
            }
        }
    }
    return out;
}

main(): int {
    assert(describe(Shape.Rect(3, 4)) == "Rect width=3 height=4", "a case's values");
    assert(describe(Shape.Point) == "Point", "a case without values");
    assert(describe(square<Shape>("Rect", 5)) == "Rect width=5 height=5", "a made case");
    assert(describe(square<Shape>("Circle", 2)) == "Circle radius=2", "one value");
    return 0;
}

Attributes

An attribute is a struct marked [Attribute]. Writing it above a declaration records a compile-time value of that struct, which has_attribute<A>() tests for and attribute<A>() returns.

# An attribute is a struct marked `[Attribute]`. Its fields hold what a
# use of it says; a field with an initializer may be left out.
[Attribute]
struct Name {
    const value: String;
}

[Attribute]
struct Ignore {}

struct User {
    [Name("user_id")]
    var id: int;
    var email: String;
    [Ignore]
    var session: int;
}

# The keys a struct's fields are written under: each field's `[Name]` if
# it has one, else the field's own name, skipping `[Ignore]`d fields.
keys<T>(): String[] {
    var keys: String[] = [];
    static (keys) {
        for (var field in reflect.properties<T>()) {
            if (!field.has_attribute<Ignore>()) {
                var key = field.name;
                if (field.has_attribute<Name>()) {
                    key = field.attribute<Name>().value;
                }
                keys.append(key);
            }
        }
    }
    return keys;
}

main(): int {
    var found = keys<User>();
    assert(found.count == 2, "the [Ignore]d field is skipped");
    assert(found[0] == "user_id", "[Name] renames the key");
    assert(found[1] == "email", "a field without [Name] keeps its name");
    return 0;
}

• Fields of an attribute struct are numbers, bool, char, String, or enums whose cases carry no values.
• Arguments are constants: literals, a negated number, an enum case (Level.High), or strings joined with +. They can be named: [Check(weight = 3)].
• Without a constructor, the arguments set the fields in order, and a field with an initializer may be left out. A constructor may take this.field parameters, or parameters its body assigns to fields (this.long_name = long_name;), which is all its body may do.
• An attribute from another module can be named through it, with or without arguments: [serialize.Ignore], [serialize.Name("key")].
• An attribute that names neither a built-in attribute nor an [Attribute] struct is an error, at its name: unknown attribute 'Optoin'.
• Built-in attributes (Target, Shared, Deprecated, …) are [Attribute] structs too, declared in the runtime (attributes), so a block reads them like its own: function.attribute<Deprecated>().message. Target holds its condition as written ("!OS(Windows)"). The compiler still checks and acts on each one itself.

struct Settings {
    var width: int;
    [Deprecated("use 'width' instead", "width")]
    var size: int;
}

# Lists the deprecated fields of `T` with what replaces each, read from the
# built-in `Deprecated` attribute like any `[Attribute]` struct.
deprecated_fields<T>(): String {
    var out = "";
    static (out) {
        for (var field in reflect.properties<T>()) {
            if (field.has_attribute<Deprecated>()) {
                out = out + $"{field.name} -> {field.attribute<Deprecated>().replacement};";
            }
        }
    }
    return out;
}

main(): int {
    assert(deprecated_fields<Settings>() == "size -> width;", "one deprecated field");
    return 0;
}

Attributes can go on structs, enums and their cases, fields, functions, methods, constructors, parameters and module-scope variables.

[Attribute]
struct Check {
    const weight: int = 1;
}

[Check]
check_addition(): bool {
    return 1 + 1 == 2;
}

[Check(weight = 3)]
check_strings(): bool {
    return "ab" + "c" == "abc";
}

# `reflect.functions_with<A>()` lists every function in the program that
# carries `A`; `check.call()` calls it directly.
run_checks(): int {
    var score = 0;
    static (score) {
        for (var check in reflect.functions_with<Check>()) {
            if (check.call()) {
                score += check.attribute<Check>().weight;
            }
        }
    }
    return score;
}

main(): int {
    assert(run_checks() == 4, "each [Check] function runs once, weighted");
    return 0;
}

Not supported yet

• Calling functions other than the reflect functions at compile time.
• Reflecting methods and constructors (reflect.methods<T>()).
• Limiting where an attribute may be used ([AttributeTargets(...)]).
• Tracing the blocks the compiler runs.
• Type information at run time.