JSON

The json module reads JSON text into a JsonValue, lets you look values up and change them, and writes them back as compact or indented text. It also converts structs to JSON and back.

# JSON: parse text, look values up, change them, and write them back.
import json;

main(): int {
    var text = "{\"name\": \"pepper\", \"server\": {\"port\": 8080}, \"tags\": [\"fast\"]}";
    var parsed = json.parse(text, path = "config.json");
    if (parsed.is_err()) {
        print(parsed.error().to_string());   # config.json:1:17: error: ...
        return 1;
    }
    var root = parsed.value();

    # Lookups chain, and a missing key gives Null instead of panicking.
    assert(root["name"].as_string().value() == "pepper", "a string");
    assert(root["server"]["port"].as_int().value_or(80) == 8080, "a number");
    assert(root["server"]["host"].as_string().value_or("localhost") == "localhost", "a default");

    # Match on the case to handle every kind.
    var described = match (root["tags"]) {
        Array(items) => $"{items.count} tag",
        Null => "no tags",
        else => "not a list",
    };
    assert(described == "1 tag", "an array");

    # Change the value in place, then write it back. Keys keep their order.
    root["tags"].append(json.JsonValue.String("safe"));
    root["debug"] = json.JsonValue.Bool(false);
    assert(root.to_string() == "{\"name\":\"pepper\",\"server\":{\"port\":8080},\"tags\":[\"fast\",\"safe\"],\"debug\":false}", "compact");

    # Build a value from scratch.
    var reply = json.JsonValue.object();
    reply["id"] = json.JsonValue.Integer(42);
    reply["ok"] = json.JsonValue.Bool(true);
    assert(reply.to_pretty_string() == "{\n  \"id\": 42,\n  \"ok\": true\n}", "indented");

    # An error names the line and column.
    var bad = json.parse("{\n  \"a\": 1\n  \"b\": 2\n}", path = "bad.json");
    assert(bad.error().to_string() == "bad.json:3:3: error: expected ',' or '}'", "where");

    # JSONC: comments and trailing commas, on request.
    var settings = json.parse("{\n  // the theme\n  \"theme\": \"dark\",\n}", comments = true);
    assert(settings.value()["theme"].as_string().value() == "dark", "JSONC");
    return 0;
}

Values

json.JsonValue is an enum whose cases carry values, so match handles each kind:

• Numbers keep their form. A number written without a fraction or an exponent is an Integer when it is within ±(2^53 − 1), the range every JSON implementation agrees on (RFC 7493). Any other number, including a larger integer, is a Number.
• Make values with the cases (JsonValue.Integer(42), JsonValue.String("a"), JsonValue.Null), and containers with JsonValue.object() and JsonValue.array(items).
• Containers are shared. An Array or Object holds its array or map by reference, so changing it through value[key] = ..., append, or the map from as_object() changes every holder of the value.

Looking values up

A lookup that finds nothing gives Null rather than panicking, so lookups chain and a default goes at the end: root["server"]["port"].as_int().value_or(8080).

Changing values

• value["key"] = member sets an object’s member. A new key goes at the end; an existing key keeps its place. It panics on a value that is not an object.
• value[index] = element replaces an array’s element, and value.append(element) adds one at the end. Both panic on a value that is not an array.
• value.remove("key") removes an object’s member and says whether it was there.

Parsing

json.parse(text) returns a Result<JsonValue, JsonError>. It is strict RFC 8259:

• The text holds one value, with only spaces, tabs and line breaks around it. A byte-order mark at the start is skipped.
• The text must be valid UTF-8; an invalid byte is an error at that byte.
• A string cannot hold a raw control character (write \n), and a \u escape of an unpaired surrogate ("\uD800") is an error.
• No comments, trailing commas, single quotes, NaN, leading zeros (01) or + signs.
• When an object repeats a key, the last value wins, in the place of the first.

parse takes optional arguments, usually passed by name:

The parser keeps its own stack rather than recursing, so deep nesting in the text cannot overflow the call stack. max_depth still bounds it, because releasing a value releases its nested values recursively. Raise it with care: Windows gives a program’s main thread a 1 MB stack, which a few thousand levels of nested objects can use up when the value is released.

Errors

A JsonError has the message, the line and column (from 1, counting bytes) and the byte offset. to_string() renders it like a compiler error:

config.json:3:3: error: expected ',' or '}'

Without a path, it starts at the line: 3:3: error: expected ',' or '}'.

Writing

• value.to_string() writes compact text, with no spaces or line breaks: {"id":42,"tags":["a","b"]}. String interpolation uses it too.
• value.to_pretty_string() puts each element and member on its own line, indented two spaces per level (to_pretty_string(indent = 4) for four), with a space after each :. An empty array or object stays [] or {}.
• Strings are written as UTF-8, escaping ", \ and the control characters. to_string(ascii_only = true) and to_pretty_string(ascii_only = true) also escape everything outside ASCII as \u, with a surrogate pair past U+FFFF.
• A Number is written as Pepper writes a double (0.5, 1000.0, 1e+16), so it reads back as a Number. NaN and the infinities, which JSON cannot spell, are written as null.
• Objects keep their keys in insertion order, so parsing a document and writing it back keeps its order.

Documents with comments

parse skips a JSONC file’s comments. To change a file people edit, such as a settings file, and write it back with its comments, read it with json.parse_document(text), which returns a Result<JsonDocument, JsonError>:

# JSON: change a JSONC settings file and write it back with its comments.
import json;

main(): int {
    var text = "{\n  // Spaces per level.\n  \"tab_size\": 4,\n  \"font\": \"Mono\", // the editor's\n  \"wrap\": true\n}";
    var parsed = json.parse_document(text, path = "settings.json");
    if (parsed.is_err()) {
        print(parsed.error().to_string());
        return 1;
    }
    var document = parsed.value();
    assert(document.to_string() == text, "unchanged text comes back as it was");

    # Change the value; each comment stays with its value.
    document.root["tab_size"] = json.JsonValue.Integer(2);
    document.set_comments("$.wrap", ["// Wrap long lines."]);
    assert(document.comments("$.tab_size")[0] == "// Spaces per level.", "by path");
    print(document.to_string());
    # {
    #   // Spaces per level.
    #   "tab_size": 2,
    #   "font": "Mono", // the editor's
    #   // Wrap long lines.
    #   "wrap": true
    # }
    return 0;
}

• document.root is the JsonValue; change it as any other.
• Each comment belongs to a value, by its path from the root $ ($.editor.tab_size, $.files[0]): the comments on the lines before it, and those on the line it ends, after its comma. Comments before a closing bracket stay at the end of their array or object, and those around the whole value stay before or after it.
• document.to_string() writes the text indented, as to_pretty_string does (to_string(indent = 4) for four spaces), with every comment back in its place. A file in that layout reads and writes back unchanged.
• A comment whose value is removed is dropped. One whose path later names another value, such as an element shifted into its index, goes with that value.
• document.comments(path) returns the comments before a value, and document.set_comments(path, ["// ..."]) replaces them.
• parse_document takes path and max_depth as parse does, and always accepts JSONC.

Structs

json.encode(value) writes a struct as JSON text, and json.decode<T>(text) reads one back. There is no code to write per type: both walk the struct’s properties with reflection while compiling, so the conversion costs what hand-written code would.

# JSON: convert structs to JSON text and back, through reflection.
import json;

enum Mode {
    Fast,
    Safe,
}

struct Tls {
    var cert: String;
}

struct ServerConfig {
    var host: String;
    [Required]
    var port: int;
    # The key is "max-connections", not the property's name.
    [Name("max-connections")]
    var max_connections: int;
    # Never written or read.
    [Ignore]
    var started_at: long;
    # `null` or missing when none.
    var tls: Tls?;
    var mode: Mode;
    var tags: Array<String>;
}

main(): int {
    var config = ServerConfig();
    config.host = "localhost";
    config.port = 8080;
    config.max_connections = 64;
    config.mode = Mode.Safe;
    config.tags = ["web"];
    var text = json.encode(config);
    assert(
        text == "{\"host\":\"localhost\",\"port\":8080,\"max-connections\":64,\"tls\":null,\"mode\":\"Safe\",\"tags\":[\"web\"]}",
        "properties in declaration order"
    );

    var decoded = json.decode<ServerConfig>("{\"port\": 443, \"tls\": {\"cert\": \"site.pem\"}}");
    if (decoded.is_err()) {
        print(decoded.error().to_string());
        return 1;
    }
    var back = decoded.value();
    assert(back.port == 443, "a number");
    assert(back.tls.value().cert == "site.pem", "a nested struct");
    assert(back.mode == Mode.Fast, "a missing key keeps its default");

    # An error names where in the document it is, by path, line and column.
    var bad = json.decode<ServerConfig>("{\"port\": 443, \"mode\": \"Slow\"}", path = "server.json");
    assert(bad.error().to_string() == "server.json:1:23: error: $.mode: 'Slow' is not a case of 'Mode'", "a path");
    var missing = json.decode<ServerConfig>("{}");
    assert(missing.error().to_string() == "1:1: error: $: missing required key 'port'", "[Required]");
    return 0;
}

A struct’s public properties become an object’s members, in declaration order. Each property’s type converts as follows, and the same rules apply at the top level (json.decode<Array<int>>("[1, 2]")):

A struct reference that holds nothing (a String, array, struct or reference enum field never assigned) is written as null, and null reads back as one, so a value round-trips. Reading a case’s value whose key is missing is an error: $.shapes[0].Circle: missing value 'radius'. null for a number, bool or enum is an error.

Attributes

The serialize module’s attributes, which json uses (they need no import of their own), change how a property is converted:

On an enum and its cases, they choose how a case is written:

# JSON: choose how an enum's cases are written with attributes.
import json;

# A tag beside the values: {"kind": "circle", "radius": 2}.
[Tag("kind")]
enum Shape {
    [Name("circle")]
    Circle(radius: int),
    Point,
}

# The values under their own key: {"type": "Click", "data": {"x": 1, "y": 2}}.
[Tag("type", content = "data")]
enum Event {
    Click(x: int, y: int),
    Close,
}

# No tag; reading takes the first case that fits.
[Untagged]
enum Size {
    Fixed(px: int),
    Auto,
}

main(): int {
    assert(json.encode(Shape.Circle(2)) == "{\"kind\":\"circle\",\"radius\":2}", "internally tagged");
    assert(json.encode(Shape.Point) == "{\"kind\":\"Point\"}", "a case with no values");
    assert(json.encode(Event.Click(1, 2)) == "{\"type\":\"Click\",\"data\":{\"x\":1,\"y\":2}}", "adjacently tagged");
    assert(json.encode(Size.Fixed(8)) == "{\"px\":8}", "untagged");
    assert(json.encode(Size.Auto) == "\"Auto\"", "untagged, no values");

    var shape = json.decode<Shape>("{\"kind\": \"circle\", \"radius\": 5}");
    if (shape.value() is Circle(radius)) {
        assert(radius == 5, "read back");
    } else {
        return 1;
    }
    var size = json.decode<Size>("\"Auto\"");
    assert(size.value() is Auto, "the first case that fits");
    var bad = json.decode<Shape>("{\"radius\": 5}");
    assert(bad.error().to_string() == "1:1: error: $: missing tag 'kind'", "a missing tag");
    return 0;
}

Reading an [Untagged] enum tries each case in order and takes the first that reads without an error (with UnknownKeys.Reject, an extra key is an error too), so a case whose values include another’s goes first. When none reads, the error says so: $: found an object, which matches no case of 'Size'. A [Tag] enum’s errors name its tag: $: missing tag 'kind', $.kind: 'Hex' is not a case of 'Shape'. A tag with the name of one of a case’s values is a compile error, since both would be written under one key.

Reading

decode makes a struct with its constructor that takes no arguments, then reads in each property it finds a key for. A missing key leaves its property at its default, unless the property is [Required]. It takes optional arguments, usually passed by name:

An error has a line and column as in Errors. One in converting a value also has its value_path, where in the document the value is as a path from the root $, and its message starts with that path. The line and column are the value’s, or the enclosing object’s for a missing key:

server.json:3:13: error: $.server.port: expected a number, found a string
server.json:7:21: error: $.tags[2]: expected a string, found null
server.json:4:24: error: $["max-connections"]: 70000 does not fit in 'u16'
server.json:1:1: error: $: missing required key 'port'

json.to_json(value) and json.from_json<T>(value) do the same conversions to and from a JsonValue rather than text (with no text, a from_json error has no line: error: $.port: ...), and json.encode(value, indent = 2) writes indented text.

Types it cannot convert

A property whose type has no JSON form, such as a pointer, a function, or a Result (a result is not a value to send over the wire), is a compile error naming the property. Notes follow the conversion back to the call that asked for it:

runtime/json/convert.pep:358:25: error: 'encode' cannot convert property 'outcome' of type 'Result<int>' to JSON
...
main.pep:10:16: note: 'encode<Job>' is used by 'main' here
   10 |     var text = json.encode(job);
      |                ^~~~~~~~~~~

Current limits

• parse skips comments; only a JsonDocument keeps them, and decode never does. A document is always written indented.
• A struct’s own conversion cannot be customized yet: a type converts as its properties do.
• Map keys must be Strings.

The reference lists every function.