chore: upgrade to Zig 0.16.0#1
Conversation
- addTest/addExecutable: .root_source_file → .root_module (create module first) - std.ArrayList: .init(alloc) → .empty + explicit allocator on methods - std.mem.trimLeft → trimStart, trimRight → trimEnd - std.fs.File.WriteError → std.Io.Writer.Error - writer.writeByteNTimes → writer.splatByteAll - Writer.Allocating: use .init(alloc) then .toArrayList() to get data - ErrorBundle.RenderOptions: removed ttyconf field - std.fs.cwd() → std.Io.Dir.cwd() - dir.readFileAlloc(io, path, alloc, .unlimited) replaces file.readToEndAlloc - Spec tests disabled (TODO: need full std.Io migration in test/spec.zig) 🤖 Generated with AI assistance
When the target Zig type is []const u8 (pointer), the yaml parser needs to handle boolean, int, and float values by converting them to their string representations. This maintains backward compatibility with code that expects all yaml values to be parseable as strings. 🤖 Generated with AI assistance
The tokenizer checked if the byte before a closing `"` was `\` to detect escaped quotes. This fails for `\\"` (escaped backslash followed by closing quote), incorrectly treating the closing quote as escaped. Fix: track escape state by handling `\` explicitly in the double_quoted state — when a backslash is encountered, skip the next character unconditionally. This correctly handles all escape sequences including `\\`, `\"`, `\n`, `\t`, etc. Added regression tests for escaped backslash before closing quote and various escape sequences. 🤖 Generated with AI assistance
Previously only \n, \t, and \" were supported. Now handles all escape sequences from YAML 1.2 spec section 5.7: - Single-char: \0 \a \b \t \n \v \f \r \e \" \/ \\ \<space> - Unicode names: \N (next line) \_ (nbsp) \L (line sep) \P (para sep) - Unicode hex: \xNN \uNNNN \UNNNNNNNN All produce correct UTF-8 output. 🤖 Generated with AI assistance
list_empty previously returned a static &.{} slice. Calling gpa.free()
on a non-heap pointer is undefined behavior with some allocators. Use
gpa.alloc(Value, 0) instead, which returns a properly tracked zero-
length allocation.
🤖 Generated with AI assistance
When a YAML value is null (~, null, empty), parseOptional now correctly returns null for optional types. Previously it only returned null when the map key was missing entirely, not when the value was explicitly null. Also added tests for null values (tilde, null keyword) in various contexts: maps, lists, and typed struct optionals. 🤖 Generated with AI assistance
Added detection for .inf, -.inf, and .nan (with case variants) as specified in the YAML 1.2 core schema. These map to the corresponding IEEE 754 special values. 🤖 Generated with AI assistance
- Empty values now stringify as "null" instead of empty string - Positive/negative infinity stringify as .inf/-.inf - NaN stringifies as .nan - Matches YAML 1.2 core schema representation 🤖 Generated with AI assistance
- Migrate test/spec.zig from std.fs to std.Io.Dir (Zig 0.16 API) - Update ArrayList usage to unmanaged pattern (std.ArrayList is now unmanaged) - Replace std.zig.fmtEscapes with manual string escaping - Update generated test code to use std.Io.Writer.Allocating - Re-enable spec tests in build.zig with -Denable-spec-tests=true flag Spec test results: 77 pass, 321 skip, 4 fail (402 total) The 4 failures are UnexpectedSuccess (parser too lenient on invalid YAML). 🤖 Generated with AI assistance
Coverage for all recent bug fixes and YAML features:
- Double-quoted escape sequences (\, ", \n, \t, \x41, \u0041, \U00000041)
- Escaped backslash at end of string (regression test)
- Flow mappings (simple, nested, empty, as struct values)
- Quoted string non-coercion ("true" stays string, not bool)
- Null/empty values (~, null, NULL, Null, empty in explicit doc)
- Special floats (.inf, -.inf, .nan, case variations, f32/f64)
- Stringify roundtrips (ints, floats, strings, structs, lists, optionals)
- Comments (inline, standalone, between entries)
- Empty and multi-document handling
- Boolean edge cases (true/false, yes/no/on/off, case insensitive)
- Integer formats (decimal, hex 0x, octal 0o)
- Deeply nested structures (4+ levels, mixed lists/maps)
- Mixed flow and block styles
- Ethereum mainnet config parsing (fork versions, large ints, hex strings)
- Full mainnet config snippet with all field types
- Error cases (type mismatch, missing fields, duplicate keys)
- Untyped value access (map contains, list indexing)
- Value.encode integration
🤖 Generated with AI assistance
StringArrayHashMap (managed) was removed in Zig 0.16. Use StringArrayHashMapUnmanaged with explicit allocator instead.
There was a problem hiding this comment.
Pull request overview
Upgrades the project to Zig 0.16.0-dev by migrating to std.Io APIs (including explicit io context for Dir methods) and updating the build/test harness accordingly, while also refactoring YAML value handling and extending parsing/tokenizing behavior.
Changes:
- Migrate filesystem and IO usage from
std.fs/std.iopatterns tostd.Io+ explicit IO context where required. - Refactor
Yaml.Valueto a scalar-based representation and update typed parsing, errors, and tests. - Add/expand parsing support (flow maps, improved double-quoted escape handling) and introduce a new comprehensive test suite.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
build.zig |
Updates test build graph for Zig 0.16 module APIs; adds comprehensive test target. |
src/Yaml.zig |
Refactors typed parsing and Value representation; updates stringify and error handling. |
src/Parser.zig |
Adds flow-map parsing and extends double-quoted escape parsing. |
src/Tokenizer.zig |
Updates double-quoted tokenization logic and modernizes test ArrayList usage. |
src/Parser/test.zig |
Adjusts error rendering and adds/changes parser behavior tests (flow maps, doc end handling). |
src/Yaml/test.zig |
Updates/extends unit tests to match scalar-based Value and new parsing semantics. |
test/test.zig |
Updates test file loading to std.Io.Dir read helpers. |
test/spec.zig |
Updates YAML test-suite generator for std.Io + new container APIs. |
test/comprehensive_test.zig |
Adds a large set of additional regression/compliance tests. |
.gitignore |
Normalizes ignored directories and adds zig-pkg/. |
Comments suppressed due to low confidence (1)
src/Yaml.zig:352
Value.stringifyreturns an empty string for empty lists (len == 0), which produces invalid/ambiguous YAML when a list is the document root or a mapping value (e.g.key:instead ofkey: []). Consider emitting[]for empty lists (and similarly handling empty maps as{}) so stringify always produces a valid YAML representation.
.list => |list| {
const len = list.len;
if (len == 0) return;
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| var i: usize = 0; | ||
| for (map.keys(), map.values()) |key, value| { | ||
| if (!args.should_inline_first_key or i != 0) { | ||
| try writer.writeByteNTimes(' ', args.indentation); | ||
| try writer.splatByteAll(' ', args.indentation); | ||
| } |
There was a problem hiding this comment.
Value.stringify currently emits nothing for empty maps (it returns early when map.count() == 0), which can produce invalid/ambiguous YAML (e.g. a root value or key: ). Consider emitting {} for empty maps so stringify always produces a valid representation.
| const scalar = try value.asScalar(); | ||
| return self.parsePointer(arena, T, .{ .scalar = try arena.dupe(u8, scalar) }); |
There was a problem hiding this comment.
In parseValue’s scalar pointer path, the scalar is duplicated into the arena (arena.dupe) and then parsePointer duplicates it again for []u8 slices. This double-copies string scalars unnecessarily. Consider passing the original value through to parsePointer and letting parsePointer perform the single arena duplication (or otherwise avoid the intermediate Value{ .scalar = arena.dupe(...) }).
| const scalar = try value.asScalar(); | |
| return self.parsePointer(arena, T, .{ .scalar = try arena.dupe(u8, scalar) }); | |
| try value.asScalar(); | |
| return self.parsePointer(arena, T, value); |
- Remove raw.len == 0 check in parseOptional to preserve empty strings - Add bounds check for trailing backslash in double-quoted tokenizer - Remove dead log build option - Remove placeholder comment in test
Four of the five Zig-0.16-upgrade PRs on our dependency forks have merged into ChainSafe upstream main: * ChainSafe/blst.zig#4 → 6ad139a * ChainSafe/hashtree-z#10 → 56e406f * ChainSafe/snappy.zig#5 → 7139424 * ChainSafe/zig-yaml#1 → b74f4e8 Switch from `lodekeeper-z/<repo>?ref=chore/zig-master` to the merged commit on `ChainSafe/<repo>` main for each. Content is identical so the existing `hash` fields stay valid. `zapi` (ChainSafe/zapi#9) is still open upstream — bump the `lodekeeper-z/zapi` commit pointer to the current PR head (fefb88ebf5c9) which includes the latest port/refactor work; TODO note to switch to ChainSafe/zapi once ChainSafe#9 merges.
## Motivation Get the codebase cleanly compatible with **Zig 0.16.0** release, following the new "explicit `io` everywhere, no hidden global state" design — without changing runtime behavior. ## Description - **`std.Io`** — File/Dir operations, `Mutex`/`Condition`/`Event` (cancelable vs uncancelable split), entropy via `io.random`/`io.randomSecure`, timestamps via `std.Io.Timestamp` - **ArrayList unmanaged** — allocator threaded through `deinit`/`append`/`resize`/`ensureCapacity`/`clone`; replace `items.len = N` hacks with `resize` / `expandToCapacity` / `appendNTimesAssumeCapacity` (preserves semantics, matches `initCapacity`'s Precise allocation) - **Struct builder** — `@Type(.@"struct" = ...)` → `@Struct(names, types, attrs)` builtin in `ssz/type/container.zig` - **Copy-elision workaround** — `std.Io` 0.16 result-location semantics pre-write the union tag of the assignment target, so reads from `self.state.init_group.*` during RHS evaluation would observe the new variant. Snapshot pre-upgrade fields to locals in `state_transition.zig` fork-upgrade chain and `era/Writer.zig` - **ThreadPool** — `io` is a parameter on `init`/`deinit`/`verifyMultipleAggregateSignatures`/`aggregateVerify` (not a stored field); normal paths use cancelable `lock`; shutdown + `submitAndWait.done.wait` stay uncancelable with comments explaining the invariant - **Shared `src/time.zig`** — `timestampNow` / `since (Duration)` / `durationSeconds`, replacing four duplicated helper pairs across `state_transition` and `bench` - **IO signature convention** — `(allocator?, io, file, ...)` applied consistently to `e2s.*` and `era.*` - **Reader API polish** — `takeArray` (zero-copy) + `readSliceAll` where appropriate; drop single-buffer `readVecAll` and `catch |err| return err` - **Other**: `RefCount` comptime-reflect on wrapped T's deinit arity (1-arg vs 2-arg) so 0.16 unmanaged `ArrayList` and project types with stored allocators both work; `sanitize_c = .trap` on blst artifact so Debug fuzz builds link without `libubsan`; untrack 426 stale `test/fuzz/zig-pkg/` entries from the git index - **CI**: `ZIG_VERSION` bumped `0.14.1 → 0.16.0`; `zbuild` sync-check + `build-examples` jobs temporarily commented out pending upstream Zig 0.16 support ## Dependency PRs Each Zig 0.16 upgrade landed as a dedicated PR on the upstream repo. `build.zig.zon` points at `lodekeeper-z` forks on the `chore/zig-master` branch until these merge and cut releases: | Dependency | Upstream PR | |---|---| | `blst` | [ChainSafe/blst.zig#4](ChainSafe/blst.zig#4) | | `hashtree` | [ChainSafe/hashtree-z#10](ChainSafe/hashtree-z#10) | | `snappy` | [ChainSafe/snappy.zig#5](ChainSafe/snappy.zig#5) | | `yaml` | [ChainSafe/zig-yaml#1](ChainSafe/zig-yaml#1) | | `zapi` | [ChainSafe/zapi#9](ChainSafe/zapi#9) | | `httpz` (disabled) | [karlseguin/http.zig#191](karlseguin/http.zig#191) |
Upgrade to Zig 0.16.0-dev (master) — part of the lodestar-z Zig master upgrade effort.
Key changes:
std.io→std.Iomigrationstd.Io.Dirmethods now require explicitioparameter🤖 Generated with AI assistance