Skip to content

chore: upgrade to Zig 0.16.0#1

Merged
wemeetagain merged 19 commits into
ChainSafe:mainfrom
lodekeeper-z:chore/zig-master
Apr 20, 2026
Merged

chore: upgrade to Zig 0.16.0#1
wemeetagain merged 19 commits into
ChainSafe:mainfrom
lodekeeper-z:chore/zig-master

Conversation

@lodekeeper-z

Copy link
Copy Markdown

Upgrade to Zig 0.16.0-dev (master) — part of the lodestar-z Zig master upgrade effort.

Key changes:

  • std.iostd.Io migration
  • std.Io.Dir methods now require explicit io parameter
  • Build system changes for 0.16 compatibility

🤖 Generated with AI assistance

- addTest/addExecutable: .root_source_file → .root_module (create module first)
- std.ArrayList: .init(alloc) → .empty + explicit allocator on methods
- std.mem.trimLeft → trimStart, trimRight → trimEnd
- std.fs.File.WriteError → std.Io.Writer.Error
- writer.writeByteNTimes → writer.splatByteAll
- Writer.Allocating: use .init(alloc) then .toArrayList() to get data
- ErrorBundle.RenderOptions: removed ttyconf field
- std.fs.cwd() → std.Io.Dir.cwd()
- dir.readFileAlloc(io, path, alloc, .unlimited) replaces file.readToEndAlloc
- Spec tests disabled (TODO: need full std.Io migration in test/spec.zig)

🤖 Generated with AI assistance
wemeetagain and others added 17 commits March 20, 2026 02:55
When the target Zig type is []const u8 (pointer), the yaml parser needs
to handle boolean, int, and float values by converting them to their
string representations. This maintains backward compatibility with code
that expects all yaml values to be parseable as strings.

🤖 Generated with AI assistance
The tokenizer checked if the byte before a closing `"` was `\` to detect
escaped quotes. This fails for `\\"` (escaped backslash followed by
closing quote), incorrectly treating the closing quote as escaped.

Fix: track escape state by handling `\` explicitly in the double_quoted
state — when a backslash is encountered, skip the next character
unconditionally. This correctly handles all escape sequences including
`\\`, `\"`, `\n`, `\t`, etc.

Added regression tests for escaped backslash before closing quote and
various escape sequences.

🤖 Generated with AI assistance
Previously only \n, \t, and \" were supported. Now handles all
escape sequences from YAML 1.2 spec section 5.7:

- Single-char: \0 \a \b \t \n \v \f \r \e \" \/ \\ \<space>
- Unicode names: \N (next line) \_ (nbsp) \L (line sep) \P (para sep)
- Unicode hex: \xNN \uNNNN \UNNNNNNNN

All produce correct UTF-8 output.

🤖 Generated with AI assistance
list_empty previously returned a static &.{} slice. Calling gpa.free()
on a non-heap pointer is undefined behavior with some allocators. Use
gpa.alloc(Value, 0) instead, which returns a properly tracked zero-
length allocation.

🤖 Generated with AI assistance
When a YAML value is null (~, null, empty), parseOptional now correctly
returns null for optional types. Previously it only returned null when
the map key was missing entirely, not when the value was explicitly
null.

Also added tests for null values (tilde, null keyword) in various
contexts: maps, lists, and typed struct optionals.

🤖 Generated with AI assistance
Added detection for .inf, -.inf, and .nan (with case variants) as
specified in the YAML 1.2 core schema. These map to the corresponding
IEEE 754 special values.

🤖 Generated with AI assistance
- Empty values now stringify as "null" instead of empty string
- Positive/negative infinity stringify as .inf/-.inf
- NaN stringifies as .nan
- Matches YAML 1.2 core schema representation

🤖 Generated with AI assistance
- Migrate test/spec.zig from std.fs to std.Io.Dir (Zig 0.16 API)
- Update ArrayList usage to unmanaged pattern (std.ArrayList is now unmanaged)
- Replace std.zig.fmtEscapes with manual string escaping
- Update generated test code to use std.Io.Writer.Allocating
- Re-enable spec tests in build.zig with -Denable-spec-tests=true flag

Spec test results: 77 pass, 321 skip, 4 fail (402 total)
The 4 failures are UnexpectedSuccess (parser too lenient on invalid YAML).

🤖 Generated with AI assistance
Coverage for all recent bug fixes and YAML features:
- Double-quoted escape sequences (\, ", \n, \t, \x41, \u0041, \U00000041)
- Escaped backslash at end of string (regression test)
- Flow mappings (simple, nested, empty, as struct values)
- Quoted string non-coercion ("true" stays string, not bool)
- Null/empty values (~, null, NULL, Null, empty in explicit doc)
- Special floats (.inf, -.inf, .nan, case variations, f32/f64)
- Stringify roundtrips (ints, floats, strings, structs, lists, optionals)
- Comments (inline, standalone, between entries)
- Empty and multi-document handling
- Boolean edge cases (true/false, yes/no/on/off, case insensitive)
- Integer formats (decimal, hex 0x, octal 0o)
- Deeply nested structures (4+ levels, mixed lists/maps)
- Mixed flow and block styles
- Ethereum mainnet config parsing (fork versions, large ints, hex strings)
- Full mainnet config snippet with all field types
- Error cases (type mismatch, missing fields, duplicate keys)
- Untyped value access (map contains, list indexing)
- Value.encode integration

🤖 Generated with AI assistance
StringArrayHashMap (managed) was removed in Zig 0.16.
Use StringArrayHashMapUnmanaged with explicit allocator instead.
@GrapeBaBa GrapeBaBa marked this pull request as ready for review April 16, 2026 16:03
Copilot AI review requested due to automatic review settings April 16, 2026 16:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Upgrades the project to Zig 0.16.0-dev by migrating to std.Io APIs (including explicit io context for Dir methods) and updating the build/test harness accordingly, while also refactoring YAML value handling and extending parsing/tokenizing behavior.

Changes:

  • Migrate filesystem and IO usage from std.fs/std.io patterns to std.Io + explicit IO context where required.
  • Refactor Yaml.Value to a scalar-based representation and update typed parsing, errors, and tests.
  • Add/expand parsing support (flow maps, improved double-quoted escape handling) and introduce a new comprehensive test suite.

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
build.zig Updates test build graph for Zig 0.16 module APIs; adds comprehensive test target.
src/Yaml.zig Refactors typed parsing and Value representation; updates stringify and error handling.
src/Parser.zig Adds flow-map parsing and extends double-quoted escape parsing.
src/Tokenizer.zig Updates double-quoted tokenization logic and modernizes test ArrayList usage.
src/Parser/test.zig Adjusts error rendering and adds/changes parser behavior tests (flow maps, doc end handling).
src/Yaml/test.zig Updates/extends unit tests to match scalar-based Value and new parsing semantics.
test/test.zig Updates test file loading to std.Io.Dir read helpers.
test/spec.zig Updates YAML test-suite generator for std.Io + new container APIs.
test/comprehensive_test.zig Adds a large set of additional regression/compliance tests.
.gitignore Normalizes ignored directories and adds zig-pkg/.
Comments suppressed due to low confidence (1)

src/Yaml.zig:352

  • Value.stringify returns an empty string for empty lists (len == 0), which produces invalid/ambiguous YAML when a list is the document root or a mapping value (e.g. key: instead of key: []). Consider emitting [] for empty lists (and similarly handling empty maps as {}) so stringify always produces a valid YAML representation.
            .list => |list| {
                const len = list.len;
                if (len == 0) return;


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/Yaml.zig
Comment on lines 382 to 386
var i: usize = 0;
for (map.keys(), map.values()) |key, value| {
if (!args.should_inline_first_key or i != 0) {
try writer.writeByteNTimes(' ', args.indentation);
try writer.splatByteAll(' ', args.indentation);
}

Copilot AI Apr 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Value.stringify currently emits nothing for empty maps (it returns early when map.count() == 0), which can produce invalid/ambiguous YAML (e.g. a root value or key: ). Consider emitting {} for empty maps so stringify always produces a valid representation.

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of scope

Comment thread src/Yaml.zig
Comment on lines +103 to +104
const scalar = try value.asScalar();
return self.parsePointer(arena, T, .{ .scalar = try arena.dupe(u8, scalar) });

Copilot AI Apr 16, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In parseValue’s scalar pointer path, the scalar is duplicated into the arena (arena.dupe) and then parsePointer duplicates it again for []u8 slices. This double-copies string scalars unnecessarily. Consider passing the original value through to parsePointer and letting parsePointer perform the single arena duplication (or otherwise avoid the intermediate Value{ .scalar = arena.dupe(...) }).

Suggested change
const scalar = try value.asScalar();
return self.parsePointer(arena, T, .{ .scalar = try arena.dupe(u8, scalar) });
try value.asScalar();
return self.parsePointer(arena, T, value);

Copilot uses AI. Check for mistakes.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

out of scope

Comment thread src/Yaml.zig Outdated
Comment thread src/Tokenizer.zig Outdated
Comment thread src/Parser/test.zig
Comment thread build.zig Outdated
Comment thread test/test.zig Outdated
- Remove raw.len == 0 check in parseOptional to preserve empty strings
- Add bounds check for trailing backslash in double-quoted tokenizer
- Remove dead log build option
- Remove placeholder comment in test
GrapeBaBa

This comment was marked as abuse.

@GrapeBaBa GrapeBaBa changed the title chore: upgrade to Zig 0.16.0-dev (master) chore: upgrade to Zig 0.16.0 Apr 17, 2026
@GrapeBaBa GrapeBaBa requested a review from wemeetagain April 17, 2026 07:47
@wemeetagain wemeetagain merged commit b74f4e8 into ChainSafe:main Apr 20, 2026
GrapeBaBa added a commit to GrapeBaBa/state-transition-z that referenced this pull request Apr 21, 2026
Four of the five Zig-0.16-upgrade PRs on our dependency forks have
merged into ChainSafe upstream main:

  * ChainSafe/blst.zig#4  → 6ad139a
  * ChainSafe/hashtree-z#10 → 56e406f
  * ChainSafe/snappy.zig#5 → 7139424
  * ChainSafe/zig-yaml#1  → b74f4e8

Switch from `lodekeeper-z/<repo>?ref=chore/zig-master` to the merged
commit on `ChainSafe/<repo>` main for each. Content is identical so
the existing `hash` fields stay valid.

`zapi` (ChainSafe/zapi#9) is still open upstream — bump the
`lodekeeper-z/zapi` commit pointer to the current PR head
(fefb88ebf5c9) which includes the latest port/refactor work; TODO
note to switch to ChainSafe/zapi once ChainSafe#9 merges.
wemeetagain pushed a commit to ChainSafe/lodestar-z that referenced this pull request Apr 24, 2026
## Motivation

Get the codebase cleanly compatible with **Zig 0.16.0** release,
following the new "explicit `io` everywhere, no hidden global state"
design — without changing runtime behavior.

## Description

- **`std.Io`** — File/Dir operations, `Mutex`/`Condition`/`Event`
(cancelable vs uncancelable split), entropy via
`io.random`/`io.randomSecure`, timestamps via `std.Io.Timestamp`
- **ArrayList unmanaged** — allocator threaded through
`deinit`/`append`/`resize`/`ensureCapacity`/`clone`; replace `items.len
= N` hacks with `resize` / `expandToCapacity` /
`appendNTimesAssumeCapacity` (preserves semantics, matches
`initCapacity`'s Precise allocation)
- **Struct builder** — `@Type(.@"struct" = ...)` → `@Struct(names,
types, attrs)` builtin in `ssz/type/container.zig`
- **Copy-elision workaround** — `std.Io` 0.16 result-location semantics
pre-write the union tag of the assignment target, so reads from
`self.state.init_group.*` during RHS evaluation would observe the new
variant. Snapshot pre-upgrade fields to locals in `state_transition.zig`
fork-upgrade chain and `era/Writer.zig`
- **ThreadPool** — `io` is a parameter on
`init`/`deinit`/`verifyMultipleAggregateSignatures`/`aggregateVerify`
(not a stored field); normal paths use cancelable `lock`; shutdown +
`submitAndWait.done.wait` stay uncancelable with comments explaining the
invariant
- **Shared `src/time.zig`** — `timestampNow` / `since (Duration)` /
`durationSeconds`, replacing four duplicated helper pairs across
`state_transition` and `bench`
- **IO signature convention** — `(allocator?, io, file, ...)` applied
consistently to `e2s.*` and `era.*`
- **Reader API polish** — `takeArray` (zero-copy) + `readSliceAll` where
appropriate; drop single-buffer `readVecAll` and `catch |err| return
err`
- **Other**: `RefCount` comptime-reflect on wrapped T's deinit arity
(1-arg vs 2-arg) so 0.16 unmanaged `ArrayList` and project types with
stored allocators both work; `sanitize_c = .trap` on blst artifact so
Debug fuzz builds link without `libubsan`; untrack 426 stale
`test/fuzz/zig-pkg/` entries from the git index
- **CI**: `ZIG_VERSION` bumped `0.14.1 → 0.16.0`; `zbuild` sync-check +
`build-examples` jobs temporarily commented out pending upstream Zig
0.16 support

## Dependency PRs

Each Zig 0.16 upgrade landed as a dedicated PR on the upstream repo.
`build.zig.zon` points at `lodekeeper-z` forks on the `chore/zig-master`
branch until these merge and cut releases:

| Dependency | Upstream PR |
|---|---|
| `blst` |
[ChainSafe/blst.zig#4](ChainSafe/blst.zig#4) |
| `hashtree` |
[ChainSafe/hashtree-z#10](ChainSafe/hashtree-z#10)
|
| `snappy` |
[ChainSafe/snappy.zig#5](ChainSafe/snappy.zig#5)
|
| `yaml` |
[ChainSafe/zig-yaml#1](ChainSafe/zig-yaml#1) |
| `zapi` | [ChainSafe/zapi#9](ChainSafe/zapi#9)
|
| `httpz` (disabled) |
[karlseguin/http.zig#191](karlseguin/http.zig#191)
|
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants