Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,11 +106,8 @@ This project uses [Gradle Dependency Verification](https://docs.gradle.org/curre
When you **add or update dependencies**, you must regenerate the verification metadata before pushing:

```bash
# Regenerate verification metadata (includes all configurations)
./gradlew build --write-verification-metadata sha256

# If you also use CycloneDX SBOM generation, update that too
./gradlew cyclonedxBom --write-verification-metadata sha256
# Regenerate verification metadata (includes build and CycloneDX SBOM configurations)
./gradlew --write-verification-metadata sha256 build cyclonedxBom -x test
```

Commit the updated `gradle/verification-metadata.xml` with your dependency changes. The CI build will **fail** if checksums are missing.
Expand Down
19 changes: 16 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
[![Release](https://github.com/toon-format/toon-java/actions/workflows/release.yml/badge.svg)](https://github.com/toon-format/toon-java/actions/workflows/release.yml)
[![Maven Central](https://img.shields.io/maven-central/v/dev.toonformat/jtoon.svg)](https://central.sonatype.com/artifact/dev.toonformat/jtoon)
![Coverage](.github/badges/jacoco.svg)
[![SPEC v3.3](https://img.shields.io/badge/spec-v3.3-fef3c0?labelColor=1b1b1f)](https://github.com/toon-format/spec)
[![SPEC v3.3.2](https://img.shields.io/badge/spec-v3.3.2-fef3c0?labelColor=1b1b1f)](https://github.com/toon-format/spec)
[![License: MIT](https://img.shields.io/badge/license-MIT-fef3c0?labelColor=1b1b1f)](./LICENSE)

Compact, human-readable serialization format for LLM contexts with **30-60% token reduction** vs JSON. Combines YAML-like indentation with CSV-like tabular arrays. Working towards full compatibility with the [official TOON specification](https://github.com/toon-format/spec).
Expand Down Expand Up @@ -351,12 +351,25 @@ Object result2 = JToon.decode(invalidToon, lenient);

**CI/CD:** GitHub Actions • Java 17 • Coverage enforcement • PR coverage comments

## Development

```bash
# Build (includes tests, checks, coverage)
./gradlew build

# Run tests only
./gradlew test

# Update dependency verification metadata (required when adding/updating dependencies)
./gradlew --write-verification-metadata sha256 build cyclonedxBom -x test
```

See [CONTRIBUTING.md](CONTRIBUTING.md) for full development guidelines.

## Project Status

This project is 100% compliant with TOON specification. Release conformance enforced on CI/CD.

See [CONTRIBUTING.md](CONTRIBUTING.md) for detailed guidelines.

## Documentation

- [📘 Full Documentation](docs/) - Extended guides and references
Expand Down
8 changes: 4 additions & 4 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ plugins {
id 'maven-publish'
id 'signing'
id 'jacoco'
id 'com.github.spotbugs' version '6.5.8'
id 'com.github.spotbugs' version '6.5.9'
id 'pmd'
id 'checkstyle'
id 'org.owasp.dependencycheck' version '12.2.2'
id 'org.cyclonedx.bom' version '3.2.4'
id 'org.cyclonedx.bom' version '3.3.0'
id 'info.solidsoft.pitest' version '1.19.0'
id 'net.ltgt.errorprone' version '5.1.0'
}
Expand Down Expand Up @@ -146,7 +146,7 @@ tasks.checkstyleTest {
dependencies {
implementation 'tools.jackson.core:jackson-databind:3.2.1'
implementation 'tools.jackson.module:jackson-module-blackbird:3.2.1'
compileOnly 'com.github.spotbugs:spotbugs-annotations:4.10.2'
compileOnly 'com.github.spotbugs:spotbugs-annotations:4.10.3'

// NullAway + Error Prone for compile-time null safety
errorprone 'com.uber.nullaway:nullaway:0.13.7'
Expand All @@ -158,7 +158,7 @@ dependencies {
}
api 'org.jspecify:jspecify:1.0.0'

testImplementation platform('org.junit:junit-bom:6.1.1')
testImplementation platform('org.junit:junit-bom:6.1.2')
testImplementation 'org.junit.jupiter:junit-jupiter'
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
testImplementation 'org.awaitility:awaitility:4.3.0'
Expand Down
246 changes: 246 additions & 0 deletions gradle/verification-metadata.xml

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,15 @@ private static Object decodeInternal(final String toon, final DecodeOptions opti
}

// Handle keyed arrays: items[2]{id,name}:
// Only match if the key part (before the bracket) doesn't contain a colon,
// because a colon indicates a key-value pair (e.g. 'a: "[2]: x"')
final Matcher keyedArray = KEYED_ARRAY_PATTERN.matcher(line);
if (keyedArray.matches()) {
return KeyDecoder.parseKeyedArrayValue(keyedArray, line, depth, context);
final String keyPart = keyedArray.group(1);
final int colonInKey = DecodeHelper.findUnquotedColon(keyPart);
if (colonInKey <= 0) {
return KeyDecoder.parseKeyedArrayValue(keyedArray, line, depth, context);
}
}
// Handle key-value pairs: name: Ada
final int colonIdx = DecodeHelper.findUnquotedColon(line);
Expand Down
3 changes: 1 addition & 2 deletions src/test/resources/conformance/decode/arrays-nested.json
Original file line number Diff line number Diff line change
Expand Up @@ -81,8 +81,7 @@
}
]
},
"specSection": "10",
"note": "Single-field list-item object: only the tabular array, no sibling fields"
"specSection": "10"
},
{
"name": "parses objects containing arrays (including empty arrays) in list format",
Expand Down
8 changes: 8 additions & 0 deletions src/test/resources/conformance/decode/arrays-primitive.json
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,14 @@
},
"specSection": "9.1"
},
{
"name": "parses quoted key containing a colon with inline array",
"input": "\"a:b\"[2]: 1,2",
"expected": {
"a:b": [1, 2]
},
"specSection": "9.1"
},
{
"name": "parses quoted key with empty array",
"input": "\"x-custom\"[0]:",
Expand Down
13 changes: 13 additions & 0 deletions src/test/resources/conformance/decode/arrays-tabular.json
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,19 @@
"count": 2
},
"specSection": "9.3"
},
{
"name": "treats a key-value line at header depth whose value contains the active delimiter as end of rows, not a row",
"input": "t[2]{a,b}:\n 1,2\n 3,4\nx: 3,4",
"expected": {
"t": [
{ "a": 1, "b": 2 },
{ "a": 3, "b": 4 }
],
"x": "3,4"
},
"specSection": "9.3",
"note": "The dedent to the header's depth closes the rows (§8); the line is a key-value pair, and the whole post-colon token is the value, non-numeric so a string (§11.2)"
}
]
}
2 changes: 1 addition & 1 deletion src/test/resources/conformance/decode/delimiters.json
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@
"note": "a,b"
},
"specSection": "11",
"note": "Object values don't require comma quoting regardless of delimiter"
"note": "Object field values are parsed as the whole post-colon token, never split on a delimiter (§11.2)"
},
{
"name": "object values in list items follow document delimiter",
Expand Down
3 changes: 1 addition & 2 deletions src/test/resources/conformance/decode/numbers.json
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,7 @@
"expected": {
"value": 5
},
"specSection": "4",
"note": "Exponent +00 results in the integer 5"
"specSection": "4"
},
{
"name": "parses zero with exponent as number",
Expand Down
52 changes: 52 additions & 0 deletions src/test/resources/conformance/decode/objects.json
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,50 @@
},
"specSection": "8"
},
{
"name": "parses quoted object value shaped like an inline array header",
"input": "a: \"[2]: x\"",
"expected": {
"a": "[2]: x"
},
"specSection": "8",
"note": "Quoted value is opaque (§4); the bracket-and-colon shape inside the quotes is data, not an array header (§6)"
},
{
"name": "parses quoted object value shaped like a count-matching inline array header",
"input": "a: \"[1]: x\"",
"expected": {
"a": "[1]: x"
},
"specSection": "8",
"note": "Quoting makes the value opaque (§4), so it is never parsed as a header – the matching count is irrelevant because a quoted scalar is never a header candidate"
},
{
"name": "parses unquoted value shaped like an inline array header after the key",
"input": "key: foo [2]: bar",
"expected": {
"key": "foo [2]: bar"
},
"specSection": "6",
"note": "An unquoted key cannot contain a colon, so the first colon ends the key; the bracket-and-colon shape in the remainder is value text, not an array header"
},
{
"name": "parses unquoted value shaped like a tabular array header after the key",
"input": "a: b [2]{x,y}: 1,2",
"expected": {
"a": "b [2]{x,y}: 1,2"
},
"specSection": "6",
"note": "An unquoted key cannot contain a colon, so the first colon ends the key; the tabular-header shape in the remainder is value text, not an array header"
},
{
"name": "parses quoted object value with bracket text and a later colon",
"input": "content: \"accept (process.argv[2]) and greet: name\"",
"expected": {
"content": "accept (process.argv[2]) and greet: name"
},
"specSection": "8"
},
{
"name": "parses quoted key with colon",
"input": "\"order:id\": 7",
Expand Down Expand Up @@ -286,6 +330,14 @@
},
"specSection": "8"
},
{
"name": "unescapes tab in key immediately followed by a colon",
"input": "\"\\t:x\": v",
"expected": {
"\t:x": "v"
},
"specSection": "8"
},
{
"name": "unescapes quotes in key",
"input": "\"he said \\\"hi\\\"\": 1",
Expand Down
36 changes: 36 additions & 0 deletions src/test/resources/conformance/decode/validation-errors.json
Original file line number Diff line number Diff line change
Expand Up @@ -185,6 +185,42 @@
"note": "[-1] is not a non-negative integer length; decoders MUST NOT interpret it as a bracket segment",
"minSpecVersion": "3.2"
},
{
"name": "throws on decimal bracket length in strict mode",
"input": "x[3.7]: a,b,c",
"expected": null,
"shouldError": true,
"options": {
"strict": true
},
"specSection": "6",
"note": "[3.7] is not an integer length (§6 length is DIGIT-only); decoders MUST NOT interpret it as a bracket segment",
"minSpecVersion": "3.2"
},
{
"name": "throws on bracket length with plus sign in strict mode",
"input": "x[+3]: a,b,c",
"expected": null,
"shouldError": true,
"options": {
"strict": true
},
"specSection": "6",
"note": "[+3] carries a sign and is not a non-negative integer length; decoders MUST NOT interpret it as a bracket segment",
"minSpecVersion": "3.2"
},
{
"name": "throws on bracket length in exponent form in strict mode",
"input": "x[1e1]: 1,2,3,4,5,6,7,8,9,10",
"expected": null,
"shouldError": true,
"options": {
"strict": true
},
"specSection": "6",
"note": "[1e1] is exponent notation, not a DIGIT-only integer length; decoders MUST NOT interpret it as a bracket segment",
"minSpecVersion": "3.2"
},
{
"name": "throws on whitespace between bracket segment and colon in strict mode",
"input": "items[2] :\n 1,2",
Expand Down
3 changes: 1 addition & 2 deletions src/test/resources/conformance/decode/whitespace.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,7 @@
{ "id": 2, "name": "Bob" }
]
},
"specSection": "12",
"note": "Values in tabular rows are trimmed"
"specSection": "12"
},
{
"name": "tolerates spaces around delimiters with quoted values",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
},
"expected": "items[1]:\n - users[2]{id,name}:\n 1,Ada\n 2,Bob\n status: active",
"specSection": "10",
"note": "YAML-style encoding for list-item objects with tabular array as first field"
"note": "Tabular header on hyphen line with rows at depth +2 and sibling fields at depth +1 (§10)"
},
{
"name": "uses list format for nested object arrays with mismatched keys",
Expand Down
2 changes: 1 addition & 1 deletion src/test/resources/conformance/encode/key-folding.json
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@
"flattenDepth": 0
},
"specSection": "13.4",
"note": "flattenDepth=0 disables all folding"
"note": "flattenDepth below 2 has no practical effect, leaving standard nesting (§13.4)"
},
{
"name": "encodes standard nesting with keyFolding=off (baseline)",
Expand Down
5 changes: 2 additions & 3 deletions src/test/resources/conformance/encode/primitives.json
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@
"input": "05",
"expected": "\"05\"",
"specSection": "7.2",
"note": "Leading zeros make it non-numeric"
"note": "Matches the numeric-like pattern (05 is a listed §7.2 example), so it MUST be quoted"
},
{
"name": "escapes newline in string",
Expand Down Expand Up @@ -198,8 +198,7 @@
"name": "encodes negative zero as zero",
"input": -0,
"expected": "0",
"specSection": "2",
"note": "Negative zero normalizes to zero"
"specSection": "2"
},
{
"name": "encodes scientific notation as decimal",
Expand Down
6 changes: 2 additions & 4 deletions src/test/resources/conformance/encode/whitespace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,7 @@
"id": 123
},
"expected": "id: 123",
"specSection": "12",
"note": "Output should not end with newline character"
"specSection": "12"
},
{
"name": "maintains proper indentation for nested structures",
Expand All @@ -37,8 +36,7 @@
"specSection": "12",
"options": {
"indent": 4
},
"note": "4-space indentation for nested objects when indent option is set to 4"
}
}
]
}
Loading