JSON with the punctuation removed. Whitespace-delimited key value pairs — no commas, no
colons, bare keys, quoted string values.
name "example-service"
version 3
listen {
host "0.0.0.0"
port 8080
}
regions ["us-west-2" "eu-central-1"]
SPEC.md is the language definition and the source of truth. This file is how to use it. examples/config.tot tours the syntax.
- String values must be quoted.
kind curlyis a parse error. Bare values are only numbers,true,false,null. Reason: two adjacent barewords are undecidable —country united stateshas no recoverable meaning. - Keys are bare unless they can't be. Reserved outside strings: whitespace, control
characters, and
, : " { } [ ] # =— plus(and), but only in a.totttemplate. Everything else is fine bare:path/to/thing,com.example.key,my-name,123. A dot is an ordinary character, not a nesting operator. A key that holds a reserved character is written quoted, escapes and all:"a\u0001b". ,and:are whitespace. Every JSON file is valid tot, byte for byte. Trailing commas are legal.#to end of line is the only comment. No block form.- The top level is an object with no braces. A document that is a single value is that value, so JSON scalar and array roots still parse.
- Integers and floats are distinct types. Float iff it has a
.or an exponent, so1 != 1.0and6e23is a float.1.and.1are legal here and normalize to1.0and0.1on the way out to JSON. - No leading zeros.
01234is an error — zip codes have to be strings. - Integers have no range limit; floats do. An integer keeps its lexeme, so a 400-digit one
survives. A float has to denote a real
f64, so1e999is a parse error — tot has no way to write an infinity. (1e-999is fine; it's zero.) - Duplicate keys are an error. No last-wins.
- No dates, anchors, tags, includes, or interpolation. Dates, anchors, and tags aren't planned. Composing documents is — see Composition — but the point of that design is to serve it from the CLI and a separate template file, so the data language keeps this rule.
Multi-line strings:
motd """
hello
"""
Indentation is stripped relative to the closing delimiter, so reindenting the block around
a string cannot change its value. No trailing newline unless you write \n.
The closing """ owns its whole line — nothing may follow it. Write \""" for a literal
""" opening a content line; otherwise it would close the string early and the error would
land somewhere further down.
cargo install --path cli # puts `tot` on your PATHOr build it in place:
cargo build --release -p tot-cli # target/release/totThe root package is the library, so a bare cargo build won't produce the binary — name the
package, or use --workspace. Rust 1.88 or newer, which is what edition 2024 plus let-chains
needs; rust-version says so in both manifests, so cargo will tell you rather than the
compiler.
tot fmt [--check] [--template] [FILE]... |
format in place; --check writes nothing and exits 1 on a diff |
tot check [--strict] [--template] [--schema=F] [FILE]... |
parse and report diagnostics |
tot build [--check] [--out=F] [--set=N=V]... FILE |
build a .tott template into a document |
tot merge [FILE]... |
fold documents together, left to right |
tot get [--raw] <PATH> [FILE] |
print the one value at PATH |
tot set <PATH> <VALUE> [FILE] |
write VALUE at PATH and print the document |
tot to <json|yaml|toml> [FILE] |
write this document as another format |
tot from <json|yaml|toml> [FILE] |
read another format and write tot |
No FILE means stdin, and so does a FILE of - (a file actually named - is ./-). For fmt
and check, every input is processed before exiting — one bad file doesn't hide the rest.
Exit codes: 0 ok, 1 the input didn't answer the request (unformatted, unparseable,
unconvertible, or no such path), 2 I/O or bad arguments. A converter refusing a document is
1 like any other unusable input — 2 is reserved for the invocation or the filesystem being
wrong, so tot from json and tot from yaml answer alike when the document is bad.
Flags: --raw (get, set), --create (set), --null=set|delete (merge, default
set), --compact (to json),
--null=omit|error (to toml, default omit). A flag that doesn't apply to the chosen format
is an error, not a no-op. A bare -- ends the flags — - is a bareword character, so --foo
is a legal key and a legal path, and a file can be named that way: tot get -- --foo config.tot.
check --strict adds one lint: a member's value must begin on its key's line. A {, [,
or """ may still run past it — only the start has to sit beside the key. Everything it
reports is legal tot, so it's opt-in. It exists because there's no separator between members:
a missing value shifts every member after it, and while quoted string values catch nearly all
of those at the offending token, keeping members on one line is what makes the error land in
the right place every time. tot fmt fixes what it reports — the formatter pulls the
value back up onto the key's line. --strict and --schema= are separate questions and
compose; a document can fail both at once.
check --schema= validates a document's shape. A schema is a tot document that looks like
the documents it describes, with a type where each value would be:
name "string"
listen {host "string" port "int" tls? "bool"}
regions ["string"]
labels {* "string"}
retries "int|null"
Types are any, string, int, float, bool, null, joined with |. ? on a key
makes the member optional — it goes on the key because presence is a property of the member,
which also means it works for a {…} or […] value. * as a key covers every other key;
without one an undeclared member is an error, since catching a typo is most of the point.
Type names are quoted because a schema is tot, and a bare word is never a value in tot. That's also what makes a schema line up with the config beside it. examples/config.schema.tot describes examples/config.tot.
Every violation is reported, not just the first, and each one names a tot get path and points
a caret at the key. A typo gives you two — the name that's missing and the name that isn't
known — because either alone sends you to the wrong place. No enumerations yet; the spec says
why.
A .tott file is tot plus one thing: a (head arg…) form, wherever a value goes,
evaluated at build time and replaced by its value. tot build turns one into a .tot
document, and the document is what you commit and everything else reads.
name "example-service"
replicas (if (param "prod") 5 1)
image (str "registry/" (param "name") ":" (param "tag"))
regions (import "regions.tot")
endpoints (map (str "https://" (it) ".example.net") (import "regions.tot"))
retries (get "limits.retries" (import "defaults.tot") 3)
tot build --set=prod=true --set-raw=tag=v1.4.2 --out=config.tot config.tottThe data language doesn't change. Parens are ordinary bareword characters in .tot and
delimiters only in .tott, so (a) 1 is still the key (a) and @type / $ref are still
bare — which is exactly what reserving @ or $ would have cost, and those are JSON-LD and
JSON Schema, the documents from json exists to read. Parens won because they never appear
in data: what's inside them is computed, what's outside isn't, and you can see which without
knowing the form set.
There are seven forms and no way to define an eighth:
(param "name") |
a build parameter, or a failure if it wasn't set |
(param "name" default) |
…or default, when it wasn't set |
(if cond then else) |
cond must be a boolean; only the branch taken is evaluated |
(str a b …) |
joins strings, numbers, and booleans |
(import "file") |
that file's value, resolved relative to the importing file |
(get path value) |
the value at path inside value |
(get path value default) |
…or default, when there's nothing there |
(map body list) |
body evaluated once per element of list |
(it) |
the element the enclosing (map …) is on |
The fixed set is the whole discipline — the moment a template can define a function, people write libraries and a config file becomes a program you have to read as one.
Things worth knowing:
-
A form goes where a value goes, and only there. A computed key would make a document's shape depend on evaluating it. Splicing one document's members into another is
merge; embedding one value isimport. -
A
paramname and animportpath are written down, not computed, so you can see what a template needs and what it pulls in without running it. -
iftakes a boolean — no truthiness here either. The branch not taken isn't evaluated, so it can import a file this configuration doesn't have. -
getreads out of a value you hand it, never out of the document being built — that would make a template's meaning depend on the order its members happened to be evaluated in, and let a document refer to itself. Its path may be computed, unlike aparamname or animportpath: those are static because they're the build's inputs, and a path isn't one. Selecting by parameter,(get (param "env") (import "environments.tot")), is the ordinary reason to want one. The third argument is the only way to reach a member that may be missing, sinceifwants a boolean and there's nohas— but it covers a member that isn't there and an index past the end, and nothing else. A step into the wrong kind of value is the document being shaped differently than the template thinks, and stays a failure with or without a default. -
mapneeds a placeholder, and(it)is a form because everything computed in tot sits inside parens._would have been shorter and would have broken the sentence the paren sigil rests on. Amapmay not appear inside anothermap's body, so(it)names the element of exactly one of them and never needs a shadowing rule; in the list argument it's fine, since that one finishes before the body runs. An import is a wall for the same reason — an imported file is parsed on its own, so(it)in one is an error rather than a reach into whatever imported it, which is also what keeps the build cache sound. Both are parse errors, sotot checkcatches them without building anything. -
Parameters come from the command line and nowhere else, so a build reproduces anywhere.
--set=N=Vtakes a tot value,--set-raw=N=Vtakes a literal string — the same split asset/set --raw. Want the environment?--set-raw=env="$ENV"puts it in plain sight. -
--checkis the one that earns its keep: it builds and compares against the document on disk (config.tott→config.tot), so CI catches a committed document that's drifted from its template, the wayfmt --checkcatches formatting. -
Imports must be acyclic, and a failure names the file it happened in plus the chain that reached it. Each file is built once however many times it's imported — its value doesn't depend on who imported it, so a build stays linear in the size of the graph instead of exponential in its depth. Sharing a fragment is the ordinary reason to have one.
-
buildrefuses a.totfile: a document is already built, and reading one as a template would report its parens as forms. Extensions are read case-insensitively, since on WindowsCONFIG.TOTandconfig.totare one file.--outwon't overwrite the template it's building from or any fragment it imports — both are files being read, and there's nothing to recover either from. A parameter set twice is refused rather than resolved, the way a duplicate key is. -
tot checkreads templates, and reading one checks its forms — unknown head, wrong argument count, computedparamname orimportpath.--strictapplies its one lint too; the parity hazard is the language's, and a form is one more value that can land on the wrong line.--schemaagainst a template is refused: a schema says what shape a document has, and(param "x")could be anything until it's built. Build first —tot build c.tott | tot check --schema=shape.tot— which is what the error tells you. -
tot fmtformats templates, so a.tottfile is kept honest like everything else here. A form is one more bracketed shape with the same rule as a collection — inline vs. block is yours, spacing is normalized, nothing is reflowed. The head stays with the opening ((str), and a block form closes with)on its own line, the way}and]do.The extension picks the dialect. Stdin has none, so it's a document unless you pass
--template— sniffing the contents would be exactly the implicit typing tot exists to avoid. One consequence worth knowing:"(a)"is unquoted to(a)in a.totfile and stays quoted in a.tottone, because unquoting it there would turn a key into a form.
examples/service.tott builds examples/service.tot, and a test keeps them in step.
merge is base-plus-overlays:
tot merge base.tot staging.tot regional/eu.totTwo objects merge member by member; anything else is replaced whole. An array replaces rather than appending — concatenation can't be undone by a later layer, so an overlay that could only ever add would be a one-way door. A change of kind replaces too, at the root or anywhere else. Base members keep their position; new ones append.
An overlay's null sets the member to null, because null is a real value here. --null=delete
removes it instead, which is how a layer takes something away — nested, so a {b null} deletes
a.b. A null at the root or inside an array is data either way.
One bad layer ends the run (there's a single output, and a partial merge is worse than none),
and comments don't survive, the same way they don't through from.
get takes a path where . selects a member and [n] an element:
tot get listen.port config.totA . nests in a path but not in a document, where it's an ordinary bareword character — so
a key holding one gets quoted: tot get '"com.example".level'. That's the only place the two
spellings of a key differ. Output is tot, so it pipes back into tot to json and friends;
--raw prints a string with no quotes or escapes, for PORT=$(tot get --raw ...). A path the
document doesn't have is exit 1 and names what was there instead. No wildcards or filters — if
you want those, tot to json | jq.
set is the dual, and takes a value spelled the way get prints one:
tot set listen.port 8080 config.tot
tot set --raw name svc config.tot # a string, without typing '"svc"'So a string needs its quotes (tot set name '"svc"') unless you use --raw, and a brace-less
host "::" port 80 is an object here exactly as it is at the top of a file.
The last step of the path may be new — adding a member is what setting is for — but the
steps before it must exist, unless --create builds them. That's the default because a
mistyped path is likelier than a genuinely missing branch, and a silent success hides the typo.
--create never replaces something already there, and never invents array elements.
merge, get, and set all read and write documents, so they chain: tot merge base.tot prod.tot | tot set replicas 3 | tot to json. None of them keep comments.
from json does no conversion. JSON is already tot, so it just reparses and reformats.
from writes a string as a """ block when it has line breaks and every line reads back
unchanged, so converted shell snippets and banners stay readable instead of collapsing into
one \n-laden line. A line ending in a space or tab falls back to a quoted literal.
The formatter preserves inline vs. block and never reflows — running fmt over minified
JSON gives you back one correctly-punctuated long line, not an exploded document. It unquotes
keys where legal, indents two spaces, and emits LF.
LF is part of the canonical form, so fmt rewrites a CRLF file. On Windows that matters:
with git's default core.autocrlf=true, fmt writes LF, git converts it back to CRLF on the
next checkout, and fmt --check reports the file unformatted forever — CI can never go green.
Check .tot files out with LF, as .gitattributes in this repo does:
*.tot text eol=lf
| JSON | Lossless both ways. Comments dropped on the way out. |
| YAML | In: aliases resolved and inlined; tags, non-string keys, and multi-document streams rejected. Out: loses nothing but float spellings, which normalize (1.00 → 1.0). |
| TOML | In: datetimes become strings. Out: nulls dropped (--null=error refuses instead), root must be an object, sub-tables hoisted below plain values because TOML's syntax demands it. |
let value = tot::parse(src)?; // -> Value
let text = tot::format(src)?; // canonical tot, source -> source
let text = tot::format_template(src)?; // the same, for a .tott source
let text = tot::format_value(&value); // Value -> tot
let folded = tot::merge(documents, tot::Nulls::Set); // or merge_into(&mut base, overlay, …)
let json = tot::json::to_string_pretty(&value);
let warns = tot::lint(src)?; // -> Vec<Warning>, all legal-but-risky
let warns = tot::lint_template(src)?; // the same, for a .tott source
let bad = tot::Schema::parse(shape)?.check(src)?; // -> Vec<Violation>
let at = tot::Path::parse("a.b[0]")?.get(&value)?;
let v = tot::parse_value(arg)?; // text in a value position, not a file
let doc = tot::Template::parse(tott)?.evaluate(¶ms)?; // or .build(¶ms, &mut imports)
let yaml = tot::yaml::to_string(&value)?; // behind the `yaml` feature
let read = tot::toml::from_str(src)?; // behind the `toml` featureTemplate keeps its own source and name, so a failure three imports deep still draws its caret
in the right file. The library does no I/O — (import …) goes through an Imports trait, so a
build can be driven from the filesystem, an archive, or a map in a test without any of them
being special.
parse_value uses the same grammar as parse; the difference is the diagnostic. A lone
bareword in a file is most likely a key that lost its value and is reported that way, but as a
value there's no key to lose one, so it's reported as a string needing quotes.
Path::parse and Path::get are separate calls because their failures are different problems
— a malformed path is your bug, a missing one is the document's answer. Spans on a path error
index into the path, not the document.
There is no JSON parser and there doesn't need to be — parse reads JSON directly.
Errors carry a span: e.render(src) gives a caret diagnostic.
To edit a parsed document, Value::get_mut chains and Map has get_mut / remove:
let mut config = tot::parse(src)?;
*config.get_mut("listen").unwrap().get_mut("port").unwrap() = Value::Integer(Integer::from_i64(9090));
config.as_object_mut().unwrap().remove("stale");Map::insert refuses a key that's already there and returns false — duplicates are a
parse error in the language, and last-wins isn't a rule tot has. Use get_mut to replace.
Numbers keep their original lexeme, so integers wider than i64 survive a round trip.
Integer equality is value equality, but Float equality is lexical — 1.0 != 1.00.
Compare as_f64() if you mean value equality.
The tot crate has no dependencies by default. serde, yaml, and toml are
off-by-default features, each pulling what it needs.
Behind the serde feature, off by default:
tot = { version = "0.2", features = ["serde"] }let config: Config = tot::from_str(src)?; // any DeserializeOwned
let text = tot::to_string(&config)?; // any Serialize
let value = tot::to_value(&config)?; // -> Value
let config: Config = tot::from_value(&value)?; // borrows, so `&'a str` fields workBoth directions go through Value rather than straight to text — the formatter and parser
already do that work well, and a streaming implementation would be a second copy of both.
- A value round-trips, not a document. Comments, blank lines, and inline-vs-block choices
live in the CST, which serde never sees.
to_stringwrites block form like any converter. - Errors name the offending value, spelled as a path:
invalid type: string "8080", expected u16 at `listen.port`. That string is a real path — hand it totot get.Error::path()gets it alone;Error::parse_error()gets the span when the document didn't parse at all. - Enums are externally tagged:
"off"for a unit variant,retry 5for anything else. nullisNone;falseand0areSome.1.0will not deserialize into au32— the language keeps integers and floats apart, and so does this. An integer will deserialize into anf64.- Integer map keys (
BTreeMap<u16, _>) work in both directions; float keys work in neither. - Infinity and NaN are a serialization error rather than a silent
null. Valueitself isSerialize + Deserialize, so part of a document can stay untyped.- Float lexemes are normalized — serde carries the number, not the spelling, so
1.comes back as1.0. SinceFloatequality is lexical, aValuewith such a lexeme is!=itself after a serde round trip. Integer lexemes are unaffected, up to 128 bits.
Behind the yaml and toml features, off by default, each pulling its parser:
tot = { version = "0.2", features = ["yaml", "toml"] }let value = tot::yaml::from_str(src)?; // -> Value
let text = tot::yaml::to_string(&value)?; // -> String
let read = tot::toml::from_str(src)?; // -> FromToml { value, datetimes }
let write = tot::toml::to_string(&v, tot::toml::NullPolicy::Omit)?; // -> ToToml { text, dropped }These are the same converters tot to and tot from run — the CLI has none of its own, so the
two cannot disagree. The side channels the CLI prints as notes are returned instead:
FromToml::datetimes names every datetime that became a string, and ToToml::dropped names
every null the policy dropped (NullPolicy::Error refuses instead). Every path either reports
is a real tot get path. Both directions carry values, not spellings: a float comes back in
its shortest form — 1.00 as 1.0 — and since Float equality is lexical, such a value is
not equal to itself after the trip. Errors are ConvertError, which carries the finished
message and no span, because the failure happened outside any tot source.
The toml feature turns on preserve_order for the whole build. Key order is the reason —
without it a converted table comes back alphabetized in both directions — but cargo resolves a
package's features once across everything it compiles, so a crate that depends on both tot
(with toml) and the toml crate directly gets preserve_order in its code too: toml::Table
becomes an IndexMap rather than a BTreeMap, and its own tables iterate and serialize in
insertion order instead of sorted order. Nothing fails to compile, and there is no way to opt
back out while depending on tot/toml — cargo features only ever add. If sorted toml::Table
iteration matters to you, sort at the point of use rather than relying on the map type.
src/ tot — library, zero dependencies
lex.rs tokenizer
parse.rs recursive descent -> Value
cst.rs lossless tree: comments, blank lines, inline/block choice
fmt.rs formatter (over the CST) + Value -> tot emitter
lint.rs opt-in checks (over the CST); nothing here is a language rule
merge.rs folding documents together, left to right
template.rs `.tott`: the data grammar plus forms, and the evaluator
path.rs `a.b[0]` paths — a CLI convenience, not part of the language
schema.rs checking a document's shape against a schema written in tot
json.rs JSON output only
yaml.rs YAML in and out, behind the `yaml` feature
toml.rs TOML in and out, behind the `toml` feature
convert.rs path spelling shared by the two converters
serde/ optional; ser.rs and de.rs, both via Value
value.rs Value, Integer, Float, Map
error.rs Span, Error, ConvertError, shared caret rendering
cli/ tot-cli — binary `tot`; no dependencies beyond tot
build.rs resolving `(import …)` against the filesystem
parse and format are separate walks over the same token stream. format validates with
parse first, then rebuilds a CST that keeps the trivia parse throws away — trivia is read
straight out of the gaps between token spans, so there's no second lexer.
cargo test --workspace --all-features
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo fmt --all --check--all-features matters — serde, yaml, and toml are off by default, and the tests
behind a feature that is off compile to nothing and pass silently.
-p tot matters too, for the opposite reason. tot-cli depends on the library with yaml
and toml on, and cargo resolves a package's features once across everything it is building, so
every --workspace command compiles the library with both converters present — there is no
such thing as a features-off --workspace build of them. Only -p tot resolves the library's
features on their own:
cargo clippy -p tot --all-targets --no-default-features -- -D warnings
cargo clippy -p tot --all-targets --features yaml -- -D warnings
cargo clippy -p tot --all-targets --features toml -- -D warningsThat is the direction the gate is easy to get wrong in, and the only way either converter gets built without the other.
Edition 2024, Rust 1.88. Formatter tests assert two properties on every fixture, not just expected output: formatting preserves the parsed value, and formatting is idempotent.
CI (.github/workflows/ci.yml) runs all of the above on Linux,
Windows, and macOS, plus three things a local run tends to skip: the MSRV the manifests claim,
the tool's own guarantees over examples/ (fmt --check, check --strict, build --check,
and the schema), and a publish dry run. That last one is not ceremony — a missing license and a
path dependency without a version are both invisible until you try to publish.
totlang.dev is built from site/ with Astro. The docs there are
these two files: /docs renders this README and /spec renders SPEC.md, so the site cannot fall
behind the language. The playground compiles the library to WebAssembly, so what it says about a
document — the formatting, the conversions, the schema violations, the caret diagnostics — is
what tot says about it.
cd site && pnpm install && pnpm buildThat needs the wasm32-unknown-unknown target and wasm-pack on top of the usual toolchain. The
wasm crate is excluded from the workspace, so cargo test --workspace never builds it and the
library goes on having no dependencies. site/README.md covers deployment.
tot merge, tot set, schema validation, the .tott template layer with all seven of its
forms, and fmt and check for templates are all built. Still open, in rough order:
- Enumerations in schemas. The most obvious missing check, with no good spelling yet —
["debug" "info"]already means an array of one element type, so it can't also mean a choice between two literals. The spec has the details. - In-place editing that keeps comments.
setandmergefold values, so they can't write back over a hand-written config without losing its comments. Doing that properly means splicing text through the CST, which is a different and larger feature. - A lint of its own for templates. A template gets the one lint the language has, since the parity hazard is the language's. Whether there's a second rule worth having — about where a form's arguments sit, say — is unmeasured.
Dual-licensed under either of
- Apache License, Version 2.0 (LICENSE-APACHE)
- MIT license (LICENSE-MIT)
at your option. This is the Rust ecosystem's convention: MIT for anyone who finds Apache-2.0's notice requirements heavy, Apache-2.0 for anyone whose legal team wants the explicit patent grant.
Unless you state otherwise, any contribution you intentionally submit for inclusion in this work shall be dual-licensed as above, with no additional terms or conditions.