Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
1ed3eea
mr-668: add GraphId newtype + Cloud-mode forward identity stubs (PR 1…
ragnorc May 25, 2026
3fb6fa3
mr-668: Omnigraph::init error-path cleanup + three failpoints (PR 2a/10)
ragnorc May 25, 2026
662cf67
mr-668: add GraphHandle + GraphRegistry data structure (PR 3/10)
ragnorc May 25, 2026
57f5f19
mr-668: router restructure + handler refactor for multi-graph (PR 4a/10)
ragnorc May 25, 2026
ecf01ef
mr-668: OpenAPI multi-mode cluster filter (PR 4b/10)
ragnorc May 25, 2026
4df361d
mr-668: multi-graph startup + mode inference (PR 5/10)
ragnorc May 25, 2026
0e5aa03
mr-668: Cedar resource-model refactor (PR 6a/10)
ragnorc May 25, 2026
94b6346
mr-668: GET /graphs endpoint + per-graph policy wire-up (PR 6b/10)
ragnorc May 25, 2026
a4e6cb6
mr-668: POST /graphs runtime create endpoint (PR 7/10)
ragnorc May 25, 2026
75514b6
mr-668: CLI omnigraph graphs list/create (PR 8/10)
ragnorc May 25, 2026
d11c18f
mr-668: composite e2e tests, race fix, v0.7.0 release (PR 9/10)
ragnorc May 25, 2026
937fd63
mr-668: remove POST /graphs and CLI graphs create (defer runtime grap…
ragnorc May 26, 2026
52f28ce
mr-668: comment cleanup and policy format style
ragnorc May 27, 2026
76ee061
mr-668: drop actor_id from PolicyRequest; pass actor as separate arg
ragnorc May 27, 2026
fc95eb8
mr-668: consolidate AppState single-mode constructors; delete with_po…
ragnorc May 27, 2026
ca474e2
mr-668: derive any_per_graph_policy on RegistrySnapshot; simplify dup…
ragnorc May 27, 2026
0da750c
mr-668: fail-fast multi-graph startup with try_collect
ragnorc May 27, 2026
dcb2dde
mr-668: drop unused State extractor from 7 read-only handlers
ragnorc May 27, 2026
dedb633
mr-668: emit info! for graph routing decision
ragnorc May 27, 2026
5b19a03
mr-668: align GET /graphs 405 body code with HTTP status
ragnorc May 27, 2026
6341feb
mr-668: drop unused axum::handler::Handler import
ragnorc May 27, 2026
d37173a
mr-668: drop unused fs2 workspace dependency
ragnorc May 27, 2026
eb0187e
mr-668: AGENTS.md Cedar row no longer hardcodes action count
ragnorc May 27, 2026
0b89342
mr-668: cfg(test)-gate GraphRegistry::insert and its mutex
ragnorc May 27, 2026
2bb6e24
mr-668: drop vestigial PolicyEngine surface
ragnorc May 27, 2026
094e868
mr-668: regression test for init re-init footgun (red)
ragnorc May 27, 2026
67a4652
mr-668: close init re-init footgun via InitOptions preflight (green)
ragnorc May 27, 2026
4e2f18a
mr-668: split PolicyEngine::load into kind-typed loaders
ragnorc May 27, 2026
c95ecff
mr-668: introduce GraphRouting, retire single_mode_handle
ragnorc May 27, 2026
00522d6
mr-668: remove ServerMode, registry field, and the SINGLE_GRAPH sentinel
ragnorc May 27, 2026
7ec448a
Merge branch 'main' into ragnorc/validate-impl-plan
ragnorc May 27, 2026
7d19a76
mr-668: regression test for nested-route path extraction (red)
ragnorc May 27, 2026
8f558e6
mr-668: named-field path-param structs for nested cluster routes (green)
ragnorc May 27, 2026
4120448
mr-668: centralize policy-requires-tokens check in the runtime classi…
ragnorc May 27, 2026
ae4121b
mr-668: close coverage gaps surfaced by the test-coverage audit
ragnorc May 27, 2026
c5a6a71
mr-668: regression test for GraphList open-mode bypass (red)
ragnorc May 28, 2026
65e5393
mr-668: server-scoped actions always require explicit policy (green)
ragnorc May 28, 2026
1aafbd9
mr-668: fold multi-graph work into v0.6.0 (no separate v0.7.0 release)
ragnorc May 28, 2026
f3d962b
Merge branch 'main' into ragnorc/validate-impl-plan
ragnorc May 28, 2026
37ec737
mr-668: sanitize internal refs from v0.6.0 release notes
ragnorc May 28, 2026
eab99e6
fix: close validated init and multi-graph gaps
ragnorc May 28, 2026
4580ec0
chore: address review cleanup comments
ragnorc May 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Tools that support `@`-imports (Claude Code) auto-include all three files via th
`CLAUDE.md` is a symlink to this file — there is exactly one source of truth. Edit `AGENTS.md`.

**Version surveyed:** 0.6.0
**Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-cli`, `omnigraph-server`
**Workspace crates:** `omnigraph-compiler`, `omnigraph` (engine), `omnigraph-policy`, `omnigraph-cli`, `omnigraph-server`
**Storage substrate:** Lance 6.x (columnar, versioned, branchable)
**License:** MIT
**Toolchain:** Rust stable, edition 2024
Expand All @@ -33,7 +33,7 @@ OmniGraph is a typed property-graph engine built as a coordination layer over ma
- **Multi-modal querying**: vector ANN (`nearest`), full-text (`search`/`fuzzy`/`match_text`/`bm25`), Reciprocal Rank Fusion (`rrf`), and graph traversal (`Expand`, anti-join `not { … }`) in one runtime.
- **Branches and commits across the whole graph**: Git-style — every successful publish appends to a commit DAG; merges are three-way at the row level.
- **Atomic per-query writes**: `mutate_as` and `load` accumulate insert/update batches into an in-memory `MutationStaging.pending` per touched table; one `stage_*` + `commit_staged` per table runs at end-of-query, then `ManifestBatchPublisher::publish` commits the manifest atomically with per-table `expected_table_versions` CAS. A mid-query failure leaves Lance HEAD untouched on staged tables — no drift, no run state machine, no staging branches. Deletes still inline-commit; D₂ at parse time prevents inserts/updates and deletes from coexisting in one query.
- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate.
- **HTTP server**: Axum + utoipa OpenAPI, bearer auth (SHA-256 hashed, optional AWS Secrets Manager). Cedar policy enforcement is engine-wide — every `_as` writer calls `Omnigraph::enforce(action, scope, actor)`, so HTTP, CLI, and embedded SDK consumers all hit the same gate. **Two modes** (v0.6.0+): single-graph (legacy flat routes) and multi-graph (`/graphs/{graph_id}/...` cluster routes + read-only `GET /graphs` enumeration). Per-graph + server-level Cedar policies. Runtime add/remove (`POST /graphs`, `DELETE /graphs/{id}`) is not exposed — operators edit `omnigraph.yaml` and restart.
- **CLI** driven by a single `omnigraph.yaml`; multi-format output (json/jsonl/csv/kv/table).

Throughout the docs, capabilities are split into **L1 — Inherited from Lance** vs **L2 — Added by OmniGraph**.
Expand Down Expand Up @@ -226,8 +226,8 @@ omnigraph policy explain --actor act-alice --action change --branch main
| Per-query atomic writes | — | In-memory `MutationStaging.pending` accumulator + `stage_*` / `commit_staged` per touched table at end-of-query + publisher CAS via `commit_with_expected` (single manifest commit per `mutate_as` / `load`); D₂ parse-time rule keeps inserts/updates and deletes from mixing |
| Three-way row-level merge | — | `OrderedTableCursor` + `StagedTableWriter`, structured `MergeConflictKind` |
| Change feeds | — | `diff_between` / `diff_commits` with manifest fast path + ID streaming |
| Cedar policy | — | 8 actions, branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as`, `ingest_as`, `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. |
| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export |
| Cedar policy | — | Per-graph actions plus server-scoped actions (see [docs/user/policy.md](docs/user/policy.md) for the current list), branch / target_branch / protected scopes, validate/test/explain CLI. **Engine-wide enforcement** (MR-722): every `_as` writer (`apply_schema_as`, `mutate_as`, `load_as`, `ingest_as`, `branch_create_as` / `branch_create_from_as`, `branch_delete_as`, `branch_merge_as`) calls `Omnigraph::enforce(action, scope, actor)` — HTTP, CLI, embedded SDK all hit the same gate. |
| HTTP server | — | Axum, OpenAPI via utoipa, bearer auth (SHA-256, AWS Secrets Manager option), `authorize_request` at the HTTP boundary (resolves bearer→actor, applies admission control), NDJSON streaming export, **multi-graph mode (v0.6.0+) with cluster routes + read-only `GET /graphs` enumeration + per-graph + server-level Cedar policies. Add/remove graphs by editing `omnigraph.yaml` and restarting.** |
| CLI with config | — | `omnigraph.yaml`, aliases, multi-format output (json/jsonl/csv/kv/table) |
| Audit / actor tracking | — | `_as` write APIs + actor map in commit graph |
| Local RustFS bootstrap | — | `scripts/local-rustfs-bootstrap.sh` one-shot S3-backed dev environment |
Expand Down
3 changes: 3 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

96 changes: 85 additions & 11 deletions crates/omnigraph-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,10 @@ use omnigraph_compiler::{
use omnigraph_server::api::{
BranchCreateOutput, BranchCreateRequest, BranchDeleteOutput, BranchListOutput,
BranchMergeOutput, BranchMergeRequest, ChangeOutput, ChangeRequest, CommitListOutput,
CommitOutput, ErrorOutput, ExportRequest, IngestOutput, IngestRequest, ReadOutput, ReadRequest,
SchemaApplyOutput, SchemaApplyRequest, SchemaOutput, SnapshotOutput, SnapshotTableOutput,
commit_output, ingest_output, read_output, schema_apply_output, snapshot_payload,
CommitOutput, ErrorOutput, ExportRequest, GraphListResponse, IngestOutput, IngestRequest,
ReadOutput, ReadRequest, SchemaApplyOutput, SchemaApplyRequest, SchemaOutput, SnapshotOutput,
SnapshotTableOutput, commit_output, ingest_output, read_output, schema_apply_output,
snapshot_payload,
};
use omnigraph_server::{
AliasCommand, OmnigraphConfig, PolicyAction, PolicyDecision, PolicyEngine, PolicyRequest,
Expand Down Expand Up @@ -72,6 +73,13 @@ enum Command {
schema: PathBuf,
/// Graph URI (local path or s3://)
uri: String,
/// Overwrite existing schema artifacts at the URI. Without
/// this flag, init refuses to touch a URI that already holds
/// `_schema.pg`, `_schema.ir.json`, or `__schema_state.json`
/// — closes the re-init footgun (MR-668 follow-up). With the
/// flag, the operator opts in to destructive semantics.
#[arg(long)]
force: bool,
},
/// Load data into a graph
Load {
Expand Down Expand Up @@ -253,6 +261,33 @@ enum Command {
#[arg(long)]
json: bool,
},
/// Manage graphs on a multi-graph server (MR-668)
Graphs {
#[command(subcommand)]
command: GraphsCommand,
},
}

/// Operations on the graph registry of a multi-graph server (MR-668).
///
/// All operations target a remote multi-graph server URL (http:// or
/// https://). Local-URI invocations return a clear error. To add or
/// remove graphs, operators edit `omnigraph.yaml` directly and restart
/// the server — runtime mutation is not exposed in v0.6.0.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Doc comment incorrectly references v0.6.0 for a command that only exists in v0.7.0. Should say v0.7.0 — the multi-graph release where runtime mutation is deferred.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At crates/omnigraph-cli/src/main.rs, line 276:

<comment>Doc comment incorrectly references v0.6.0 for a command that only exists in v0.7.0. Should say v0.7.0 — the multi-graph release where runtime mutation is deferred.</comment>

<file context>
@@ -273,7 +273,7 @@ enum Command {
 /// https://). Local-URI invocations return a clear error. To add or
 /// remove graphs, operators edit `omnigraph.yaml` directly and restart
-/// the server — runtime mutation is not exposed in v0.7.0.
+/// the server — runtime mutation is not exposed in v0.6.0.
 #[derive(Debug, Subcommand)]
 enum GraphsCommand {
</file context>

#[derive(Debug, Subcommand)]
enum GraphsCommand {
/// List every graph registered with the multi-graph server.
List {
/// Remote server URL (e.g. `https://server.example.com`).
#[arg(long)]
uri: Option<String>,
#[arg(long)]
target: Option<String>,
#[arg(long)]
config: Option<PathBuf>,
#[arg(long)]
json: bool,
},
}

#[derive(Debug, Subcommand)]
Expand Down Expand Up @@ -694,7 +729,7 @@ fn resolve_policy_engine(config: &OmnigraphConfig) -> Result<PolicyEngine> {
let policy_file = config
.resolve_policy_file()
.ok_or_else(|| color_eyre::eyre::eyre!("policy.file must be set in omnigraph.yaml"))?;
PolicyEngine::load(&policy_file, &policy_graph_id(config))
PolicyEngine::load_graph(&policy_file, &policy_graph_id(config))
}

/// Open a local-URI graph and, when `policy.file` is configured in
Expand Down Expand Up @@ -1305,12 +1340,12 @@ fn print_commit_human(commit: &CommitOutput) {
println!("created_at: {}", commit.created_at);
}

fn print_policy_explain(decision: &PolicyDecision, request: &PolicyRequest) {
fn print_policy_explain(decision: &PolicyDecision, actor_id: &str, request: &PolicyRequest) {
println!(
"decision: {}",
if decision.allowed { "allow" } else { "deny" }
);
println!("actor: {}", request.actor_id);
println!("actor: {}", actor_id);
println!("action: {}", request.action);
if let Some(branch) = &request.branch {
println!("branch: {}", branch);
Expand Down Expand Up @@ -1718,10 +1753,15 @@ async fn main() -> Result<()> {
print_embed_human(&output);
}
}
Command::Init { schema, uri } => {
Command::Init { schema, uri, force } => {
let schema_source = fs::read_to_string(&schema)?;
ensure_local_graph_parent(&uri)?;
Omnigraph::init(&uri, &schema_source).await?;
Omnigraph::init_with_options(
&uri,
&schema_source,
omnigraph::db::InitOptions { force },
)
.await?;
scaffold_config_if_missing(&uri)?;
println!("initialized {}", uri);
}
Expand Down Expand Up @@ -2443,13 +2483,12 @@ async fn main() -> Result<()> {
let config = load_cli_config(config.as_ref())?;
let engine = resolve_policy_engine(&config)?;
let request = PolicyRequest {
actor_id: actor,
action,
branch,
target_branch,
};
let decision = engine.authorize(&request)?;
print_policy_explain(&decision, &request);
let decision = engine.authorize(&actor, &request)?;
print_policy_explain(&decision, &actor, &request);
}
},
Command::Optimize {
Expand Down Expand Up @@ -2556,6 +2595,41 @@ async fn main() -> Result<()> {
);
}
}
Command::Graphs { command } => match command {
GraphsCommand::List {
uri,
target,
config,
json,
} => {
let config = load_cli_config(config.as_ref())?;
let bearer_token =
resolve_remote_bearer_token(&config, uri.as_deref(), target.as_deref())?;
let uri = resolve_uri(&config, uri, target.as_deref())?;
if !is_remote_uri(&uri) {
bail!(
"`omnigraph graphs list` requires a remote multi-graph server URL \
(http:// or https://). To enumerate local graphs, read `omnigraph.yaml` \
directly."
);
}
let payload = remote_json::<GraphListResponse>(
&http_client,
Method::GET,
remote_url(&uri, "/graphs"),
None,
bearer_token.as_deref(),
)
.await?;
if json {
print_json(&payload)?;
} else {
for entry in payload.graphs {
println!("{}\t{}", entry.graph_id, entry.uri);
}
}
}
},
}
Ok(())
}
Expand Down
44 changes: 44 additions & 0 deletions crates/omnigraph-cli/tests/cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2040,3 +2040,47 @@ fn schema_plan_parity_cli_and_sdk() {
);
assert_eq!(cli_payload["supported"], plan.supported);
}

// ─── MR-668 PR 8 — omnigraph graphs subcommand ─────────────────────────────

/// `omnigraph graphs --help` lists only the read-only `list`
/// subcommand. Runtime add (`create`) and remove (`delete`) are
/// deferred — operators add/remove graphs by editing `omnigraph.yaml`
/// and restarting. This test pins the deferral against accidental
/// re-introduction.
#[test]
fn graphs_subcommand_help_lists_list_only() {
let output = output_success(cli().arg("graphs").arg("--help"));
let stdout = stdout_string(&output);
assert!(
stdout.contains("list"),
"expected `list` subcommand in help output:\n{stdout}"
);
let lowered = stdout.to_lowercase();
assert!(
!lowered.contains("create a new graph"),
"graph create should not be in v0.6.0 help; got:\n{stdout}"
);
assert!(
!lowered.contains("delete a graph"),
"graph delete should not be in v0.6.0 help; got:\n{stdout}"
);
}

/// `omnigraph graphs list` against a local URI errors with a clear
/// message — the CLI only operates against remote multi-graph servers.
#[test]
fn graphs_list_against_local_uri_errors_with_remote_only_message() {
let output = output_failure(
cli()
.arg("graphs")
.arg("list")
.arg("--uri")
.arg("/tmp/local"),
);
let stderr = String::from_utf8_lossy(&output.stderr).into_owned();
assert!(
stderr.contains("remote multi-graph server URL"),
"expected 'remote multi-graph server URL' rejection in stderr; got:\n{stderr}"
);
}
120 changes: 120 additions & 0 deletions crates/omnigraph-cli/tests/system_remote.rs
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,17 @@ rules:
target_branch_scope: protected
"#;

const GRAPH_LIST_SERVER_POLICY_YAML: &str = r#"
version: 1
groups:
admins: [act-admin]
rules:
- id: admins-can-list-graphs
allow:
actors: { group: admins }
actions: [graph_list]
"#;

fn yaml_string(value: &str) -> String {
format!("'{}'", value.replace('\'', "''"))
}
Expand Down Expand Up @@ -888,3 +899,112 @@ query insert_person($name: String, $age: I32) {
assert_eq!(verify["row_count"], 1);
assert_eq!(verify["rows"][0]["p.name"], "PolicyRemote");
}

// ─── MR-668 PR 8 — omnigraph graphs list end-to-end ────────────────────────

/// Multi-graph server + CLI `omnigraph graphs list` end-to-end.
///
/// Steps:
/// 1. Init a graph `alpha` on disk and write an `omnigraph.yaml`
/// whose `graphs:` map references it.
/// 2. Spawn the server with `--config <yaml>`.
/// 3. `omnigraph graphs list` — expect to see `alpha`.
///
/// Ignored by default — spawning servers needs loopback socket
/// permissions some sandboxes lack.
#[test]
#[ignore = "requires loopback socket permissions in sandboxed runners"]
fn graphs_list_against_multi_graph_server() {
let cfg_dir = tempfile::tempdir().unwrap();
let schema_path = fixture("test.pg");

// Init `alpha` on disk.
let alpha_uri = cfg_dir.path().join("alpha.omni");
tokio::runtime::Runtime::new().unwrap().block_on(async {
Omnigraph::init(
alpha_uri.to_str().unwrap(),
&fs::read_to_string(&schema_path).unwrap(),
)
.await
.unwrap();
});

fs::write(
cfg_dir.path().join("server-policy.yaml"),
GRAPH_LIST_SERVER_POLICY_YAML,
)
.unwrap();

// Server config with `graphs:` map and no `server.graph` selector
// — multi mode (rule 4 of the inference matrix). `GET /graphs` is a
// server-scoped action, so the success path needs an explicit server
// policy and bearer token.
let server_config_path = cfg_dir.path().join("omnigraph.yaml");
fs::write(
&server_config_path,
format!(
"\
server:
policy:
file: ./server-policy.yaml
graphs:
alpha:
uri: {}
",
yaml_string(&alpha_uri.to_string_lossy())
),
)
.unwrap();

let server = spawn_server_with_config_env(
&server_config_path,
&[(
"OMNIGRAPH_SERVER_BEARER_TOKENS_JSON",
r#"{"act-admin":"admin-token"}"#,
)],
);

// Client config — the CLI's `--target dev` resolves to `server.base_url`.
let client_config_path = cfg_dir.path().join("client.yaml");
fs::write(
&client_config_path,
format!(
"\
graphs:
dev:
uri: {}
bearer_token_env: GRAPH_LIST_TOKEN
cli:
graph: dev
auth:
env_file: ./.env.omni
",
yaml_string(&server.base_url)
),
)
.unwrap();
fs::write(
cfg_dir.path().join(".env.omni"),
"GRAPH_LIST_TOKEN=admin-token\n",
)
.unwrap();

// `graphs list` lists `alpha`.
let payload = parse_stdout_json(&output_success(
cli()
.arg("graphs")
.arg("list")
.arg("--config")
.arg(&client_config_path)
.arg("--json"),
));
let ids: Vec<&str> = payload["graphs"]
.as_array()
.unwrap()
.iter()
.map(|g| g["graph_id"].as_str().unwrap())
.collect();
assert_eq!(ids, vec!["alpha"]);

drop(server);
}
Loading