tokenjam-fleet

A believable company fleet of runnable example LLM agents, each instrumented with OpenTelemetry. They emit standard gen_ai.* spans (token usage, model, cost) via OpenLLMetry and export them over OTLP to whatever observability backend you point them at.

The fleet belongs to a made-up company, Northwind AI, organized by department (Engineering, Marketing, Sales, Finance, Product, HR). Seven of its agents each deliberately exercise one LLM-cost anti-pattern — an oversized model for a trivial task, a long identical prompt prefix re-sent every call, a bloated system prompt, verbose output for a terse task, and so on — while the rest are healthy clean baselines, so the fleet looks like a real company: mostly well-behaved, with a few agents worth optimizing. Run the whole fleet once and watch a cost tool light up on all seven patterns at once.

Illustrative example code only: a made-up company (Northwind AI) with no real business data — no real customers, prospects, financials, or strategy. The anti-patterns are clearly-labeled teaching examples ("this agent deliberately …").

The reusable piece is tokenjam_fleet/otel.py — a one-call OpenTelemetry bootstrap you can drop into your own agent.

---

Quick start

1. Install

pip install -e .        # or: pip install -r requirements.txt

2. Point it at an OTLP backend

cp .env.example .env

Set the standard OTLP exporter variables — any OTLP-compatible backend works (a local OpenTelemetry Collector, a hosted platform, etc.):

OTEL_EXPORTER_OTLP_ENDPOINT=<your OTLP endpoint>
OTEL_EXPORTER_OTLP_HEADERS=<header auth, if your backend needs it>

Also set ANTHROPIC_API_KEY and/or OPENAI_API_KEY for the agents (an agent whose key is missing is skipped, not fatal).

3. Run the whole fleet once

python -m tokenjam_fleet

This drives every agent once so telemetry populates immediately. Prefer to validate without spending tokens? Build every prompt and emit nothing:

python -m tokenjam_fleet --dry-run   # no API calls, no keys, no backend needed

Or run a single agent:

python -m tokenjam_fleet.agents.invoice_parser

---

The fleet

Seventeen agents across six departments. Each agent sets its identity as the OTel service.name and its department as service.namespace (an OTel resource attribute), so any backend can group the fleet by team. Grouping keys on service.namespace.

Department	Agent	Task	Anti-pattern it exercises	Analyzer it lights up
Engineering	pr-summarizer	one-line PR summaries	— (clean baseline)	—
Engineering	code-reviewer	review tiny snippets	spawns a sub-agent on an oversized model for light work	subagent
Engineering	ci-triage-bot	one-line CI-failure cause	— (clean baseline)	—
Engineering	test-generator	one unit test for a small function	— (clean baseline)	—
Marketing	content-drafter	short blog intro	— (clean baseline)	—
Marketing	seo-analyzer	write a meta description	verbose output for a task needing one terse line	verbosity
Marketing	campaign-copy	one short ad headline	— (clean baseline)	—
Sales	lead-qualifier	score inbound leads	same fixed (tool, arg-shape) per lead	script
Sales	email-drafter	draft cold emails	re-derives the same plan skeleton every run	reuse
Sales	deal-summarizer	two-sentence deal status	— (clean baseline)	—
Finance	invoice-parser	extract invoice fields	re-sends a long identical prefix every call, no cache_control	cache
Finance	expense-categorizer	categorize expenses	bloated system prompt full of low-signal text	trim
Product	roadmap-assistant	trivial KB lookup (OpenAI Agents SDK)	runs on an oversized model for a tiny task	downsize
Product	spec-writer	feature idea → 3-bullet spec	— (clean baseline)	—
HR	resume-screener	score a resume against a role	— (clean baseline)	—
HR	policy-qa-bot	answer a short HR-policy question	— (clean baseline)	—
HR	onboarding-assistant	a new-hire day-1 checklist item	— (clean baseline)	—

Seven anti-patterns; the other ten agents are clean baselines for contrast — a believable company is mostly well-behaved agents with a few worth optimizing.

These are structural patterns, not guarantees — an oversized model is a candidate to review, not proof a smaller one would have worked. A cost tool such as TokenJam surfaces them as downsize / cache / script / trim / reuse / subagent / verbosity findings, framed as estimated recoverable, never guaranteed savings.

---

Instrument your own agent

Two lines:

from tokenjam_fleet.otel import init_telemetry   # or paste otel.py into your app

init_telemetry("my_agent", department="engineering")
# everything after this emits gen_ai.* over OTLP

init_telemetry(agent_name, department) sets service.name (the agent's identity) and service.namespace (the owning team). Under the hood the bootstrap installs OpenLLMetry — which patches the Anthropic / OpenAI SDKs to emit OTel GenAI-semconv spans — and points the OTLP/HTTP exporter at your configured endpoint.

---

OTLP config

The bootstrap reads the standard OTLP exporter environment (OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS) and passes it to Traceloop.init(...). No Traceloop SaaS key is required — the endpoint override routes OTLP to your own backend. Header auth (if any) goes in OTEL_EXPORTER_OTLP_HEADERS as comma-separated key=value pairs.

Point it at TokenJam Cloud

Set the endpoint to your Cloud ingest URL and the auth headers to your org + ingest key (see .env.example for a worked example):

OTEL_EXPORTER_OTLP_ENDPOINT=<your TokenJam Cloud ingest URL>
OTEL_EXPORTER_OTLP_HEADERS=X-TokenJam-Org=<your-org-id>,Authorization=Bearer <your-ingest-key>

Then python -m tokenjam_fleet — the fleet shows up grouped by department, with a finding on each anti-pattern agent.

---

Links

• OpenLLMetry: https://github.com/traceloop/openllmetry
• TokenJam (open-source LLM cost tool): https://github.com/Metabuilder-Labs/tokenjam

License

MIT — see LICENSE.