Skip to content

feat: extract @kilocode/worker-utils shared package#683

Merged
iscekic merged 17 commits intomainfrom
chore/worker-utils
Mar 1, 2026
Merged

feat: extract @kilocode/worker-utils shared package#683
iscekic merged 17 commits intomainfrom
chore/worker-utils

Conversation

@iscekic
Copy link
Contributor

@iscekic iscekic commented Feb 28, 2026

Summary

Utilities extracted

# Utility Copies replaced Description
1 withDORetry 6 DO retry with exponential backoff
2 backendAuthMiddleware 5 Hono bearer auth middleware
3 withTimeout 2 Async operation timeout
4 createR2Client 2 R2 presigned URL factory
5 resSuccess/resError 2 Typed JSON response helpers
6 zodJsonValidator 2 Hono Zod body validator
7 timingSafeEqual 3 Timing-safe string comparison
8 formatError 2 Error formatting for logging
9 extractBearerToken 7 Bearer header extraction
10 createErrorHandler 5 Hono app.onError factory
11 createNotFoundHandler 5 Hono app.notFound factory
12 Owner type 3 Shared owner type
13 MCPServerConfig type 2 Shared MCP server config type

Design decisions

  • withDORetry logger injection: The shared implementation accepts an optional logger parameter (defaults to console). Workers with structured loggers (cloud-agent, cloud-agent-next, webhook-agent-ingest) use thin local wrappers that pre-bind their WorkersLogger, preserving structured logging without touching call sites.
  • createErrorHandler includeMessage option: Defaults to true (includes message: err.message in 500 responses). Workers that previously had no message field (cloud-agent-next, deploy-infra/builder) opt out with { includeMessage: false }.
  • extractBearerToken: Trims the header before slicing, case-insensitive prefix match per RFC 6750.
  • backendAuthMiddleware: Migrated in all 5 workers that had identical inline auth (auto-fix, auto-triage, code-review, deploy-infra/builder, plus the existing session-proxy usage).

Dead dependency cleanup

  • Removed aws4fetch from cloud-agent and cloud-agent-next (moved to worker-utils)
  • Removed @hono/zod-validator from cloudflare-o11y (never imported)
  • Removed zod from cloudflare-auto-fix-infra (never imported)

Not migrated (intentionally)

Follow-up

Create packages/worker-utils/ with 14 shared utilities extracted from
individual workers, eliminating ~2900 lines of duplicated code:

- withDORetry: DO retry with exponential backoff (6 copies → 1)
- backendAuthMiddleware: Hono bearer auth (4 copies → 1)
- withTimeout: async operation timeout (2 copies → 1)
- createR2Client: R2 presigned URL factory (2 copies → 1)
- resSuccess/resError: typed JSON response helpers (2 copies → 1)
- zodJsonValidator: Hono Zod body validator (2 copies → 1)
- timingSafeEqual: timing-safe string comparison (3 copies → 1)
- formatError: error formatting for logging (2 copies → 1)
- extractBearerToken: Bearer header extraction (7 inline → 1)
- encryption: Web Crypto port of RSA-OAEP + AES-256-GCM envelope
  encryption, replacing node:crypto impl (3 copies → 1)
- createErrorHandler/createNotFoundHandler: Hono handler factories
- Owner, MCPServerConfig shared types

Also removes dead deps (@hono/zod-validator from o11y, zod from
auto-fix) and moves aws4fetch from cloud-agent/cloud-agent-next
into worker-utils.
@iscekic iscekic self-assigned this Feb 28, 2026
@kilo-code-bot
Copy link
Contributor

kilo-code-bot bot commented Feb 28, 2026

Code Review Summary

Status: 2 Suggestions | Recommendation: Merge (address suggestions as follow-up)

Overview

Severity Count
CRITICAL 0
WARNING 0
SUGGESTION 2
Other Observations (not in diff)

Issues found in unchanged code or about missing files that cannot receive inline comments:

File Line Issue
packages/worker-utils/src/ [SUGGESTION]: Missing unit tests — only do-retry.test.ts exists. extractBearerToken, backendAuthMiddleware, createErrorHandler, zodJsonValidator, and formatError have no tests. extractBearerToken in particular handles security-sensitive parsing (RFC 6750 bearer token extraction) and would benefit from edge-case coverage.
packages/worker-utils/src/do-retry.ts 32 [SUGGESTION]: as RetryableError cast — per project coding style, as should be avoided. Could use 'retryable' in error && (error as {retryable: unknown}).retryable === true or a type guard function. This is pre-existing code that was moved, not newly introduced.

Notes

This is a well-executed refactoring PR that consolidates duplicated utilities from 8+ workers into a shared @kilocode/worker-utils package. Key observations:

  • Deduplication is thorough: withDORetry, createR2Client, withTimeout, resSuccess/resError, formatError, zodJsonValidator, backendAuthMiddleware, extractBearerToken, createErrorHandler, createNotFoundHandler, Owner, and MCPServerConfig are all properly centralized.
  • Behavioral consistency preserved: Workers that previously used console for logging still use console (via the default logger parameter). Workers with structured loggers (cloud-agent, cloud-agent-next, webhook-agent-ingest) pass their logger through local wrappers.
  • scheduler.wait() fallback: The shared do-retry.ts now gracefully falls back to setTimeout when scheduler is unavailable, which is correct for both Cloudflare Workers and test environments.
  • Type changes are safe: Owner and MCPServerConfig changed from interface to type (per project coding style preference). ApiResponse changed from { success: boolean; data?; error? } to a discriminated union — callers only use resSuccess/resError functions, so this is compatible.
  • Jest config properly updated: moduleNameMapper for @kilocode/worker-utils, .js extension stripping, and testPathIgnorePatterns are all correct.
  • typecheck script now runs recursively: pnpm -r --filter '!kilocode-backend' run typecheck ensures all workspace packages are type-checked.
  • z.any() in gastown: The change from z.unknown() to z.any() is well-documented — it's needed for Cloudflare's Rpc.Serializable<T> type inference.

Previous review comments about setTimeout vs scheduler.wait(), unused dependencies, duplicate imports, and as ArrayBuffer casts appear to have been addressed in follow-up commits.

Files Reviewed (78 files)
  • packages/worker-utils/package.json — new shared package
  • packages/worker-utils/tsconfig.json — new
  • packages/worker-utils/src/index.ts — barrel exports
  • packages/worker-utils/src/do-retry.ts — consolidated DO retry logic
  • packages/worker-utils/src/do-retry.test.ts — moved + updated tests
  • packages/worker-utils/src/backend-auth-middleware.ts — new shared middleware
  • packages/worker-utils/src/error-handler.ts — new shared error handler
  • packages/worker-utils/src/not-found-handler.ts — new shared 404 handler
  • packages/worker-utils/src/extract-bearer-token.ts — new shared token extractor
  • packages/worker-utils/src/format-error.ts — consolidated error formatter
  • packages/worker-utils/src/r2-client.ts — moved R2 client
  • packages/worker-utils/src/res.ts — consolidated response helpers
  • packages/worker-utils/src/timeout.ts — moved timeout utility
  • packages/worker-utils/src/types.ts — shared types (Owner, MCPServerConfig)
  • packages/worker-utils/src/zod-json-validator.ts — consolidated validator
  • packages/db/package.json — added typecheck script
  • package.json — added worker-utils dep + recursive typecheck
  • pnpm-workspace.yaml — added worker-utils package
  • jest.config.ts — module mapping + ignore patterns
  • pnpm-lock.yaml — lock file updates
  • cloud-agent/package.json — dep changes
  • cloud-agent/src/streaming.ts — import update
  • cloud-agent/src/utils/do-retry.ts — thin wrapper
  • cloud-agent/src/utils/do-retry.test.ts — deleted (moved)
  • cloud-agent/src/utils/image-download.ts — import update
  • cloud-agent/src/utils/timeout.ts — deleted (moved)
  • cloud-agent/src/workspace.ts — import update
  • cloud-agent-next/package.json — dep changes
  • cloud-agent-next/src/server.ts — use shared handlers
  • cloud-agent-next/src/streaming.ts — import update
  • cloud-agent-next/src/utils/do-retry.ts — thin wrapper
  • cloud-agent-next/src/utils/image-download.ts — import update
  • cloud-agent-next/src/utils/r2-client.ts — deleted (moved)
  • cloud-agent-next/src/workspace.ts — import update
  • cloudflare-ai-attribution/package.json — added dep
  • cloudflare-ai-attribution/src/ai-attribution.worker.ts — import update
  • cloudflare-ai-attribution/src/util/res.ts — deleted (moved)
  • cloudflare-app-builder/package.json — added dep
  • cloudflare-app-builder/src/utils/logger.ts — re-export formatError
  • cloudflare-auto-fix-infra/package.json — dep changes
  • cloudflare-auto-fix-infra/src/index.ts — use shared middleware/handlers
  • cloudflare-auto-fix-infra/src/types.ts — import shared types
  • cloudflare-auto-triage-infra/package.json — added dep
  • cloudflare-auto-triage-infra/src/index.ts — use shared middleware/handlers
  • cloudflare-auto-triage-infra/src/types.ts — import shared types
  • cloudflare-code-review-infra/package.json — added dep
  • cloudflare-code-review-infra/src/index.ts — use shared middleware/handlers
  • cloudflare-code-review-infra/src/types.ts — import shared types
  • cloudflare-db-proxy/package.json — added dep
  • cloudflare-db-proxy/src/logger.ts — re-export formatError
  • cloudflare-deploy-infra/builder/package.json — added dep
  • cloudflare-deploy-infra/builder/src/index.ts — use shared middleware/handlers
  • cloudflare-gastown/package.json — added dep
  • cloudflare-gastown/container/package.json — typecheck fix
  • cloudflare-gastown/src/db/tables/agent-metadata.table.ts — z.any() fix
  • cloudflare-gastown/src/db/tables/beads.table.ts — z.any() fix
  • cloudflare-gastown/src/middleware/auth.middleware.ts — use extractBearerToken
  • cloudflare-gastown/src/middleware/mayor-auth.middleware.ts — use extractBearerToken
  • cloudflare-gastown/src/types.ts — checkpoint: any
  • cloudflare-gastown/src/util/res.util.ts — re-export from worker-utils
  • cloudflare-o11y/package.json — replaced @hono/zod-validator
  • cloudflare-o11y/src/alerting/config-routes.ts — import update
  • cloudflare-o11y/src/api-metrics-routes.ts — import update
  • cloudflare-o11y/src/util/validation.ts — deleted (moved)
  • cloudflare-session-ingest/package.json — dep changes
  • cloudflare-session-ingest/src/index.ts — import update
  • cloudflare-session-ingest/src/routes/api.ts — consolidated imports
  • cloudflare-session-ingest/src/services/session-export.ts — import update
  • cloudflare-session-ingest/src/session-ingest-rpc.ts — import update
  • cloudflare-session-ingest/src/util/do-retry.test.ts — deleted (moved)
  • cloudflare-session-ingest/src/util/do-retry.ts — deleted (moved)
  • cloudflare-webhook-agent-ingest/package.json — added dep
  • cloudflare-webhook-agent-ingest/src/index.ts — import update
  • cloudflare-webhook-agent-ingest/src/routes/api.ts — import update
  • cloudflare-webhook-agent-ingest/src/routes/callbacks.ts — import update
  • cloudflare-webhook-agent-ingest/src/routes/inbound.ts — import update
  • cloudflare-webhook-agent-ingest/src/util/auth.ts — import update
  • cloudflare-webhook-agent-ingest/src/util/do-retry.ts — thin wrapper
  • kiloclaw/package.json — added dep
  • kiloclaw/src/routes/platform.ts — import update
  • kiloclaw/src/util/do-retry.ts — deleted (moved)

Fix these issues in Kilo Cloud

…cryption

- Restore JSDoc comments on do-retry (Cloudflare retry semantics, backoff
  formula, @param/@returns/@throws/@example, .overloaded note)
- Add do-retry.test.ts with 18 tests (ported from cloud-agent-next + session-ingest)
- Replace `as ArrayBuffer` casts in encryption.ts with safe .buffer.slice()
- Parallelize independent encryptKiloClawSecret calls with Promise.all
…kages

- Rewrite timingSafeEqual with JS polyfill fallback (removes global
  SubtleCrypto augmentation, works in Node.js/Next.js without
  crypto.subtle.timingSafeEqual)
- Migrate timingSafeEqual consumers: kiloclaw, db-proxy, webhook-agent-ingest
- Migrate formatError consumers: db-proxy, app-builder
- Fix encryption.ts type errors (CryptoKey|CryptoKeyPair narrowing,
  ArrayBuffer|JsonWebKey narrowing) without using 'as' casts
- Fix missing await on async mergeEnvVarsWithSecrets in both cloud-agent
  and cloud-agent-next session-service.ts
- Fix deploy-builder env-decryptor: decryptWithPrivateKey is now async
- Fix backendAuthMiddleware generic constraint so concrete Env types work
- Parallelize buildWorkerChannelsPatch encryption calls
- Add typecheck scripts to packages/db, session-ingest, o11y
- Add typecheck:all script to root (pnpm -r run typecheck)
- Add typescript devDep to cloudflare-session-ingest

All 21 workspace packages pass pnpm typecheck:all.
@iscekic
Copy link
Contributor Author

iscekic commented Feb 28, 2026

Addressed the two unanswered review comments about unused deps in cloudflare-db-proxy and cloudflare-app-builder — both now have actual imports from @kilocode/worker-utils:

  • db-proxy: imports timingSafeEqual and formatError
  • app-builder: imports formatError

Also completed the remaining migrations and fixed pre-existing type errors. Full details in b38963c.

Base automatically changed from chore/catalog-cleanup to main March 1, 2026 14:53
iscekic added 2 commits March 1, 2026 16:02
… extractBearerToken)

- Replace gastown's res.util.ts with re-export shim from worker-utils
- Use extractBearerToken from worker-utils in auth middleware
- Make extractBearerToken case-insensitive per RFC 6750 §2.1
- Add @kilocode/worker-utils workspace dependency to gastown
… wrappers, auth migrations, split encryption

- createErrorHandler: add includeMessage option (default true); cloud-agent-next
  and deploy-infra/builder opt out to preserve their existing no-message behavior
- extractBearerToken: trim header before slicing to match old .trim() behavior
- withDORetry: create thin logger-binding wrappers in cloud-agent,
  cloud-agent-next, and webhook-agent-ingest so structured logging is preserved
- Complete backendAuthMiddleware migration in auto-fix, auto-triage, and
  deploy-infra/builder (removes inline auth boilerplate)
- Complete extractBearerToken migration in gastown mayor-auth
- Remove encryption changes (Web Crypto migration) — split to separate PR
iscekic added 3 commits March 1, 2026 17:03
These functions were changed from async to sync during the worker-utils
extraction but call sites still had await, triggering await-thenable lint.
tsc is available directly via the typescript devDependency.
iscekic added 2 commits March 1, 2026 19:02
…ow-up PR

Restore local timingSafeEqual implementations in cloudflare-db-proxy,
cloudflare-webhook-agent-ingest, and kiloclaw to keep this PR focused
on non-crypto utilities. The shared extraction will land in PR #697.
…ioral contracts

- withDORetry and createErrorHandler now use logger.withFields() when
  available, preserving structured log output for WorkersLogger consumers
- Restore original log message strings for grep/alert compatibility
- extractBearerToken trims the extracted token, matching old gastown behavior
- createErrorHandler falls back to 'Unknown error' for empty messages
@iscekic iscekic force-pushed the chore/worker-utils branch from 2685a0f to b9fb942 Compare March 1, 2026 18:29
Copy link
Contributor

@pandemicsyn pandemicsyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-approving, but pretty sure scheduler.wait is what we want.

Ci's also complaining about a typecheck failure linting gastown.

…ble type inference

The PR's broadened typecheck now includes gastown's individual tsgo check,
which revealed that `unknown` in Agent.checkpoint and BeadRecord.metadata
makes Cloudflare's Rpc.Serializable<T> resolve DO RPC return types to
`never`. Recursive JSON value types cause 'excessively deep' instantiation.
Using z.any() is the pragmatic fix — Zod still validates at runtime.
Prefer Cloudflare's scheduler.wait() over setTimeout for backoff delays
in withDORetry, falling back to setTimeout in non-Workers environments.
Consolidate duplicate @kilocode/worker-utils imports in session-ingest.
@iscekic iscekic enabled auto-merge March 1, 2026 21:02
@iscekic iscekic merged commit 23e0b41 into main Mar 1, 2026
12 checks passed
@iscekic iscekic deleted the chore/worker-utils branch March 1, 2026 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants