mpp: add Payment HTTP Authentication Scheme support#220
Merged
Conversation
Code reviewNo issues found. Checked for bugs and CLAUDE.md compliance. |
Roasbeef
force-pushed
the
mpp-payment-auth-scheme
branch
3 times, most recently
from
c79b67a to
7fedb86
Compare
In this commit, we introduce the `mpp` package which implements the wire
format and cryptographic primitives for the Payment HTTP Authentication
Scheme (draft-httpauth-payment-00). This new protocol enables HTTP
resources to require payment challenges using a 402 Payment Required
flow that is payment-method agnostic.
The package provides several foundational components:
- types.go: Core protocol types including ChallengeParams (the server's
WWW-Authenticate parameters), Credential (the client's Authorization
token), ChallengeEcho (the echoed challenge in credentials), and
Receipt (the Payment-Receipt response header).
- jcs.go: A minimal JSON Canonicalization Scheme (RFC 8785) implementation
that produces deterministic JSON serialization by sorting object keys
lexicographically and stripping whitespace. This is critical for the
HMAC challenge binding since the base64url-encoded request parameter
must produce identical bytes across implementations.
- header.go: HTTP header encoding/decoding for all three protocol
headers: WWW-Authenticate (challenge), Authorization (credential),
and Payment-Receipt (receipt). Includes base64url encoding without
padding per RFC 4648 Section 5, auth-param parsing per RFC 9110,
and generic EncodeRequest/DecodeRequest helpers for JCS+base64url
round-tripping.
- challenge.go: Stateless HMAC-SHA256 challenge ID binding per spec
Section 5.1.2.1.1. The HMAC input uses seven fixed positional slots
(realm|method|intent|request|expires|digest|opaque) joined with pipe
delimiters, where absent optional fields appear as empty strings.
This avoids ambiguity between different combinations of optional
fields and enables the server to verify challenge integrity without
any database lookup.
…etails This commit extends the mpp package with the Lightning-specific protocol types for both registered intents, along with structured error responses. For the charge intent (draft-lightning-charge-00), we define ChargeRequest (containing amount, currency, and methodDetails with the BOLT11 invoice, payment hash, and network), ChargePayload (the preimage proof), and ChargeReceipt. The charge flow maps closely to the existing L402 pattern: server issues a BOLT11 invoice as a challenge, client pays and presents the preimage. For the session intent (draft-lightning-session-00), we define SessionRequest (per-unit cost, deposit invoice, payment hash, idle timeout), SessionPayload (discriminated by action: open/bearer/topUp/ close), SessionReceipt (with refundSats and refundStatus for close), and NeedTopUpEvent (SSE event data for mid-stream balance exhaustion). The session intent enables prepaid sessions where clients deposit a lump sum, authenticate subsequent requests with the preimage as a bearer token, top up when needed, and receive a refund of unspent balance on close. The problem.go file defines RFC 9457 Problem Details types and constants for all error URIs specified in the three drafts, including the paymentauth.org problem namespace and Lightning-specific sub-types like invalid-preimage, session-not-found, and insufficient-balance.
In this commit, we extend the auth package's interface surface to support the new Payment HTTP Authentication Scheme alongside the existing L402 flow. Three new interfaces and one new type are introduced: ReceiptProvider is an optional interface that authenticators can implement to supply response headers (specifically Payment-Receipt) for successfully authenticated requests. The proxy checks for this via Go's interface satisfaction after Accept returns true, keeping the core Authenticator interface unchanged and L402 unaffected. SessionStore defines the persistence contract for MPP prepaid sessions, with methods for creating sessions, querying by ID, atomically updating deposit/spent balances, and closing sessions. The Session struct tracks the payment hash, deposit/spent sats, return invoice, and open/closed status. PaymentSender abstracts Lightning payment sending, used by the session authenticator to refund unspent balance to the client's return invoice when a session closes.
The MultiAuthenticator wraps multiple Authenticator implementations and tries each in sequence, returning true on the first match. On the challenge side, FreshChallengeHeader calls every sub-authenticator and merges all returned WWW-Authenticate headers into a single response, allowing 402 responses to include challenges for L402, LSAT, and the new Payment scheme simultaneously. Clients then pick whichever scheme they support. For receipt generation, the MultiAuthenticator tracks which sub- authenticator accepted the credential (via mutex-protected index) and delegates ReceiptHeader to that authenticator if it implements the optional ReceiptProvider interface. This ensures Payment-Receipt headers are only generated for MPP credentials, not L402.
The MPPAuthenticator handles the Lightning "charge" intent of the Payment HTTP Authentication Scheme. It reuses the existing mint.Challenger for BOLT11 invoice creation and InvoiceChecker for settlement verification, meaning no new LND infrastructure is needed for the charge flow. On the challenge side, FreshChallengeHeader creates a BOLT11 invoice via the challenger, wraps it in a ChargeRequest (amount, currency, invoice, paymentHash, network), JCS-serializes and base64url-encodes it as the request parameter, then computes the stateless HMAC-SHA256 challenge ID binding all parameters together. On the verification side, Accept parses the Payment credential, verifies the HMAC challenge ID (ensuring no parameter tampering), checks the expiry window, decodes the charge payload's preimage, confirms SHA256(preimage) == paymentHash from the echoed request, and finally verifies the invoice is settled in the LND backend. This provides the same security guarantees as L402 (preimage proves payment) but through the new standardized wire format. ReceiptHeader generates a Payment-Receipt with status, method, timestamp, payment hash reference, and challenge ID for audit correlation. The test suite covers valid acceptance, invalid preimage, HMAC tampering, expired challenges, unsettled invoices, wrong-intent rejection, and a full end-to-end flow (generate challenge → build credential → verify).
This commit adds the persistent storage layer for MPP prepaid sessions. The migration (000004_mpp_sessions) creates the mpp_sessions table with columns for session_id (the payment hash hex, used as the primary lookup key), the raw payment_hash blob, deposit_sats/spent_sats for balance tracking, the return_invoice for refunds, and an open/closed status flag. The sqlc queries provide atomic operations critical for concurrent session safety: InsertMPPSession, GetMPPSessionByID, UpdateMPPSessionDeposit (atomic increment of deposit_sats), UpdateMPPSessionSpent (atomic increment of spent_sats), and CloseMPPSession (conditional update only if status='open'). MPPSessionsStore implements the auth.SessionStore interface following the same TransactionExecutor pattern used by SecretsStore. The DeductSessionBalance method performs a read-then-write within a single transaction to enforce balance constraints, preventing overdraft even under concurrent requests.
The MPPSessionAuthenticator handles the full lifecycle of Lightning prepaid sessions as defined in draft-lightning-session-00. It manages four distinct credential actions: Open: Verifies the HMAC challenge binding and preimage-to-paymentHash match, confirms the deposit invoice is settled, validates the client's zero-amount return invoice, and creates a new session in the store with the deposit balance. Bearer: Looks up the session by ID, verifies SHA256(preimage) matches the session's payment hash, and confirms the session is still open. Bearer verification intentionally does not deduct from the balance; that responsibility belongs to the streaming/metering layer. TopUp: Verifies a fresh HMAC-bound challenge for the top-up invoice, confirms the top-up preimage matches and the invoice is settled, then atomically increments the session's deposit balance. Close: Verifies session ownership via preimage, atomically marks the session as closed (before attempting refund to prevent concurrent spending), computes the refund as depositSats - spentSats, and attempts a single best-effort payment to the return invoice via PaymentSender. The refund outcome (succeeded/failed/skipped) is cached for inclusion in the Payment-Receipt header. The test suite covers the full lifecycle: open → bearer → wrong preimage rejection → topUp balance increase → close with refund verification → double-close rejection, plus challenge header generation with correct deposit multiplier and idle timeout.
LndPaymentSender implements the auth.PaymentSender interface using LND's SendPaymentSync RPC. When a session closes with unspent balance, the session authenticator uses this to pay the client's return invoice with the refund amount. The PaymentClient interface is a narrow subset of lnrpc.LightningClient, requiring only SendPaymentSync — this keeps the dependency surface small and testable.
This commit updates the reverse proxy to support the Payment HTTP Auth Scheme's response-side requirements. Receipt injection: After Accept returns true, the proxy checks if the authenticator implements ReceiptProvider and stores any receipt headers in the request context. The ModifyResponse callback picks these up and injects them into the proxied response, along with Cache-Control: private per the spec's requirement that receipted responses not be stored by shared caches. Problem Details: When a 402 response includes a Payment scheme challenge, the response body is now RFC 9457 Problem Details JSON with Content-Type application/problem+json, rather than the plain text "payment required" used for L402-only responses. This enables structured error handling per the spec. Cache-Control: All 402 responses now include Cache-Control: no-store to prevent caching of challenge responses containing single-use invoices. CORS: The Payment-Receipt header is added to both Access-Control-Expose- Headers and Access-Control-Allow-Headers so browser clients can read receipts. Director: MPP credentials in the Authorization header are left as-is during forwarding, since unlike L402 they don't need format normalization. Two integration tests verify the full proxy flow: TestProxyMPP402Response confirms the 402 includes all three schemes (LSAT, L402, Payment), Problem Details body, Cache-Control, CORS, and valid HMAC binding. TestProxyMPPChargeEndToEnd tests the complete challenge → credential → 200 with Payment-Receipt + Cache-Control: private cycle.
This commit ties everything together by adding configuration, wiring, and logger registration for the MPP Payment HTTP Auth Scheme. Configuration: AuthConfig gains EnableMPP (activates the Payment scheme alongside L402), MPPRealm (protection space for challenges, defaults to listen address), EnableSessions (activates the session intent), SessionDepositMultiplier (units per deposit, default 20), and SessionIdleTimeout (seconds, default 300). Wiring: When EnableMPP is set, createProxy builds a MultiAuthenticator composing the L402 authenticator with an MPPAuthenticator (charge intent) and optionally an MPPSessionAuthenticator (session intent). The HMAC secret for challenge binding is derived deterministically from a well-known key hash stored via the existing SecretStore, ensuring challenges survive server restarts. Session stores are created for both postgres and sqlite backends using the TransactionExecutor pattern. Logger: The mpp package's MPAY subsystem is registered in the global logger setup.
Locally we had sqlc v1.30.0 which only differs in the version comment at the top of each generated file. Regenerating with v1.25.0 (the version pinned in the Docker-based CI check) makes the diff clean.
Fix all golangci-lint failures: extract the "open" session status string into a sessionStatusOpen constant (goconst), remove the unused paymentAuthScheme const and receiptHeader test field (unused), add nolint annotation for the mpp logger var that's initialized but not yet read within the package, and run gofmt on files with alignment issues.
The postgres and sqlite cases intentionally mirror each other since they create the same set of TransactionExecutor-backed stores from different concrete DB types. Adding the MPP session store pushed the block size past the dupl threshold. This is the same pre-existing pattern used for secrets, onion, and LNC stores.
This commit addresses 18 findings from the multi-agent code review: Race condition fixes: - Replace lastCloseResult singleton with sync.Map keyed by sessionID to prevent cross-session refund data leakage under concurrent closes. - Replace lastAcceptedIdx on MultiAuthenticator with a try-all-providers approach in ReceiptHeader, eliminating the race between Accept and ReceiptHeader on concurrent requests. Security hardening: - Add HMAC challenge ID verification to handleBearer and handleClose (previously only open and topUp verified the challenge binding). - Wire DeductSessionBalance into handleBearer so session balance is actually consumed on each bearer request. - Add challenge expiry (15 min default) to all FreshChallengeHeader calls. - Validate return invoice as BOLT11 with no encoded amount using zpay32. - Add credential size limit (64KB) in ParseCredential. - Add realm and request field validation in ParseCredential. - Add positive-amount validation on UpdateSessionBalance. - Add integer overflow check for deposit calculation. SQL improvements: - Change SQL UPDATEs to :execresult and check RowsAffected to surface silent no-ops (closed session, not found). - Make DeductSessionBalance atomic with balance check in WHERE clause, eliminating the read-then-write TOCTOU race on PostgreSQL. Wiring fixes: - Wire PaymentSender into createProxy using admin macaroon LND client so session close refunds actually work in production. - Upgrade deriveHMACSecret to return error instead of silently falling back to an ephemeral random secret. - Add config validation: EnableSessions requires EnableMPP. - Add context.WithTimeout(10s) to session authenticator Accept. Cleanup: - Remove unused ChargeReceipt type. - Add concurrent tests for session close and bearer operations.
This commit adds per-service auth scheme control, allowing operators to configure whether each service uses L402, MPP, or both authentication schemes. Previously, MPP was only toggled globally via EnableMPP config. Proto changes: - Add AuthScheme enum (AUTH_SCHEME_L402, AUTH_SCHEME_MPP, AUTH_SCHEME_L402_MPP) to admin.proto - Add auth_scheme field to Service, CreateServiceRequest, and UpdateServiceRequest messages - Add mpp_enabled, sessions_enabled, mpp_realm to GetInfoResponse Admin server: - Wire MPPEnabled, SessionsEnabled, MPPRealm into ServerConfig and GetInfo response - Add authSchemeToString/stringToAuthScheme conversion helpers - Update CreateService/UpdateService/ListServices to handle auth_scheme - Refactor ServiceStore.UpsertService to take ServiceParams struct instead of positional string args Database: - Add 000007_services_auth_scheme migration (auth_scheme TEXT DEFAULT 'l402') - Update sqlc queries and regenerate models Dashboard: - Add AuthScheme type, labels map, and scheme selector to service forms - Show auth scheme column in services table - Add auth scheme config row to service detail page - Update InfoResponse type with MPP fields
Update admin API docs with: - auth_scheme field in service CRUD operations - GetInfo MPP fields (mpp_enabled, sessions_enabled, mpp_realm) - Per-service auth scheme examples (L402, MPP, L402+MPP) Update dashboard docs to mention auth scheme selector in services page and detail page.
Add auth_type column to l402_transactions table (modified migration 000004 in-place since it hasn't been released). MPP charge payments are now recorded alongside L402 transactions, enabling revenue tracking in the admin dashboard for all payment schemes. DB changes: - Add auth_type TEXT column (default 'l402') to l402_transactions - Make token_id nullable (MPP transactions have no macaroon token) - Add auth_type index and filter to filtered queries - Add auth_type filter support to ListFiltered/CountFiltered Store changes: - Add RecordMPPTransaction method to L402TransactionsStore - Add TransactionFilter struct to replace positional args in ListFiltered/CountFiltered (prevents arg ordering bugs) - Existing RecordTransaction now explicitly sets auth_type='l402' Auth changes: - Add TransactionRecorder interface to auth package - MPPAuthenticator accepts optional TransactionRecorder, records pending mpp_charge transactions in FreshChallengeHeader - Settlement callback already handles MPP via payment hash matching Wiring: - Pass concrete txnStore as TransactionRecorder to createProxy - Admin server uses TransactionFilter struct for queries
Roasbeef
force-pushed
the
mpp-payment-auth-scheme
branch
from
7fedb86 to
68846a2
Compare
Fix UpdateService to always apply auth_scheme rather than skipping when the value is AUTH_SCHEME_L402 (proto enum zero value). This allows resetting a service back to L402-only via the admin API. Add migration 000008 to add auth_type column to existing databases that were created before the in-place modification of migration 000004. New databases get the column from 000004 directly; existing databases pick it up via this upgrade migration.
Fix a bug where services loaded from the database via mergeServicesFromDB were not passed to the proxy. The createProxy function was using cfg.Services (config-only) instead of the merged initialServices list, so DB-persisted services would appear in the admin API but not be routable by the proxy. Also update migration 000008 to fully recreate the l402_transactions table with nullable token_id for existing databases. SQLite does not support ALTER COLUMN, so the migration renames the old table, creates a new one with the correct schema, copies data, and drops the old table.
Fix gofmt whitespace in services_test.go (24*time.Hour spacing) and run prettier --write on dashboard files to match CI format check.
This commit addresses findings from a comprehensive 6-agent code review of the MPP Payment HTTP Authentication Scheme implementation. Security fixes: - Fix TOCTOU race in session close refund by adding atomic CloseSessionAndGetBalance SQL query that returns the remaining balance within the same UPDATE...RETURNING statement - Wire per-service AuthScheme into proxy Accept flow via SchemeTagged interface and AcceptForScheme on MultiAuthenticator - Make auth_scheme optional in UpdateServiceRequest proto to prevent silent downgrade on partial updates - Replace unbounded sync.Map with neutrino LRU cache for close results to prevent memory leaks when ReceiptHeader is never called - Add challenge expiry checks to bearer and close handlers for consistency with open and topUp - Pre-check credential encoded length before base64 decode to avoid allocating memory for oversized payloads - Fix ReadOrCreateRootKey to only regenerate on os.IsNotExist, not on any file error (prevents silent macaroon invalidation) - Validate return invoice expiry at session open to ensure refunds are possible on close Operational improvements: - Fire session refund payments asynchronously so close handler doesn't block the HTTP response during LN payment routing - Log warning when MultiAuthenticator operates in degraded mode with partial scheme set - Delete redundant migration 000008 (000004 already includes auth_type) - Add MPP, session, and admin config sections to sample-conf.yaml - Add consolidated schema generation tool (cmd/merge-sql-schemas) that applies all migrations to in-memory SQLite and dumps final schema
Restructure main() to use a helper function that returns errors instead of calling log.Fatalf after defer db.Close().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In this PR, we add support for the Payment HTTP Authentication Scheme (
draft-httpauth-payment-00), a new payment-method-agnostic challenge-response protocol that gives HTTP 402 its long-awaited semantics. The scheme is designed to coexist alongside the existing L402 protocol, letting clients choose whichever authentication scheme they support.The protocol works like this: a server returns
402 Payment Requiredwith aWWW-Authenticate: Paymentchallenge containing anid,realm,method,intent, and a base64url-encodedrequestparameter. The client fulfills the payment (e.g., pays a BOLT11 invoice), then retries the request with anAuthorization: Paymentcredential containing the proof. The server verifies the proof and returns aPayment-Receiptheader on success.For Lightning specifically, two intents are registered: charge (one-time BOLT11 payment, similar to L402 but with a standardized wire format) and session (prepaid sessions with deposit, bearer token, top-up, and close-with-refund lifecycle). The charge intent maps almost 1:1 to the existing L402 flow, reusing the same
mint.ChallengerandInvoiceCheckerinfrastructure. The session intent is entirely new functionality that enables use cases like metered streaming (e.g., LLM token generation) where per-request Lightning round-trips would be too slow.See each commit message for a detailed description w.r.t the incremental changes.
Protocol Package (
mpp/)The new
mpppackage implements the wire format and cryptographic primitives. We wrote a minimal JCS (RFC 8785) canonicalizer rather than pulling in an external dependency, since our JSON objects are flat or simply-nested and the full spec complexity isn't needed. Challenge IDs use stateless HMAC-SHA256 binding per Section 5.1.2.1.1 of the spec: seven positional slots (realm, method, intent, request, expires, digest, opaque) joined with pipes, then HMAC'd. This means the server never needs to store challenge state for the charge intent; it just recomputes the HMAC on verification.Auth Layer
The
MultiAuthenticatorcomposes any number ofAuthenticatorimplementations. On a 402, it callsFreshChallengeHeaderon all sub-authenticators and merges the returnedWWW-Authenticateheaders, so a single response can containLSAT,L402, andPaymentchallenges simultaneously. On credential verification, it tries each authenticator in order and tracks which one accepted (for receipt delegation via the newReceiptProviderinterface).The
MPPAuthenticatorhandles the charge intent: HMAC verification, expiry check,SHA256(preimage) == paymentHash, and invoice settlement confirmation. TheMPPSessionAuthenticatorhandles the full session lifecycle (open, bearer, topUp, close) with the session store providing atomic balance operations and thePaymentSenderhandling refunds on close.Proxy Changes
The proxy now injects
Payment-Receiptheaders into proxied responses via request context (set afterAccept, picked up inModifyResponse). 402 responses includeCache-Control: no-storeand RFC 9457 Problem Details JSON when a Payment challenge is present. Receipted responses getCache-Control: privateto prevent shared cache storage. CORS headers exposePayment-Receipt.Configuration
Three new config flags control the feature:
EnableMPPactivates the Payment scheme alongside L402,EnableSessionsadds the session intent, andMPPRealmsets the protection space. The HMAC secret is derived deterministically from a well-known key hash stored via the existingSecretStore, so challenges survive restarts.Testing
80+ tests cover the new code: JCS canonicalization, header round-trips, HMAC binding with tamper detection and optional field permutations, charge authenticator (valid/invalid preimage, HMAC tampering, expiry, unsettled invoice, wrong intent, end-to-end flow), session authenticator (open, bearer, wrong preimage, top-up balance increment, close with refund, double-close rejection), multi-authenticator (first-match semantics, header merging, receipt delegation), and two proxy integration tests that spin up real HTTP servers to verify the full 402 → credential → 200 + receipt cycle.