feat(snapshot): supply/value conservation invariants on import (#125 follow-up) - #126
Merged
Conversation
simulated_node::recent_blocks read `b->witness` from a signed_block, a leftover Steem/Golos field name. VIZ's block_header exposes the producer as `validator` (block_header.hpp:18), so the consensus_sim harness failed to compile against current headers — the suite had never been built in a real environment (CI does not build BUILD_CONSENSUS_TESTS, local builds are blocked by Boost 1.90). Fixing this unblocks the harness build so the is_wedged truth-table test (added in #125) can actually run.
Adds two post-import checks to load_snapshot(), the deferred follow-up from #125 (the incident class: an export-side incomplete snapshot with a missing account passes the checksum and count reconciliation, then wedges the node when a canonical block references the absent account). - SHARES: dgp.total_vesting_shares == sum of every account's own vesting_shares. Delegation objects redistribute already-counted vests and are intentionally excluded. - TOKEN: dgp.current_supply == sum of every liquid/locked TOKEN pool (account balance + reserved, escrow balance + pending fee, invite balance, validator pending reward, vesting/reward/committee funds). Both are strict equalities: an account missing from the export that held stake or a balance drops the sum below the independently-tracked dgp total, so a short export no longer reconciles. On mismatch they raise the existing retryable FC_ASSERT (retry another trusted peer). Per-component subtotals are logged unconditionally for diagnosis. Verified satoshi-exact (delta=0) against six real mainnet snapshots spanning blocks 81334800-81620400, including the height-varying validator pending-reward term.
On1x
approved these changes
Jul 14, 2026
On1x
left a comment
On1x
left a comment
Member
There was a problem hiding this comment.
Approve. This lands the export-side value invariant deferred from #125 (review finding #4) — the real detector for the incident class.
Verified the accounting is complete, not just delta=0 on the six tested heights:
- TOKEN pools — checked every
asset(TOKEN_SYMBOL)field in the codebase. The sum covers all real pools:account.balance+reserved_balance,escrow.token_balance+pending_fee,invite.balance,validator.pending_stakeholder_reward, and the three dgp funds. The fields left out are correctly left out:account.current_bid— the bid TOKEN is held inreserved_balance(database.cpp:4704-4705reserved_balance -= current_bid; balance += current_bid); summing it would double-count.committee_request.remain_payout_amount— the TOKEN stays incommittee_funduntil paid; payout decrements both together (database.cpp:3245/3250). It's a claim schedule, not a holding.account_offer_price/last_bid/subaccount_offer_price— price/record fields, no TOKEN held.paid_subscription.amount— a price (share_type), paid by direct transfer each period (database.cpp:3080), no escrow pool.
- SHARES —
Σ account.vesting_shares == dgp.total_vesting_sharesis correct: delegation doesn't mint vests, it only moves voting power (delegated_/received_vesting_sharesare separate fields;vesting_sharesis the account's own minted stake), so the delegation objects must be excluded to avoid double-counting. The satoshi-exact match on real mainnet snapshots (which have active delegations) confirms this. - Harness fix —
signed_block::validator(not the Steem-ism->witness) is correct for current VIZ headers; unblocks theis_wedgedtest from #125.
Both are strict, retryable FC_ASSERTs on the snapshot-import path only (not block processing), with per-component subtotal logging for attribution — good.
Forward-looking note (non-blocking): these sums are now a maintenance coupling — any future hardfork that introduces a new TOKEN/VESTS pool must be added here or valid snapshots will start failing the strict equality. Worth a comment near the pool list pointing back to the supply-accounting sites, and ideally folding the eventual database::validate_invariants() definition into the same source of truth.
4 tasks
On1x
added a commit
that referenced
this pull request
Jul 14, 2026
…126) Brings the wedged-behind-network watchdog (#125) and the snapshot post-import completeness/supply invariants (#126) onto the PM branch. The #125 checks (count reconciliation, referential integrity, singletons) and the #126 SHARES conservation check are PM-agnostic and stay fatal. The #126 TOKEN supply invariant is extended for PM: PM is zero-sum, so a bet / liquidity / lazy-deposit / commit-escrow / dispute-fee / oracle- insurance / leverage-collateral moves TOKEN out of account.balance into a PM object field. Those pools are now summed (minimal non-overlapping set, duplicates such as bets_sum/reserves, empty-provider lazy LP, and deposit.principal excluded). Because this PM accounting is not yet satoshi-validated against real PM snapshots, the TOKEN check is LOG-ONLY on this branch (per-component delta logged, import NOT failed) until reconciled to delta==0 and re-armed as a fatal FC_ASSERT. Tracked in #127 with a TODO(pm) at the check. Conflicts: tests/consensus_sim/CMakeLists.txt (kept both test_pm_lifecycle and test_wedge_predicate scenarios).
On1x
added a commit
that referenced
this pull request
Aug 4, 2026
…ing it (PR #124 F1) The B3 winners_pool floor stopped the uint64 wrap but still emitted exactly |winners_pool| tokens: winners got their principal back out of tokens the losers'+forfeit pot never funded, while settle_liquidity returned LP principal unconditionally, so nothing absorbed the shortfall. Any market whose accumulated leverage profit (negative forfeit_pool) outran ~96% of the losing stakes over-emitted at settlement. compute_settlement now reports the shortfall as settle_result::uncovered, and settle_liquidity charges it against LP principal pro-rata (LPs are the leverage counterparty, capped ≤ their principal, last LP absorbs the rounding remainder). The header zero-sum identity now holds unconditionally. The strict supply invariant (#126) remains the backstop for the pathological uncovered > Σ-principal case (pos_cap should prevent it). Found by @chiliec in the PR #124 fix-round review (finding 1).
On1x
added a commit
that referenced
this pull request
Aug 4, 2026
…review) Three issues in the F1 uncovered->LP-principal sink, all found by @chiliec: 1. Escape in the dispute-grace window (the serious one). pm_resolve_market sets status=3 but settle_market runs from the deferred cron sweep only after result_expiration - pm_dispute_grace_sec (~12h). pm_withdraw_liquidity unlocked on status>=2 and skipped the pm_min_liquidity floor at status 3, so an LP could pull 100% of principal in that window, empty settle_liquidity's set (early return before the charge) and dodge its share of — forfeit_pool is public on get_market, so the exit is a rational, computable choice. Gate on finalized_time instead: it is 0 until a terminal transition (settle/void/expire), so LP principal stays locked through resolve-> settle while already-finalized markets are still served. 2. Remainder could exceed a small last LP. The floor-charge remainder was dumped on the last LP, which a naive small last LP couldn't absorb -> principal_ret clamp -> re-emission. Spread it over LPs with headroom instead (total headroom always covers it). 3. uncovered > Σ principal is a real bounded over-emission that #126 only catches on a later import; wlog it at the settlement site where it happens (pos_cap should keep it unreachable).
On1x
added a commit
that referenced
this pull request
Aug 22, 2026
* feat(pm plan): add opt-in batch and commit-reveal betting modes for front-running protection
- Introduce two optional execution modes per bet: batch (mode 1) and commit-reveal (mode 2)
- Preserve instant per-bet execution (mode 0) as default for best UX
- Define global and per-market parameters for configuration and governance control
- Implement batch epochs with unified uniform-price settlement to prevent intra-batch front-running
- Add commit-reveal operations: bet commitment, reveal, penalty for no-reveal with transfer into winners' pool
- Design batch settlement math preserving liquidity provider invariant and fairness via canonical CPMM aggregation
- Add database schema changes for bets, commitments, and market flags supporting new modes and states
- Extend market logs to track batch-related actions: commit, reveal, batch settlement, and forfeits
- Specify layered defenses against front-running including epoch snapshot pricing and min_tokens enforcement
- Define phased rollout plan: batch mode first, then commit-reveal, followed by tiering and client policies
- Document open questions and future improvements like encrypted sealed bids (mode 3) and cancellation policy
* feat(pm docs): add Prediction Market params (HF14) and leverage risk-off strategy
- Add new chain_properties versions 4 (hf13) and 5 (hf14 PM) for governance params
- Introduce 40+ Prediction Market consensus parameters including fees, disputes, batch settings, cron budget, lazy pool, and leverage
- Define market fee structure and governance cap rules focusing on oracle fee cap
- Document versioning, median calculation, and validator publishing rules for new params
- Implement leverage fund as sub-allocation of lazy pool free balance with detailed accounting and protocol state transitions
- Present full mathematical leverage model: CPMM/LMSR calculations, cancel values, liquidation thresholds, safety margin, and leverage constraints
- Describe atomic liquidation mechanism ensuring pool protection before opposing bets execute, including cascade logic and cascade loop handling
- Analyze risk types and mitigation: price-movement risk elimination via atomic liquidation, safety margins, and dynamic caps; outcome risk remains inherent
- Provide detailed architecture overview, fund allocation parameters, frontend UI terminology (Boost vs Leverage), and risk analysis
- Outline protocol operations, API endpoints, database schema, pre-calculation and slider logic for leverage positions
- Address MEV considerations for liquidation rebalancing and sandwich attacks with mitigation plans for VIZ DLT implementation
* docs(pm-workflow): add detailed prediction market workflow documentation by role
- Add comprehensive README outlining canonical scenarios for normal and disputed resolves
- Document master ledger calculations for payout distributions and zero-sum properties
- Include role-specific subfolders with interaction diagrams and signed/virtual operations
- Provide detailed token flow tables for normal and disputed market outcomes
- Outline leverage mechanics including open, close, liquidation, and settlement steps
- Describe dispute scenarios: disputer wins, loses, and forced auto-close with penalties
- Verify all operations and virtual operations present in code with references
- Add instructions on how to observe states and events via API plugin methods
- Cover edge cases like time penalties, liquidations, and refund mechanics
- Provide an index of workflow document folders by participant role for easy navigation
* chore(thirdparty): update fc submodule to latest commit
- Updated fc submodule pointer from 99b5d133 to 5a9d84a1
- Ensured thirdparty dependencies are current and consistent
* chore(deps): add mermaid and related plugins to devDependencies
- Added mermaid package version 11.4.1
- Added vitepress-plugin-mermaid version 2.0.17 for Mermaid integration
- Added multiple new dependencies related to mermaid and diagram rendering
- Included types packages for d3 and related libraries for better type support
- Updated package-lock.json to reflect new dependencies and their versions
- Removed several optional and deprecated dependencies to clean up lock file
* feat(hardfork): add support for hardfork 14 in prediction markets
- Increase total number of hardforks from 13 to 14
- Introduce hardfork 14 with features for prediction markets (binary CPMM, multi LMSR)
- Add oracles, dispute mechanisms, batch/commit-reveal, and lazy liquidity pools
- Define placeholder activation times for mainnet and testnet
- Use version 4.0.0 for hardfork 14 release candidate
* feat(pm): integrate Prediction Markets core logic and HF14 support
- Add chain_properties_pm median evaluator and integrate into median calculation
- Implement lazy-pool logic for DAO-committee voting weight in committee_processing
- Register PM evaluators for all PM operations in database initialization
- Add core indexes for PM-related objects for consensus and chain state tracking
- Implement PM liquidity settlement, market resolution, leverage liquidation, and recall mechanics
- Add HF14 hardfork initialization including lazy-liquidity pool singleton creation
- Update CMakeLists.txt to include PM source and header files with proper compiler flags
- Enhance database.cpp with PM processing hooks and vote weight adjustments for lazy pool stake
- Provide detailed internal helpers for PM operation including settle market, refund bets, and liquidity allocation
* feat(prediction_market_api): add prediction market plugin with extensive APIs
- Implement prediction_market_api plugin with full lifecycle management
- Provide APIs to query markets, outcomes, bets, positions, and liquidity
- Support account leverage positions and creator ban status retrieval
- Offer oracle info and list oracles with reliability scoring
- Include dispute info and vote tallying with projected verdict calculation
- Record and prune metadata and kline time-series data on block application
- Add support for lazy pool and deposit queries along with chain properties
- Implement metadata parsing and filtered market listing by category and jurisdiction
- Integrate with chain plugin database and handle post-operation kline recording
- Configure plugin with pmm-ttl-days option for metadata retention days
- Setup CMake build configuration for prediction_market_api plugin library
* feat(snapshot): add import support for HF14 Prediction Market objects
- Introduce functions to import pm_oracle, pm_market, pm_outcome, and pm_dispute objects with
shared_string member handling
- Add export and import logic for all HF14 PM related indices in snapshot processing
- Clear existing PM objects before importing new ones during database initialization
- Enhance snapshot deserialization to handle absent PM objects in pre-HF14 snapshots
- Log import counts for each PM object type during snapshot loading to aid diagnostics
* feat(wallet): add HF14 prediction market API support
- Include prediction_market_api plugin in wallet build dependencies
- Add remote_prediction_market_api binding with optional connection handling
- Implement pm_api() accessor for prediction_market_api proxy with assertion
- Introduce prediction market helper methods for oracle registration, update, market creation,
bet placement, commitment hashing, bet commit/reveal/cancel, liquidity management, market resolution,
dispute creation/voting/resolution, position transfer, lazy deposit/withdraw, and leverage operations
- Add read API passthrough methods for markets, oracles, bets, positions, liquidity, disputes, lazy pool,
chain properties, market metadata, and market kline data
- Extend fc::api remote_node_api.hpp with prediction market API message signature class and FC_API definition
- Update wallet.hpp and wallet.cpp with full prediction market API support and method declarations
- Ensure wallet starts normally even if prediction_market_api plugin is unavailable on connected node
* feat(vizd): add prediction_market_api plugin registration
- Include prediction_market_api in CMakeLists.txt for vizd program
- Add prediction_market_api header inclusion in main.cpp
- Register prediction_market_api plugin in appbase application initialization code
* test(pm): add comprehensive prediction-market lifecycle integration tests
- Add tests covering oracle registration after HF14 activation
- Implement full binary market lifecycle: creation, betting, resolution, payout
- Test committee dispute scenario with outcome overturning via voting
- Verify lazy-pool stake contributes to dispute voting weight and quorum
- Ensure external oracle rejection refunds seed liquidity exactly once
- Add bet cancellation scenario reversing CPMM reserves and bets sum
- Adjust consensus_sim harness to support new pm tests and sanitizer flags conditionally
- Expose direct database access in simulated_node for test assertions
- Fix simulated_node block witness field to validator for accuracy in tests
* feat(prediction-markets): add HF14 prediction markets and v5 chain properties support
- Introduce HF13 distribution epoch length and HF14 prediction markets features in governance docs
- Add detailed median-voted HF14 prediction-market parameters and kill-switch flags explanation
- Add new `prediction_market_api` plugin with extensive JSON-RPC read-only methods for markets, bets,
oracles, disputes, lazy pool, and governance data access
- Include computed DTOs and charting support for prediction markets with offset-from-newest pagination
- Document prediction market concepts analysis comparing Onix protocol to theoretical models
- Update advanced hardfork and chain properties docs to cover new prediction market functionality
* docs(prediction-markets): add prediction market API plugin documentation
- Introduce readonly JSON-RPC plugin for HF14 prediction markets state access
- Document market-related API methods including markets, outcomes, bets, liquidity, and metadata
- Describe position, leverage, oracle, dispute, lazy pool, and governance methods
- Provide details on kline/time series for market weight history and pagination approach
- Explain computed DTOs representing bets, oracles, votes, and payout structures
- Include example usage and code snippets for API calls and data processing
- Link to relevant protocol operations and chain property documentation
docs(governance): update chain properties with HF13 and PM parameters
- Add chain_properties_hf13 with distribution_epoch_length parameter
- Introduce chain_properties_pm (v5) for ~30 prediction market parameters and kill-switch flags
- Detail all median-voted parameters for oracle, market, batch, dispute, time penalty, lazy pool, leverage, and fairness
- Clarify live kill-switch flags to disable commit-reveal, lazy pool, or leverage without hardfork
docs(advanced): extend hardfork management with HF13 and prediction markets
- Add entries for HF13 epoch length and HF14 prediction markets including CPMM/LMSR, oracles, disputes, commit-reveal, lazy-pool, and chain properties v5
docs(prediction-markets): add comprehensive analysis of conceptual mapping of Onix PM protocol
- Provide detailed table comparing 90 theoretical prediction market concepts against VIZ Onix on-chain implementation
- Categorize concepts as solved, inherent, not needed, partial/roadmap, client layer, or open risks
- Discuss information theory, mechanism design, liquidity and trading aspects in depth
- Highlight Onix innovations: risk-free LP, CPMM binary, LMSR multi, commit-reveal batch bets, optional leverage subsystem, lazy pool governance voting weight
- Explain architectural decisions omitting orderbooks, combinatorial markets, and peer prediction
* chore(chainbase): update submodule to latest commit
- Updated chainbase submodule commit from 39ab2c2 to d429230
- Ensures third-party library is aligned with latest upstream changes
* feat(prediction-markets): extend HF14 with oracle rebuttals, ban lifecycle, coverage floors and thin-client APIs
Consensus (HF14 follow-up ops, appended so operation indices stay stable):
- pm_dispute_oracle_respond (op 22): the market oracle posts a public rebuttal
onto an open dispute; stored on the dispute object (public-hearing model),
allowed only while open and within oracle_response_deadline, re-post overwrites.
- pm_unban (op 23): the resolver that imposed an account-mode ban (banned_by)
may lift it early; sets banned_until to epoch and clears banned_by.
- pm_ban_expired (virtual): the per-block cron sweeps temporary oracle/creator
bans at banned_until, clears them and emits the lift for history/indexers.
On-chain state:
- pm_market gains decision_url/decision_reason — the oracle's resolution
statement stored on-chain (set by pm_resolve_market / pm_no_contest reason),
readable via get_market with no history scan.
- pm_resolve_market_operation gains decision_reason (reflected on the wire).
- pm_dispute gains oracle_response/oracle_response_time.
- pm_oracle and pm_creator_ban gain banned_by; pm_creator_ban gains a
by_ban_expiry index so the cron sweeps expired bans oldest-first (cleared
bans sort into the 0-bucket, permanent bans past now, both skipped).
Chain properties (witness-median tunables):
- pm_listing_min_coverage_percent (2.5x): hide under-insured markets from the
default catalog (enforced by the API plugin, revealed via show_risky).
- pm_betting_min_coverage_percent (1.5x, advisory): client risk-confirm
threshold; validated betting <= listing.
Thin-client read APIs (non-consensus, for the viz-js client):
- get_leverage_quote / get_leverage_close_preview / get_leverage_convert_preview
reuse the frozen pm::leverage math to mirror the open/close/convert evaluators.
- get_market_categories (taxonomy + live counts), get_market_full (one-call
enriched, account-scoped), get_lazy_allocations / get_market_lazy_allocation.
- Wallet remote_node_api bindings for all of the above.
Docs & tests:
- EN + ru + zh-CN docs updated (chain-properties, prediction-market-api,
specification, operations overview/prediction-markets/validators,
virtual-operations); library-integration spec + thin-client plan added.
- test_pm_lifecycle: cases #58-#63 cover oracle rebuttal + decision_reason,
no-contest rationale, manual unban and its guards, and ban auto-expiry vop.
* fix(wallet): return fc::variant from prediction_market_api read pass-throughs
The cli_wallet build failed because remote_prediction_market_api and the
wallet_api pm_get_*/pm_list_* methods returned the node's typed objects. Those
chainbase state objects (pm_market_object, pm_bet_object, ...) and the API DTOs
embedding them are not default-constructible (deleted default ctor / shared_string
members require a segment manager), so fc::api's client deserializer (T tmp;
var.as<T>()) could not instantiate them.
Return fc::variant instead: the node already emits fully-formed JSON and cli_wallet
prints the variant unchanged, so the read surface is identical.
* fix(wallet): drop dead prediction_market_api plugin include from remote_node_api
cli_wallet failed to compile because remote_node_api.hpp pulled in
<graphene/plugins/prediction_market_api/prediction_market_api.hpp> transitively,
but programs/cli_wallet has no include path to that plugin. After the read
pass-throughs switched to fc::variant, the header (and the pmapi alias) are no
longer referenced anywhere in the wallet, so remove them. graphene_wallet still
builds; the public wallet header no longer leaks a plugin-only dependency.
* feat(prediction-markets): add oracle accept window and lazy-pool min LP fee
Add two HF14 median-voted consensus parameters and their enforcement:
- pm_oracle_accept_window_sec (default 1h): a pending market the named
oracle never accepts nor rejects is voided by the per-block cron once
now >= created_time + window. The creators seed liquidity is refunded
(return_liquidity); the non-refundable creation fee stays with the DAO
fund. Tracked via a new pm_market_object.accept_deadline field and a
by_accept_deadline index; emits the new pm_market_expired virtual op
(op-id 101, appended last in the operation variant to keep tags stable).
- pm_lazy_min_liquidity_fee_percent (default 2%): the lazy pool skips
markets whose liquidity_fee_percent is below this reward floor, so it
never subsidizes depth it is not paid enough to provide.
Wired into calc_median and chain_properties_pm::validate().
* docs(prediction-markets): document accept window and lazy-pool min LP fee
Cover the new pm_oracle_accept_window_sec / pm_market_expired lifecycle
and the pm_lazy_min_liquidity_fee_percent reward-floor gate across:
- EN docs (chain-properties, specification, operations, virtual-operations)
- RU and zh-CN localizations (@l10n) at full parity with the EN source
- library integration spec (delta section + property/vop tables, op-id 101)
and thin-client plan
- Onix paper EN + RU (state machine, acceptance flow, lazy-pool gate);
PDFs rebuilt via pandoc + xelatex (EN 30pp, RU 32pp, 0 missing glyphs).
* docs(integration): clarify fee units and remove permille from protocol
- Added warning that the live protocol uses basis points (bp), not permille (‰)
- Explained the conversion from original PHP prototype’s permille to bp in on-chain code
- Specified that all fee fields (oracle_fee_percent, creator_fee_percent, liquidity_fee_percent, etc.) use bp (10000 = 100%)
- Highlighted the use of `fromBP` parser for fee fields and rejection of markets exceeding fee sum 10000
- Warned that using deprecated `fromPermille` leads to incorrect fee values, off by a factor of 10
* fix(prediction-markets): resolve ternary type mismatch on accept_deadline
The ?: between time_point_sec() and (now + fc::seconds(...)) has no common
type — the latter yields fc::time_point, and each type converts to the other,
which GCC rejects as ambiguous. Wrap the second branch in an explicit
time_point_sec(), matching the copy-init conversion already used for the
reveal/dispute deadlines in this file.
* fix(account_history): mark impacted accounts for prediction-market operations
The generic impacted-account visitor only collected signing authorities, so
prediction-market events were missing from the histories of accounts that did
not sign them:
- signed ops lost their counterparties (pm_create_market -> oracle,
pm_transfer_position -> recipient, pm_unban -> target, oracle auto-accept
whitelist);
- virtual ops carry no authority at all, so payouts, forfeits, liquidations,
oracle penalties, market accept/expire and ban expiry were invisible to the
affected users.
Add explicit get_impacted_account_visitor overloads for the PM user and
virtual operations, inserting every account field they carry. Market-only
virtual ops that reference a market by id but carry no account name
(pm_batch_settle / pm_dispute_finalize / pm_dispute_auto_close /
pm_lazy_recall) are intentionally left to the generic handler.
* refactor(prediction-market): move market metadata off-chain (prunable, per-node)
The free-form `metadata` JSON was stored in the consensus `pm_market_object`
(shared_string) permanently — never pruned — even though consensus never reads
it (it is written once and only parsed off-chain by the prediction_market_api
plugin). That let a market permanently bloat every node's chainbase/shared
memory with unbounded, unvalidated data.
Move it out of consensus entirely:
- pm_market_object: drop the `metadata` field (member, ctor, FC_REFLECT). The
operation `pm_create_market_operation.metadata` is unchanged — clients still
send it and it lives in the block log, exactly like custom_operation.json.
- pm_create_market_evaluator: stop persisting metadata into state.
- prediction_market_api: ingest metadata off-chain from the create operation
(post_apply_operation) into the existing prunable pm_market_meta_object,
instead of reading it back from the consensus object in on_block.
- snapshot: drop the metadata import/export for pm_market (auto-excluded from
the reflected dump; import of legacy snapshots ignores the field).
Because it is now non-consensus, each node prunes it on its own schedule via
--pmm-ttl-days (default lowered 7 -> 5; 0 keeps it forever for archival nodes).
No consensus length/UTF-8 cap is needed — the blob no longer touches state.
* feat(prediction-market): garbage-collect finalized markets after a fixed retention
A resolved+settled market (status 3, payout_status 3) is immutable — no betting,
dispute, resolve or payout can touch it again; it only lingered in chainbase
"for history", growing shared-memory state without bound.
process_pm_markets() now GCs such markets and their whole object cluster
(outcomes, bets, liquidity, commits, dispute votes, leverage positions, the
dispute and lazy-allocation rows) once they have been closed for a FIXED protocol
constant PM_CLOSED_MARKET_RETENTION_SEC = 5 days (measured from
result_expiration + dispute grace). The retention is hardcoded and identical on
every node, so pruning is fully deterministic: every node deletes exactly the
same markets at the same block, keeping shared-memory state and snapshots in
lock-step network-wide (a node syncing from a snapshot ends up with the same
market set as everyone else). Work is bounded by the existing per-block cap.
Only status-3/payout-3 markets are collected; disputed (payout_status 2) and
never-settled markets are left untouched. Nothing holds an id-reference to a
settled market, so there are no dangling references after removal.
* feat(prediction-market): GC every terminal market, not just resolved ones
Extend the market garbage collector to reclaim ANY dead market a fixed 5 days
after it becomes terminal — not only resolved+paid ones. A market is dead once
nothing can act on it: resolved and paid out, void/no-contest, oracle-rejected,
or the oracle never accepted and the accept window expired.
To anchor the retention on the actual moment of death (rather than the declared
result_expiration), add a `finalized_time` field to pm_market_object, set to the
head-block time at every terminal transition:
- oracle rejects the market (status -1)
- accept window expires, market voided (pm_market_expired)
- oracle misses resolution, refund (pm_oracle_missed_penalty)
- dispute auto-close refund
- settlement / auto-payout (covers resolved, no-contest, post-dispute)
A new by_finalized index (finalized_time, id) lets process_pm_markets() sweep
terminal markets in time order, skipping the finalized_time==0 live bucket, and
delete each cluster PM_CLOSED_MARKET_RETENTION_SEC (5 days) later. Retention is a
fixed protocol constant identical on every node, so pruning stays deterministic
and snapshots identical network-wide. Snapshot import reads finalized_time when
present. Work stays bounded by the per-block cap.
* test(prediction-market): cover all five terminal-market GC paths
Add consensus_sim scenarios asserting that every way a market can die gets its
whole object cluster reclaimed from state after the retention window:
- gc_resolved_market_after_retention (resolved + auto-paid)
- gc_oracle_reject_after_retention (oracle rejects the terms)
- gc_accept_window_expired_after_retention (oracle never accepts, window expires)
- gc_oracle_missed_after_retention (oracle misses the resolution deadline)
- gc_dispute_auto_close_after_retention (dispute filed, nobody votes, auto-close)
Each drives the market to its terminal state, asserts finalized_time is stamped
and the cluster is still present, then advances past the retention and asserts
the market plus its outcomes/bets/liquidity/commits/dispute-votes/leverage/
dispute/lazy-allocation rows are all gone (market_cluster_absent helper).
To make the 5-day retention testable, expose it as a median-voted chain
property pm_closed_market_retention_sec (default 432000 = 5 days) instead of a
hardcoded constant. It stays identical on every node at any block (median), so
GC remains deterministic and snapshot-safe, while tests can shorten it to 30s.
The GC cron now reads mp.pm_closed_market_retention_sec.
* feat(validator): add disable-minority-fork-detection for single-operator forks
On a testnet fork where one operator controls all validators, the
minority-fork detector loops forever: healthy participation (>=33%)
auto-clears the enable-stale-production override every tick, then
"last 21 blocks all ours" triggers a reset-to-LIB resync. Add an
explicit disable-minority-fork-detection flag that bypasses both the
standard and DLT detection blocks and is never auto-cleared.
* docs(validator): add disable-minority-fork-detection option for single-operator forks
- Introduce `disable-minority-fork-detection` config to fully skip minority fork detection
- Document usage warnings: only for testnets or single-operator forks, never enable in public networks
- Clarify difference from `enable-stale-production` which auto-clears at high participation
- Update validator plugin docs and configuration references to include new flag
- Explain behavior in validator-node docs and warnings for detection loop avoidance
- Note watchdog and fork detection interactions with new flag in documentation tables
* pm-api: index market title/image/condition_id in metadata plugin
The prediction_market_api plugin extracted only category/subcategory/tags/
banned_jurisdictions from a market's metadata JSON and discarded title, image
and the source condition_id, so thin clients had no human-readable question or
icon to render (markets showed as 'Market #N'). Add title, image and
condition_id to pm_market_meta_object + parse_market_metadata + FC_REFLECT and
copy them in ingest_market_meta. Non-consensus prunable index — no hardfork;
a replay re-extracts these for existing markets from the block log. condition_id
also enables reliable client back-linking of a mirrored market to its source.
Tests extended (meta_parse_test).
* pm_api: enrich market read APIs with parsed metadata (title/image)
The consensus pm_market_object deliberately does NOT carry the free-form
metadata; the plugin parses it off-chain into pm_market_meta_index. So clients
that read market.metadata (title/image/category) got nothing — cards and the
market detail showed 'Market #N' with no thumbnail.
Add market_card(db, market): serialize the market and inject a reconstructed
'metadata' object (title/image/category/subcategory/tags[]/banned_jurisdictions[]/
condition_id) plus flat title/image/category fields. Route the market-returning
read APIs through it: get_market, list_markets, list_markets_by_oracle,
list_markets_by_creator (return vector<fc::variant>), and get_market_full (overlay
the enriched card onto the .market field). Existing clients that parse
market.metadata now work unchanged; no consensus/state change, no replay.
remote_node_api already types these as fc::variant, so the wallet/CLI is unaffected.
* pm_api: add list_markets_awaiting_resolution(oracle, from, limit)
Read-only query for the markets that need an oracle's result now: active
(status 1) markets whose betting window has closed (betting_expiration <=
head_block_time) and are therefore not yet resolved. "Awaiting" is not a
distinct status — a market stays active from open through betting-close until
it is resolved — so it can't be filtered by status alone. Walk the
by_betting_expiration index (keyed status, betting_expiration, id) over the
bounded prefix of active markets past their betting deadline and keep the
given oracle's rows. This lets an Oracle Console list pending-resolution
markets without scanning the oracle's full (mostly resolved) history client-side.
Plugin-only read method — no consensus change.
* pm_api: index short resolution rules (metadata.description) into market meta
The parser already ships each market's short rules text under metadata.description
(Polymarket description / Kalshi rules_primary), but meta_parse dropped it. Add
description to the meta whitelist: parse it, store it in pm_market_meta_object, and
return it in market_card's metadata. The on-chain url still points to the full legal
terms at the source; description is the short "how the oracle resolves" text for
clients. Display/indexing only — no consensus change, no hardfork.
* pm_api: newest order for list_markets + one-shot DLT meta backfill
list_markets gains an optional order arg ("oldest" default = legacy, "newest"
= id desc via reverse traversal of the by_status equal-range). Discovery feeds
need newest-first without a full scan.
Meta backfill: after --replay-from-snapshot, off-chain market metadata is rebuilt
only for markets whose create op fell in the reindex window. The DLT rolling block
log can reach further back than the snapshot, so those older create ops are still
on disk. A one-shot pass (run from on_block, budgeted 500 blocks/apply) scans the
DLT log and re-ingests meta for any market still missing it. Op->market mapping is
positional per block (meta ingest is all-or-nothing per block, so the k-th create
op created the k-th market of that block); create_meta_for() is idempotent.
Runs post-replay (reindex_from_dlt does not flush applied_block), no-op when nothing
is missing (normal restart) or when there is no DLT log.
* pm_api: fix meta backfill op->market mapping (identity key, not positional)
The first backfill mapped create ops to markets positionally by block timestamp,
assuming market.created_time == the DLT block's own timestamp. created_time is set
from head_block_time() which lags by a block, so the timestamp bucket was off and
titles landed on the wrong markets (observed shift on testnet).
Replace with a strong identity key (creator + url + betting/result expiration +
outcome count) — all consensus fields the evaluator copies verbatim from the op.
A key mismatch now simply leaves a market's meta empty; it can never mis-assign a
title to the wrong market. Duplicate keys resolve in id order via a small list.
Recovery: replay-from-snapshot starts with an empty meta index, so rebuilding the
node on this image re-ingests all meta correctly (the previous wrong meta is gone).
* pm: open-ended markets (betting_expiration=0, open until oracle resolves)
A market created with betting_expiration == 0 keeps betting open until the
oracle resolves it; result_expiration (<= now + pm_max_market_duration, i.e.
<= 1 year) becomes a pure emergency backstop. If the oracle never resolves by
then, the existing missed-resolution path in process_pm_markets refunds every
bet and slashes the oracle insurance — so the "oracle abandoned the market"
case has a trustless recourse without a dispute.
- create: betting_expiration==0 branch requires allow_early_resolution and a
result_expiration within (now, now + pm_max_market_duration]. Non-open-ended
markets keep the existing checks.
- betting / liquidity gates (place, commit, reveal, add, withdraw): treat 0 as
"open while the market is active".
- leverage: available on open-ended markets too (the extra volatility is the
bettor's own risk). The expiration-buffer check only applies when there is a
real betting deadline; guarding it also avoids the epoch-0 underflow of
(betting_expiration - buffer).
- prediction_market_api: mirror the leverage buffer guard; report
auto_close_time = 0 (no force-close point) for open-ended markets.
Resolve evaluator and the missed-resolution backstop are reused unchanged.
Gated by HF14, no separate feature flag.
* pm: leverage funding rate (perpetual carry cost on the loan)
Adds a time-based funding cost to leverage positions so a long-held loan (now
possible on open-ended markets) pays the lazy pool for the capital it ties up.
- New median-voted chain property pm_leverage_funding_rate_ppm_per_day (uint32,
ppm of the loan per 24h; default 50 = 0.005%/day; 0 disables). Validated <= 100%/day.
- pm_leverage_position_object gains funding_paid (cumulative) and funding_due_time
(next 24h boundary), plus a by_lev_funding_due index (status, funding_due_time, id).
- accrue_leverage_funding() charges whole 24h periods (one-shot catch-up) from the
bettor's equity into funding_paid, which raises the effective obligation
(liquidation_threshold + funding_paid) and thus pulls up the liquidation point.
Funding flows to LP yield through the existing pool_profit path.
- process_pm_markets() gets a funding sweep: for each due active position, accrue,
reprice at current reserves, and liquidate (reason 3) if now underwater.
- open sets funding_due_time = created_time + 24h; close/convert/liquidate accrue
first and settle funding to the pool. cascade trigger includes funding_paid.
- prediction_market_api leverage quote exposes funding_rate_ppm_per_day;
get_pm_chain_properties returns the new property automatically.
Snapshot uses generic FC_REFLECT import/export so the new fields are covered.
* pm_api: index metadata.event (parent grouping) + list_markets_by_event
Sibling markets of one real-world event (a match/game) carry the same opaque
metadata.event key. Index it off-chain in the prediction_market_api plugin
(non-consensus, prunable — same mechanism as category/description), mirroring
by_meta_category with a by_meta_event index, and expose:
list_markets_by_event(event, from, limit) -> full market cards, oldest-first
market_card and get_market_meta now surface the event field; wallet CLI gets
pm_list_markets_by_event. Parsers already emit meta.event (Polymarket event_id,
Kalshi event_ticker). Enables client event pages / parent-child nesting.
Plugin TU compiles clean; meta_parse_test covers the new field.
* pm: raise multi-outcome limits for testnet (pm_max_outcomes default 10->64, MAX_PM_OUTCOMES_PER_MARKET 16->128)
* pm: raise lmsr n cap 16->128 to match MAX_PM_OUTCOMES_PER_MARKET (completes multi-outcome limit bump)
* pm-api: index & serve metadata.event_title (readable event label for event pages/cards)
* pm-props: default pm_leverage_enabled=true on pm/testnet branch
Fresh chains built from this branch enable leverage out of the box. On an
already-running testnet the median value persists in state, so validators
must still vote pm_leverage_enabled on to activate it there.
* pm-api: serve real binary outcome labels in weight-sums (was hardcoded A/B)
make_weight_sums hardcoded binary outcome labels as A/B, so clients could
not show the actual outcome names (team names, Yes/No, Over/Under) even
though pm_create_market stores real labels on-chain in pm_outcome_object
for binary markets too. Read those labels for binary as well (fallback to
A/B only when a label is blank). Feeds get_market_weight_sums/get_market_full
so Forecaster can render named outcome bars.
* pm-api: case-insensitive tag filter in list_markets_by_category
Tag source casing is inconsistent ("Dota 2" vs "counter strike 2") and
clients lowercase tags, so a case-sensitive match made a 'dota 2' query
return nothing. Add meta_csv_contains_ci (both sides ASCII-lowercased) and
use it for the tag filter; comma-boundary semantics unchanged. Category /
jurisdiction / subcategory matching left as-is.
* pm-api: add get_category_tag_counts(category) for stable per-category tag counts
get_market_categories only exposes GLOBAL top-20 hot tags, so a browse UI had
to count tags from whatever page it loaded — the number jumped (e.g. Dota 2
shows 1 in the newest-page view, 540 once the tag is opened). New method
returns authoritative per-tag counts within one category (all tags, scanning
only that category's index slice), so the UI can show a stable count that
doesn't depend on the loaded page. Tags returned in hot_tags (categories empty).
* pm-api(list_markets_by_category): attach volume to volume/expiration-sorted rows
The category listing already loads each market to sort by volume/expiration, but returned only
the meta_object (no financials), so clients had to fire a per-market get_market_weight_sums just
to show a volume badge and could not rank markets globally across categories (Forecaster Popular
fell back to round-robin). Return each row as a variant and, when the market was loaded, attach
volume (= bets_sum, raw shares). Wire-compatible: existing meta fields unchanged, newest/oldest
rows omit the field (client fills lazily). Return type vector<pm_market_meta_object> -> vector<fc::variant>.
* pm-api: hide child/prop markets from list_markets_by_category by default (hide_children arg, meta.child field)
* lazy-pool: never overpay withdrawals — queue (FIFO) the amount owed and pay from free_balance as capital returns; support partial withdraw; add pending_withdrawals + get_lazy_withdraw_requests
* docs(lazy-pool): document withdrawal FIFO queue + pending_withdrawals + free_balance≥0 invariant (Transitions 9/11/12)
* lazy-pool: price deposit shares against pool equity (free+allocated-pending_withdrawals), not free_balance alone — fixes over-issuance when capital is deployed; docs updated
* chain: log block-gen tx-apply exceptions instead of swallowing silently
The block-generation loop wrapped each pending-tx apply in a try/catch whose
wlog lines were commented out, so a transaction that passes pending push but
fails _apply_transaction at block generation was dropped with no trace in the
node log (valid-at-pending / never-included / sync-broadcast-hangs, and nothing
logged). Re-enable the diagnostic wlog so such drops surface the fc::exception.
* pm-api: attach live status/payout_status/resolved_outcome to list_markets_by_category rows
* pm-api: list_markets_by_category emits array tags + metadata object (no raw CSV in listings)
* network_broadcast_api: elevate swallowed accept_transaction failures dlog->elog (surface silent apply/assert drops)
* pm: fix leverage_open insert failure — add id tiebreaker to by_lev_bet unique index (bet field is always default 0)
* pm-api: aggregate per-oracle listing risk floor + apply to category/event listings
Insurance backs an oracle's whole book, not each market. Rework below_risk_floor
into oracle_below_risk_floor (per-oracle, cached per head block):
(A) insurance < pm_min_oracle_insurance -> hide the ENTIRE book, incl. zero-bet
markets. Fixes broke/slashed oracles staying fully visible because every
fresh market had bets_sum==0 and so never tripped the old per-market check.
(B) insurance*100 < coverage% * sum(bets_sum over the oracle's OPEN markets).
Also apply the filter (show_risky opt-out) to list_markets_by_category and
list_markets_by_event, which Forecaster uses to browse and previously bypassed
the floor entirely. Non-consensus (API plugin) -> API-node rebuild, no replay.
* issue #127: TEMP snapshot invariant instrumentation — per-status PM token breakdown
Logs per-status token sums for bet/leverage/commit/dispute/liquidity plus a
bet_held-vs-Σmarket.bets_sum drift, right before the TOKEN invariant line, to
localize the delta=-9079 undercount. Read-only; revert after localizing.
Branch off origin/pm (5230165d). -fsyntax-only clean.
* issue #127: TEMP instr round 2 — sum market-level token holders (reserve_a/b, lmsr_subsidy, liquidity_fee_earned, oracle_fixed_fee) to localize -9079 residual
* pm(issue #127): route leverage-exit curve residual to forfeit_pool
At leverage open, collateral+loan (total_bet) leaves circulation and is tracked
as the position (counted while active). At close/liquidate only cancel_value cv
returns to pool+bettor; the remainder total_bet-cv (AMM spread + kdiv floor) was
left frozen in the virtual reserves and, on never-settling markets, became an
untracked token deficit (the static ~9079-raw snapshot residual, issue #127).
Route that remainder to market.forfeit_pool at both exit sites so token supply
reconciles exactly: it is counted by the snapshot invariant (pm_forfeit) and
distributed to winners at settlement, matching the design intent that price-impact
accrues to the rest of the market. pm_leverage_convert already conserves (position
becomes a real bet, bets_sum += total_bet), so only liquidate_position and
pm_leverage_close needed the fix.
* pm(issue #127): re-arm PM supply invariant, anchored to testnet legacy residual
Localization done + leverage-exit residual fixed (6b002c14), so re-arm the PM token
supply invariant: it now ++invariant_failures (fails snapshot import) on ANY divergence.
Removes the TEMP per-status / market-fields instrumentation probes.
TEMP TESTNET ANCHOR: PM_SUPPLY_LEGACY_RESIDUAL = -9079. The current testnet snapshot
carries a frozen PRE-FIX leverage-floor residual that code cannot retro-heal; anchoring
the check to that known constant makes the invariant live again and catches any NEW
drift, while tolerating the legacy value. Set the anchor to 0 for mainnet / after a clean
state rebuild → strict ==0 enforcement. See q#199=B.
* pm: per-account frozen-funds counters on account_object (foundation)
Add three display-only running aggregates to account_object so clients can
read how much of an account's own funds is frozen in prediction markets in a
single get_accounts call, instead of summing tens of thousands of markets:
pm_liquidity_committed - own LP/creator liquidity in live markets
pm_bets_staked - own stake in open/queued bets
pm_leverage_collateral - own collateral in active leverage positions
This commit lays the foundation only:
- fields on account_object + account_api_object (get_accounts DTO)
- database::pm_adjust_frozen(account, kind, delta) maintenance helper
(clamped at zero; NEVER gates consensus - cosmetic telemetry only)
- database::pm_seed_frozen_counters() one-time walk of existing
pm_liquidity / pm_bet / pm_leverage_position objects
- seed trigger + guard flag dgpo.pm_frozen_counters_seeded at the top of
process_pm_markets(), so counters are seeded without a full replay
(VIZ testnet is DLT-only, no genesis)
Not yet wired: the ~25 incremental lock/unlock/move touchpoints in
pm_evaluator.cpp. Until those land the counters seed once and then drift,
so this branch is NOT deploy-ready. Single-TU -fsyntax-only clean.
* pm: instrument frozen-counter touchpoints + drift-check invariant
Maintain the account_object frozen aggregates incrementally at every PM
lock/unlock/move point in pm_evaluator.cpp:
liquidity: create_market seed, add_liquidity (LOCK); withdraw_liquidity,
settle_liquidity, return_liquidity (UNLOCK principal only)
bets: place_bet binary+LMSR, reveal (LOCK); void refund, winner/loser
settle, refund_all_bets, cancel_bet, batch-settle slippage
(UNLOCK); transfer_position full/partial (MOVE)
leverage: leverage_open (LOCK); liquidate_position, leverage_close
(UNLOCK); leverage_convert (MOVE collateral->bets as total_bet)
Fees to recipients (oracle/creator/committee/void-compensation) and the
loan portion of leverage stay out - only the account's own principal
entering/leaving the locked set moves a counter.
Add database::pm_verify_frozen_counters(): re-derives the expected totals
from live objects and diffs the stored aggregates, logging any drift.
Called once right after seeding to log a baseline (clean/MISMATCH); safe to
invoke later to detect drift. Single-TU -fsyntax-only clean.
* pm: add by_oracle_status composite index + list_markets_by_oracle_status API
Adds a (oracle, status, id) composite index on pm_market plus a read-only
list_markets_by_oracle_status(oracle, status, from, limit) API (plugin +
wallet). Lets clients pull one oracle's markets filtered to a single status
(active/resolved/...) in a bounded index walk instead of fetching by_oracle
and filtering client-side. No hardfork, read-only.
* docs(pm-api): document list_markets_by_oracle_status(oracle, status, from, limit)
Add the new per-oracle-per-status listing to the prediction-market-api reference
table + description, and to the library-integration spec. Docs only (no build).
* pm(#266): idempotent corrective re-seed of frozen counters (fix over-count)
The live testnet pm_liquidity_committed read ~3.76x a physically-impossible value
(30.46M VIZ for polymarket vs a ceiling of markets*seed = 124,984*100 = 12.5M; real
object sum ~8.1M) — legacy/inflated seed state. The runtime lock/unlock touchpoints
(create/add/withdraw/settle_liquidity/return_liquidity, bet place/refund/settle,
leverage) are correct, so the drift is baked into the seeded value, not ongoing.
Fix: make pm_seed_frozen_counters() idempotent — zero every account's three counters
first, then re-sum from the live objects — and trigger it once more via a new
dgpo flag pm_frozen_counters_reseeded_v1. Counters are display-only (never gate
consensus), so recomputing is safe and needs no HF/replay. On the next block after
deploy the counters converge to the exact object totals (drift-check logs clean).
* prediction_market_api: list_markets order='expiration' — global soonest-closing feed (by_betting_expiration, still-open only)
* snapshot: carry PM metadata across snapshots (DLT-safe, config-optional)
The pm_market_meta index (titles/images/tags/event) is non-consensus plugin
state rebuilt from pm_create_market ops. On DLT nodes (testnet and mainnet — no
genesis, bounded block-log) that rebuild loses meta once the create op rotates
out of the stored block window, and snapshots never carried it, so meta was lost
on snapshot hand-off (empty titles).
- Move the pm_market_meta object/index TYPE from the prediction_market_api plugin
into libraries/chain (graphene/chain/pm_meta_object.hpp), namespace graphene::chain,
so the snapshot plugin can serialize it with no plugin-to-plugin dependency. The
old plugin header becomes a thin shim (using namespace graphene::chain) — zero
churn in prediction_market_api.cpp; runtime registration (add_plugin_index) stays
in the plugin. Object-type id unchanged (space 30) so shared_memory stays compatible.
- snapshot plugin: new config option snapshot-include-pm-meta (default true).
Export a pm_market_meta section, guarded on include_pm_meta && has_index<> (only
when the PM API plugin is loaded). Dedicated import handler (10 shared_string
fields → not the generic fc::from_variant path), called under
state.contains("pm_market_meta") && has_index<>.
- Fully backward/forward compatible: old snapshots lack the key -> import skips it
-> node falls back to rebuild-from-DLT; old nodes reading new snapshots ignore
the extra key.
Verified: single-TU -fsyntax-only on plugins/snapshot/plugin.cpp and
plugins/prediction_market_api/prediction_market_api.cpp (both exit 0).
* pm: force-close leverage when betting ends, not at settlement
A leveraged position bets on the market price (crowd sentiment) and settles at
cancel_value, independent of the oracle outcome. It should not linger open through
resolution + dispute grace, bleeding funding and remaining funding-liquidatable
while nobody can bet anymore.
process_pm_markets now force-closes an open position the moment new betting is
impossible: at betting_expiration for fixed-deadline markets (before the oracle
resolves), or at resolve/void (status>=3) for open-ended markets. Anchored on the
status-0 position set via by_lev_funding_due so each position closes exactly once;
settle_market/return_liquidity remain idempotent backstops.
Docs (specification 16a, whitepaper 5.5, workflows) updated to state leverage does
not wait for the oracle result.
* docs(prediction-markets): link published SSRN version of Onix whitepaper
* docs(prediction-markets): note leverage-exit residual routes to forfeit_pool (pool curve dust)
* pm: add O(1) per-oracle active_markets counter (oracle_object -> get_oracle)
Live count of an oracle's markets currently in the active(1) state, so
get_oracle/watchdogs read it in O(1) instead of paging list_markets.
- pm_oracle_object.active_markets (uint32, display-only, never gates consensus),
auto-exposed via the get_oracle API DTO (embeds+reflects the whole object).
- Maintained incrementally: +1 on create-active (self-oracle/auto-accept) and
manual accept (status 0->1); -1 on resolve, no_contest and missed-resolution
void (1->3). Guarded dec helper is idempotent for already-terminal markets.
- One-time seed from live markets on the first block after upgrade, gated by the
new dgpo.pm_active_markets_seeded flag (old frozen-counters gate already spent),
plus a debug drift-check pm_verify_oracle_active_markets(). Mirrors the
frozen-counters seed pattern; no full replay needed.
Single-TU -fsyntax-only clean.
* docs(prediction-markets): use formal SSRN citation for the Onix whitepaper
* fix(pm): price-neutral CPMM liquidity add/withdraw (PR #124 B4/B5)
Binary CPMM liquidity ops mishandled the reserves:
- add split the deposit 50/50, moving the odds on unbalanced curves;
- withdraw shrank liquidity_sum but left reserve_a/b/k untouched, so an
add->withdraw round-trip handed out ~30% free weight in the winners'
pool (the exploit Babin and Vyacheslav both hit).
Scale both reserves by (L +/- amount) / L, L = liquidity_sum before the op,
so the reserve ratio (odds) is unchanged and only depth tracks capital;
the round-trip is now exact. Applied to pm_add_liquidity,
pm_withdraw_liquidity and the lazy-pool recall (its mirror path). The
lazy-pool activation deploy stays 50/50 - it only runs at genesis when
reserves are already balanced, where proportional == 50/50.
The withdraw guard condition was in fact correct (early withdrawal allowed
during betting, positions locked once betting closes until resolution -
spec section 9); only its assert *message* was misleading. Fix the message
rather than invert the condition (inverting would have removed the
documented Early Withdrawal feature and allowed withdrawal in the
post-expiration lock window). Enforce the spec's min-liquidity floor on
early withdrawal - now required, since reserves track liquidity_sum and a
full exit would otherwise drain the curve to zero.
Reserves are the virtual pricing curve; real VIZ = liquidity_sum + bets_sum
is untouched, so token conservation and the PM supply invariant are
unaffected. Verified single-TU -fsyntax-only. Docs updated (spec section 9).
* fix(pm): PR #124 review fixes B3/B6/B7/B1 (mint, k-corruption, cancel gate, snapshot anchor)
B3 (critical, mint) - a leveraged position that closes in profit routes a
NEGATIVE curve_residual (cv > total_bet) into forfeit_pool. That is the
correct signed accounting - the surplus was paid to the winner out of the
pool, so it must net down the parimutuel winners' pool - but compute_settlement
then cast a negative winners_pool to uint64_t, wrapping to ~1.8e19 and minting
(seen live: market 19 forfeit_pool = -69227). Fix in compute_settlement: floor
winners_pool at 0 before any uint64 cast, so a net-negative pool just returns
winners their principal and never mints. Keep the signed routing (clamping the
residual instead would break token conservation for winning closes - the surplus
would not be deducted and settlement would over-distribute). Verified against
issue #127 conservation model.
B6 (critical, k-corruption) - pm_cancel_bet reversed the nominal stake out of
the CPMM reserve without a floor; intervening opposite-side bets can move the
curve so the stake no longer fits, underflowing share_type and corrupting
k = reserve_a x reserve_b. Assert the reserve covers the stake, else refuse the
cancel (the bet can still settle).
B7 (high, cancel gate) - pm_cancel_bet had no betting_expiration gate, so a
losing bet could be unwound after the outcome became known. Gate it to the
pre-close window (open-ended markets stay cancellable while active).
B1 (release blocker, snapshot) - the PM supply invariant anchor was hardcoded
to -9079, so every healthy/fresh chain (delta == 0) failed import. Make it the
config option snapshot-pm-legacy-residual (default 0 = strict); a testnet
carrying frozen pre-fix dust sets it to that constant, mainnet stays strict.
B4/B5 landed earlier (172d87c4). B2 (pm_lazy_withdraw_request in snapshot) and
B9 (time_penalty) are tracked separately. Verified single-TU -fsyntax-only on
all three changed TUs.
* feat(pm): wire up B9 late-bet time penalty (PR #124 review)
The anti-sniping penalty (spec section 8) was fully scaffolded - market stores
time_penalty_type/value/curve, medians exist, pm_bet_object.time_penalty is read
at settlement (parimutuel profit deduction) - but no bet-creation path ever
COMPUTED and assigned it, so it stayed 0 and the protection was inert. A late
"sniper" could bet on a near-certain outcome and take a parimutuel share of the
losers' pool without having taken real risk, diluting honest early bettors.
Add compute_time_penalty() (integer-only, spec section 8: fixed/percentage
window, linear/quadratic curve, scaled by median pm_max_time_penalty) and assign
it at every bet-creation site, keyed to when the market exposure was actually
taken:
- pm_place_bet (binary + LMSR): placement time;
- pm_reveal_bet: the blind COMMIT time (not the reveal - honest late reveals
within the window are not punished);
- pm_leverage_convert: the leverage OPEN time;
- pm_transfer_position: inherited from the source bet.
Dormant by default (markets created with time_penalty_value = 0 get 0; open-ended
markets have no window), so it only activates when a creator opts in - no change
to existing markets. Verified single-TU -fsyntax-only. Spec section 8 updated
with the risk-time source table.
* fix(pm): include pm_lazy_withdraw_request in snapshots (PR #124 B2)
The pending lazy-pool withdrawal requests were the only PM consensus index left
out of snapshot export/import. On reload the request objects vanished while
pm_lazy_pool.pending_withdrawals still carried their total, so the FIFO payout
queue no longer matched its accounting - a consensus divergence between replayed
and snapshot-loaded nodes, with the owed funds stuck (counted as pending but no
request object to pay out).
Serialize it exactly like its seven PM siblings: EXPORT_INDEX in the writer and
import_simple_objects in the reader (section "pm_lazy_withdraw_request"). Its
per-section count joins header.object_counts automatically. Verified single-TU
-fsyntax-only. Round-trip (snapshot with open withdraw requests -> reload ->
queue intact) to be confirmed on the owner's testnet redeploy.
Completes the PR #124 review: B1-B9 all addressed.
* fix(pm): bound pm_max_time_penalty in validate + clamp penalty at profit (PR #124 F3)
B9 wired compute_time_penalty to spend pm_max_time_penalty as profit*penalty/1e6 in
compute_settlement, but chain_properties_pm::validate() never bounded it (unlike every sibling
ratio). A median vote > 1e6 would make a winner's payout negative (settle_market drops it) while
lp_bonus still carries the phantom penalty. Bound it <= 1e6 in validate(), and defensively clamp
penalty <= profit in compute_settlement so no median value can break settlement.
Found by @chiliec in the PR #124 fix-round review (finding 3).
* fix(pm): charge parimutuel shortfall to LP principal instead of emitting it (PR #124 F1)
The B3 winners_pool floor stopped the uint64 wrap but still emitted exactly |winners_pool| tokens:
winners got their principal back out of tokens the losers'+forfeit pot never funded, while
settle_liquidity returned LP principal unconditionally, so nothing absorbed the shortfall. Any
market whose accumulated leverage profit (negative forfeit_pool) outran ~96% of the losing stakes
over-emitted at settlement.
compute_settlement now reports the shortfall as settle_result::uncovered, and settle_liquidity
charges it against LP principal pro-rata (LPs are the leverage counterparty, capped ≤ their
principal, last LP absorbs the rounding remainder). The header zero-sum identity now holds
unconditionally. The strict supply invariant (#126) remains the backstop for the pathological
uncovered > Σ-principal case (pos_cap should prevent it).
Found by @chiliec in the PR #124 fix-round review (finding 1).
* fix(pm): curve-price binary bet cancellation instead of nominal reversal (PR #124 F2)
pm_cancel_bet reversed the nominal (amount, weight) pair recorded at bet time and rebuilt k from
the result. On any curve that had moved since the bet — an opposing bet is enough, no LP op or
leverage needed — this corrupted k = reserve_a*reserve_b and left the retained position an inflated
claim weight (chained bet/cancel reached +75% self-referentially, up to 103x with asymmetric
stakes) that settles as real money, while also handing a free option to unwind a losing bet at 100%.
A cancel is now the mirror of the buy: sell bet.weight back at the current reserves,
new_reserve_in = k/(reserve_out + weight), refund = reserve_in - new_reserve_in. k stays invariant
(no reserve can underflow, subsuming the B6 guard), min_return protects the bettor from an adverse
move, and the gap between the original stake and the curve-priced refund routes to forfeit_pool
(signed) so VIZ is conserved: a loss accrues to the market, a gain is charged to LP principal at
settlement via the F1 shortfall path. Type-1/LMSR cancellation is unchanged (nominal, out.q based).
Found by @chiliec in the PR #124 fix-round review (finding 2, the serious one).
* test(pm): import @chiliec standalone replay harness for PR #124 review (#129)
19 standalone replay programs under tests/pm/replay/ that reproduce each PR #124 review finding and
verify the fix round, plus a build.sh, an fc::uint128_t shim (native __int128, bit-identical to
fc/src/uint128_t.cpp:224,257) and a README. They link against the real pure consensus math
(parimutuel.cpp/leverage.cpp) with no node/libfc; evaluator transitions are modelled inline and
cite their line numbers. Not part of the CMake build (each is a main()); kept as reference next to
the CI-wired unit tests. The findings they surfaced (F1/F2/F3) are fixed on this branch; the
CI-wired regression guards live in parimutuel_test.cpp.
Authored by @chiliec (Vladimir Babin), handed over as PR #129.
* test(pm): CI-wired regression guards for PR #124 F1/F3 + fix invariant comment
Fold the pure-math half of @chiliec's replay findings into the existing Boost.Test target
(parimutuel_test.cpp, already run by ctest):
- negative_winners_pool_reports_uncovered: a forfeit_pool more negative than the losers' pot yields
settle_result::uncovered = |winners_pool| and pays winners exactly their principal (F1).
- negative_forfeit_still_covered_no_uncovered: floor never engages while the pot covers it.
- time_penalty_clamped_at_profit: a time_penalty > 1e6 is clamped so payout never drops below
principal (F3).
check_conservation now counts on the INPUT side (it is an external top-up charged to LP
principal by the caller), and the parimutuel.hpp identity comment is corrected to match.
F2 (curve-priced cancel) lives in pm_evaluator.cpp (needs the database), so its regression belongs in
a DB-backed test; @chiliec's standalone t11/t12 replay cases cover it in tests/pm/replay/ meanwhile.
* fix(pm): actually assign settle_result::uncovered before the floor (PR #124 F1)
The F1 producing side was left out of 5ff694e — compute_settlement floored winners_pool without
ever setting r.uncovered, so the pro-rata LP charge in settle_liquidity was dead code and the
shortfall was still emitted. Set r.uncovered = -winners_pool before flooring so the caller charges
it to LP principal and the zero-sum identity holds. With F2, a same-side follow-on bet moving the
curve makes the curve-priced refund exceed the stake, driving forfeit_pool negative with no leverage
at all, so this path is reachable in ordinary betting, not only via leverage.
Caught by @chiliec (PR #130, t14_f2_ledger).
* test(pm): add @chiliec t14_f2_ledger replay case (PR #130)
Standalone replay that models pm_cancel_bet as rewritten in f3ff915 and pm_place_bet's CPMM buy,
then settles through the real compute_settlement. Part A verifies the F2 curve-priced cancel is an
exact mirror of the buy (k invariant, weight inflation closed, chain now has a real cash cost).
Part B is the ledger that caught the F1 dead-code bug (fixed in bd1dd67d). Same standalone style as
the rest of tests/pm/replay/, not part of the CMake build.
Authored by @chiliec (Vladimir Babin), handed over as PR #130.
* fix(pm): make the F1 LP charge inescapable + robust remainder (PR #124 review)
Three issues in the F1 uncovered->LP-principal sink, all found by @chiliec:
1. Escape in the dispute-grace window (the serious one). pm_resolve_market sets status=3 but
settle_market runs from the deferred cron sweep only after result_expiration - pm_dispute_grace_sec
(~12h). pm_withdraw_liquidity unlocked on status>=2 and skipped the pm_min_liquidity floor at
status 3, so an LP could pull 100% of principal in that window, empty settle_liquidity's
set (early return before the charge) and dodge its share of — forfeit_pool is public on
get_market, so the exit is a rational, computable choice. Gate on finalized_time instead: it is 0
until a terminal transition (settle/void/expire), so LP principal stays locked through resolve->
settle while already-finalized markets are still served.
2. Remainder could exceed a small last LP. The floor-charge remainder was dumped on the last LP,
which a naive small last LP couldn't absorb -> principal_ret clamp -> re-emission. Spread it over
LPs with headroom instead (total headroom always covers it).
3. uncovered > Σ principal is a real bounded over-emission that #126 only catches on a later import;
wlog it at the settlement site where it happens (pos_cap should keep it unreachable).
* docs(pm): correct payout_status enum comment and refresh t14 epilogue (PR #124 review)
payout_status comment said '2 paid, 3 disputed' but the code is the other way round: 2 is disputed
(pm_dispute_create, the only writer) and 3 is finalized/paid (settle sweep, closed-no-payout). Swap
the comment to match. Also refresh the now-stale epilogue in t14_f2_ledger.cpp: uncovered is assigned
as of bd1dd67, so the ledger reads out - uncovered == held with no emission. Both noted by chiliec.
* fix(pm): close the F1 withdraw escape for open-ended markets too (PR #124 review)
The finalized_time gate half-closed it: clause 1 (betting_expiration == 0) is unconditionally true
for open-ended markets, which are first-class (betting_expiration = 0 is legal and requires
allow_early_resolution). Such a market resolves early to status 3 with forfeit_pool public, and the
withdraw path had no status check, so clause 1 alone let an LP exit through the whole resolve->settle
window (floored only to pm_min_liquidity = 100 VIZ) and re-emit uncovered - 100 -- and open-ended is
exactly where uncovered is most likely, since the leverage expiration buffer is waived there.
New gate (per chiliec): withdrawable iff finalized, OR still pre-resolution AND inside the betting
window. status < 2 makes resolution itself the lock trigger regardless of a betting deadline; the
betting-window clause keeps early withdrawal working during betting, including open-ended.
Also: hoist the no-LP diagnostic above settle_liquidity's active.empty() early return -- the one case
that emits 100% of uncovered was the only one that logged nothing. And document that early resolution
deliberately does not advance result_expiration (disputers keep the advertised window; it also sets
the LP lock / exposure length).
* feat(pm): pull result_expiration forward on early resolution (PR #124 review, q#326=B)
Early resolution previously left result_expiration at its advertised (possibly
far-future) value, so LP principal stayed locked and the market unsettled until
result_expiration + dispute grace — up to ~pm_max_market_duration for open-ended
markets. Per owner decision, an early oracle report now shifts result_expiration
earlier by exactly the margin it beat the deadline (new value = now), mirroring
pm_no_contest. Disputers keep the full pm_dispute_grace_sec window, re-anchored to
the announcement. The shift is one-directional: a late report (now >= result_expiration)
leaves the window untouched, never extending a disputer's/settle deadline.
Refresh the F1-escape comment and the spec state-transition table accordingly.
* fix(pm): give the missed-resolution void a grace so pm_resolve_market is reachable (PR #124 review)
process_pm_markets() runs at the end of a block, after dgp.time has advanced to this
block's timestamp, while in-block transactions still saw the previous block's time.
The missed-resolution cron voided any active market at result_expiration <= now with
zero slack, so it fired one block before the earliest resolve transaction whose clock
could reach result_expiration. A fixed-deadline market without allow_early_resolution
(legal at creation, only required for open-ended markets) was therefore impossible to
resolve via pm_resolve_market: it always terminated as missed-resolution, refunding
bets/liquidity and slashing the oracle for a deadline it had no reachable block to meet.
Only can_resolve_early and pm_no_contest were reachable. Found by @chiliec.
Void only once result_expiration + pm_dispute_grace_sec has elapsed (the same cutoff the
settle sweep in §5 uses), leaving the oracle a real window [result_expiration,
result_expiration + grace] to report. This also makes the one-directional late-report
branch of the result_expiration pull-forward (aa3a1e9c) re…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #125. Adds the deferred export-side completeness detector for snapshot import, and fixes a stale field that blocked the consensus-sim harness build.
Why
The #125 incident class is export-side incompleteness: a serving node serializes state with an account missing, the download matches its checksum, count-reconciliation passes (the recorded counts derive from the same short arrays), import succeeds, and the node wedges the moment a canonical block references the absent account (
out_of_range "viz-social-bot"). Neither the checksum nor the count check can see this. A value invariant can: a missing balance/stake holder drops the summed total below the independently-tracked dgp supply.What
plugins/snapshot/plugin.cpp— two post-import checks inload_snapshot():dgp.total_vesting_shares == Σ account.vesting_shares. Delegation objects only redistribute already-counted vests → excluded.dgp.current_supply == Σall liquid/locked TOKEN pools (account balance + reserved, escrow balance + pending_fee, invite balance, validator pending reward, vesting/reward/committee funds).Both are strict equalities that raise the existing retryable
FC_ASSERTon mismatch (→ retry another trusted peer), consistent with the other completeness checks. Per-component subtotals are logged unconditionally so any future mismatch is attributable to a specific pool.tests/consensus_sim/harness/simulated_node.cpp: readsigned_block::validatorinstead of the stale->witness(Steem-ism). The harness had never compiled against current VIZ headers; this unblocks the suite so theis_wedgedtest from #125 can run.Verification
Built on a real Linux/Boost env (Docker,
Dockerfile-production). Ran the new vizd against six real mainnet snapshots (blocks 81334800–81620400): both invariants reconcile delta=0 to the satoshi at every height, including the height-varying validator pending-reward term. Thewedge_predicateunit test (from #125) also builds and passes 8/8 with the harness fix.