Checking systems...
Per-source health for the engine and every oracle it reads, plus a permanent incident log. We publish every issue our own monitoring catches: a watcher you can't audit isn't a watcher.
Per-source uptime. Last 90 days.
One bar per day, colored by the state Pegana observed. Earlier days show as no-data: we never paint history we didn't record.
| Live channel engine -> /v1/ws heartbeat | 90 days agoToday | - · No data |
| Alert delivery telegram / webhook dispatch | 90 days agoToday | waiting for stats · No data |
| Sanctum LST intrinsic · on-chain stake-pool value | 90 days agoToday | 15s · No data |
| Jupiter Market quote · best route | 90 days agoToday | 30s · No data |
| Pyth Live price oracle | 90 days agoToday | ~1s · No data |
| Hylo On-chain CR + stability pool | 90 days agoToday | 30s · No data |
Alert activity. Rolling 24h.
Calibration. Are the bands still right?
Each asset's trailing-8d p95 spread measured against its own drift / depeg bands.
See the full per-asset table →
Incidents. Permanent log.
1 user-facing incident. We still publish every issue our own monitoring caught internally.
A new freeze detector briefly mislabelled JupSOL as Unknown
We shipped an engine 'freeze detector' meant to publish UNKNOWN when a price basis stops moving (a real failure mode we'd just found on another asset). Its threshold, 20 consecutive identical samples, about 3 minutes, was too low for thinly-traded liquid-staking tokens, whose price ratio is naturally constant for minutes between trades. Six such assets, including the public JupSOL, were briefly and wrongly marked Unknown.
Detection. Caught minutes later by our own post-deploy full-state verification. Assets had gone Unknown with no corroborating market move.
Root cause. The pre-deploy validation measured the total number of distinct values over 6 hours, not the maximum consecutive-identical run the detector actually keys on. So it missed that a healthy but illiquid asset routinely sits identical for a few minutes. A threshold tuned to minutes cannot tell 'frozen for hours' (the real failure) from 'no trade for a few minutes' (normal).
The fix. Reverted the freeze detector and redeployed the engine; every state recovered within ~9 minutes. The separate containment of the genuinely-frozen asset (hyloSOL+) was unaffected. The detector will be redesigned around a multi-hour window, or to detect the upstream source freeze directly, and re-validated on the exact metric it uses, before any redeploy.
JupUSD briefly read CRITICAL on a single bad market quote
A single Jupiter market quote returned a zero output amount, so the market price computed as $0 and the discount pinned to exactly 1.0 (−100%). That one bad sample poisoned the smoothing filter (EWMA, α=0.3) and JupUSD briefly classified CRITICAL even though the market had not moved.
Detection. Flagged by our own post-launch T+24h health scorecard. A CRITICAL on an otherwise calm asset with no corroborating market move stood out immediately.
Root cause. Two systemic gaps: the plausibility check accepted an impossible reading (a market price of exactly $0) instead of rejecting it, and the source adapter had no floor to drop a zero / near-zero quote before it entered the pipeline.
The fix. The indexer now skips zero and near-zero Jupiter output amounts and applies a NAV floor at the source, and the plausibility guard was tightened to strictly < 1.0 so a fully-zeroed price can never reach the smoothing filter again.
Uncalibrated assets could have reached the public alert feed
A calibration downgrade marked 11 uncalibrated assets as monitoring-only, but the flag was display-only: the engine and dispatcher had no awareness of it, so those assets could in principle have emitted alerts on the public feed.
Detection. Found during a pre-promotion review of the full alert egress path, before any uncalibrated asset transitioned.
Root cause. The monitoring-only status was stored as metadata with no corresponding enforcement at the public egress surfaces. A status flag without a gate.
The fix. An egress gate now filters monitoring-only assets at every public surface: the alerts list, the audit ledger (index and CSV), the stats counters, the dispatcher fan-out, and the engine's on-chain commit. Internal rows are still recorded for the calibration retro.