← All lessons/Administration
57
Administration

Sherapd & Alabay Watchdogs

SapixDB's built-in performance and security watchdogs: query their strands, understand severity levels, and follow the ops runbook for each alert type.

Prerequisite: Lesson 56 complete

What you'll learn

  • Sherapd (performance): throughput, latency P50/P95, disk growth, cron health
  • Alabay (security): auth failures, chain integrity passes, injection detection, cage anomalies
  • Both are named agents — query with GET /v1/agents/sherapd/strand/records
  • Severity: info (TRACE only, not on strand), warn, alert, critical
  • SaQL filter to alert-only: where severity eq alert on the watchdog agent
  • Sherapd alert runbook: export, checkpoint, scale disk
  • Alabay alert runbook: sapix chain verify, investigate source IP, escalate critical findings
  • In normal healthy operation both strands are empty
Challenge

Query the Sherapd strand for any warn or alert records. If none exist (healthy system), use sapix chain verify on an agent to trigger a verification pass. Then SaQL-filter Alabay records by severity=alert.

## Sherapd & Alabay — Internal Watchdogs

SapixDB ships two built-in watchdog agents that monitor itself from the inside and write their findings as immutable strand records.

WatchdogFocusQuery
SherapdPerformance & healthGET /v1/agents/sherapd/strand/records
AlabaySecurity & integrityGET /v1/agents/alabay/strand/records

Both start automatically at process boot. No configuration required.

---

## Sherapd — Performance Watchdog

Sherapd monitors write throughput, query latency, disk growth, and agent health. It writes a strand record for every warn or alert observation. Routine info findings are logged at TRACE only — the strand stays empty during normal healthy operation.

What Sherapd monitors

  • Write throughput and group-commit batch sizes
  • Query latency (P50, P95 sliding window)
  • Disk usage per strand directory
  • Per-agent record growth rate
  • Cron job and trigger execution times
  • SSE subscriber count

Query Sherapd's history

curl "http://localhost:7475/v1/agents/sherapd/strand/records?limit=50" \
  -H "Authorization: Bearer $KEY"

Sample record payload: `json { "severity": "warn", "task": "disk_growth", "message": "Strand directory exceeds 80% of configured limit", "agent_id": "orders", "value_mb": 8192 } `

SaQL: filter to alerts only

curl -X POST http://localhost:7475/v1/query \
  -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "sherapd",
    "type": "latest",
    "limit": 100,
    "where": [{ "field": "severity", "op": "eq", "value": "alert" }]
  }'

---

## Alabay — Security Watchdog

Alabay detects intrusion attempts, runs scheduled chain integrity passes, and monitors access anomalies. Its findings are strand records secured by the same BLAKE3 + Ed25519 chain as all other data — an attacker cannot erase them without breaking every hash that follows.

What Alabay monitors

  • Repeated authentication failures (brute-force detection)
  • Chain integrity: background passes checking parent_hash links and Ed25519 signatures
  • Unusual write rates that could indicate injection attacks
  • Cage state changes (agent caged unexpectedly)
  • Schema drift from approved Mutant proposals

Query Alabay's history

curl "http://localhost:7475/v1/agents/alabay/strand/records?limit=50" \
  -H "Authorization: Bearer $KEY"

Sample finding: `json { "severity": "alert", "check": "auth_failure_rate", "message": "15 failed auth attempts in 60 seconds", "source_ip": "203.0.113.42", "count": 15 } `

Severity levels

LevelWritten to strand?Example
infoNo (TRACE log only)Chain verified clean
warnYesDisk at 75%, elevated latency
alertYesDisk at 90%, brute-force detected
criticalYesConfirmed chain tampering

Ops runbook

warn on Sherapd: Check disk with df -h, review query load with GET /v1/status.

alert on Sherapd: Export the strand now (sapix agents export) before disk fills. Consider a checkpoint.

warn on Alabay: Review auth logs. Check if the source IP belongs to a known service.

alert/critical on Alabay: Immediate investigation. Run sapix chain verify <agent_id> to confirm or rule out tampering.

---

Sherapd and Alabay write their findings as standard strand records — you can query them with any SaQL query, including time-range scans for a specific incident window. See also: Lesson 24 (chain verification), Lesson 56 (CLI), Lesson 58 (DBA Agent).

← Previous
Lesson 56: CLI (sapix)
Next →
Lesson 58: DBA Agent