Skip to content

v1.3.41.0 -- Read pool: reads leave the writer connection

argosVersion 1.3.40.4 -> 1.3.41.0. Minor bump because it is an architecture change, not a patch: the panel now holds two SQLite handles on the same file.

What changes

  • db.OpenReadOnly: a second handle with mode=ro, PRAGMA query_only, busy_timeout(5000) and two connections (db.ReadPoolSize). A write through it fails at the SQLite level.
  • Handlers.ReadDB + h.reader(): every GET handler, the Authenticate middleware's session lookup and cookie-domain read, and the read helpers shared with mutating handlers (requireHost, loadOIDCConfigOrError, safeReturnTo, scenariosReader, resolveHostDomains) read through the pool. 58 call sites in internal/api.
  • Components: NotifRepo.ReadDB (list / get / stats / recent alerts / push-sub lists / ActiveRulesFor), backup.Manager.ReadDB (List, Get), country.Expander.ReadDB (List), JobRunner.SetReadDB (Get, ListByCountry); dashboard.Queries, appsec.StatusReader and hardening.TimeoutCache are constructed with the read handle.
  • Stays on the writer: session.Touch, OIDCCallback, ForwardAuth, Login / Logout / TOTP flows, every mutating handler, publicip.Detector, SystemHealth's h.DB.Stats(), and all background writers. The full table with one reason per row is in docs/architecture/storage.md, which also corrects its old claim that argos already used WAL's concurrent readers.
  • ARGOS_READ_POOL (compose, .env.example, env-vars reference). 0 is the kill-switch: the second handle is not opened and every reader falls back to the writer (v1.3.40.4 behaviour). Apply with .env + make deploy-prod; no image rollback. The boot log says read pool enabled or read pool disabled by ARGOS_READ_POOL=0.
  • Restore: both handles open after backup.ApplyPending. A restore (POST /api/backups/{id}/restore or argos restore) writes a flag, the process exits, and the next boot replaces the file before either handle exists, so no connection ever points at a replaced inode. Nothing to add.
  • scripts/smoke/read-pool.sh and TestOpenReadOnly.

Why

v1.3.40.1 to v1.3.40.4 showed what one connection costs: a purge batch, a checkpoint fsync or the pinned dashboard refresh holds it and every request waits, including the session lookup that runs before any handler. The pre-flight on the dense demo seed (v1.3.40 planning doc, section 8) found three causes; v1.3.40.4 removed the CPU one (cursor plan), this release removes the other two for readers (the writer keeps paying the fsync, as it must).

Measured on the dense demo (500k rows, final image, dd IO pressure)

read-pool.sh with IO_PRESSURE=1 (a dd loop writing 256 MB files with O_DIRECT + fsync on the same disk for the whole run; host /proc/pressure/io avg10 mean 22 %, peak 44 %). /api/hosts sampled every 100 ms; gate p99 <= 50 ms.

gate pool on (ARGOS_READ_POOL=1) pool off (ARGOS_READ_POOL=0, same image)
idle, pinned 24 h refresh p50 1.5 ms, p99 6.2 ms, max 66 ms, n=1,082 PASS p50 1.5 ms, p99 79 ms, max 310 ms FAIL
strip 427,423 rows in 497 s; p50 1.6 ms, p99 8.0 ms, max 141 ms, n=4,456 PASS 17,759 rows in 12.6 s; p99 648 ms, max 668 ms FAIL
export x3, 24 h access (68k rows, 16 MB, 4.8-5.2 s each) aggregated n=152: p50 2.1 ms, p99 13.2 ms, max 62 ms PASS n=102: p50 35 ms, p99 219 ms, max 273 ms FAIL
cap purge with COUNT(*) 50,004 rows in 23.4 s; p50 1.5 ms, p99 15.5 ms, max 23 ms PASS 20,003 rows in 16.6 s; n=3, p50 1.66 s, p99 14.7 s FAIL

Panel RSS peak 72 MB with the pool (44-63 MB in the pre-flight), WAL peak 8.6 MB, no checkpoint starvation. The pool-off column is the kill-switch proving it flips the behaviour: same image, one .env line, and the v1.3.40.4 numbers are back.

Before / after on the demo without IO pressure is in the planning doc section 8 (before: idle p99 340 ms, strip 433 ms, export 357 ms, cap purge 487 ms).

Gate on prod

Deploy scheduled for the morning of 2026-09-27 (operator decision: five panel deploys on 2026-09-26 and the crowdsec alignment recreate at 23:30 local; an architecture change does not overlap a bouncer window). Sequence: make deploy-prod with the image already built, logical rollback ARGOS_READ_POOL=0 + make deploy-prod, image rollback to 1.3.40.4 only if the panel is not healthy in 60 s.

After-smokes: read-pool.sh with IO_PRESSURE=0 and CAP_PURGE_ROWS=0 (gates 0, 1, 2, 3; no dd on prod and no rows deleted to pass a gate: the IO-pressure and cap-purge gates are demo-only, covered above), dashboard-latency.sh strict, panel-boot.sh, waf-sources-agree.sh at 1 %. Tag after the operator's check in the panel. Results recorded here once run.

Known issues carried

Unchanged from v1.3.40.4: Until shown as "expired" on imported decisions; blocked_requests_24h mixing; dashboard/cache_test timing flake; the 48 h DB-size reading. New: the CSV export pages with LIMIT/OFFSET on the read pool without a snapshot transaction, as before (rows inserted during the export can shift pages); rollup territory (v1.3.41.x).