Skip to content

v1.3.39.0 -- The panel shows the WAF that is running

First release of v1.3.39. No schema change; the blocking path (bouncer, AppSec mode, Coraza toggle) is untouched. argosVersion 1.3.38.5 -> 1.3.39.0.

PHASE 0 (2026-09-26): why waf_audit was empty since April

The panel has two WAF engines: the per-host Coraza WAF embedded in Caddy (writes /var/log/caddy/waf-audit.log, tailed into the waf_audit table) and CrowdSec AppSec, global, inline through the bouncer (alerts in the LAPI). On the operator's prod every host has Coraza off; host_security.updated_at tops at 2026-04-25 16:29:37, the minute waf-audit.log (0 bytes) was last touched, the day AppSec became the WAF (v1.3.12 argos/appsec-block, v1.3.19, block mode since 2026-04-26). The live Caddy config has no waf handler at all, only crowdsec + appsec on 17 hosts. The chain image -> SecAuditLog -> volume (rw in caddy, ro in the panel, 0644) -> tailer -> parser -> waf_audit is intact and has no producer.

Meanwhile AppSec: 1,127 kind=waf alerts in 8 days, 261 blocked in 24 h, cscli metrics 770k processed / 1.98k blocked since the container started. The Dashboard said "No WAF events in range", the Security overview "Blocked 24h: 0".

Pre-flight against the live LAPI (v1.7.7)

GET /v1/alerts: the 500 cap was ours (limit=500 in the client); the LAPI default is 100 and 5,000 is accepted and honoured. No pagination (offset/page are a 500 "unknown filter"); since/until are relative durations only (RFC3339 is a 500); kind (waf/crowdsec/capi) and scenario are exact matches; include_capi=false drops the CAPI list alerts and their thousands of decisions (7.9 MB -> 4.2 MB on 7 d). Cost: 24 h = 259 alerts / 0.7 MB / 0.09 s; 7 d = 1,270 / 4.2 MB / 0.67 s. Every kind=waf alert carries target_fqdn in its events (1,127/1,127), so hits can be attributed to hosts. The alert does not say whether the request was blocked; attribution stays by mode boundary, as since v1.3.12.

What ships

  • crowdsec.AlertsQuery / QueryAlerts: the query the panel may build, pinned by TestQueryAlertsContract (since as minutes, kind, include_capi=false, with_decisions=false, limit=5000, and nothing else). Alert.Remediation read; WasBlocked honours it.
  • appsec.Provider.Alerts: one cached 24 h fetch (30 s) for every consumer; 7 d on demand (60 s) with the seven-window fallback past 5,000. Summarize aggregates hits, bans, per-host, per-IP, per-rule, per-path and time buckets. Burst detector on the shared refresh (10 hits from one IP in 60 s -> waf_attack_burst, engine: appsec, once per burst, history before boot ignored).
  • Dashboard security: waf_engines, three named series, both engines in every table and on the map; "No WAF events" only when neither engine had any.
  • Security overview: blocked_24h_total = Coraza + AppSec hits + AppSec bans with blocked_24h_by_engine; per host engine, coraza_24h, appsec_hits_24h (by target_fqdn); the engines' state shown as state ("Coraza: off on 19 hosts, AppSec: block").
  • Logs: Coraza presets named as such; empty waf_audit result explains itself and links to /appsec?window= (now honoured); waf_audit (Coraza) in the source select.
  • AppSec page: hits / bans card; "(by mode)" on blocked / detected with the limitation as a tooltip.
  • log ingestor started names the waf_audit tailer.

Measured

Before on prod (1.3.38.5), waf-sources-agree.sh, same 24 h: dashboard 0, overview 0, AppSec page 260, LAPI 266 (245 hits + 21 bans): FAIL.

After, on the demo (argos-prod-argos:1.3.39.0-rc, AppSec not functional there, so 29 synthetic alerts with the real shape were posted to its LAPI: 22 kind=waf hits on three hosts including a burst of 12 from one IP inside 30 s, 2 appsec-native bans, 5 hits older than 24 h): waf-sources-agree.sh PASS, dashboard 24 / overview 24 / AppSec 24 / LAPI 24 (22 + 2); AppSec windows 1h = 17, 6h = 24; per-host blocked_24h 8 / 10 / 4 by target_fqdn; the 5 old hits excluded by every view; the burst produced one waf_attack_burst delivery (engine: appsec) against the demo's webhook rule. The LAPI log shows the exact query include_capi=false&limit=5000&since=1440m&with_decisions=false every 24 s from the pinned dashboard view and nothing else from the AppSec page or the overview.

Two defects found and fixed during that run: the dashboard's 24 h range reaches the provider as 24 h plus its bucket snap and was classified as a 7 d window (now short up to 25 h), and the panel windowed alerts by created_at while the LAPI filters started_at (now Alert.StartedAt everywhere; on prod the two differ by at most 9 s, on synthetic alerts by hours).

After on prod (deploy 09:25:32Z, image sha256:1530ca... -> sha256:81c930..., healthy in 10 s, verify-deploy PASS):

Smoke Result
waf-sources-agree.sh PASS: dashboard 260 / overview 260 / AppSec page 260 / LAPI 266 (245 kind=waf + 21 bans), 2 %
dashboard-latency.sh MAX_SECONDS=1 EXPECT_CACHE=hit PASS: health 0.0009 s hit, overview 0.061 s; the pinned security view, now fed by the LAPI too, is X-Argos-Cache: hit
panel-boot.sh PASS: boot-to-listen 0.097 s, first purge +122.2 s
Threats bytes per response (curl, page=1&per_page=100) 29,670 B
crowdsec container over the 10 min after deploy (cap 256 MB) see CHANGELOG "Measured after deploy"

/api/dashboard/security 24 h on prod: waf_engines.appsec mode block, 245 hits, 15 bans, top rules vpatch-env-access 142, experimental-no-user-agent 57, vpatch-CVE-2025-29927 21; security/overview blocked_24h_total 260, 17 hosts with events, per-host hits summing to 245.

The 6-alert gap to the LAPI (2 %): crowdsecurity/crowdsec-appsec-outofband does not match the crowdsecurity/appsec- prefix looksLikeAppSec tests, so the panel (and the AppSec page, since v1.3.4) never counted it while the smoke's LAPI leg does. One-line fix (add the crowdsecurity/crowdsec-appsec- prefix) queued for v1.3.39.1.

Smoke gate

scripts/smoke/waf-sources-agree.sh: dashboard/security waf_engines.appsec.events, security/overview appsec_hits_24h + appsec_bans_24h, appsec/metrics total_hits and the LAPI count through cscli alerts list --since 24h (kind=waf plus crowdsecurity/appsec-native and crowdsecurity/crowdsec-appsec-outofband) all > 0 and within 5 %.

Gates before commit: gofmt -l clean, go vet ./..., go test ./..., npm run build, npm run lint, check-no-personal-data.sh, mkdocs build --strict.

Known issues

  • blocked_requests_24h on the dashboard overview counts every 403 at the edge (bans and AppSec alike; 5,873 on prod): the access log has no field that separates them. Proposal for v1.3.40/41: a response header or a per-handler log field from Caddy.
  • Threats Until column expired for imported decisions (carried).

Files changed

  • backend/internal/crowdsec/client.go, types.go (+ client_test.go).
  • backend/internal/appsec/alerts.go (new), metrics.go, types.go (+ alerts_test.go).
  • backend/internal/dashboard/types.go, backend/internal/api/ dashboard.go, waf_merge.go (new, + test), security_overview.go, logs.go, backend/cmd/argos/main.go.
  • frontend/src/api/client.ts, pages/Dashboard.tsx, pages/SecurityOverview.tsx, pages/Logs.tsx, pages/AppSec.tsx.
  • scripts/smoke/waf-sources-agree.sh, docs/reference/api.md, CHANGELOG.md, docs/release-notes/v1.3.39.0.md, mkdocs.yml, docs/operations/verification-report.md, frontend/package.json.