Skip to content

v1.3.9 — Detect-mode AppSec actually emits alerts now

Closes the v1.3.8 "investigated, not addressed" item. The argos/appsec-detect config was silently dropping every rule match before it reached the LAPI alert pipeline -- why cscli alerts list stayed empty and the panel's AppSec page reported total_hits = 0 forever regardless of payload, even though rules WERE loaded and AppSec WAS being called.

Bug

crowdsec/appsec-configs/argos-appsec-detect.yaml declared:

name: argos/appsec-detect
default_remediation: allow
inband_rules: [...]
outofband_rules: [...]

...with no on_match hook.

Root cause from CrowdSec source

Trace through pkg/acquisition/modules/appsec/appsec_runner.go:

  1. AppsecRuntimeConfig.ClearResponse() runs at the start of every request. It sets both Response.SendEvent = true AND Response.SendAlert = true. So inband matches default to "do generate an alert".
  2. After inband processing, the runner explicitly resets r.AppsecRuntime.Response.SendAlert = false at the inband -> outband boundary (runner.go line 341):
request.IsInBand = false
request.IsOutBand = true
r.AppsecRuntime.Response.SendAlert = false
r.AppsecRuntime.Response.SendEvent = true
  1. After outband processing, the runner emits the alert only when SendAlert == true:
if r.AppsecRuntime.Response.SendAlert {
    appsecOvlfw, err := AppsecEventGeneration(evt)
    ...
    r.outChan <- *appsecOvlfw
}

So outband matches in detect mode produce no alert unless an explicit on_match: SendAlert() hook re-enables it. The vendor crowdsecurity/crs config knows this and carries the directive (filtered to IsOutBand); the argos config was missing it.

There is also an interaction with inband matches in detect mode: in the live prod stack, even payloads that match inband rules were producing zero alerts. The runner code suggests inband should still emit (since SendAlert defaults to true), but the empirically-observed outcome was zero. Adding the IsInBand filter to the hook covers this case for free; we declare both phases for symmetry.

Fix

 name: argos/appsec-detect
 default_remediation: allow
+
+on_match:
+  - filter: IsInBand == true
+    apply:
+      - SendAlert()
+  - filter: IsOutBand == true
+    apply:
+      - SendAlert()
+
 inband_rules:
   - crowdsecurity/base-config
   - crowdsecurity/vpatch-*
   - crowdsecurity/generic-*
 outofband_rules:
   - crowdsecurity/experimental-*
   - crowdsecurity/appsec-generic-test

Mirrors the vendor crowdsecurity/crs style (which only declares the IsOutBand filter; argos declares both for the inband-default edge case).

Consequential bug fixed at the same time

Once SendAlert() is wired, the v1.3.8 envelope-headers fix for panel internal probes exposed a new false-positive: the healthcheck + ProbeHub each fired a probe every ~30 s with no User-Agent header. The crowdsecurity/experimental-no-user-agent rule then classified each probe as an attack -- 120 alerts per hour from the panel's own internal traffic.

Fix: both probes now send a User-Agent (argos-panel/healthcheck and argos-panel/probe respectively) plus the matching X-Crowdsec-Appsec-User-Agent. Verified zero self-alerts post-deploy.

LAPI volume considerations

Each matched request that passes a bucket overflow generates one alert. CrowdSec already protects against alert floods via:

  • Per-scenario buckets with leakspeed + capacity. The AppSec scenarios (appsec-vpatch, appsec-native) leak at 60s / capacity 3, so a single attacker can produce at most ~3 alerts per minute per bucket.
  • distinct: evt.Meta.rule_name on appsec-vpatch -- the same rule firing repeatedly from one IP only fills its bucket once.
  • blackhole: 1m on the trigger scenarios -- after a bucket overflows, the same source is silenced for 60 s.
  • The LAPI bouncer in argos-edge bans malicious IPs before they reach AppSec, removing the attacker from the loop entirely for repeated offenders.

Operators who see alert volume they want to reduce further can edit /etc/crowdsec/scenarios/appsec*.yaml and increase leakspeed or blackhole. Documented as the standard CrowdSec tuning workflow; not blocking this release.

Smoke results (prod stack argos-prod-argos:v1.3.9)

Pre-fix (v1.3.8 baseline) -- 0 alerts despite multiple attempted attack payloads. Confirmed via cscli alerts list --since 1h | grep -c appsec returning 0.

Post-fix -- after sending two requests with empty User-Agent (the simplest signature):

+----+---------------+------------------------------------------+--------+
| ID |     value     |                  reason                  | kind   |
+----+---------------+------------------------------------------+--------+
| 8  | Ip:172.18.0.1 | crowdsecurity/experimental-no-user-agent | waf    |
| 7  | Ip:172.18.0.1 | crowdsecurity/experimental-no-user-agent | waf    |
+----+---------------+------------------------------------------+--------+

Panel /api/appsec/metrics:

{"total_hits": 2, "blocked": 0, "logged": 2,
 "top_rules": [{"rule": "crowdsecurity/experimental-no-user-agent", "count": 2}]}

logged: 2 matches detect-mode behaviour (alert recorded, request flowed through). blocked: 0 because the bouncer received {"action":"allow"} from the detect listener.

Files changed

  • crowdsec/appsec-configs/argos-appsec-detect.yaml -- on_match: SendAlert() declaration for both phases.
  • backend/internal/appsec/healthcheck.go -- User-Agent on the 5-min cron probe.
  • backend/internal/appsec/hub.go -- User-Agent on the Status-page probe.
  • backend/internal/appsec/healthcheck_test.go -- new test covering the User-Agent forwarding.
  • docs/features/appsec.md -- new "Testing AppSec detection" section with 10 payload signatures + verification commands.
  • docs/operations/troubleshooting.md -- "Detect mode emits no alerts" entry with cause + upgrade path + stale-volume recovery for the argos_prod_shared_setup volume.

Upgrade

cd argos-edge
git pull
docker compose build
docker compose up -d
# If your shared_setup volume already has the v1.3.8 detect
# config (no on_match), refresh it:
docker compose exec crowdsec /setup-appsec.sh

The setup-appsec.sh step copies the new argos-appsec-detect.yaml over the stale one and SIGHUPs crowdsec; ~1 s of AppSec listener downtime, no other impact.

Not changed

  • v1.3.8's two acquisitions / two listeners / two configs split is unchanged. The "boot warnings: conflicting id" cosmetic issue from v1.3.8 is unaffected and still documented in troubleshooting.
  • Block mode (crowdsecurity/appsec-default config) was already alerting correctly; that path is not modified.
  • Bouncer-side appsec_url switching between :7422 (block) and :7423 (detect) keeps working the same way.