diff --git a/docs/deliverability.md b/docs/deliverability.md index d28440f..5d9decf 100644 --- a/docs/deliverability.md +++ b/docs/deliverability.md @@ -202,6 +202,21 @@ doing since the data finally arrives. already blocklists hard bounces after 1 (`bounce.actions hard->blocklist`), so these self-clean, but pre-scrubbing the dirtiest segments before send avoids the reputation hit. See `data/` segment exports. +- **Consumer-domain exclusion (two layers).** The authoritative list lives in + `scripts/_email_exclusions.py` (`BLOCKED_EMAIL_DOMAINS`): gmail/google, the full + Yahoo/Verizon-Media family, Microsoft consumer, **Apple/iCloud (added 2026-06-19)**, + dead/legacy ISPs, and the legal do-not-contact list. + 1. *NEW selections:* the per-vertical builders filter it out of audience SQL and + `listmonk_import.py` refuses to import a blocked address. + 2. *Already-imported subs:* LIST-BASED campaigns (FCC Direct Contacts list 3, + CRTC/USF blasts) can still hit consumer subs imported BEFORE a domain joined + the list. `scripts/scrub_listmonk_consumer.py` reconciles the live subscriber + table against the exclusion list and blocklists any ENABLED match (idempotent; + `--dry-run` supported; both `listmonk` + `listmonk_hc`). Runs daily 06:30 UTC + via `/etc/cron.d/pw-listmonk-scrub` (tracked at `infra/cron/pw-listmonk-scrub`). + First run 2026-06-19 blocklisted **7,943** trucking + **21** HC stale consumer + subs (1,321 iCloud, 267 gmail, etc.) that were leaking via the running CRTC + campaign. Re-run the scrub whenever you add a domain to the exclusion list. - **Don't re-expand IPs** until Postmaster Tools shows recovered reputation. - **Volume discipline:** keep the global 200/hr sliding window until reputation is green; concentrated low volume on one warm IP beats bursts. diff --git a/infra/cron/pw-listmonk-scrub b/infra/cron/pw-listmonk-scrub new file mode 100644 index 0000000..4ace351 --- /dev/null +++ b/infra/cron/pw-listmonk-scrub @@ -0,0 +1,11 @@ +# Daily Listmonk consumer-domain reconciliation. Blocklists any ENABLED +# subscriber whose address is on the authoritative exclusion list +# (scripts/_email_exclusions.py) -- consumer mailboxes (gmail/yahoo/microsoft/ +# apple/etc.), dead/legacy ISPs, and the legal do-not-contact list. The +# per-vertical builders already exclude these from NEW selections and the +# importer refuses to import them, but LIST-BASED campaigns (FCC Direct +# Contacts, CRTC/USF blasts, etc.) can still hit subscribers imported before a +# domain joined the exclusion list. This keeps the live subscriber table in +# sync so the exclusion applies retroactively, protecting sender reputation. +# Runs at 06:30 UTC, before the 07:00/07:45/08:00 UTC campaign builders. +30 6 * * * deploy cd /opt/performancewest && docker compose exec -T workers python3 -m scripts.scrub_listmonk_consumer >> /var/log/pw-listmonk-scrub.log 2>&1 && LM_HC=$(docker compose exec -T workers printenv DATABASE_URL | sed 's#/[^/]*$#/listmonk_hc#') && docker compose exec -T -e LISTMONK_DATABASE_URL="$LM_HC" workers python3 -m scripts.scrub_listmonk_consumer >> /var/log/pw-listmonk-scrub.log 2>&1