mail: DMARC aggregate-report parser + dedicated dmarc@ mailbox ingestion
Tool 2 of the deliverability monitoring pair (Tool 1 = mail_reputation_monitor). DMARC rua reports from dozens of operators (Google, Yahoo, Comcast, Cox, Bell, Mimecast, Cisco ESA, GMX, mail.com, ...) were landing in ops@ (dmarc@ was a DL), burying real mail and never parsed. Now ingested + queryable: - dmarc@performancewest.net converted DL -> dedicated Carbonio mailbox; isolated IMAP creds in server .env, surfaced to workers in docker-compose.yml (mirrors OPS_IMAP_*). 29 historical reports moved ops@ -> dmarc@ via IMAP. - scripts/dmarc_report_parser.py: IMAP fetch unseen -> decompress .gz/.zip/.xml (namespace-agnostic: classic + urn:ietf:params:xml:ns:dmarc-2.0 GMX/mail.com) -> parse aggregate XML -> upsert dmarc_report (keyed (org_name,report_id), no-op on re-parse) + dmarc_record per source IP. dmarc_pass = dkim_aligned OR spf_aligned. Marks \Seen. --dry-run/--all/--alert (7d per-IP summary + Telegram if one of OUR IPs <95% pass, or EXTERNAL IP sends >=20 failing msgs as us = spoofing under p=reject). psycopg2 imported lazily so --dry-run runs without the driver. - api/migrations/102_dmarc_aggregate.sql: dmarc_report + dmarc_record tables. - infra/cron/pw-dmarc-parser: 06:20 UTC daily --alert (after reputation, before scrub). - docs/deliverability.md: DMARC section DONE; query examples. Verified: dry-run --all parses all 28 reports (1 non-report test probe), 0 unknown after the namespace fix.
This commit is contained in:
parent
b45332b5f7
commit
8e5590b492
5 changed files with 509 additions and 8 deletions
|
|
@ -119,6 +119,11 @@ services:
|
|||
- OPS_IMAP_PORT=${OPS_IMAP_PORT:-993}
|
||||
- OPS_IMAP_USER=${OPS_IMAP_USER}
|
||||
- OPS_IMAP_PASS=${OPS_IMAP_PASS}
|
||||
# DMARC aggregate-report ingestion mailbox (scripts.dmarc_report_parser)
|
||||
- DMARC_IMAP_HOST=${DMARC_IMAP_HOST:-mail.performancewest.net}
|
||||
- DMARC_IMAP_PORT=${DMARC_IMAP_PORT:-993}
|
||||
- DMARC_IMAP_USER=${DMARC_IMAP_USER:-dmarc@performancewest.net}
|
||||
- DMARC_IMAP_PASS=${DMARC_IMAP_PASS}
|
||||
- FROM_EMAIL=Performance West <noreply@performancewest.net>
|
||||
- CRYPTO_SWEEP_ADMIN_EMAIL=${ADMIN_EMAIL:-ops@performancewest.net}
|
||||
- USAC_USERNAME=${USAC_USERNAME}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue