Commit graph

14 commits

Author SHA1 Message Date
justin
9dd6f53eb2 infra(mail): remove 18 dormant snowshoe IPs from postfix + host
Consolidate the outbound mail footprint to match the SPF intent (already
trimmed to .94/.107 on 2026-06-19). A 20-IP sending footprint reads as
snowshoe spam to receivers and was contributing to domain-reputation
throttling (Microsoft 451 4.7.500, Gmail low-reputation).

Removed from /etc/postfix/master.cf: transports yahooslow, out02-04,
out06-20, rehab02-04, HC submission ports 2527/2528, hcout2/hcout3.
Removed from /etc/network/interfaces (+ live ip addr del): host bindings
.90-.93, .95-.106, .108-.109. Kept: .94 (trucking/out05), .107 (HC/hcout1),
.71/.72 (infra).

Verified live: postfix check OK, both streams still status=sent post-change,
SSH session on .71 unaffected, transport_maps still routes via out05.

Snapshots: infra/postfix/live-snapshots/master.cf, infra/network/interfaces.
Live backups on server: /root/{master.cf,interfaces}.bak_snowshoe_*.
2026-06-23 23:45:41 -05:00
justin
8e5590b492 mail: DMARC aggregate-report parser + dedicated dmarc@ mailbox ingestion
Tool 2 of the deliverability monitoring pair (Tool 1 = mail_reputation_monitor).
DMARC rua reports from dozens of operators (Google, Yahoo, Comcast, Cox, Bell,
Mimecast, Cisco ESA, GMX, mail.com, ...) were landing in ops@ (dmarc@ was a DL),
burying real mail and never parsed. Now ingested + queryable:

- dmarc@performancewest.net converted DL -> dedicated Carbonio mailbox; isolated
  IMAP creds in server .env, surfaced to workers in docker-compose.yml (mirrors
  OPS_IMAP_*). 29 historical reports moved ops@ -> dmarc@ via IMAP.
- scripts/dmarc_report_parser.py: IMAP fetch unseen -> decompress .gz/.zip/.xml
  (namespace-agnostic: classic + urn:ietf:params:xml:ns:dmarc-2.0 GMX/mail.com) ->
  parse aggregate XML -> upsert dmarc_report (keyed (org_name,report_id), no-op on
  re-parse) + dmarc_record per source IP. dmarc_pass = dkim_aligned OR spf_aligned.
  Marks \Seen. --dry-run/--all/--alert (7d per-IP summary + Telegram if one of OUR
  IPs <95% pass, or EXTERNAL IP sends >=20 failing msgs as us = spoofing under
  p=reject). psycopg2 imported lazily so --dry-run runs without the driver.
- api/migrations/102_dmarc_aggregate.sql: dmarc_report + dmarc_record tables.
- infra/cron/pw-dmarc-parser: 06:20 UTC daily --alert (after reputation, before scrub).
- docs/deliverability.md: DMARC section DONE; query examples.

Verified: dry-run --all parses all 28 reports (1 non-report test probe), 0 unknown
after the namespace fix.
2026-06-19 08:50:20 -05:00
justin
bd7ba23841 docs(deliverability): Yahoo CFL ENROLLED for both domains (reporting fbl@)
performancewest.net + send.performancewest.net both show Enrolled in the Yahoo
Sender Hub, reporting email fbl@. All three FBLs (Google Postmaster, MS SNDS+JMRP,
Yahoo CFL) now complete.
2026-06-19 08:29:12 -05:00
justin
b8b6444084 docs(deliverability): Yahoo CFL verification keys added for both domains
Added yahoo-verification-key TXT records via Hestia for performancewest.net
(apex) and send.performancewest.net; both propagated to all HE.net slaves +
public resolvers. Ready to click Verify in the Yahoo CFL form, complaint dest fbl@.
2026-06-19 02:13:48 -05:00
justin
a9bbfbf59b docs(deliverability): Microsoft MANUAL 2 fully DONE — SNDS access + JMRP both set
SNDS access requested/granted for 207.174.124.94 + .107; JMRP feeds registered
with complaint dest fbl@. Section marked complete. SNDS data populates in ~24-48h.
2026-06-19 02:03:30 -05:00
justin
f293466519 docs(deliverability): JMRP complaint dest set to fbl@performancewest.net
Corrected: JMRP feed destination was set to fbl@ directly (no forward needed);
ARF complaints route to ops@.
2026-06-19 01:00:16 -05:00
justin
60540f949d docs(deliverability): JMRP done — both IPs registered (pw1/.94, pw2/.107)
Note JMRP delivers ARF complaints to the signed-in MS account's email, not
automatically to fbl@; set a forward if that account isn't fbl@performancewest.net.
2026-06-19 00:59:49 -05:00
justin
776817c727 docs(deliverability): correct SNDS entry URL (snds.microsoft.com does not resolve)
Use the legacy sendersupport.olc.protection.outlook.com/snds/ (308-redirects) or
the direct substrate.office.com/ip-domain-management-snds/SNDS app URL. Flag that
snds.microsoft.com has no DNS.
2026-06-19 00:46:25 -05:00
justin
7828ee4587 docs(deliverability): fix SNDS/JMRP URLs for Microsoft's 2026 substrate migration
SNDS moved off sendersupport.olc.protection.outlook.com to
substrate.office.com/ip-domain-management-snds/. The old /snds/ and /pm/ links
308-redirect there. Document that the footer/help links going to microsoft.com
are boilerplate (not broken), and that you must Log in FIRST or the Request
Access / JMRP links bounce to login.microsoftonline.com (expected, not dead).
Add working direct links + canonical https://snds.microsoft.com entry point.
2026-06-19 00:45:59 -05:00
justin
e18f23634a docs(deliverability): document consumer-domain exclusion two-layer model + scrub
Records the Apple/iCloud addition, the builder-vs-list-based distinction, the
scrub_listmonk_consumer reconciliation tool + daily cron, and the 2026-06-19
first-run numbers (7,943 trucking + 21 HC stale consumer subs blocklisted).
2026-06-19 00:01:17 -05:00
justin
49842bddbb docs(deliverability): Microsoft #1 priority + role mailboxes created (Carbonio)
Created postmaster@/abuse@/fbl@/dmarc@ as Carbonio DLs -> ops@ (they previously
REJECTED 5.1.1, which would have blocked SNDS verification AND was silently
dropping all DMARC aggregate reports). Verified accept-at-MX + delivered E2E.
Reframe Microsoft as the #1 monitoring priority (85% of audience), Yahoo as
lowest (<1%); add Carbonio admin access note; note DMARC parser now worth building.
2026-06-18 23:31:20 -05:00
justin
3ca960aca5 docs+infra(deliverability): document bulk subdomain; ansible signs send.performancewest.net
- infra/ansible/roles/mail: refactor OpenDKIM to support multiple signing domains
  via opendkim_signing_domains list (root + send.performancewest.net). Loops
  keygen/ownership/keytable/signingtable so the live two-domain setup is
  reproducible from ansible.
- infra/ansible group_vars: add bulk_mail_subdomain + campaign_from_* +
  campaign_reply_to documentation vars (map to CAMPAIGN_FROM / HC_CAMPAIGN_FROM
  env read by the builder scripts). smtp_from (transactional) stays on root.
- docs/deliverability.md: rewrite TL;DR with the carrierone-vs-performancewest
  A/B proof (same server/IPs, different From domain -> Inbox vs Junk) and the
  ~85% Microsoft / 14% Google / <1% Yahoo audience mix; add the bulk-subdomain
  section, SPF trim, rehab-disabled, and the Hestia DNS automation runbook.
2026-06-18 23:12:05 -05:00
justin
1056705cf9 docs(deliverability): Google Postmaster TXT added+verified via Hestia DNS master
DNS is fully automatable: Hestia (cp.carrierone.com, zone owner = justin user)
is the DNS master, HE.net are slaves. Added google-site-verification TXT (id
14464) via v-add-dns-record as root; verified resolving on public resolvers +
HE.net slaves. Owner just clicks Verify in the Postmaster console. Documents the
v-add-dns-record path for future records.
2026-06-18 22:05:01 -05:00
justin
5253f16675 docs: deliverability runbook (incident, IP consolidation, monitoring setup)
Documents the 2026-06-18 reputation incident (snowshoe -> Gmail domain-rep
blocks, RBLs all clean), the single-IP-per-stream consolidation, and
fill-in-the-blanks setup steps for Google Postmaster Tools, Microsoft SNDS/JMRP,
and Yahoo CFL (all require owner account login + HE.net DNS). Plus ongoing
hygiene + how to re-expand IPs once reputation recovers.
2026-06-18 17:46:28 -05:00