From 62292b96af5aacce3a220cab7da3d690a5626667 Mon Sep 17 00:00:00 2001 From: justin Date: Mon, 22 Jun 2026 11:59:29 -0500 Subject: [PATCH] docs(deliverability): document Jun 22 re-send of never-delivered DKIM-window audience Records the MAIN_EXCLUDE_OPERATORS=google override, the resend_dkim_backup_20260622 rollback table, the past-send_at HTTP 400 gotcha (use --send-hour for same-day re-runs), and the exact revert SQL. 6461-row backup; ~2999 re-sent Jun 22, rest drain on subsequent daily runs (Gmail excluded, Microsoft/Hotmail included). --- docs/email-deliverability-runbook.md | 30 ++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/email-deliverability-runbook.md b/docs/email-deliverability-runbook.md index 3982589..e544f5f 100644 --- a/docs/email-deliverability-runbook.md +++ b/docs/email-deliverability-runbook.md @@ -171,6 +171,36 @@ sudo opendkim-testkey -d performancewest.net -s mail -vvv # expect "key OK" - Dead M365 tenant scrub: HC defers are mostly `451 4.4.4` against dead M365 tenants + `421` LuxSci throttle. Identify and suppress dead tenants. +### Re-send of the never-delivered (unsigned) audience — Jun 22 2026 + +The ~79k cold sends made during the broken window (Jun 1 - Jun 18 00:31 UTC) were +stamped `listmonk_sent_at` at send time, so the builder permanently excluded them +even though they were junked/blocked unsigned. With DKIM now fixed we re-send to +the now-deliverable subset, **excluding Gmail** (Google consumer reputation is +still recovering) but **including Microsoft/Hotmail** (the bulk of the list). + +What was done (all reversible): +1. `MAIN_EXCLUDE_OPERATORS` env override added to the builder (commit `5a3063e`): + when set it REPLACES the default `WARMUP_EXCLUDE_OPERATORS`. Set to `google` in + the `workers` service env so cold sends go to everything except Google, driving + both the SQL exclude and the per-operator daily cap (google cap=0, others 120). +2. Backed up the reset target to `performancewest.resend_dkim_backup_20260622` + (6,461 rows = broken-window AND `email_verify_result IN (smtp_valid, + send_confirmed)` AND `mx_provider <> google`), then `UPDATE fmcsa_carriers SET + listmonk_sent_at = NULL` for exactly those rows so the builder re-queues them. +3. Ran the builder with `--send-hour 17 --send-minute 30` (the default per-tz hours + 09-12 UTC were already past; **Listmonk rejects a past `send_at` with HTTP 400 + "Scheduled date should be in the future"** — always override the hour for a + same-day manual re-run after the normal window). Result: 30 campaigns, + queued_recipients=3000 (warmup cap), ~2,999 re-stamped. Provider mix: Microsoft + 1,272 / Comcast / Charter / Proofpoint / long-tail; **zero Google**. + +The remaining ~3.5k of the 6,461 backup set drain on subsequent daily runs under +the same cap. To revert a row: `UPDATE fmcsa_carriers c SET listmonk_sent_at = +b.old_listmonk_sent_at FROM resend_dkim_backup_20260622 b WHERE c.dot_number = +b.dot_number;`. To resume normal warmup exclusion later, unset +`MAIN_EXCLUDE_OPERATORS` (reverts to Google+Microsoft+consumer-MX held to day 30). + ### Follow-up hardening — DONE (Jun 17-18 2026) All discovered during the post-incident technical audit; each fix is codified.