Commit graph

9 commits

Author SHA1 Message Date
justin
9dd6f53eb2 infra(mail): remove 18 dormant snowshoe IPs from postfix + host
Consolidate the outbound mail footprint to match the SPF intent (already
trimmed to .94/.107 on 2026-06-19). A 20-IP sending footprint reads as
snowshoe spam to receivers and was contributing to domain-reputation
throttling (Microsoft 451 4.7.500, Gmail low-reputation).

Removed from /etc/postfix/master.cf: transports yahooslow, out02-04,
out06-20, rehab02-04, HC submission ports 2527/2528, hcout2/hcout3.
Removed from /etc/network/interfaces (+ live ip addr del): host bindings
.90-.93, .95-.106, .108-.109. Kept: .94 (trucking/out05), .107 (HC/hcout1),
.71/.72 (infra).

Verified live: postfix check OK, both streams still status=sent post-change,
SSH session on .71 unaffected, transport_maps still routes via out05.

Snapshots: infra/postfix/live-snapshots/master.cf, infra/network/interfaces.
Live backups on server: /root/{master.cf,interfaces}.bak_snowshoe_*.
2026-06-23 23:45:41 -05:00
justin
1e9dcfcfd1 mail(rampcap): step trucking cap back up to 400/h (day 19-20), 500/h ceiling
The day-9 Gmail block that forced the 200/h hold is resolved: per-MX throttling
shipped, Google is excluded entirely (MAIN_EXCLUDE_OPERATORS=google), and the
OpenDKIM signing bug is fixed. With Google out of the mix, 400/h (~4k/day) is
within the envelope these IPs cleanly sustained at 68-76% delivery with zero
blocks. Lets the post-DKIM re-send backlog drain in ~1 day instead of ~3.
2026-06-22 12:49:54 -05:00
justin
545e6f7ed7 infra(mail): consolidate sending IPs (kill snowshoe) now that DKIM is fixed
The multi-IP rotation was built to spread risk while DKIM was broken (fixed
2026-06-17) and after the May 30-31 over-volume blast. With DKIM signing
correctly, spreading ~3k trucking msgs/day across 12 IPs (.94-.105) + ~1.2k
healthcare msgs/day across 3 IPs (.107-.109) gave each IP far too little
per-receiver volume to build reputation. Gmail/Outlook read it as snowshoe spam
and reputation-blocked ~200 msgs/day ("very low reputation of the sending
domain") -> 0 human clicks, 0 sales.

Consolidate to ONE IP per stream so each accrues real reputation:
 - trucking: pw-mta-warmup ALL=(out05) -> randmap collapses to {out05:} = .94
 - healthcare: listmonk-hc SMTP servers 2/3 (ports 2527/2528 -> .108/.109)
   disabled in DB; all HC mail now egresses .107 (hcmta01). [applied live]

Applied live: transport_maps now randmap:{out05:}; listmonk-hc restarted.
To re-expand later: add transports back to ALL + re-enable the HC SMTP servers.
2026-06-18 17:41:07 -05:00
justin
dd4ed3ea38 warmup: ROLL BACK main pool to 200/h after Gmail spam-blocked IPs at 400/h
Day 9 (2026-06-13) alert: main pool 54% delivery, 202 Gmail spam-blocks
(550-5.7.1 'Gmail has detected') on warming IPs .94-.98. The 4k/day (400/h)
ramp was too aggressive AND the trucking pool lacks the per-MX throttling the HC
pool got -- Google-Workspace-hosted business domains (weberfarms.net, uatruck.com,
etc.) concentrated and Gmail blocked us. Held at 200/h (~2k/day) through day 20 to
recover, then slow step to 300/h. Applied live (cap already set to 200/h).
2026-06-13 20:10:13 -05:00
justin
887bf9a14a warmup: grow main (trucking) pool faster -- 3k -> 4k/day now, 5k at day 14
The main sending IPs are cleanly warmed: today 3,845 sent at 0.18% bounce,
ZERO deferrals, ZERO ISP rate-limit/blocklist/Spamhaus hits. The script's own
note records these IPs historically sustained ~2,500/day at 68-76% delivery;
collapses only ever came from 17k-29k spikes. So we have ample headroom to
accelerate the trucking ramp safely:
  day 7-13: 300/h -> 400/h (~4,000/day)   [applied now, day 8]
  day 14+:  new    500/h    (~5,000/day)   [hard ceiling, well under ~17k]

Also vendored pw-listmonk-rampcap into the repo (infra/postfix/) -- it
previously lived only on the server at /usr/local/bin. Live script updated and
applied (listmonk cap now 400/h).
2026-06-11 00:13:41 -05:00
justin
90d8b94f3f feat(email): wire listmonk-hc into deploy + dev override + hc ramp-cap
- deploy.sh/deploy-dev.sh: bring up listmonk-hc (upstream image, excluded from
  build); document the one-time listmonk_hc DB create + --install.
- docker-compose.dev.override.yml: dev-only override (committed) that drops the
  prod host-port bindings and pins dev's own postgres volume (dev-pgdata) via
  compose !override tags. deploy-dev ships it as docker-compose.override.yml so
  syncing the canonical compose to the shared host no longer breaks dev's
  api-postgres (port :5432 clash + volume switch). Discovered + fixed while
  validating listmonk-hc on dev.
- pw-hc-rampcap.sh: healthcare analogue of pw-listmonk-rampcap, ramps the
  listmonk_hc cap 100->1000/h off /etc/postfix/hc-warmup-start, fully
  independent of the trucking ramp/cap.
2026-06-05 19:19:45 -05:00
justin
70d742df08 feat(mta): healthcare HOT-stream Postfix setup (dedicated hc IPs, isolated)
Adds 3 hc submission ports (2526/2527/2528) in the single Postfix instance,
each content_filter'd onto a dedicated hc transport (hcout1/2/3) binding the
hc IPs .107/.108/.109 with hc HELO identity (hcmta01-03) and hotter concurrency.
listmonk-hc round-robins the 3 ports.

Discovered + documented the constraint that drove this shape: transport_maps
randmap is owned by the shared trivial-rewrite(8) and is global, so neither a
per-smtpd -o transport_maps nor a FILTER randmap:{...} can scope a separate IP
pool (FILTER parses randmap as a literal transport). content_filter=hcoutN:
(empty nexthop) overrides transport_maps and keeps the real recipient domain.

Verified end-to-end on the server: :2527 -> hcout2 (.108) -> real gmail MX;
trucking transport_maps (.94-.96) untouched. Idempotent, postfix-check gated
with auto-rollback.
2026-06-05 19:07:02 -05:00
justin
b48fc3a406 Retire burned MTA IPs in warmup script 2026-06-03 23:37:27 -05:00
justin
2fab98c0a8 postfix: multi-IP warmup sending pool (20 IPs, gradual rotation)
- 20 IPs (.90-.109 / mta01-mta20) with FCrDNS + SPF in HestiaCP
- .90 (mta01) dedicated Yahoo/AOL recovery IP (yahooslow, 20s trickle)
- .91-.109 (out02-out20) rotation pool via transport_maps randmap
- pw-mta-warmup: cron-driven scheduler grows the active rotation pool
  3 -> 5 -> 8 -> 12 -> 16 -> 19 IPs over ~25 days
- mta_setup.sh: idempotent installer (backups + postfix-check-gated reload)

New IPs verified clean on Spamhaus/Barracuda/SpamCop/SORBS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 19:03:30 -05:00