new-site/infra/mail/pw-mail-ips.service
justin 4276adab80 infra(mail): fix warmed sending IPs dropping off ens18 on reboot (Jun 24 outage)
Unattended kernel-upgrade reboot (Jun 24 04:04) left only .71 bound because
classic ifupdown applies just the first 'address' line. Postfix then failed to
bind .94/.107 ('Cannot assign requested address') and silently egressed from
.71 -- which is NOT in SPF (every fallback msg failed SPF) and is on RLR621 +
Trend ERS-QIL. ~37h of bypassed IP-warming + a near-zero sales day.

Fixes:
- /etc/network/interfaces: explicit up/down ip-addr hooks for .72/.94/.107
- pw-mail-ips.service: systemd oneshot re-binds IPs + flushes queue on boot
- pw-mail-ip-watchdog: */5 cron re-binds missing IPs + flushes, also catches
  'Cannot assign' bind failures
- runbook: full incident writeup + reboot-test lesson

Host already remediated live; this commits the host artifacts + docs.
2026-06-25 17:28:33 -05:00

13 lines
428 B
Desktop File

[Unit]
Description=Ensure Performance West mail sending IPs are bound to ens18
After=network-online.target networking.service
Wants=network-online.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c "for ip in 207.174.124.72 207.174.124.94 207.174.124.107; do ip addr show ens18 | grep -q \"$ip/\" || ip addr add $ip/23 dev ens18; done"
ExecStart=/usr/sbin/postqueue -f
[Install]
WantedBy=multi-user.target