4dc5690666
The entire outbound campaign pipeline lived ONLY on the host and was never in
IaC -- a fresh rebuild would have silently shipped NO campaigns, NO IP warmup/
ramp, and NO bounce processing. New mail-pipeline role + deploy-mail-pipeline.yml
playbook deploy it from the canonical repo copies:
cron.d (infra/cron/):
- pw-trucking-campaign-builder, pw-ifta-campaign, pw-ucr-campaign
- pw-hc-campaign, pw-hc-nppes, pw-hc-refresh
- pw-mta-warmup, pw-listmonk-rampcap, pw-hc-rampcap
- pw-ip-rehab, pw-warmup-tg-alert
helper scripts (-> /usr/local/bin):
- pw-mta-warmup, pw-listmonk-rampcap, pw-hc-rampcap, pw-warmup-tg-alert
- postfix-bounce-notify.sh, postfix-hc-bounce-notify.sh, listmonk-bounce-sync.py
systemd services:
- pw-bounce-watcher.service (was missing from repo), pw-hc-bounce-watcher.service
Also creates the deploy-owned {{project_dir}}/logs dir (deploy can't write
/var/log, so a missing dir made cron redirects fail). Added the 6 cron.d files
that existed only on the host, the trucking bounce-watcher unit, and synced
infra/cron/pw-hc-refresh to the live version (revalidation download + enrich
steps). Role wired into site.yml after the mail (OpenDKIM) role.
Part of the email-deliverability incident hardening.
18 lines
1.9 KiB
Text
18 lines
1.9 KiB
Text
# NOTE: logs go to /opt/performancewest/logs/ (deploy-owned). The deploy user
|
|
# cannot write /var/log, so a /var/log redirect makes cron silently fail before
|
|
# the command runs. Ensure /opt/performancewest/logs exists + is deploy-owned.
|
|
# Healthcare data refresh: re-check every emailable NPI against the live
|
|
# government sources (CMS Revalidation list, OIG LEIE) + MX re-classification
|
|
# (Google-host detection) so warmup sends never go stale. Runs Mon/Wed/Fri 06:00
|
|
# Central, ~1h before the 07:00 weekday send. Mon/Wed/Fri (vs weekly) shrinks the
|
|
# CMS data-lag window to ~2-3 days, so a provider who just completed their
|
|
# revalidation stops being targeted faster (fewer "already done" replies).
|
|
# Takes ~8 min. SAM is opt-in (--sam-pages); SAM exclusions rarely carry an NPI,
|
|
# so OIG LEIE is the NPI-bearing exclusion source. Pipeline:
|
|
# 1. hc_data_refresh.py -- re-verify NPIs vs CMS/OIG + MX reclassify
|
|
# 2. download CMS revalidation_base.csv (institutional revalidation dates)
|
|
# 3. enrich_institutional_revalidation.py -- merge reval dates into the
|
|
# institutional CSV consumed by the pw-hc-nppes builder
|
|
# 4. build_healthcare_campaigns_cron.py --prune-only -- evict newly-Google-
|
|
# hosted + suppressed subscribers from the warmup lists
|
|
0 6 * * 1,3,5 deploy cd /opt/performancewest && python3 -u scripts/hc_data_refresh.py >> /opt/performancewest/logs/pw-hc-refresh.log 2>&1 && curl -s "https://data.cms.gov/sites/default/files/2026-05/96484587-20ec-4070-a4de-cd7de3ec0093/revalidation_base.csv" -o data/npi_build/revalidation_base.csv 2>>/opt/performancewest/logs/pw-hc-refresh.log && python3 -u scripts/enrich_institutional_revalidation.py data/hc_nppes_institutional_verified.csv data/npi_build/revalidation_base.csv data/hc_nppes_institutional_enriched.csv >> /opt/performancewest/logs/pw-hc-refresh.log 2>&1 && python3 -u scripts/build_healthcare_campaigns_cron.py --prune-only >> /opt/performancewest/logs/pw-hc-refresh.log 2>&1
|