justin/new-site
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
new-site/scripts
History
justin 8e5590b492 mail: DMARC aggregate-report parser + dedicated dmarc@ mailbox ingestion 
Tool 2 of the deliverability monitoring pair (Tool 1 = mail_reputation_monitor).
DMARC rua reports from dozens of operators (Google, Yahoo, Comcast, Cox, Bell,
Mimecast, Cisco ESA, GMX, mail.com, ...) were landing in ops@ (dmarc@ was a DL),
burying real mail and never parsed. Now ingested + queryable:

- dmarc@performancewest.net converted DL -> dedicated Carbonio mailbox; isolated
  IMAP creds in server .env, surfaced to workers in docker-compose.yml (mirrors
  OPS_IMAP_*). 29 historical reports moved ops@ -> dmarc@ via IMAP.
- scripts/dmarc_report_parser.py: IMAP fetch unseen -> decompress .gz/.zip/.xml
  (namespace-agnostic: classic + urn:ietf:params:xml:ns:dmarc-2.0 GMX/mail.com) ->
  parse aggregate XML -> upsert dmarc_report (keyed (org_name,report_id), no-op on
  re-parse) + dmarc_record per source IP. dmarc_pass = dkim_aligned OR spf_aligned.
  Marks \Seen. --dry-run/--all/--alert (7d per-IP summary + Telegram if one of OUR
  IPs <95% pass, or EXTERNAL IP sends >=20 failing msgs as us = spoofing under
  p=reject). psycopg2 imported lazily so --dry-run runs without the driver.
- api/migrations/102_dmarc_aggregate.sql: dmarc_report + dmarc_record tables.
- infra/cron/pw-dmarc-parser: 06:20 UTC daily --alert (after reputation, before scrub).
- docs/deliverability.md: DMARC section DONE; query examples.

Verified: dry-run --all parses all 28 reports (1 non-report test probe), 0 unknown
after the namespace fix.
2026-06-19 08:50:20 -05:00
..
document_gen fix(mcs150): point intake email to per-slug wizard (not sales page) + add Trailers field 2026-06-16 16:21:57 -05:00
email email trust signals: add data-safety + guarantee + social-proof strip to HC, telecom (campaign_template), and trucking (6 source + active campaigns via injector). Vertical accents: teal/blue/orange 2026-06-06 04:13:16 -05:00
formation fix(email): add text/plain part to every transactional + telecom email 2026-06-17 21:07:40 -05:00
templates Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
tests Add portable Line-us pen-arm support to ink-signature pipeline 2026-06-07 03:45:46 -05:00
workers test(workers): NPI recurring-cycle fulfillment path (13 assertions) 2026-06-18 09:38:26 -05:00
_email_exclusions.py feat(deliverability): exclude Apple consumer mail + scrub stale consumer subs from Listmonk 2026-06-18 23:55:58 -05:00
_email_plaintext.py email: handle unquoted hrefs in plaintext converter + add tests 2026-06-17 20:28:15 -05:00
alert.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
backup-db.sh Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
bounce-watcher.sh feat(deliverability): send bulk campaigns from dedicated subdomain send.performancewest.net 2026-06-18 23:07:23 -05:00
build_healthcare_campaigns.py hc: unlock the full 62k verified institutional pool for broad offers 2026-06-14 01:07:40 -05:00
build_healthcare_campaigns_cron.py feat(deliverability): send bulk campaigns from dedicated subdomain send.performancewest.net 2026-06-18 23:07:23 -05:00
build_ifta_quarterly_campaign.py campaigns: auto-rollout catch-all pool gated by warmup day + live bounce rate 2026-06-18 01:39:09 -05:00
build_npi_outreach_lists.py feat(healthcare): split outreach list into 3 outbound streams 2026-06-05 18:59:44 -05:00
build_otc_campaign.py campaigns: disable daily discount by default — test normal-price deals 2026-06-17 22:51:28 -05:00
build_trucking_campaigns.py feat(deliverability): send bulk campaigns from dedicated subdomain send.performancewest.net 2026-06-18 23:07:23 -05:00
build_ucr_annual_campaign.py campaigns: auto-rollout catch-all pool gated by warmup day + live bounce rate 2026-06-18 01:39:09 -05:00
burner_list_verify.py fix(verifier): mx_unreachable was mislabeling live big-ISP mailboxes 2026-06-17 05:48:08 -05:00
campaign_template.html can-spam: add full street address to ALL email templates + wire HC personal variant 2026-06-13 21:27:16 -05:00
check-service-catalog-drift.py feat(healthcare): OIG/SAM exclusion screening as $79/mo Stripe Subscription 2026-06-18 07:54:38 -05:00
check_npi_slug_consistency.sh test(npi): add slug consistency check across all wiring places 2026-06-05 01:35:04 -05:00
clean_hc_warmup_list.py hc-warmup: add list-hygiene script (drop undeliverable addrs, smtp_valid first) 2026-06-07 18:08:36 -05:00
create_agent_jaykordic.cjs Add referral/discount code to FCC carrier page + REF-JAYK05 agent 2026-06-02 14:31:22 -05:00
create_deficiency_report_campaign.py Pivot CRTC offering on FCC carrier page for A-Z wholesale carriers 2026-06-02 13:26:39 -05:00
create_deficiency_source_campaigns.py Add DOT check CTA to trucking deficiency emails 2026-06-04 18:29:01 -05:00
create_state_campaigns.py Lower trucking compliance pricing across product + marketing surfaces 2026-06-02 10:45:07 -05:00
deploy-dev.sh feat(email): wire listmonk-hc into deploy + dev override + hc ramp-cap 2026-06-05 19:19:45 -05:00
deploy-go-live.sh Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
deploy.sh Use --no-cache in deploy to prevent stale Docker builds 2026-04-27 07:57:14 -05:00
dmarc_report_parser.py mail: DMARC aggregate-report parser + dedicated dmarc@ mailbox ingestion 2026-06-19 08:50:20 -05:00
Dockerfile fix(docker): COPY SC COC Form.pdf into workers image 2026-06-16 09:23:43 -05:00
e2e-formation-order.mjs fix(formation): add working /name-search worker route + e2e harness 2026-06-09 07:51:54 -05:00
e2e-paypal-portal-fix.mjs test(e2e): fix compliance_orders seed columns (no total_cents); regression PASS 2026-06-09 14:35:04 -05:00
enrich_institutional_revalidation.py hc: unlock the full 62k verified institutional pool for broad offers 2026-06-14 01:07:40 -05:00
gap_tracker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
gen-service-catalog.py feat(healthcare): OIG/SAM exclusion screening as $79/mo Stripe Subscription 2026-06-18 07:54:38 -05:00
generate_all_permutations.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
generate_all_templates.py Fix flagged items: CRTC email submission, BITS todo, selector docs, stale plans 2026-05-04 11:33:45 -05:00
generate_canada_carrier_guide_pdf.py CRTC: ERPNext as portal source of truth + harden discount expiry + carrier guide PDF 2026-06-17 23:34:13 -05:00
harvest_clia_renewals.py clia: new CLIA certificate renewal service, order page, email template + harvest 2026-06-13 22:10:51 -05:00
harvest_nppes_mailable.py hc: NPPES endpoint mailable-inbox harvester (institutional/consumer, HISP-filtered) 2026-06-12 20:03:12 -05:00
harvest_otc_issuers.py otc: domain->email scraper + filing-agent domain filtering 2026-06-14 06:56:45 -05:00
hc-bounce-watcher.sh feat(deliverability): send bulk campaigns from dedicated subdomain send.performancewest.net 2026-06-18 23:07:23 -05:00
hc_data_refresh.py fix(hc deliverability): MX-based Google-host exclusion during warmup 2026-06-08 03:32:12 -05:00
healthcare_email_streams.py fix(npi): two-tier Direct/HISP classifier so real Direct-Primary-Care/counseling practices stay institutional (was wrongly parked); add classifier unit tests 2026-06-06 00:09:42 -05:00
inject_healthcare_nav.py fix: maintain Services dropdown header from one canonical source 2026-06-05 14:27:24 -05:00
inject_order_headers.py feat(site): vertical-specific order-page headers (trucking/telecom/healthcare/corporate) via unified VerticalOrderHeader; apply to all 49 order pages; retire TruckingOrderHeader 2026-06-06 01:52:22 -05:00
ip_rehab.py warmup(ip-rehab): bias recipients to multi-subscriber business domains (cut bounce) 2026-06-09 20:31:45 -05:00
listmonk-bounce-sync.py UPL-proof document templates + reliable bounce sync 2026-05-21 15:06:29 -05:00
load_npi_companion_data.py fix(npi): lenient CSV decoding in companion loader (CMS exports have stray latin-1 bytes) 2026-06-05 01:38:02 -05:00
mail_reputation_monitor.py feat(deliverability): mail reputation monitor (SNDS-equivalent from postfix logs) 2026-06-19 08:35:45 -05:00
match_clia_to_nppes.py clia: new CLIA certificate renewal service, order page, email template + harvest 2026-06-13 22:10:51 -05:00
methodology_campaign.py Wire createCommission() into compliance batch checkout 2026-05-11 11:00:38 -05:00
mx_tag_carriers.py mx_tag: bulk UPDATE via temp-table join (per-domain UPDATE full-scanned 1.49M rows each time) 2026-06-14 21:29:52 -05:00
ollama_client.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
otc_lead_pull.py scripts(otc): dedupe by CIK; commit the 861-company lead list 2026-06-09 07:10:54 -05:00
populate_deficiency_list.py Add shared DOCX style module + campaign tools 2026-05-04 08:52:07 -05:00
populate_new_carrier_startup_campaign.py campaigns: exclude full Yahoo/Verizon-Media domain family from cold email 2026-06-02 12:14:43 -05:00
probe_npi_undetected.py feat(healthcare): route NPPES/PECOS Playwright flows through residential SOCKS proxy 2026-06-05 14:36:01 -05:00
product_facts.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
recon_gckey.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
reddit-monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
reissue-onboarding-links.mjs fix(portal): onboarding/login links last 7 days, not 60 min 2026-06-09 22:50:09 -05:00
requirements.txt add fax filing pipeline: VitalPBX sender, attestation cover page with digital signature, compliance checker pending filing override 2026-05-30 18:32:01 -05:00
rescue-mark.mjs fix(portal): onboarding/login links last 7 days, not 60 min 2026-06-09 22:50:09 -05:00
rescue-mitchell-email.mjs fix(portal): onboarding/login links last 7 days, not 60 min 2026-06-09 22:50:09 -05:00
rescue-mitchell.mjs chore: export ensureComplianceSalesOrder for rescue/backfill use 2026-06-09 14:44:28 -05:00
rescue-paul-correct.mjs fix(checkout): create Postgres customers row on order completion (PayPal login bug) 2026-06-09 14:28:19 -05:00
rescue-paul-set-password.mjs scripts: one-off fresh password-set link for Paul Wilson (ERPNext auth) 2026-06-17 10:19:53 -05:00
rescue-paul.mjs fix(portal): onboarding/login links last 7 days, not 60 min 2026-06-09 22:50:09 -05:00
scrape_otc_emails.py otc: domain->email scraper + filing-agent domain filtering 2026-06-14 06:56:45 -05:00
scrub_listmonk_consumer.py feat(deliverability): exclude Apple consumer mail + scrub stale consumer subs from Listmonk 2026-06-18 23:55:58 -05:00
send_test_campaigns.py Add shared DOCX style module + campaign tools 2026-05-04 08:52:07 -05:00
setup_erpnext_payments.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
setup_trucking_campaigns.py Lower trucking compliance pricing across product + marketing surfaces 2026-06-02 10:45:07 -05:00
sync_nav.py fix: maintain Services dropdown header from one canonical source 2026-06-05 14:27:24 -05:00
test_email_plaintext.py email: handle unquoted hrefs in plaintext converter + add tests 2026-06-17 20:28:15 -05:00
test_healthcare_e2e.py Reframe healthcare filing as standard vs expedited; e2e test + bug fixes 2026-06-05 03:58:46 -05:00
test_healthcare_email_streams.py fix(npi): two-tier Direct/HISP classifier so real Direct-Primary-Care/counseling practices stay institutional (was wrongly parked); add classifier unit tests 2026-06-06 00:09:42 -05:00
verify_csv_emails.py verify: tag each address with its MX provider for per-operator warmup throttling 2026-06-12 20:06:44 -05:00