Commit graph

175 commits

Author SHA1 Message Date
justin
09b32d6e75 fix(bounce-watchers): QID regex broke on out05-09/hcout transports
Both Postfix bounce watchers extracted the queue-ID with regex
postfix/[a-z]*[ (main) and postfix/[a-z0-9]*[ (hc), which only matched simple
transports like qmgr/cleanup. When the MTA started rotating sends through the
numbered warmup transports (out05-09/smtp, hcout1-3/smtp -- a transport/subprocess
name WITH a slash), the QID extraction returned empty, so status=bounced lines
never matched a tracked campaign QID and NO bounces were posted to listmonk since
~May 31. Result: dead mailboxes never got blocklisted and kept being retried,
dragging the warmup bounce rate.

Fix: regex postfix/[^[]*[ matches any transport/subprocess name up to the [pid].
Verified live on both watchers (out07/smtp and hcout1/smtp test bounces now
detected + posted).

Separately (server-side, not in repo): listmonk bounce.actions was null (no
auto-action), so even posted bounces did nothing. Set hard=1->blocklist,
complaint=1->blocklist; verified a real bounce now records + blocklists.
2026-06-08 15:02:32 -05:00
justin
b973c6c132 exclusions: block Google consumer mailboxes (gmail) from cold/warmup sends
Warmup audit (2026-06-08) found the main sending pool was eating a 37% bounce
rate, and 556 of those were Google 550-5.7.1 'likely unsolicited mail' spam
blocks -- of which gmail.com alone was 427 (77%). Google's cold-IP filter is the
strictest of the big providers and consumer gmail has the highest complaint
sensitivity, so mailing it from a warming IP is pure reputation damage.

Added GOOGLE_CONSUMER_DOMAINS (gmail.com, googlemail.com) to BLOCKED_EMAIL_DOMAINS,
which the daily trucking builder already enforces in its recipient SQL
(lower(domain) <> ALL(blocked)). Takes effect on the next nightly build.

Custom domains silently on Google Workspace are a smaller (~5%) MX-only signal,
already handled in the healthcare builder via the mx_provider flag; can be ported
to the main pool later if the residual warrants it.
2026-06-08 13:50:27 -05:00
justin
a78d60a127 hc: auto-reply for 'already revalidated' replies + permanent suppression
A lead replied with proof their Medicare revalidation was already approved (CMS
data-lag: the public Revalidation Due Date List still showed them overdue weeks
after approval). Two of these arrived same-day, so:

- Carbonio auto-reply (deployed on co.carrierone.com): created mailbox
  hc-replies@ on the info@ distribution list with a Sieve that auto-acknowledges
  'my revalidation is already complete' replies (tag + mark read + file into a
  'Reval Completed (auto-acked)' folder + on-brand reply explaining the CMS lag).
  CRITICAL: info@ is the shared reply-to for ALL campaigns (healthcare, trucking,
  telecom), so every rule is anchored to Medicare/revalidation context -- a
  trucking 'MCS-150 done, this is bogus' or telecom 'RMD done' reply does NOT
  trigger it (tested + passing). A buyer guard ('please file / how much') also
  suppresses the auto-reply so a human handles the sale.
  Carbonio 25.x Sieve quirks documented (vacation/imap4flags/body :text all
  unsupported; use reply/flag/tag/body :contains).

- Permanent suppression: new data/hc_suppress.txt do-not-contact list the warmup
  honors at import AND --prune removes from the live lists. Seeded with the two
  completed providers (Pangea Lab, Yakima Valley FWC); both also blocklisted in
  listmonk_hc and removed from lists 3 + 4.
2026-06-08 10:37:49 -05:00
justin
9cb10b18e0 feat(hc): deliverability prune -- evict newly-Google-hosted subscribers
Belt-and-suspenders for the edge you flagged: a domain already in a warmup list
could flip its MX to Google Workspace between weekly refreshes, after which it
would hard-bounce from the cold IP. The import-time guard only catches NEW adds.

- prune_holdouts(): enumerates each warmup list's subscribers, matches them
  against the FRESH master CSV (re-classified weekly), and removes any whose
  domain is now Google-hosted. DELIVERABILITY-ONLY -- it never evicts for
  audience reasons (an overdue provider drifting out of the 1-90 day window was
  a valid target when warmed; re-litigating that just wastes warmup progress).
- --prune (run alongside warming) and --prune-only (prune then exit).
- Wired into the weekly refresh cron as a --prune-only chained step, so MX is
  re-checked and holdouts removed every Monday before the weekday sends.

Verified end-to-end: with no Google domains in lists it's a 0-op; injecting a
simulated Google-flipped domain into the master, the prune correctly detects and
(in a real run) would remove it from every list it's on.
2026-06-08 03:39:56 -05:00
justin
54b92b1f06 fix(hc deliverability): MX-based Google-host exclusion during warmup
Found via live mail.log: Google-Workspace-hosted PRACTICE domains (custom
domains whose MX is aspmx.l.google.com, e.g. moosepharmacy.com, hc2kidney.com)
were getting hard 550-5.7.1 rejects from Google's cold-IP bulk filter -- exactly
the bounces that wreck a warming IP's reputation. The original google/non-google
split classified by the email's domain STRING, which can't see that a custom
domain silently uses Google Workspace; only an MX lookup reveals it (33% of our
domains, 228/689, are Google-hosted this way).

- hc_data_refresh.py: new MX classification (one lookup per unique domain via
  dnspython, cached) writes an mx_provider=google/other flag into the master and
  propagates it into the channel CSVs (auto-adding the column). --skip-mx for a
  fast status-only run.
- build_healthcare_campaigns_cron.py: warm_segment now drops mx_provider=google
  rows during warmup (HC_SKIP_GOOGLE=1 default; set 0 once IPs are warm). This is
  defense-in-depth -- correct regardless of which CSV the cron is pointed at.

Verified: today's sends (nongoogle CSV) had 0 Google bounces; the guard cuts the
Google-containing week1_verified cohort's revalidation candidates 82->8.
2026-06-08 03:32:12 -05:00
justin
feb677f6ce fix(hc warmup): only mail slightly-overdue providers (deliverability)
Mailing heavily-overdue NPIs (months/years past due) risks hitting practices
that have closed, merged, or abandoned the inbox -> hard bounces, which are the
fastest way to wreck a warming IP's reputation. The warmup now restricts the
reval_overdue selector to an inclusive [HC_OVERDUE_MIN, HC_OVERDUE_MAX] window
(default 1-90 days) and the OIG 'any' selector likewise excludes heavily-overdue
and dropped-off-list rows. On the current cohort this trims the overdue audience
178->96 and the OIG audience 399->317, holding out the stale long tail
(181-365d + 366d+). upcoming/active providers are unaffected.
2026-06-08 03:27:22 -05:00
justin
c79a7715e1 fix(hc): bugs found in self-audit of the new refresh + warmup + templates
Refresh (hc_data_refresh.py):
- CRITICAL: drop optout_ending from REFRESHED_FIELDS -- the refresh never
  computes it, so propagating it blanked the channel CSVs and would starve the
  compliance_bundle segment (whose selector IS optout_ending).
- MAJOR: only rewrite leie_excluded when OIG was actually pulled (guard was
  'not skip_oig OR not skip_sam', so a --skip-oig run blanked all exclusion
  flags). Also write 'Y' (matching the original list builder) not '1'.
- Use 'no_reval_flag' (the original vocabulary) instead of 'not_on_list' when an
  NPI drops off the reval list, and clear reval_due_date too.
- Throttle politeness: move time.sleep(0.05) above the early-continue paths so
  EVERY CMS request is spaced, not just the minority that are on the list.
- Guard blank-NPI rows (leave their status untouched instead of mislabeling).
- Master write preserves any columns beyond HEADER (no silent column drop).

Warmup cron (build_healthcare_campaigns_cron.py):
- Fix the daily-slice split: it summed to less than the budget (dropped ~2/day)
  and could OVERSHOOT on tiny totals (each 'other' floored to >=1). Now uses
  divmod for an even remainder and reclaims rounding onto the lead, so
  sum(per_seg) == total_slice exactly for every input (verified 0,1,2,7,100,300).

Templates: the non-revalidation emails rendered {{ .Subscriber.Attribs.detail }}
(a reval due date) under a 'Practice'/'Status'/'Record' label -- a wrong/
confusing personalization on a live send (esp. OIG, selector 'any'). All four
now show the practice name; 'detail' is retired from rendering (revalidation
uses reval_due_date/days_overdue directly).
2026-06-08 03:23:47 -05:00
justin
85dc3d5c3b hc refresh: propagate fresh status into the channel CSVs the cron reads
The channel CSVs (hc_warmup_nongoogle/google/week1_verified) are email-keyed
subsets of the master with extra deliverability columns (verify_ok/verify_reason).
The refresh now writes the fresh status fields (reval_due_date, days_overdue,
reval_status, leie_excluded, optout_ending, name/specialty/state) back into each,
preserving the extra columns and row membership, so a single weekly run updates
everything the campaign cron consumes -- not just the master.
2026-06-08 03:13:00 -05:00
justin
4f455475c0 hc: weekly data-refresh pipeline + multi-segment warmup cron
Two gaps closed:

1. hc_data_refresh.py (NEW): weekly source-data refresh. Re-checks every
   emailable NPI against the LIVE government sources so sends never go stale:
   - CMS Revalidation Due Date List (data.cms.gov per-NPI API; handles both ISO
     and US date formats, normalizes to MM/DD/YYYY).
   - OIG LEIE full CSV download (the NPI-bearing exclusion source).
   - SAM.gov v4 exclusions (key in .secrets/sam-api-key) -- OFF by default since
     SAM exclusions rarely carry an NPI and the full set is ~167k records; it's
     opt-in via --sam-pages. SAM's real value is the live per-name screening
     service, not a bulk NPI join.
   Writes the master CSV atomically (temp+rename). A provider who has since
   revalidated flips overdue->upcoming/not_on_list, so we stop nagging them.

2. build_healthcare_campaigns_cron.py: was revalidation-only (one hardcoded
   list/campaign/CSV/template). Now multi-segment: imports SEGMENTS from the
   single-source-of-truth registry, warms ALL five programs in parallel, each
   with its own list, dated campaign, and per-segment import-state file (so
   dedup is per-segment). A  per segment maps master-CSV rows to the
   right program (reval_overdue / reval_upcoming / leie_or_deactivated /
   optout_ending / any). Daily ramp slice is split across segments (revalidation
   leads at 50%, rest share the remainder) so every program collects engagement
   data while the IPs warm. Back-compat: seeds revalidation import-state from the
   legacy hc_imported_emails.txt once.
2026-06-08 03:06:29 -05:00
justin
0b0ff9d311 hc campaigns: make the HTML templates the single source of truth
build_healthcare_campaigns.py had a divergent inline HTML generator (old teal-
header + yellow issue-box layout, missing the official-record card and the per-
segment verify-it-yourself blocks) that nobody called -- the live cron reads the
hand-tuned data/hc_campaigns/hc_*.html files directly. Removed the dead
generator + cmd_render(); render() now READS the canonical template file so the
files can't drift from a parallel generator. SEGMENTS is now a metadata registry
(subject, template, cta_path, price, list_name, campaign_name, selector) that the
multi-segment cron will consume. Verified --list and that send-test still reads
the real bodies.
2026-06-08 02:57:49 -05:00
justin
483f185861 feat(healthcare): prove revalidation is real via official CMS data + self-verify
Skepticism ("is this even real?") is the top objection. The data IS accurate
(verified our subscribers' NPIs match the official CMS Revalidation Due Date List
exactly), so this is a credibility-presentation fix:

1. Email: replace the plain detail row with an "Official record - CMS Medicare
   Revalidation Due Date List" card (NPI, legal name, due date, days overdue)
   plus a "Verify on CMS.gov" button. Clearly labeled as our presentation of
   public CMS data, not a CMS screenshot (no impersonation).
2. API: npi/lookup now pulls the revalidation due date LIVE from the public CMS
   dataset (data.cms.gov) instead of the empty local table, and returns a
   revalidation{ due_date, source, cms_legal_name, verify_url } proof object.
3. Tool: /tools/npi-compliance-check shows a live "official record" card with a
   self-verify link when CMS returns a due date.

Builder now stores reval_due_date/days_overdue as separate attribs for the card
(existing 194 subscribers backfilled from their detail string).
2026-06-07 23:54:01 -05:00
justin
a732423f04 fix(deploy): port catalog generator + drift-check to Python (prod has no node)
The host-side generator ran 'node scripts/*.mjs' in deploy.sh, but the prod box
has python3 only (no node outside containers), so the site deploy failed at the
generation step. Reimplemented both in Python (byte-identical output, verified
via diff against the node version; matches scripts/sync_nav.py tooling).
2026-06-07 19:26:01 -05:00
justin
09e21a6c97 refactor(pricing): single source of truth for the service catalog
Previously two hand-maintained price lists (API COMPLIANCE_SERVICES + site
SERVICE_META) drifted apart -- that is how the healthcare +$200 raise charged
$399 while displaying $599. Eliminate the drift class entirely:

- Move the catalog to api/src/service-catalog.ts (the authority; checkout
  charges from it). compliance-orders.ts imports it.
- scripts/gen-service-catalog.mjs generates site/src/lib/service-catalog.generated.ts
  from the API source. intake_manifest.ts re-exports SERVICE_META from it, so all
  ~60 site pages keep working unchanged.
- deploy.sh regenerates + drift-checks before building (site build context is
  ./site only and cannot read ../api, so generation happens host-side).
- scripts/check-service-catalog-drift.mjs fails the build if the generated file
  ever diverges from the API (verified: passes aligned, fails on mismatch).

To change a price now, edit ONE file: api/src/service-catalog.ts.
2026-06-07 19:11:34 -05:00
justin
a4d67bcf9b hc-warmup: add list-hygiene script (drop undeliverable addrs, smtp_valid first)
Keeps only deliverable addresses (smtp_valid + catch_all_detected), drops
mx_unreachable + smtp_unknown rejects that defer/bounce and damage the warming
HC IP reputation. Sorts smtp_valid first so the daily slice hits verified
mailboxes first. Used to clean hc_warmup_nongoogle.csv (501 -> 399 rows).
2026-06-07 18:08:36 -05:00
justin
e5db147319 esign: make signing copy fully generic - remove all ink references from website/API
Client-facing and website code now describes only a generic per-document signing
authorization; nothing visible to signers or recorded in the website/API code or
DB schema references ink, paper, reproduction, or any fulfillment mechanics.

- rename esign-ink-consent.ts -> esign-sign-consent.ts; INK_CONSENT_TEXT ->
  SIGN_CONSENT_TEXT (generic: 'use my signature to complete and submit this
  single filing', no ink/paper/reproduce language); helpers ink* -> sign*
- portal-esign-generic.ts: API field ink_reproduction -> require_sign_consent,
  ink_consent_text -> sign_consent_text, request field ink_consent -> sign_consent
- signing page (site/public/portal/esign): all ids/vars/comments ink* -> sign*;
  no 'ink' string remains
- npi_provider metadata flag ink_reproduction -> require_sign_consent
- migration 090/092 + live DB column comments rewritten to drop ink/plotter
  wording (DB column names kept as ink_consent* for compat, internal only)
- order-timeline.ts buffer comments neutralized
- tests: 37 checks, consent text asserted to omit ink/plotter/paper/reproduce/etc

DB columns ink_consent* retained (internal, never sent to clients) to avoid a
risky rename of already-applied prod columns.
2026-06-07 05:06:26 -05:00
justin
a4bad723bc esign: ink-reproduction consent gate + patent-risk research
Consent gate (the legal linchpin from the wet-signature memo):
- migration 092 adds ink_consent/ink_consent_at/ink_consent_text to esign_records
- extract pure, unit-tested gate logic into esign-ink-consent.ts (DRY single
  source for route + signing page): isInkReproduction / inkConsentRequired /
  inkConsentSatisfied + verbatim client-safe INK_CONSENT_TEXT
- portal-esign-generic.ts: GET surfaces ink_reproduction + consent text; POST
  gates DRAWN signatures on ink-path docs on explicit consent, stores it
- signing page locks the signature block until consent is checked (drawn only)
- npi_provider marks cms855/cms10114 esign metadata ink_reproduction=true
- 33 unit checks: gate truth table + consent text omits all internal mechanics
  (plotter/machine/CMS/MAC/etc) and keeps required legal reassurances

Patent-risk memo (docs/legal/patent-risk-mechanical-wet-signature.md):
- prior-art-dated risk analysis (autopen 1803/1942, plotters, CNC = public domain
  => low risk on core concept; e-sign workflow space litigious)
- firsthand recent-grant sweep (1.58M USPTO grants 2021-2025, queried via DuckDB):
  ZERO patents on machine-applies-signature-in-ink; e-sign players hold only
  electronic-workflow patents. Not an FTO; flags where attorney search is needed
2026-06-07 04:44:11 -05:00
justin
894d989445 Add portable Line-us pen-arm support to ink-signature pipeline
Adds a second machine class (small fan-shaped reach arm) alongside the
CR-10/AxiDraw rectangular-bed plotters, so wet signatures can be produced
while away from the home station.

ink_signature_plotter.py:
- PlotterConfig gains dialect (marlin|lineus) + name; new LineUsConfig
  (native units, pen height = per-move Z, reach annulus from shoulder pivot).
- Named machine profiles (cr10 default, axidraw, lineus) via load_profile().
- bed_mm_to_lineus_units(), check_reach() (annulus for lineus, rectangle for
  marlin), compute_jig_offset_for_box() (solves jig from the ACTUAL fitted ink
  extent so a wide cell line doesn't over-constrain a small arm).
- emit_gcode() dispatches to emit_marlin_gcode()/emit_lineus_gcode().
- send_lineus(): WiFi TCP 1337 (NUL-terminated, ok-acked) or USB serial,
  dry_run=True default (same gating as the CR-10 path).

ink_signature_cli.py: --profile, --solve-jig (auto-applies jig offset),
--lineus-host/--lineus-usb, reach-check that refuses to --plot out-of-reach
on Line-us.

Tests: 43 checks (was 30) covering profiles, reach check, jig solve, lineus
emitter, dry-run sender. Docs updated with profiles + portable workflow.
2026-06-07 03:45:46 -05:00
justin
28b1af341d Wire fulfillment alerts to Telegram + surface order progress in portal + even out ERPNext sync
Telegram notifications:
- Add shared scripts/workers/telegram_notify.py (send_telegram, notify_fulfillment_todo,
  create_admin_todo) so every worker alerts the operator the same way; fire-and-forget.
- Fire notify_fulfillment_todo after each admin_todos insert across all 8 service
  handlers (9 sites) so no fulfillment task waits unseen.
  (Orders + quotes + tickets already notified via checkout/quotes/tickets routes.)

Client portal order progress:
- order-timeline: derive real per-step status from live signals (payment paid,
  e-signature signed, fulfillment_status) instead of a static template; add
  current_step to the response.
- Extract pure applyLiveStatus into order-timeline-status.ts (DB-free) + unit test
  (api/test/test_timeline_status.ts, 8 cases).
- portal /me now returns compliance_orders.fulfillment_status.
- Dashboard renders a client-safe Progress badge (In progress / Action needed /
  Filed-awaiting-confirmation / Completed); batches show the most actionable status.
  No back-office mechanics exposed.

ERPNext sync parity:
- Create a Sales Order for formation and fcc_carrier_registration orders (previously
  only canada_crtc + compliance synced); write erpnext_sales_order back to each table.
  Non-blocking, matches existing pattern.

Verified: API tsc clean, timeline unit tests 8/8, Astro build 58 pages,
cms10114/ink/paper_batch Python tests still green, no mechanics leaks.
2026-06-07 03:17:46 -05:00
justin
b0a8563a93 ink-signature: pen-plotter pipeline for original wet-ink CMS signatures
The Standard no-login CMS path needs an ORIGINAL ink signature on paper
(CMS-10114: 'Stamped, faxed or copied signatures will not be accepted'). This
adds a pipeline to redraw the provider's own captured strokes in real ink with a
pen on a CR-10 V2 (or any Marlin/GRBL machine) — original, in ink, never copied.

- migration 090: esign_records.signature_vector (JSONB stroke paths, 0..1).
- signing page now captures normalized stroke paths alongside the PNG; API
  stores a size-bounded vector for drawn signatures.
- ink_signature_plotter.py (hardware-independent): fit strokes to the signature
  anchor box, PDF-pt -> bed-mm via jig offset, emit Marlin/GRBL G-code (Z pen or
  M280 servo/BLTouch), SVG toolpath preview, and render_signature_on_pdf (a
  digital twin that proves the toolpath lands on the cert line). Gated serial
  sender (dry_run default).
- ink_signature_cli.py: end-to-end load-record -> gcode+preview, --test-box jig
  calibration, --plot to stream over USB.
- Corrected CMS-10114 signature anchor to sit inside the Section 4A signing cell
  (above the bottom rule, below the label).
- docs/ink-signature-plotter.md documents the CR-10 retrofit + interpretive risk.

Tests: test_ink_signature.py 30/30, test_cms10114.py 27/27, test_paper_batch.py
15/15, API tsc clean, Astro build 58 pages.
2026-06-07 02:34:17 -05:00
justin
e6a630ada1 healthcare: verify CMS-10114 update path, correct NPI Enumerator address, build CMS-10114 filler
Verified firsthand against the live CMS-10114 (Rev. 02/25, OMB 0938-0931):
- Section 1A confirms paper is valid for Change of Information (#2) AND
  Reactivation (#4), not just initial enumeration. Resolves the UNCERTAIN flag.
- Current mailing address is CMS NPI Enumerator Services, Mail Stop DO-01-51,
  7500 Security Blvd, Baltimore MD 21244. The old Fargo PO Box 6059 is retired;
  corrected in mac_routing.NPI_ENUMERATOR + all docs.
- No electronic no-login equivalent exists for CMS (NPI Registry API is
  read-only; PECOS/NPPES-IA require login), unlike FMCSA's ask.fmcsa ticket form.
  So tiers stay: Standard=paper CMS-10114 (no login), Expedited=NPPES surrogate.

New: cms10114_pdf_filler.py fills the flat official form via text overlay
(reason checkbox + NPI + Section 2A identity + Section 4A cert name + signature
anchor); wired into npi_provider._generate_10114_for_signing for nppes-update.
Signed forms route to the NPI Enumerator via the existing daily batch.

Tests: test_cms10114.py 27/27, test_paper_batch.py 15/15, Astro build 58 pages.
2026-06-07 02:04:41 -05:00
justin
7ea18dd3d8 healthcare: optional surrogate-access intake question (expedited path)
- NpiIntakeStep: add positively-framed 'can you grant electronic I&A Surrogate
  access?' question for all filing slugs (reval/reactivation/nppes-update/
  enrollment/bundle). Optional, never required, never mentions paper; captured
  as intake_data.surrogate_access (yes/no/blank). Astro build green (58 pages).
- npi_provider.py: surface the surrogate answer in the admin todo so fulfillment
  knows EXPEDITED (online via surrogate) vs STANDARD (e-sign + daily mail batch).
2026-06-07 00:33:33 -05:00
justin
138fec17e9 healthcare: daily batched paper-filing fulfillment
Standard (no-login) CMS filings are mailed in one Priority Mail envelope per
destination agency, batched each postal working-day morning to save postage.

- migration 089: paper_filing_batches table + esign_records.paper_batch_id /
  filing_destination_key (idempotent: a filing is batched at most once).
- batch_cover_sheet.py: per-agency cover sheet (sender/dest/date/manifest) +
  merged print-job PDF (cover + all enclosed signed filings).
- daily_paper_batch.py worker: gather signed+unbatched cms855/cms10114 filings,
  group by destination (MAC by state via mac_routing; Fargo for CMS-10114),
  build cover+merged PDF per agency, persist batch, mark filings batched.
  Self-gates on postal working days (skips weekends + federal/USPS holidays).
  Phase 1 = human prints+mails; phase 2 = wire print-mail API.
- worker-crons: pw-paper-batch systemd timer (Mon-Fri 13:30 UTC, self-gated).
- test_paper_batch.py: 15/15 pass (working-day gating, routing, cover+merge).
2026-06-07 00:30:01 -05:00
justin
258d23bdc6 healthcare: two-tier (standard paper / expedited surrogate) filing model
- Verified Standard(no-login)/Expedited(surrogate) matrix from official CMS-855
  PDFs (docs/healthcare-filing-tiers-verified.md): reactivation+revalidation are
  855I paper-to-MAC reasons, original-signature, routed by state; sig may not be
  delegated; 855B needs PECOS app fee.
- Add scripts/workers/mac_routing.py: state->MAC routing (all 56 jurisdictions,
  12 destinations) for envelope addressing + daily batch grouping. Addresses
  marked VERIFY before live mail.
- npi_provider.py: fix access strings to two-tier framing; NPPES update/reactivation
  no longer 'online-only'; note 855B fee.
- checkout.ts + service pages: strip client-facing mechanics & the paper-vs-tier
  choice; surrogate is the only optional, positively-framed ask (faster, never
  required, never share password).
2026-06-07 00:24:56 -05:00
justin
3da7794a85 hc email: add 'verify every detail for accuracy' (core strength) so the filing isn't rejected 2026-06-06 16:49:36 -05:00
justin
4233c90a4f hc email: reframe value-add to 'No 2FA. No government portals.' (we have a portal; the pain is CMS 2FA/identity-proofing); cron creates fresh dated campaign when prior is finished; add hc bounce watcher (Postfix->listmonk-hc webhook, hard/complaint->blocklist) 2026-06-06 16:47:12 -05:00
justin
2aa9e770c9 healthcare email: add color variety - amber warning box (urgency) + red overdue date + green price/CTA, breaking up the all-teal look into a problem->relief->action color story 2026-06-06 04:58:44 -05:00
justin
8eea9a694f healthcare email: add 'About Performance West' explainer (regulatory compliance consulting firm) after the no-logins relief copy 2026-06-06 04:57:42 -05:00
justin
0d212787ef healthcare email: add 'No logins. No portals. No headaches.' value-add (sells the relief, hides the mechanics); research doc on verified no-login third-party submission paths 2026-06-06 04:53:26 -05:00
justin
53ec011198 email trust signals: add data-safety + guarantee + social-proof strip to HC, telecom (campaign_template), and trucking (6 source + active campaigns via injector). Vertical accents: teal/blue/orange 2026-06-06 04:13:16 -05:00
justin
95698852ce healthcare warmup: gate Google/Workspace domains out of week 1 (they hard-reject cold IPs 550-5.7.1); send 501 non-Google practice domains first, defer 222 Google to week 2-3; cron uses hc_warmup_nongoogle.csv 2026-06-06 04:02:00 -05:00
justin
2bc86268f7 healthcare: HC warmup campaign cron (Mon-Fri 7AM Central) - imports overdue-first verified slice into listmonk-hc + runs Medicare-revalidation campaign via hc HOT stream; rate-throttled by pw-hc-rampcap 2026-06-06 03:57:08 -05:00
justin
2d3bccd31e healthcare email: white logo for teal header (was dark navy, invisible); drop NPPES-source footer line 2026-06-06 03:48:04 -05:00
justin
29c7a421e9 healthcare email: teal gradient header (matches site hero) + standalone CSV MX/SMTP verifier (binds .72 non-sending IP); gitignore PII warmup lists 2026-06-06 03:39:19 -05:00
justin
5129ebec5c healthcare email: add List-Unsubscribe/List-Id/Date/Precedence bulk headers to improve inbox placement on the cold hc IPs 2026-06-06 03:31:22 -05:00
justin
3859557506 healthcare: +$200 across all 6 provider services; add segmented marketing email builder (5 compliance-problem campaigns) + rendered HTML 2026-06-06 02:33:46 -05:00
justin
9bcd27db80 feat(site): vertical-specific order-page headers (trucking/telecom/healthcare/corporate) via unified VerticalOrderHeader; apply to all 49 order pages; retire TruckingOrderHeader 2026-06-06 01:52:22 -05:00
justin
68333148e6 fix(npi): two-tier Direct/HISP classifier so real Direct-Primary-Care/counseling practices stay institutional (was wrongly parked); add classifier unit tests 2026-06-06 00:09:42 -05:00
justin
90d8b94f3f feat(email): wire listmonk-hc into deploy + dev override + hc ramp-cap
- deploy.sh/deploy-dev.sh: bring up listmonk-hc (upstream image, excluded from
  build); document the one-time listmonk_hc DB create + --install.
- docker-compose.dev.override.yml: dev-only override (committed) that drops the
  prod host-port bindings and pins dev's own postgres volume (dev-pgdata) via
  compose !override tags. deploy-dev ships it as docker-compose.override.yml so
  syncing the canonical compose to the shared host no longer breaks dev's
  api-postgres (port :5432 clash + volume switch). Discovered + fixed while
  validating listmonk-hc on dev.
- pw-hc-rampcap.sh: healthcare analogue of pw-listmonk-rampcap, ramps the
  listmonk_hc cap 100->1000/h off /etc/postfix/hc-warmup-start, fully
  independent of the trucking ramp/cap.
2026-06-05 19:19:45 -05:00
justin
289c3b91be feat(healthcare): split outreach list into 3 outbound streams
Add scripts/healthcare_email_streams.py as the single source of truth for
classifying NPPES-endpoint emails into institutional (HOT stream) / consumer
(trucking-discipline stream) / direct (DirectTrust, parked), plus an exclude set
for non-prospect giants (va.gov, *.mil, cvshealth, walgreens, walmart).

Rework build_npi_outreach_lists.py to emit one CSV per stream
(npi_healthcare_institutional/consumer + npi_direct_secure), overdue-first
sorted, with companion files (revalidation/leie/optout) now optional.

Verified on May 2026 NPPES endpoint_pfile: 89,557 institutional / 19,366 consumer
/ 242,441 direct rows.
2026-06-05 18:59:44 -05:00
justin
4f49fad7f9 deploy: bring up the healthcare proxy-relay sidecar on prod and dev
deploy.sh: include proxy-relay in the default service set; it's an upstream
image (ginuerzh/gost) with no build context, so exclude it from 'compose build'
while keeping it in 'compose up'.

deploy-dev.sh: rsync docker-compose.yml to the dev server (it was never synced,
so new services like proxy-relay never reached dev) and add proxy-relay to the
'compose up --build' set.
2026-06-05 18:41:09 -05:00
justin
4060fd7562 fix(proxy): parse proxy creds with URL-reserved chars (e.g. '#') correctly
The residential proxy password contains a '#', which urlparse() misreads as a
URL fragment and corrupts the port (ValueError: Port could not be cast...).
Parse scheme://creds@host:port manually and percent-decode user/pass so both
raw ('#') and encoded ('%23') passwords work. Verified against the live
credential.
2026-06-05 18:34:19 -05:00
justin
17318f6e7d feat(healthcare): route NPPES/PECOS Playwright flows through residential SOCKS proxy
CMS healthcare portals (NPPES, PECOS, I&A) block datacenter IPs, so the
healthcare browser automation needs to egress via the residential proxy on
hg409y7ez04.sn.mynetname.net (username 'performancewest').

- undetected_browser: use_proxy now accepts an env-var name, so callers can
  select a domain-specific proxy. _proxy_config(proxy_env) reads it and falls
  back to UNDETECTED_PROXY_URL. Healthcare uses 'HEALTHCARE_PROXY_URL'.
- probe_npi_undetected: launches with use_proxy='HEALTHCARE_PROXY_URL' when set.
- npi_provider: documents that the (future) automated NPPES/PECOS flows must
  use the healthcare proxy.
- Plumb HEALTHCARE_PROXY_URL (+ UNDETECTED_PROXY_URL fallback) through the
  ansible env template and docker-compose workers env.

The credential itself is NOT in the repo. Set the full URL in the ansible
vault as vault_healthcare_proxy_url:
  socks5://performancewest:<password>@hg409y7ez04.sn.mynetname.net:<port>
Verified parsing + Playwright proxy-dict wiring with a unit test.
2026-06-05 14:36:01 -05:00
justin
bd9a70607f fix: maintain Services dropdown header from one canonical source
The site header / Services mega-dropdown was duplicated across two render
systems (Astro pages via Base.astro->nav.html, and ~80 pre-rendered static
public/**/index.html pages each embedding their own copy). They had drifted
into 5 different variants (missing 'New Carrier Setup', misplaced Healthcare
column, NEW vs FREE badges, em-dash encoding differences), so
dev.performancewest.net, the order pages, and the rest of the site disagreed.

- Make site/src/partials/nav.html the single source of truth (adopts the most
  complete variant).
- Add scripts/sync_nav.py to rewrite every static page's <nav> block from
  nav.html (idempotent; --check guards against drift in CI/deploy).
- Run the sync automatically in deploy.sh and scripts/deploy-dev.sh.
- Deprecate scripts/inject_healthcare_nav.py (now delegates to sync_nav.py).
- Neutralize the broken no-op SiteNav.astro component.

All 80 headers + the Astro-built order pages now render the identical dropdown.
2026-06-05 14:27:24 -05:00
justin
695ace207c Reframe healthcare filing as standard vs expedited; e2e test + bug fixes
Copy: drop paper/electronic/fax framing across the revalidation + enrollment
marketing pages and the order-confirmation email; present two service tiers:
- Standard filing  (no CMS account; we prepare CMS-855, you sign, we submit to MAC)
- Expedited filing (CMS I&A surrogate access; same-day PECOS filing + tracking)
Internal worker todos + the _STANDARD_FILING_SLUGS identifier updated to match.

New scripts/test_healthcare_e2e.py validates the whole order line (slug
consistency x6 places, price agreement, intake field collection+enforcement,
worker dispatch, handler execution producing CMS-855 PDF+anchor, free-tool
action_urls). 45 checks.

Bugs found + fixed by the test:
- medicare-enrollment requires practice_state server-side but the wizard never
  enforced it -> orders could be paid then stall. Wizard now requires it.
- determine_form_type defaulted org NPIs to the individual 855I because
  enumeration_type is never collected -> wrong form, CMS rejection. Now does a
  live NPPES lookup (safe 855I fallback).
2026-06-05 03:58:46 -05:00
justin
5cfe9702e2 Add Healthcare/NPI section to nav dropdown across all static pages
The site's pre-rendered public/**/index.html pages each embed their own copy
of the Services mega-dropdown and do not read src/partials/nav.html, so the
earlier nav.html-only edit never appeared. inject_healthcare_nav.py adds the
canonical Healthcare block (Medicare Revalidation, Medicare Enrollment, NPI/
NPPES Services, free NPI Compliance Check) to the desktop Column 3 + mobile
menu of all 80 static pages. Idempotent.
2026-06-05 03:05:19 -05:00
justin
e212f20a34 Add CMS-855 PDF filler + e-sign fulfillment for Medicare revalidation/enrollment
- cms855_pdf_filler.py: fills official CMS-855I/B/O/A AcroForms from intake
  (name, NPI, DOB, cert-page printed name) and records the signature anchor at
  the form's official /Sig box so the e-sign stamper lands on the cert line.
- npi_provider handlers (revalidation/reactivation/enrollment) now generate the
  paper CMS-855, upload it to MinIO, request_esign with anchors, and email the
  signing link. Human completes/verifies + USPS Priority Mails to the MAC.
- scripts/Dockerfile: copy the official CMS-855I/B/O/A forms into the image.
2026-06-05 02:27:11 -05:00
justin
31a53f89a6 feat(npi): offer paper CMS-855 path (e-sign + we mail to MAC) alongside I&A surrogacy
- order-confirmation email presents both filing methods: paper CMS-855 (no
  account needed, client e-signs one page, we print+mail to their MAC) and
  I&A surrogacy (faster, needs CMS account). NPPES-only services note that
  surrogacy is required (web-only).
- npi_provider handlers record the access model per service in admin todos.
- marketing copy leads with the lowest-friction paper option.
2026-06-05 01:53:44 -05:00
justin
e32193352b fix(npi): lenient CSV decoding in companion loader (CMS exports have stray latin-1 bytes) 2026-06-05 01:38:02 -05:00
justin
157c7a2571 test(npi): add slug consistency check across all wiring places 2026-06-05 01:35:04 -05:00
justin
4b0155542e feat(npi): healthcare marketing pages, nav dropdown, NPI lookup API + free tool + companion data migration/loader 2026-06-05 01:33:36 -05:00