Commit graph

586 commits

Author SHA1 Message Date
justin
68e6b60951 fix: worker emails (localhost:25 -> SMTP relay) + create ERPNext SO on webhook payment
Two bugs found tracing Mitchell Allen's batch CB-95BA6C90 (5 DOT services, card):

1) Worker authorization/signing-link/status emails were sent via
   smtplib.SMTP('localhost', 25), which has no MTA in the workers container ->
   every send failed '[Errno 111] Connection refused', so customers never got
   their e-sign links and orders sat 'awaiting client signature' forever. Routed
   all 9 hardcoded localhost:25 sites (state_trucking, mcs150_update, boc3_filing,
   hazmat_phmsa, mailbox_setup, dot_esign, completion_emails) through the
   authenticated SMTP relay (SMTP_HOST/PORT/STARTTLS/login) + added a shared
   worker_email.send_worker_email helper.

2) The ERPNext Sales Order for compliance/compliance_batch was only created in
   the /checkout/create-session endpoint, but CARD orders confirm via the Stripe
   WEBHOOK -> handlePaymentComplete, which never created the SO. Result: every
   webhook-confirmed order had erpnext_sales_order=NULL and workers logged
   'Sales Order not found 404' then built from PG. Added idempotent
   ensureComplianceSalesOrder() to handlePaymentComplete so ALL payment methods
   (card-webhook, PayPal, crypto) create + link the SO.
2026-06-09 14:40:46 -05:00
justin
220f301453 test(e2e): fix compliance_orders seed columns (no total_cents); regression PASS
e2e-paypal-portal-fix.mjs now passes against live prod: completing a compliance
order creates the customers row (id, name=E2E Tester, company from intake_data,
no password) -> customer can register/reset + log in. PayPal login bug fixed.
2026-06-09 14:35:04 -05:00
justin
3c65dd8748 fix(checkout): pull company from intake_data (compliance has no customer_company col)
compliance_orders stores company in intake_data JSON, not a column; read it from
there (company/legal_name/entity_name) with graceful fallback. Fix e2e test seed
accordingly.
2026-06-09 14:31:42 -05:00
justin
9987b1e30d fix(checkout): create Postgres customers row on order completion (PayPal login bug)
Portal login + forgot-password read the Postgres customers table (bcrypt), NOT
ERPNext. ensureCompliancePortalUser (the common path for Stripe/PayPal/crypto via
handlePaymentComplete) only provisioned the ERPNext customer/website-user and
never created the customers row -- so customers (notably PayPal, who reach this
path directly) had no account to log into or reset a password against. Now upserts
the customers row (no password; ON CONFLICT keeps any existing hash) with name +
company so they can register/reset and log in immediately.

Also: narrowed the placeholder-email skip from 'any synthetic@ or pipeline.com' to
exactly 'synthetic@pipeline.com' (the FMCSA-census placeholder) so real customers
on those real consumer domains aren't wrongly skipped -- which is what bit Paul
Wilson. Added cc support to sendEmail. e2e-paypal-portal-fix.mjs is the regression
test (seeds a compliance order, runs handlePaymentComplete, asserts the customers
row is created). Rescue scripts for the affected customer included.
2026-06-09 14:28:19 -05:00
justin
b437f66bc8 docs(dexit): name search fixed (TX open-data API) / honest (NV unknown); rm probes
E2E harness re-run = ALL PASS: TX returns real availability via the open-data API,
NV returns unknown (available=None) to flag manual verification, both flow through
to correct ERPNext Sales Orders. Removed one-off portal-probe scripts.
2026-06-09 08:46:28 -05:00
justin
20c11e6180 fix(formation/NV): name search returns unknown (admin-verify), not a fake result
Nevada's entity search (esos.nv.gov/SilverFlume) is behind Imperva Incapsula bot
protection that blocks headless + the residential proxy IPs, and NV has no public
open-data API/bulk dataset. The old adapter scraped the blocked challenge page and
returned available=False for everything (would tell customers a free name is taken)
and also had a NameError bug (f"${CODE}..."). Now NV detects the Incapsula
challenge and returns available=None ('could not determine') with a note to verify
manually on SilverFlume -- never a false 'taken', so it never wrongly blocks an
order. TX remains fully automated via the open-data API.
2026-06-09 08:41:22 -05:00
justin
f94ad1682b fix(formation/TX): name search via Texas open-data API, not scraping
The TX Comptroller web search is now a JS form (old input#entityName selector
dead) and SOSDirect is login-gated, so the scraper returned garbage. Replaced
search_name with the Texas Socrata 'Active Franchise Taxpayers' dataset
(data.texas.gov/resource/9cir-efmm.json) over SoQL -- free, no-auth, no-login,
no bot-blocks. Exact normalized match => unavailable; no rows => available; API
error => available=None (never a false 'taken'). Verified: unique name = 0 rows
(available), 'APPLE INC.' = exact match (taken).
2026-06-09 08:34:37 -05:00
justin
561ad78ea8 docs(dexit): note NV adapter also mis-parses happy path (not just errors) 2026-06-09 08:14:09 -05:00
justin
76c4d55603 fix(formation): name-search returns null (not false) on adapter error
E2E harness exposed that the NV name-search adapter times out on a stale input
selector and search_name() swallowed the error and returned available=False --
i.e. it would tell customers an AVAILABLE name is taken. Now returns
available=None ('could not determine') on adapter error / unknown state, which the
API already maps to null. The NV/TX portal selectors still need a scraping fix
(separate task; e2e harness is the acceptance test) before enabling a self-serve
formation checkout. Documented full e2e results + the bugs caught (missing ERPNext
Items, entity_type case, missing /name-search route) in the readiness doc.
2026-06-09 08:06:43 -05:00
justin
4c0decd175 fix(formation): add working /name-search worker route + e2e harness
Two latent bugs the e2e harness caught:
1. api entities.ts GET /states/:code/name-search calls WORKER_URL/name-search,
   but job_server had NO such route -> 404 -> silently fell back to stale
   entity_cache on every live name check. Added a synchronous /name-search route
   returning {available,exact_match,similar_names,state}.
2. both the new route AND the existing async handle_name_search imported a
   nonexistent search_name_sync(); fixed to drive the real async search_name()
   via an event loop (same pattern as /entity-status).

scripts/e2e-formation-order.mjs: replays the real formation order flow (live
name search -> formation_orders insert -> ERPNext customer + Sales Order with
BUSINESS-FORMATION + STATE-FILING-FEE line items -> verify SO total + DB linkage
-> cleanup) without a real Stripe charge or state filing. Run in the api container.
Also created the missing ERPNext Items (BUSINESS-FORMATION, STATE-FILING-FEE,
FOREIGN-QUAL-SINGLE/MULTI) that the formation SO references.
2026-06-09 07:51:54 -05:00
justin
c0344769a0 docs(dexit): handle foreign-qualification complication on a move
A DE corp foreign-qualified in CA/NY/etc must update EVERY foreign registration
when it domesticates (withdraw in the destination state since it becomes domestic
there; amend or re-file in the rest to reflect the new home state) -- getting it
wrong = doing business unregistered (default judgments/penalties). We already have
the building block (foreign-qualification-single/multi SKU + ForeignQualificationHandler
that fans out per state + migration 066/073 schema); missing is amend/withdraw modes
+ an intake step capturing the list of qualified states. Product = multi-part, priced
per state touched (revenue multiplier), scoped at intake.
2026-06-09 07:39:50 -05:00
justin
b5b2e6e6c3 site: add DEXIT corporate-services page + readiness assessment + cited filings
New page /services/corporate/dexit-reincorporation (matches CRTC service-page
structure): explains DEXIT, the DE franchise-tax dollar driver (real Oracle Health
proxy: $23,600 -> ~$1,000), NV/TX/FL destination guidance, 6-step how-it-works,
3 cited real SEC reincorporation filings (Oracle Health, FG Financial, LogicMark)
with verbatim quotes + EDGAR links, honesty callout, and a lead-gen CTA ('Get my
DEXIT estimate' -> /contact?topic=dexit, NOT a buy-now checkout). Linked from the
corporate services index (new card) + the global Services dropdown across the site.

docs/dexit-cited-filings.md: the filing excerpts + verified gov/statute links.
docs/dexit-readiness-assessment.md: HONEST e2e readiness -- new NV/TX formation is
built (checkout order_type=formation -> formation_orders -> ERPNext SO ->
formation_worker -> TX/NV adapters) but unverified e2e; the 'move a company'
(conversion/domestication) flow + corporate annual-report automation are NOT built;
EIN is kept on a conversion (our ein_worker does NEW EINs only). Page stays lead-gen
until the generic entity-conversion SKU + admin-assisted handler are built+tested.
2026-06-09 07:35:12 -05:00
justin
bcedf2b318 docs(otc): add audience + DEXIT motivation + appeal strategy (sec 4d)
Grounded in real reincorporation proxies. WHO: microcap CEO/founder/CFO/secretary
(no legal dept -> decision-maker, not gatekeeper). WHAT they want: (1) cut the DE
franchise tax -- the hard-dollar driver; real proxy shows a pre-revenue startup
paid $23,600 DE franchise tax vs ~$1,000 in NV; (2) stronger D&O liability shield
(NRS 78.138); (3) escape DE's 2024 case-law shift; (4) TXSE/story optionality.
HOW to convert: lead with their dollar math not features, productize the dread
(flat fee, we do filing legwork, lawyer just signs), recommend the destination,
stack recurring RA + annual-report revenue, de-risk with guarantee + verifiable
sources.
2026-06-09 07:16:34 -05:00
justin
0ccc323af7 data(otc): add display_name + short_name merge fields for outreach
EDGAR issuer names are messy (ALL-CAPS, trailing /DE/ /NV state tags, entity
suffixes) and render badly in a personalized subject line. Added:
  display_name -- title-cased, state-tag-stripped, suffix kept
  short_name   -- suffix dropped too, for a punchy subject merge field
    e.g. 'WINDTREE THERAPEUTICS INC /DE/' -> 'Windtree Therapeutics'
So 'Is DEXIT in the cards for {{short_name}}?' renders clean. ~98% of short_names
are <=40 chars (survive mobile subject truncation).
2026-06-09 07:14:06 -05:00
justin
37393e5bbc scripts(otc): dedupe by CIK; commit the 861-company lead list
The master file lists warrants/units as separate tickers under one CIK, so the
pull now dedupes to one row per company (other tickers kept in all_tickers).

data/otc_leads.csv: 861 unique active US-domestic microcap OTC issuers
(<$75M float, all actively filing, 100% with business address + phone). By
incorporation: DE 365, NV 325 (DE+NV=690 = the reincorporation targets), WY 44,
FL 39, MD 38. Dropped from the 2,771 OTC universe: 1,672 foreign, 62
accelerated/large filers, 73 delinquent/dark. EDGAR has no email -> phone +
address captured for enrichment / direct mail / call.
2026-06-09 07:10:54 -05:00
justin
1b3cbf2fbf scripts(otc): SEC EDGAR lead-pull + reincorporation-destination research
scripts/otc_lead_pull.py: pulls company_tickers_exchange.json + per-issuer
submissions/CIK*.json from SEC EDGAR (10 req/sec, declared User-Agent), filters
to active US-domestic microcaps (drops foreign ADRs, accelerated/large filers
that keep counsel on retainer, and delinquent/dark shells), writes a CSV with
state of incorporation, business/mailing address, phone, SIC, filer size bucket,
last filing date. DE/NV prioritized.

docs 4c: where companies actually reincorporate TO -- Nevada #1 (281 filings),
Texas the fast riser (99 all-time, but 27 vs NV 33 since 2024), Florida modest
(23), Wyoming niche (8). Lead with 'leaving Delaware?' and let client pick
NV/TX/FL; same flat-fee conversion productizes across all three.
2026-06-09 06:58:01 -05:00
justin
ee44800934 docs(otc): add size analysis -- skip large filers, target the ~93% microcaps
SEC filer-category data (n=139 US-domestic OTC): ~93-95% are sub-$75M-float
Smaller Reporting / non-accelerated microcaps; only ~4-5% are accelerated/large
(those keep securities counsel on retainer -- not our lane). 91% actively filing.
Recommend filtering OUT Large accelerated/Accelerated + delinquent/dark -> ~700-850
active microcap prospects. Pitch framing: not 'replace your lawyer' but 'flat-fee
commoditized state filings so counsel only does what needs a lawyer'.
2026-06-09 06:49:16 -05:00
justin
497a4d4409 docs: research OTC Markets/pink sheets as a corporate-services lead source
SEC EDGAR (free, public, bulk-OK at 10 req/sec) is the goldmine -- per-issuer
state of incorporation, business/mailing address, phone, SIC, entity type via
company_tickers_exchange.json + submissions/CIK*.json. ~2,771 OTC SEC filers;
~35% US-domestic (~970), of which DE+NV = 73% -> the reincorporate-to-Texas /
registered-agent / annual-report / foreign-qualification target list. EDGAR has
no email (enrich from IR pages or direct mail/call). Texas reincorporation is a
real early trend (48/43 EDGAR filings; TBOC Ch.10 conversion, Texas Business
Court, TXSE). CAN-SPAM compliant B2B; filter to US states to avoid CASL/GDPR.
Do NOT scrape OTCMarkets.com (ToS prohibits; unneeded).
2026-06-09 06:43:15 -05:00
justin
9b9d317916 infra/k8s: shkeeper liveness+readiness probes (fix recurring crypto.performancewest.net downtime)
crypto.performancewest.net kept going down because the shkeeper-deployment web
pod periodically HANGS (HTTP server deadlocks while the apscheduler background
thread keeps the process alive). The helm chart (shkeeper-1.7.15) ships NO
liveness or readiness probe, so k8s saw the hung pod as Running and never
restarted it, and kept routing traffic to the dead backend -> site down until a
manual restart.

Added HTTP probes on / :5000 (302 = healthy): liveness auto-restarts a hung pod,
readiness pulls it from the Service endpoints. Applied live via kubectl patch
(chart does not expose probes via values; re-apply after any helm upgrade --
command in the file header). Verified: new pod comes up READY 1/1 (probe passes)
and crypto.performancewest.net serves 302 again.
2026-06-09 04:57:50 -05:00
justin
a308aeed6b fix(checkout): batch SO custom_order_type must be 'compliance' not 'compliance_batch'
ERPNext's PW Order Type select field only allows formation/canada_crtc/bundle/
compliance. A batch is a compliance order (multi-service), so use 'compliance';
the multi-service nature is already captured by the line items + external order
id.
2026-06-09 00:27:17 -05:00
justin
baa40443de fix(checkout): create ERPNext Sales Order for compliance_batch orders
Batch orders (CB-XXXX, used by the trucking new-carrier flow and any multi-
service cart) never created an ERPNext Sales Order -- the SO-creation branch was
gated to order_type 'compliance' only. So those paid orders never reached
ERPNext for fulfillment/accounting (0 of all paid batch orders had an
erpnext_sales_order). Added a compliance_batch branch that creates ONE Sales
Order with a line item per service in the batch (+ government-fee + processing-
fee lines), then stamps the SO name on every batch row. Non-blocking like the
others. Also created the missing ERPNext Items the new slugs reference
(DOT-NEW-CARRIER-BUNDLE, LLC-FORMATION, CORP-FORMATION, NEW-CARRIER-BUNDLE which
was missing too, GOVERNMENT-FILING-FEE).
2026-06-09 00:23:15 -05:00
justin
90bccfda32 fix(checkout): route dot-new-carrier-bundle on success page + worker pipeline
Follow-on to the trucking new-carrier slug fix:
- success page: add dot-new-carrier-bundle to DOT_SLUGS + NEW_CARRIER_SLUGS so
  the order-confirmation 'what to expect' messaging classifies it as trucking.
- pipeline_orchestrator: the trucking onboarding PIPELINE was keyed under the
  bare 'new-carrier-bundle' slug, which is the TELECOM bundle's slug (also a
  collision at the worker layer). Re-keyed to 'dot-new-carrier-bundle' so a
  trucking bundle never runs the telecom pipeline (and vice versa).
2026-06-08 23:48:56 -05:00
justin
c6819371d8 fix(checkout): trucking new-carrier ordered the wrong (telecom) product + ACH broke
Two reported bugs, plus two related ones found while tracing:

1. WRONG PRODUCT (Stripe showed 'FCC setup package' for a trucking order): the
   trucking new-carrier form reused the slug 'new-carrier-bundle', which is the
   TELECOM VoIP onboarding bundle (FRN+499+RMD+CPNI+CALEA, $1799). So trucking
   customers were charged the telecom product/price and saw FCC on their receipt.
   Added a distinct 'dot-new-carrier-bundle' (USDOT+MC+BOC-3+MCS-150+Drug&Alcohol,
   $599 + FMCSA gov fees) and pointed the trucking page at it.

2. ACH 500 error: the Stripe session requested the Financial Connections
   'balances' permission, which isn't activated on the account -> Stripe rejected
   the whole session (invalid_request_error). Removed 'balances' (+prefetch); we
   only need 'payment_method' to collect+charge the bank account.

Also fixed (found while tracing):
3. The telecom new-carrier-bundle's BUNDLE_COMPONENTS listed TRUCKING slugs by
   mistake (copy/paste) -- corrected to its real FCC components.
4. The trucking page offered llc-formation / corp-formation / foreign-qual which
   did not exist in the catalog (batch would 400). Added llc-formation +
   corp-formation; remapped foreign-qual -> foreign-qualification-single.

Catalog regenerated (66 -> 69 services), drift-check + tsc clean.
2026-06-08 23:42:36 -05:00
justin
7c39a858cc monitoring: daily warmup IP-reputation Telegram alert
End-of-day (20:00 Central) check of campaign deliverability across both sending
pools (main out05-09 + healthcare hcout). Sends a Telegram alert ONLY when there
is a reputation problem -- delivery below 65% or a spam/policy-block (550-5.7.1)
spike above 150/day -- so healthy days stay silent. Reuses the existing
TELEGRAM_BOT_TOKEN/CHAT_ID from /opt/performancewest/.env. Logs every run to
/var/log/pw-warmup-healthcheck.log for history. Excludes internal/probe noise so
the delivery figure reflects real external recipients.
2026-06-08 21:06:41 -05:00
justin
09b32d6e75 fix(bounce-watchers): QID regex broke on out05-09/hcout transports
Both Postfix bounce watchers extracted the queue-ID with regex
postfix/[a-z]*[ (main) and postfix/[a-z0-9]*[ (hc), which only matched simple
transports like qmgr/cleanup. When the MTA started rotating sends through the
numbered warmup transports (out05-09/smtp, hcout1-3/smtp -- a transport/subprocess
name WITH a slash), the QID extraction returned empty, so status=bounced lines
never matched a tracked campaign QID and NO bounces were posted to listmonk since
~May 31. Result: dead mailboxes never got blocklisted and kept being retried,
dragging the warmup bounce rate.

Fix: regex postfix/[^[]*[ matches any transport/subprocess name up to the [pid].
Verified live on both watchers (out07/smtp and hcout1/smtp test bounces now
detected + posted).

Separately (server-side, not in repo): listmonk bounce.actions was null (no
auto-action), so even posted bounces did nothing. Set hard=1->blocklist,
complaint=1->blocklist; verified a real bounce now records + blocklists.
2026-06-08 15:02:32 -05:00
justin
b973c6c132 exclusions: block Google consumer mailboxes (gmail) from cold/warmup sends
Warmup audit (2026-06-08) found the main sending pool was eating a 37% bounce
rate, and 556 of those were Google 550-5.7.1 'likely unsolicited mail' spam
blocks -- of which gmail.com alone was 427 (77%). Google's cold-IP filter is the
strictest of the big providers and consumer gmail has the highest complaint
sensitivity, so mailing it from a warming IP is pure reputation damage.

Added GOOGLE_CONSUMER_DOMAINS (gmail.com, googlemail.com) to BLOCKED_EMAIL_DOMAINS,
which the daily trucking builder already enforces in its recipient SQL
(lower(domain) <> ALL(blocked)). Takes effect on the next nightly build.

Custom domains silently on Google Workspace are a smaller (~5%) MX-only signal,
already handled in the healthcare builder via the mx_provider flag; can be ported
to the main pool later if the residual warrants it.
2026-06-08 13:50:27 -05:00
justin
2156a5e05f hc refresh: run Mon/Wed/Fri instead of weekly to shrink CMS data-lag
The 'already revalidated' replies come from the CMS data-lag window (a provider
completes their revalidation but CMS's public Due Date List still shows them
overdue for weeks). Running the refresh 3x/week instead of weekly shrinks that
window from up to 7 days to ~2-3, so a provider who just completed stops being
targeted faster. No change to the overdue window or audience size -- this is the
lever that reduces stale-data complaints without losing prospects.
2026-06-08 10:53:36 -05:00
justin
a78d60a127 hc: auto-reply for 'already revalidated' replies + permanent suppression
A lead replied with proof their Medicare revalidation was already approved (CMS
data-lag: the public Revalidation Due Date List still showed them overdue weeks
after approval). Two of these arrived same-day, so:

- Carbonio auto-reply (deployed on co.carrierone.com): created mailbox
  hc-replies@ on the info@ distribution list with a Sieve that auto-acknowledges
  'my revalidation is already complete' replies (tag + mark read + file into a
  'Reval Completed (auto-acked)' folder + on-brand reply explaining the CMS lag).
  CRITICAL: info@ is the shared reply-to for ALL campaigns (healthcare, trucking,
  telecom), so every rule is anchored to Medicare/revalidation context -- a
  trucking 'MCS-150 done, this is bogus' or telecom 'RMD done' reply does NOT
  trigger it (tested + passing). A buyer guard ('please file / how much') also
  suppresses the auto-reply so a human handles the sale.
  Carbonio 25.x Sieve quirks documented (vacation/imap4flags/body :text all
  unsupported; use reply/flag/tag/body :contains).

- Permanent suppression: new data/hc_suppress.txt do-not-contact list the warmup
  honors at import AND --prune removes from the live lists. Seeded with the two
  completed providers (Pangea Lab, Yakima Valley FWC); both also blocklisted in
  listmonk_hc and removed from lists 3 + 4.
2026-06-08 10:37:49 -05:00
justin
9cb10b18e0 feat(hc): deliverability prune -- evict newly-Google-hosted subscribers
Belt-and-suspenders for the edge you flagged: a domain already in a warmup list
could flip its MX to Google Workspace between weekly refreshes, after which it
would hard-bounce from the cold IP. The import-time guard only catches NEW adds.

- prune_holdouts(): enumerates each warmup list's subscribers, matches them
  against the FRESH master CSV (re-classified weekly), and removes any whose
  domain is now Google-hosted. DELIVERABILITY-ONLY -- it never evicts for
  audience reasons (an overdue provider drifting out of the 1-90 day window was
  a valid target when warmed; re-litigating that just wastes warmup progress).
- --prune (run alongside warming) and --prune-only (prune then exit).
- Wired into the weekly refresh cron as a --prune-only chained step, so MX is
  re-checked and holdouts removed every Monday before the weekday sends.

Verified end-to-end: with no Google domains in lists it's a 0-op; injecting a
simulated Google-flipped domain into the master, the prune correctly detects and
(in a real run) would remove it from every list it's on.
2026-06-08 03:39:56 -05:00
justin
54b92b1f06 fix(hc deliverability): MX-based Google-host exclusion during warmup
Found via live mail.log: Google-Workspace-hosted PRACTICE domains (custom
domains whose MX is aspmx.l.google.com, e.g. moosepharmacy.com, hc2kidney.com)
were getting hard 550-5.7.1 rejects from Google's cold-IP bulk filter -- exactly
the bounces that wreck a warming IP's reputation. The original google/non-google
split classified by the email's domain STRING, which can't see that a custom
domain silently uses Google Workspace; only an MX lookup reveals it (33% of our
domains, 228/689, are Google-hosted this way).

- hc_data_refresh.py: new MX classification (one lookup per unique domain via
  dnspython, cached) writes an mx_provider=google/other flag into the master and
  propagates it into the channel CSVs (auto-adding the column). --skip-mx for a
  fast status-only run.
- build_healthcare_campaigns_cron.py: warm_segment now drops mx_provider=google
  rows during warmup (HC_SKIP_GOOGLE=1 default; set 0 once IPs are warm). This is
  defense-in-depth -- correct regardless of which CSV the cron is pointed at.

Verified: today's sends (nongoogle CSV) had 0 Google bounces; the guard cuts the
Google-containing week1_verified cohort's revalidation candidates 82->8.
2026-06-08 03:32:12 -05:00
justin
feb677f6ce fix(hc warmup): only mail slightly-overdue providers (deliverability)
Mailing heavily-overdue NPIs (months/years past due) risks hitting practices
that have closed, merged, or abandoned the inbox -> hard bounces, which are the
fastest way to wreck a warming IP's reputation. The warmup now restricts the
reval_overdue selector to an inclusive [HC_OVERDUE_MIN, HC_OVERDUE_MAX] window
(default 1-90 days) and the OIG 'any' selector likewise excludes heavily-overdue
and dropped-off-list rows. On the current cohort this trims the overdue audience
178->96 and the OIG audience 399->317, holding out the stale long tail
(181-365d + 366d+). upcoming/active providers are unaffected.
2026-06-08 03:27:22 -05:00
justin
c79a7715e1 fix(hc): bugs found in self-audit of the new refresh + warmup + templates
Refresh (hc_data_refresh.py):
- CRITICAL: drop optout_ending from REFRESHED_FIELDS -- the refresh never
  computes it, so propagating it blanked the channel CSVs and would starve the
  compliance_bundle segment (whose selector IS optout_ending).
- MAJOR: only rewrite leie_excluded when OIG was actually pulled (guard was
  'not skip_oig OR not skip_sam', so a --skip-oig run blanked all exclusion
  flags). Also write 'Y' (matching the original list builder) not '1'.
- Use 'no_reval_flag' (the original vocabulary) instead of 'not_on_list' when an
  NPI drops off the reval list, and clear reval_due_date too.
- Throttle politeness: move time.sleep(0.05) above the early-continue paths so
  EVERY CMS request is spaced, not just the minority that are on the list.
- Guard blank-NPI rows (leave their status untouched instead of mislabeling).
- Master write preserves any columns beyond HEADER (no silent column drop).

Warmup cron (build_healthcare_campaigns_cron.py):
- Fix the daily-slice split: it summed to less than the budget (dropped ~2/day)
  and could OVERSHOOT on tiny totals (each 'other' floored to >=1). Now uses
  divmod for an even remainder and reclaims rounding onto the lead, so
  sum(per_seg) == total_slice exactly for every input (verified 0,1,2,7,100,300).

Templates: the non-revalidation emails rendered {{ .Subscriber.Attribs.detail }}
(a reval due date) under a 'Practice'/'Status'/'Record' label -- a wrong/
confusing personalization on a live send (esp. OIG, selector 'any'). All four
now show the practice name; 'detail' is retired from rendering (revalidation
uses reval_due_date/days_overdue directly).
2026-06-08 03:23:47 -05:00
justin
167c4a3847 infra/cron: multi-segment hc warmup + weekly data-refresh cron
Tracks the deployed cron.d files in the repo:
- pw-hc-campaign: updated comment to reflect the now multi-segment warmup
  (revalidation + OIG + NPPES + reactivation + bundle); command unchanged.
- pw-hc-refresh (NEW): Mon 06:00 Central weekly data refresh, ~1h before the
  07:00 weekday send, so every send uses fresh CMS/OIG status.
2026-06-08 03:15:47 -05:00
justin
85dc3d5c3b hc refresh: propagate fresh status into the channel CSVs the cron reads
The channel CSVs (hc_warmup_nongoogle/google/week1_verified) are email-keyed
subsets of the master with extra deliverability columns (verify_ok/verify_reason).
The refresh now writes the fresh status fields (reval_due_date, days_overdue,
reval_status, leie_excluded, optout_ending, name/specialty/state) back into each,
preserving the extra columns and row membership, so a single weekly run updates
everything the campaign cron consumes -- not just the master.
2026-06-08 03:13:00 -05:00
justin
4f455475c0 hc: weekly data-refresh pipeline + multi-segment warmup cron
Two gaps closed:

1. hc_data_refresh.py (NEW): weekly source-data refresh. Re-checks every
   emailable NPI against the LIVE government sources so sends never go stale:
   - CMS Revalidation Due Date List (data.cms.gov per-NPI API; handles both ISO
     and US date formats, normalizes to MM/DD/YYYY).
   - OIG LEIE full CSV download (the NPI-bearing exclusion source).
   - SAM.gov v4 exclusions (key in .secrets/sam-api-key) -- OFF by default since
     SAM exclusions rarely carry an NPI and the full set is ~167k records; it's
     opt-in via --sam-pages. SAM's real value is the live per-name screening
     service, not a bulk NPI join.
   Writes the master CSV atomically (temp+rename). A provider who has since
   revalidated flips overdue->upcoming/not_on_list, so we stop nagging them.

2. build_healthcare_campaigns_cron.py: was revalidation-only (one hardcoded
   list/campaign/CSV/template). Now multi-segment: imports SEGMENTS from the
   single-source-of-truth registry, warms ALL five programs in parallel, each
   with its own list, dated campaign, and per-segment import-state file (so
   dedup is per-segment). A  per segment maps master-CSV rows to the
   right program (reval_overdue / reval_upcoming / leie_or_deactivated /
   optout_ending / any). Daily ramp slice is split across segments (revalidation
   leads at 50%, rest share the remainder) so every program collects engagement
   data while the IPs warm. Back-compat: seeds revalidation import-state from the
   legacy hc_imported_emails.txt once.
2026-06-08 03:06:29 -05:00
justin
42c6b9607f home: add healthcare to meta description + hero vertical lists
The homepage meta description (description/og/twitter) and hero paragraph listed
'trucking, telecom, data privacy, TCPA, and corporate' but omitted healthcare,
even though healthcare is now a first-class vertical (5-areas strip, full
service pages, active email program). Added healthcare to both vertical lists
and a healthcare-specifics clause (Medicare revalidation, NPI/NPPES, enrollment,
OIG/SAM) mirroring the existing DOT/FMCSA clause.
2026-06-08 02:58:49 -05:00
justin
0b0ff9d311 hc campaigns: make the HTML templates the single source of truth
build_healthcare_campaigns.py had a divergent inline HTML generator (old teal-
header + yellow issue-box layout, missing the official-record card and the per-
segment verify-it-yourself blocks) that nobody called -- the live cron reads the
hand-tuned data/hc_campaigns/hc_*.html files directly. Removed the dead
generator + cmd_render(); render() now READS the canonical template file so the
files can't drift from a parallel generator. SEGMENTS is now a metadata registry
(subject, template, cta_path, price, list_name, campaign_name, selector) that the
multi-segment cron will consume. Verified --list and that send-test still reads
the real bodies.
2026-06-08 02:57:49 -05:00
justin
aa195e6c18 hc emails: add source-grounded 'verify it yourself' trust blocks to all programs
The revalidation email had a 'check the official CMS record yourself' proof
block (the strongest trust signal), but the other four healthcare programs had
none -- just the generic SOC2/guarantee footer. Each now points the provider to
the actual public government source that backs its claim:

- NPPES outdated -> 'Look up my NPI on NPPES' (npiregistry.cms.hhs.gov, fully
  public; shows the exact address/taxonomy/contact payers and CMS see).
- OIG screening -> 'Search OIG LEIE / Search SAM.gov' (exclusions.oig.hhs.gov +
  sam.gov), with an honest note that a one-time self-search isn't the documented
  recurring screening CMS expects.
- Reactivation (deactivated) -> deactivation isn't a single public dataset, so
  this is framed honestly: most deactivations follow a lapsed revalidation
  (public CMS Revalidation list) and show in NPPES; also 'are your claims
  paying?' as a self-check. No fabricated 'deactivated record' card.
- Compliance bundle -> all four official sources (CMS Revalidation, NPPES, OIG
  LEIE, SAM.gov) it monitors year-round.

All four government URLs verified reachable (200/302). No paper/mail filing
mechanics revealed; CMS/NPPES/OIG/SAM public names are fine and signal
expertise.
2026-06-08 02:43:02 -05:00
justin
19785629d1 checkout: add shared trust band (guarantee + security) to order flow
High-friction conversion points (payment step, review step, order intro) had
almost no trust reinforcement at the moment of payment. Adds a shared,
regulator-agnostic CheckoutTrustBand component used across all four verticals:

- Payment step: 'full' variant -- money-back-if-we-fail-to-file guarantee +
  256-bit TLS / Stripe / SOC 2 / PCI / fixed-price security badges + the right
  'not affiliated with <agency>' disclaimer for the vertical.
- Review step: 'compact' variant -- guarantee + disclaimer (no badges).
- Order intro (VerticalOrderHeader, shared by all 49 order pages): thin green
  'Secure checkout / Fixed price / Money-back guarantee' bar.

Guarantee copy is a real promise (full refund if we cannot file), worded so it
never overpromises a regulatory outcome (agency approval is not ours to give).

Vertical is inferred from the intake-step list via slugVertical() (single
source of truth, no hand-maintained slug table), with an explicit corporate
slug set since corporate services share the generic 'entity' step. Note: the
'dc_agent' step is the telecom D.C. process-agent designation, not corporate.

Also fixes two pre-existing mislabeled order-page headers surfaced by an
exhaustive header-vs-disclaimer audit: rmd-filing (Robocall Mitigation DB) and
new-carrier-bundle (VoIP carrier onboarding) are telecom, not healthcare/
trucking.
2026-06-08 02:36:38 -05:00
justin
ada9e01321 checkout(healthcare): collapse + reposition surrogate how-to when not granted
The order confirmation asked providers to optionally grant CMS I&A Surrogate
access, then showed the full how-to to everyone high up. Now it respects their
intake answer:
- Granted (surrogate_access=yes): keep the how-to prominent up top so they can
  complete the step they chose.
- Not granted / undecided: top block just confirms 'we're handling it, nothing
  else to do' (no nagging); the speed-it-up how-to moves lower and is collapsed
  behind a 'Show me how' <details> expander (still available, just out of the way).
2026-06-08 00:36:41 -05:00
justin
978b36ad92 home: replace TCPA with Healthcare in the five-areas strip, Healthcare first
Per priority shift to the healthcare vertical: drop the TCPA card from the
'Five areas of compliance expertise' strip and add Healthcare as the first
card (teal accent, links to /services/healthcare). Still five cards.
2026-06-08 00:27:56 -05:00
justin
25cf23dded feat(orders): reduce friction & chargebacks across order flow
1. Email: add a 'Problem with your order? We're here to help' support band to
   the shared htmlEmail() footer, so EVERY transactional email (confirmation,
   portal link, receipts) has a prominent 'Get help with your order' button
   linking to /contact. Less silent frustration -> fewer chargebacks.

2. NPI order form: entering a 10-digit NPI now auto-fills provider name, practice
   state, and specialty from the live NPPES lookup (same API as the free
   compliance-check tool), with a 'Found: <name>' confirmation. Only fills empty
   fields so it never clobbers edits.

3. NPI order form: read ?npi= from the URL so the email 'Start my revalidation'
   click lands with the NPI prefilled and the rest auto-filled (was being
   ignored entirely before).

4. Support FAB: add the floating help button + panel to 27 static public pages
   that were missing it (order, portal, trucking, survey, upload pages), so help
   is one click away everywhere.
2026-06-08 00:24:17 -05:00
justin
80e07aecbb email(healthcare): brighter barber stripe + center the official-record header
Brighten the stripe to #2563eb/#ef4444, lighten the text band to 58% so the
colors show through more, and center the official-record header text.
2026-06-08 00:14:20 -05:00
justin
5edc6151cf email(healthcare): restore CMS-855/PECOS terms + style the service-fee row as a card
- Keep public CMS terms (CMS-855, PECOS) in client copy; the rule is about not
  exposing the paper/mail filing mechanic, not public form/system names.
- Wrap the bare 'service fee / $599' row in a bordered card with a prominent
  green price so it no longer floats awkwardly under the verify box.
2026-06-08 00:12:09 -05:00
justin
6be066ccc9 email(healthcare): brighten official-record stripe to blue/red barber pole
Swap the dark slate stripe for a bright blue/white/red diagonal barber-pole
pattern; keep the header text readable via a translucent dark band behind it.
2026-06-08 00:11:04 -05:00
justin
22d7c72ab3 email(healthcare): restore the no-login/no-2FA convenience blurb
Adds a teal 'No logins, no 2FA codes, no headaches' card (between the fee and
the CTA): we do the whole revalidation, you never share a password or chase a
two-factor code, just a one-minute e-signature. Mirrors the npi-revalidation
service page's convenience pitch, kept clean of form numbers.
2026-06-08 00:09:50 -05:00
justin
5b78141997 email(healthcare): add diagonal stripe pattern behind the official-record header
Adds a subtle barber-shop diagonal stripe (repeating-linear-gradient over the
solid slate bg) to the CMS official-record card header. Layered over the solid
#1e293b so clients that ignore the gradient (Outlook) still get the dark bar.
2026-06-08 00:07:41 -05:00
justin
022407e807 email(healthcare): add not-affiliated disclaimer to all HC campaigns + scrub mechanics
- Add the 'Performance West is an independent compliance firm, not affiliated
  with CMS or Medicare' footer disclaimer to the 4 remaining HC emails
  (reactivation, NPPES, OIG/SAM, bundle), matching the revalidation email.
  OIG email also names the OIG and SAM.gov it references.
- Scrub client-facing mechanics: drop the CMS-855 form number from the
  reactivation CTA and the PECOS system name from the revalidation CTA; clean
  the same out of source comments.
2026-06-08 00:06:29 -05:00
justin
a91d7c8513 email(healthcare): move 'not affiliated with CMS' disclaimer to footer
Keeps the official-record card clean (just the data.cms.gov source line) and
puts the not-affiliated disclaimer in the standard footer alongside the
company line.
2026-06-08 00:03:24 -05:00
justin
483f185861 feat(healthcare): prove revalidation is real via official CMS data + self-verify
Skepticism ("is this even real?") is the top objection. The data IS accurate
(verified our subscribers' NPIs match the official CMS Revalidation Due Date List
exactly), so this is a credibility-presentation fix:

1. Email: replace the plain detail row with an "Official record - CMS Medicare
   Revalidation Due Date List" card (NPI, legal name, due date, days overdue)
   plus a "Verify on CMS.gov" button. Clearly labeled as our presentation of
   public CMS data, not a CMS screenshot (no impersonation).
2. API: npi/lookup now pulls the revalidation due date LIVE from the public CMS
   dataset (data.cms.gov) instead of the empty local table, and returns a
   revalidation{ due_date, source, cms_legal_name, verify_url } proof object.
3. Tool: /tools/npi-compliance-check shows a live "official record" card with a
   self-verify link when CMS returns a due date.

Builder now stores reval_due_date/days_overdue as separate attribs for the card
(existing 194 subscribers backfilled from their detail string).
2026-06-07 23:54:01 -05:00