Main pool is calendar-day 12 but reputation is wrecked (54% delivery, Gmail+
Outlook blocks) -- NOT warmed. MX tagging confirmed the cause: 702k carriers on
Google + 135k on Microsoft = the warmup was hammering exactly the two operators
blocking us. Hold Google/MS/Proofpoint/etc. OUT entirely until day 30 (configurable),
sending only to the long-tail operators (yahoo/comcast/charter/centurylink/etc.)
that don't bot-throttle, so reputation can recover; then re-introduce big
operators gradually via mx_daily_caps. 1.24M/1.49M carriers now tagged.
The real bottleneck was the write, not DNS: each per-domain UPDATE full-scanned
fmcsa_carriers (no functional index on lower(split_part(email,'@',2))). Resolve
all domains concurrently into a list, load a temp table, then ONE join-UPDATE =
single table scan. Tags ~12k domains -> hundreds of thousands of carriers fast.
The serial path (verifier's 8s+6s lifetime per domain) was far too slow for
bulk tagging -- 0 tagged in 3 min on dead domains. Self-contained fast resolver
+ ThreadPoolExecutor(40) resolves thousands of domains in minutes.
The Jun 13-14 Gmail+Outlook block storm came from the main/trucking pool having
NO per-MX throttling (only HC had it) -- it concentrated warmup volume on
Google/Microsoft-Workspace-hosted business domains. Port the HC fix:
- migration 097: fmcsa_carriers.mx_provider column.
- mx_tag_carriers.py: resolve MX once per distinct domain (reuses the verifier's
classifier+cache), tag every carrier with that domain's operator. Bounded per
run, prioritizes unsent verified carriers.
- build_trucking_campaigns: during warmup (day<=6) EXCLUDE tagged Google/MS/
Proofpoint/etc. carriers in fetch_carriers; per-MX cap in select_sendable_
carriers so known operators never dominate the quota. Untagged carriers pass
(not collapsed onto one bucket) until tagging fills in. mx_daily_caps ramps
with the main warmup day; MAIN_SKIP_BIG_MX=0 disables once warmed.
The HC warmup crons were '* * 1-5' (Mon-Fri), silently skipping weekends -- but a
proper warmup needs CONTINUOUS daily volume for 21 days (mailbox providers reward
consistency; gaps stall reputation). The Jun 14 'HC 0 sent' alert was just a
skipped Sunday, but the weekend skips also broke ramp continuity.
- pw-hc-campaign + pw-hc-nppes: '* * 1-5' -> '* * *' (daily), vendored + applied live.
- Re-aligned the warmup start stamp from calendar-day 9 to send-day 5 so the
volume ramp matches reputation actually built (it had skipped ~4 weekend days,
running the ramp ahead of real history).
- Fixed the stale 'Mon-Fri only' comment in daily_slice().
- Vendored nppes cron now carries the enriched-CSV + 4-segment config.
otc_reincorporation.html: redomesticate-to-Texas hook (Business Court + TXSE +
DE franchise-tax cost) personalized by state_inc_name/company/ticker, cross-sell
RA/foreign-qual/annual-report/franchise-tax, same-day coupon, lead-capture CTA to
/contact?service=reincorporation (high-touch corporate service, not self-serve),
careful 'not a law firm / not legal advice' disclaimers + CAN-SPAM address.
build_otc_campaign.py: emails only verified-email issuers from the harvest+scrape
+verify pipeline, --de-nv-only for the best reincorp fit, reuses trucking sender
plumbing + coupon. Per-deal value is high so capped modestly (400/run default).
scrape_otc_emails.py: fetch each issuer domain's IR/contact pages (gzip,
HTML-only, early-abort, prefer ir@/investor@/info@), extract a contact email.
Skip filing-agent domains (DFN/Donnelley/Broadridge/etc.) that leak into the
extracted domain -- those are not the issuer's site. Same filter added to the
harvester's DOMAIN_NOISE for future runs. Phone (100%) is the fallback channel
for email misses.
Pilot -> production: harvest_otc_issuers.py pulls the OTC/None universe (2,771),
keeps US-domestic (requires BOTH a US state-of-incorporation AND a US-state
business address -- disambiguates the 'DE'=Delaware-vs-Germany trap that leaked
Infineon etc.), and extracts each issuer's website DOMAIN directly from its
latest 10-K/8-K/DEF-14A filing (free, no scrape; ~58-60% find rate in testing).
Outputs cik,name,ticker,state_inc,phone,city,state,zip,domain -- ready for the
domain->email scrape + verify step. Phone is 100% (clean fallback call channel).
Reincorporation-to-TX / RA / foreign-qual / franchise-tax / annual-report fit.
Ran the email-findability pilot we should have run for CLIA. SEC/OTC is viable:
~940 US-domestic OTC issuers, domain recoverable from the 10-K/8-K filing itself
at ~100% (free, no scrape), email via site scrape ~25-50%, phone 100%. High
per-deal value (reincorporation/RA/foreign-qual/franchise tax). Documented the
build plan.
The OIG-screening + NPPES-update segments were effectively limited to ~1,437
providers because the warmup 'any' selector excluded not-on-reval-list rows as a
deliverability proxy -- but that excludes almost the ENTIRE institutional list
(org NPIs aren't individual Medicare enrollees). Since we already SMTP-verified
all 63k inboxes, add an 'institutional_verified' selector that trusts our own
verification instead of reval-list presence. Result: OIG + NPPES-update now
address 62,422 (43x more), giving multiple broad offers to test engagement on.
- enrich_institutional_revalidation.py: fast local join of the institutional
list to the CMS Revalidation Due Date List bulk file (revalidation_base.csv)
by NPI -> adds reval_due_date/days_overdue/reval_status. ~1,437 are genuine
Medicare enrollees (197 overdue / 164 due-soon) -> flagship $599 reval pitch.
- npi_reactivation stays on leie_or_deactivated (only REAL deactivations -- no
false 'your NPI is deactivated' claims to active orgs).
Synthesize this session's findings into a ranking of candidate verticals by the
one thing that actually gates cold email: a reliable public bulk source of
deliverable emails. Tier 1 (email native): FCC, FMCSA. Tier 2 (one free hop):
healthcare ORG NPIs (already harvested 63k verified), SEC/OTC corporate. Tier 3
(domain-scrape): FMC OTI, state business entities by trigger. Tier 4 (phone/mail
only, NOT email): CLIA (0.3% match proven), EPA RCRA, individual NPIs.
UCR (Unified Carrier Registration) is annual: opens Oct 1, due Dec 31, mandatory
for interstate carriers (op A, same ~628k pool as IFTA) -> recurring revenue.
- build_ucr_annual_campaign.py: 3-touch business-day cadence (30/12/4 bd before
Dec 31, wider than IFTA since it's once a year), escalating tone, same-day
coupon, 'I already did it' suppression. Reuses build_trucking_campaigns +
IFTA business-day/token helpers (DRY). Per-year cycle reset.
- ucr_annual_reminder.html: deadline + fines/OOS risk + 'we figure out your fee
tier' + coupon + filed link + CAN-SPAM. Source campaign 473.
- migration 096: ucr_reminded_at / ucr_touch_no / ucr_self_filed_at.
- ifta.ts: add GET /api/v1/ucr/filed (shares the HMAC token scheme).
- checkout.ts: order-placement Telegram now shows 'Source: campaign (code X)'
when a discount code is present, so IFTA/UCR/CLIA conversions are visible.
(Confirmed order-alert Telegram already fires from handlePaymentComplete for
all compliance orders via both webhook + session paths.)
The handler is an API route; performancewest.net/api/* proxies to the Astro site
(:4322), only api.performancewest.net serves the API. Build the filed link from
PUBLIC_API_URL (default https://api.performancewest.net). Verified end-to-end:
valid token -> 200 confirmation page, invalid -> 403.
- Multi-touch reminders at 10/7/4 BUSINESS days before each deadline (weekends
skipped; biz-day math so a touch never lands purely on a weekend with no
runway). Escalating tone soft -> urgent -> last-chance, with the 'almost too
late to DIY, we can still file it' angle so it's a convenience sale, not a free
reminder service. ifta_touch_no tracks the highest touch sent so each touch
hits only carriers below that level; never repeats a touch.
- 'I already filed it' one-click link: HMAC-tokenized GET /api/v1/ifta/filed
(token matches between Python builder and api/src/routes/ifta.ts -- verified
identical output), records ifta_self_filed_at, friendly confirmation page,
stops further touches this cycle + gives DIY-vs-prospect signal. Builder
excludes self-filed carriers.
- migration 094 (ifta_touch_no) + 095 (ifta_self_filed_at); cycle reset clears
both each new quarter. Verified: biz-day touch schedule, token cross-match.
add_subscriber only attached existing subscribers to the new list without
updating attribs, so a carrier emailed in a prior campaign kept STALE attribs --
meaning the daily coupon code and IFTA merge fields (ifta_due_date, ifta_quarter,
lp_link) rendered BLANK for any repeat recipient. Now merge+PUT the fresh attribs
on the existing subscriber before attaching. Affects all trucking campaigns, not
just IFTA. Verified: IFTA preview now persists ifta_quarter/ifta_due_date/lp_link.
IFTA returns are due on fixed dates (Apr30/Jul31/Oct31/Jan31) and every
interstate carrier (op code A, ~628k sendable) files 4x/year forever -- pure
recurring revenue, no per-carrier deadline data needed.
- build_ifta_quarterly_campaign.py: self-gates to the reminder window (~21d
before each deadline), selects interstate carriers, mints the same-day coupon,
builds+schedules the campaign reusing build_trucking_campaigns plumbing (DRY:
one source of truth for sending/suppression/coupon). Per-quarter cycle reset
(ifta_reminder_cycle marker) so each quarter re-reminds the full pool; marks
ifta_reminded_at to avoid double-sends within a cycle.
- ifta_quarterly_reminder.html: deadline + penalties + 'we do the math' + coupon
+ CAN-SPAM. Listmonk source campaign id 469.
- migration 094: fmcsa_carriers.ifta_reminded_at column + partial index.
Verified: deadline/window logic correct, imports reuse tc helpers, migration
applied on prod.
Capture the full decision trail and chosen approach for making CLIA labs
emailable: why NPI->NPPES (0.3%) and DirectTrust failed, datacenter-IP search
blocking, the $99 B2B-list -> email-domain -> scrape-current-email -> verify
pipeline (durable domain even when the mailbox is stale), hard rules protecting
the warming mail pool, gzip/HTML-only bandwidth optimization, residential proxy
options, the sample-validation gate before committing, what's already built
(harvest, service, order page, email template), and the postcard fallback.
Set up the CLIA recurring-renewal vein (every clinical lab renews its CLIA cert
on a 2-year cycle; CMS publishes the full lab file with expiration dates):
- service-catalog: clia-renewal ($449, discountable) + order page (npi-intake
steps) + intake manifest entry.
- harvest_clia_renewals.py: parse the CMS Provider-of-Services CLIA file, filter
to labs expiring within a window (default 120d), emit name/address/phone/expiry.
676k labs -> ~70k expiring in the next ~4 months.
- match_clia_to_nppes.py: CLIA has no NPI/email, so bridge to emailable NPPES
orgs by normalized name+zip to recover NPI+email (yield TBD; labs that do not
match still have clean phone+postal for a phone/mail channel).
- hc_clia_renewal.html: warm turnover-safety-net email with the striped official-
record card (CLIA #, expiry, status), verify-on-CMS-QCOR, founder guarantee
card, full CAN-SPAM address.
- Upgrade the plain teal record banner to the authoritative barber-pole 'Official
record' banner in the personal/turnover/overdue-personal templates (the switch
to personal templates had dropped the striped look from live revalidation sends).
- nppes_outdated: replace plain info table with the striped 'Official record -
NPPES NPI Registry' card (status honestly labeled as our compliance flag).
- Wire revalidation_overdue -> hc_revalidation_overdue_personal.html with a direct
past-due subject ('Your Medicare revalidation is past due - let's get it filed')
and PAST DUE status + days-overdue in the record card; due_soon stays warm.
- Striped card now on all 7 templates that show a real record; oig_screening and
compliance_bundle correctly omit it (no specific record to display).
Match the authoritative 'official record' look of the revalidation emails on the
deactivated/NPI-reactivation template: striped banner + structured NPPES record
card. Kept it accurate -- NPI/name are labeled NPPES (the real public source);
the deactivation status is labeled as our compliance flag (not an NPPES field),
since deactivation is not a single public dataset, with a 'confirm via official
sources' footnote.
Per your call: add the same personal founder card (headshot linked to /about,
service-neutral satisfaction-guarantee quote, signature, title) to the four
remaining HC templates for a consistent trust signal across all healthcare
outreach. Kept the factually-direct subjects where the situation IS past-due/
deactivated (npi_reactivation) -- only the framing softens, not the facts.
All HC templates now use the v2 signature.
CAN-SPAM requires a valid physical postal address in every commercial email.
All 8 HC campaign templates and the FCC campaign_template.html only had
'Cheyenne, WY' (no street) -- added the full
'525 Randall Ave Ste 100-1195, Cheyenne, WY 82001' to match the (already-correct)
trucking templates. Audited every Listmonk source/sent campaign + wrapper
templates: all active sends carry address + unsubscribe.
Also: revalidation segments now use hc_revalidation_personal.html with subject
'Let's make sure your Medicare revalidation is handled in time'.
Adds hc_revalidation_personal.html: the turnover safety-net email plus a
personal guarantee card from Justin Hannah -- round headshot (links to /about so
readers can confirm a real person stands behind it), an italic satisfaction-
guarantee quote ('I will personally make it right... that is my promise'), a
rendered 'Justin Hannah' signature (Dancing Script, SIL OFL), and his title
(Founder & Principal Consultant). Signature image generated via PIL and added to
site/public/images/justin-signature.png. Test sent to justin@.
New HC template (hc_revalidation_turnover.html) that gets the warm, 'someone who
has our back' feel WITHOUT falsely claiming a prior business relationship (which
would be a deceptive practice under FTC/UDAP and is especially risky with
compliance-minded healthcare admins). Instead it leans on:
- the real staff-turnover insight ('whoever last handled this may have moved on')
- genuine relevance (their actual NPI + CMS revalidation due date)
- the safety-net positioning ('we keep an eye on this so it does not become your
problem' / 'we will make sure it gets done right no matter who handled it')
- true social proof (trusted by providers nationwide) + verify-on-CMS.gov
Every claim is true and defensible. Test sent to justin@.
The sales we got came at $79 + a 24hr coupon; cutting MCS-150 to $39 flat
removed urgency and conversions did NOT improve (a permanent low price sets a
new anchor and lets people defer). Restore the higher anchor and let an
expiring discount create the now-or-lose-it decision.
- Restore MCS-150 anchor $39 -> $79 (catalog single source + regenerated).
- build_trucking_campaigns.py: mint ONE random 5-letter coupon per send-day
(40% off, valid through 23:59:59 ET that day) into the existing discount_codes
table; inject coupon_code/pct/expires + a ?code= LP link into every email.
Idempotent per day; service-fee-only scope (gov/pass-through fees never cut).
- Listmonk MCS-150 (186) + Inactive USDOT (188) templates: lead with the
struck-through anchor + sale price + code + 'expires tonight', and point the
primary CTA at the order page (with code) instead of the 'free check' tool.
- OrderPriceBanner: validates ?code= via /api/v1/discount and shows
was/now + expiry; Wizard forwards the code to order creation.
- Verified: code gen, expiry math, scope enforcement, discount API
(40% off $79 = $47.40), site+api builds clean.
Day 9 (2026-06-13) alert: main pool 54% delivery, 202 Gmail spam-blocks
(550-5.7.1 'Gmail has detected') on warming IPs .94-.98. The 4k/day (400/h)
ramp was too aggressive AND the trucking pool lacks the per-MX throttling the HC
pool got -- Google-Workspace-hosted business domains (weberfarms.net, uatruck.com,
etc.) concentrated and Gmail blocked us. Held at 200/h (~2k/day) through day 20 to
recover, then slow step to 300/h. Applied live (cap already set to 200/h).
Adds /etc/cron.d/pw-hc-nppes (weekdays 07:30) that imports the verified NPPES
institutional general-compliance base into the OIG screening segment, throttled
per MX operator. Separate from the 07:00 reval-segment run so the two pipelines
stay independent. Vendored the cron file under infra/cron/.
Reputation is tracked per receiving mail operator, not per recipient domain, so
the daily warmup slice is now distributed across MX operators with per-operator
daily caps (ramping with the warmup day): Microsoft/Google/Proofpoint/etc. capped
individually, long-tail operators each get a generous default. This lets total
daily volume be much higher than a flat cap without hammering any single system.
mx_throttled() respects the mx_provider column the verifier now writes; falls back
to flat slicing if absent.
The saved Goal + Funnel reports used a pre-v3.1.0 parameters shape (urls/dateRange)
that umami 3.1.0's report schema rejects (400 -> 'Something went wrong'). Migrated
both reports in the umami DB to the current funnelReportSchema/goalReportSchema
(steps[], top-level dates, goal type/value). Verified funnel route now passes schema
validation. TODO updated/closed.
Reputation is tracked per receiving mail operator (Microsoft 365, Google
Workspace, Proofpoint, etc.), not per recipient domain -- so warmup can safely
send far more total volume if it's spread across many MX operators and throttled
per-operator. The verifier now classifies each domain's (already-cached) MX into
a provider label and writes an mx_provider column, so the warmup importer can
cap sends per operator per day. NPPES institutional sample distribution:
Microsoft 33%, Google 11%, Proofpoint ~16%, long tail across dozens of others.
Extracts cold-mailable provider inboxes from the NPPES endpoint_pfile, dropping
Direct/HISP gateway domains (not deliverable from a normal MTA). From the
June 2026 NPPES file: 88,728 institutional + 19,355 consumer mailable
candidates. Institutional is the warmup-safe slice (consumer webmail is held
back -- aggressive filtering would hurt the warming IP).
add_subscriber only attached an already-existing subscriber to the new list
without updating attribs, so the due-soon template's days_until merge field was
blank for providers already imported by another segment. Now PUT the merged
attribs (existing + this segment's npi/practice/due-date/days_until) before
adding to the list.
The HC warmup pool is supply-constrained (~400 verified providers, all fed by
the same narrow 'revalidation 1-90 days OVERDUE' slice). This adds a mirror-image
proactive segment that targets providers whose Medicare revalidation is UPCOMING
within the next 1-90 days, drawn from the same CMS Revalidation Due Date List --
no new data source needed. 'Handle it before your deadline' is a strong pitch and
roughly doubles the deliverable pool.
- New selector reval_due_soon (status=upcoming, days_until in [HC_DUE_SOON_MIN,
HC_DUE_SOON_MAX] default 1-90).
- New segment revalidation_due_soon reusing the existing /order/npi-revalidation
service ($599) with template hc_revalidation_due_soon.html.
- attribs_for now exposes days_until (positive days to due date).
- Added to ACTIVE_SEGMENTS.
David Sgro (PA OAG complaint BCP-26-05-025816) opted out 2026-04-13; response
emailed to the AG 2026-06-11. To make the suppression bulletproof and keep the
response's representations true:
- Added a legal do-not-contact list (DO_NOT_CONTACT_DOMAINS/_EMAILS) to
_email_exclusions.py with dataspindle.com / dave@dataspindle.com; folded into
BLOCKED_EMAIL_DOMAINS and is_blocked().
- listmonk_import.upsert_subscriber now refuses to import/re-confirm any
suppressed address. This closes the exact gap that re-added him on 2026-04-26:
the duplicate-import branch re-added an existing unsubscribed subscriber to
lists with status=confirmed, overriding the opt-out.
Draft response to PA OAG Bureau of Consumer Protection mediation request.
Core arguments: (1) address came from his own public FCC RMD filing, not
scraping; (2) commercial email is governed by CAN-SPAM (opt-out, permits B2B),
not the fax/telemarketing 'Unsolicited Telecommunication Advertisement Act' he
cites; (3) opt-out honored same day (manual suppression Apr 13), now permanent;
(4) no purchase/harm; (5) the post-opt-out 'emails' he complains of were our
replies to HIS own argumentative emails, not solicitations. Marked DRAFT FOR
ATTORNEY REVIEW with bracketed items to confirm before sending.
The main sending IPs are cleanly warmed: today 3,845 sent at 0.18% bounce,
ZERO deferrals, ZERO ISP rate-limit/blocklist/Spamhaus hits. The script's own
note records these IPs historically sustained ~2,500/day at 68-76% delivery;
collapses only ever came from 17k-29k spikes. So we have ample headroom to
accelerate the trucking ramp safely:
day 7-13: 300/h -> 400/h (~4,000/day) [applied now, day 8]
day 14+: new 500/h (~5,000/day) [hard ceiling, well under ~17k]
Also vendored pw-listmonk-rampcap into the repo (infra/postfix/) -- it
previously lived only on the server at /usr/local/bin. Live script updated and
applied (listmonk cap now 400/h).
Add ezstorehost to trusted_admin in both layers — the nft input set and
the DOCKER-USER iptables chain (Forgejo is containerised; DNAT means the
post-DNAT dport 22 rule applies). Required for static-tenant deploys from
ezStorehost-infra to clone repos over ssh://.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- MCS150UpdateHandler is the catch-all for many admin-assisted DOT services
(UCR, MC authority, audit prep, ETA, name reservation, registered agent,
annual report). It was filling an MCS-150 PDF for ALL of them -- e.g. a UCR
order produced a wrong MCS-150 PDF. Now only MCS150_FORM_SLUGS fill the form;
others get an admin-review todo (PDF 'not generated') for manual handling.
Signature flow was already correctly scoped (UCR is not in DOT_SIGNING).
- handle_process_compliance_service forced the Sales Order workflow_state to
'Review' via set_value, which bypasses ERPNext's allowed transitions and
threw WorkflowPermissionError (Received -> Review) on every run. The Postgres
fulfillment_status is the source of truth; the ERPNext workflow_state is a
cosmetic mirror. Now try the proper apply_workflow action and stay quiet
(debug, not warning) when no valid Review transition exists.
