Adds hc_revalidation_personal.html: the turnover safety-net email plus a
personal guarantee card from Justin Hannah -- round headshot (links to /about so
readers can confirm a real person stands behind it), an italic satisfaction-
guarantee quote ('I will personally make it right... that is my promise'), a
rendered 'Justin Hannah' signature (Dancing Script, SIL OFL), and his title
(Founder & Principal Consultant). Signature image generated via PIL and added to
site/public/images/justin-signature.png. Test sent to justin@.
New HC template (hc_revalidation_turnover.html) that gets the warm, 'someone who
has our back' feel WITHOUT falsely claiming a prior business relationship (which
would be a deceptive practice under FTC/UDAP and is especially risky with
compliance-minded healthcare admins). Instead it leans on:
- the real staff-turnover insight ('whoever last handled this may have moved on')
- genuine relevance (their actual NPI + CMS revalidation due date)
- the safety-net positioning ('we keep an eye on this so it does not become your
problem' / 'we will make sure it gets done right no matter who handled it')
- true social proof (trusted by providers nationwide) + verify-on-CMS.gov
Every claim is true and defensible. Test sent to justin@.
The sales we got came at $79 + a 24hr coupon; cutting MCS-150 to $39 flat
removed urgency and conversions did NOT improve (a permanent low price sets a
new anchor and lets people defer). Restore the higher anchor and let an
expiring discount create the now-or-lose-it decision.
- Restore MCS-150 anchor $39 -> $79 (catalog single source + regenerated).
- build_trucking_campaigns.py: mint ONE random 5-letter coupon per send-day
(40% off, valid through 23:59:59 ET that day) into the existing discount_codes
table; inject coupon_code/pct/expires + a ?code= LP link into every email.
Idempotent per day; service-fee-only scope (gov/pass-through fees never cut).
- Listmonk MCS-150 (186) + Inactive USDOT (188) templates: lead with the
struck-through anchor + sale price + code + 'expires tonight', and point the
primary CTA at the order page (with code) instead of the 'free check' tool.
- OrderPriceBanner: validates ?code= via /api/v1/discount and shows
was/now + expiry; Wizard forwards the code to order creation.
- Verified: code gen, expiry math, scope enforcement, discount API
(40% off $79 = $47.40), site+api builds clean.
Day 9 (2026-06-13) alert: main pool 54% delivery, 202 Gmail spam-blocks
(550-5.7.1 'Gmail has detected') on warming IPs .94-.98. The 4k/day (400/h)
ramp was too aggressive AND the trucking pool lacks the per-MX throttling the HC
pool got -- Google-Workspace-hosted business domains (weberfarms.net, uatruck.com,
etc.) concentrated and Gmail blocked us. Held at 200/h (~2k/day) through day 20 to
recover, then slow step to 300/h. Applied live (cap already set to 200/h).
Adds /etc/cron.d/pw-hc-nppes (weekdays 07:30) that imports the verified NPPES
institutional general-compliance base into the OIG screening segment, throttled
per MX operator. Separate from the 07:00 reval-segment run so the two pipelines
stay independent. Vendored the cron file under infra/cron/.
Reputation is tracked per receiving mail operator, not per recipient domain, so
the daily warmup slice is now distributed across MX operators with per-operator
daily caps (ramping with the warmup day): Microsoft/Google/Proofpoint/etc. capped
individually, long-tail operators each get a generous default. This lets total
daily volume be much higher than a flat cap without hammering any single system.
mx_throttled() respects the mx_provider column the verifier now writes; falls back
to flat slicing if absent.
The saved Goal + Funnel reports used a pre-v3.1.0 parameters shape (urls/dateRange)
that umami 3.1.0's report schema rejects (400 -> 'Something went wrong'). Migrated
both reports in the umami DB to the current funnelReportSchema/goalReportSchema
(steps[], top-level dates, goal type/value). Verified funnel route now passes schema
validation. TODO updated/closed.
Reputation is tracked per receiving mail operator (Microsoft 365, Google
Workspace, Proofpoint, etc.), not per recipient domain -- so warmup can safely
send far more total volume if it's spread across many MX operators and throttled
per-operator. The verifier now classifies each domain's (already-cached) MX into
a provider label and writes an mx_provider column, so the warmup importer can
cap sends per operator per day. NPPES institutional sample distribution:
Microsoft 33%, Google 11%, Proofpoint ~16%, long tail across dozens of others.
Extracts cold-mailable provider inboxes from the NPPES endpoint_pfile, dropping
Direct/HISP gateway domains (not deliverable from a normal MTA). From the
June 2026 NPPES file: 88,728 institutional + 19,355 consumer mailable
candidates. Institutional is the warmup-safe slice (consumer webmail is held
back -- aggressive filtering would hurt the warming IP).
add_subscriber only attached an already-existing subscriber to the new list
without updating attribs, so the due-soon template's days_until merge field was
blank for providers already imported by another segment. Now PUT the merged
attribs (existing + this segment's npi/practice/due-date/days_until) before
adding to the list.
The HC warmup pool is supply-constrained (~400 verified providers, all fed by
the same narrow 'revalidation 1-90 days OVERDUE' slice). This adds a mirror-image
proactive segment that targets providers whose Medicare revalidation is UPCOMING
within the next 1-90 days, drawn from the same CMS Revalidation Due Date List --
no new data source needed. 'Handle it before your deadline' is a strong pitch and
roughly doubles the deliverable pool.
- New selector reval_due_soon (status=upcoming, days_until in [HC_DUE_SOON_MIN,
HC_DUE_SOON_MAX] default 1-90).
- New segment revalidation_due_soon reusing the existing /order/npi-revalidation
service ($599) with template hc_revalidation_due_soon.html.
- attribs_for now exposes days_until (positive days to due date).
- Added to ACTIVE_SEGMENTS.
David Sgro (PA OAG complaint BCP-26-05-025816) opted out 2026-04-13; response
emailed to the AG 2026-06-11. To make the suppression bulletproof and keep the
response's representations true:
- Added a legal do-not-contact list (DO_NOT_CONTACT_DOMAINS/_EMAILS) to
_email_exclusions.py with dataspindle.com / dave@dataspindle.com; folded into
BLOCKED_EMAIL_DOMAINS and is_blocked().
- listmonk_import.upsert_subscriber now refuses to import/re-confirm any
suppressed address. This closes the exact gap that re-added him on 2026-04-26:
the duplicate-import branch re-added an existing unsubscribed subscriber to
lists with status=confirmed, overriding the opt-out.
Draft response to PA OAG Bureau of Consumer Protection mediation request.
Core arguments: (1) address came from his own public FCC RMD filing, not
scraping; (2) commercial email is governed by CAN-SPAM (opt-out, permits B2B),
not the fax/telemarketing 'Unsolicited Telecommunication Advertisement Act' he
cites; (3) opt-out honored same day (manual suppression Apr 13), now permanent;
(4) no purchase/harm; (5) the post-opt-out 'emails' he complains of were our
replies to HIS own argumentative emails, not solicitations. Marked DRAFT FOR
ATTORNEY REVIEW with bracketed items to confirm before sending.
The main sending IPs are cleanly warmed: today 3,845 sent at 0.18% bounce,
ZERO deferrals, ZERO ISP rate-limit/blocklist/Spamhaus hits. The script's own
note records these IPs historically sustained ~2,500/day at 68-76% delivery;
collapses only ever came from 17k-29k spikes. So we have ample headroom to
accelerate the trucking ramp safely:
day 7-13: 300/h -> 400/h (~4,000/day) [applied now, day 8]
day 14+: new 500/h (~5,000/day) [hard ceiling, well under ~17k]
Also vendored pw-listmonk-rampcap into the repo (infra/postfix/) -- it
previously lived only on the server at /usr/local/bin. Live script updated and
applied (listmonk cap now 400/h).
Add ezstorehost to trusted_admin in both layers — the nft input set and
the DOCKER-USER iptables chain (Forgejo is containerised; DNAT means the
post-DNAT dport 22 rule applies). Required for static-tenant deploys from
ezStorehost-infra to clone repos over ssh://.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- MCS150UpdateHandler is the catch-all for many admin-assisted DOT services
(UCR, MC authority, audit prep, ETA, name reservation, registered agent,
annual report). It was filling an MCS-150 PDF for ALL of them -- e.g. a UCR
order produced a wrong MCS-150 PDF. Now only MCS150_FORM_SLUGS fill the form;
others get an admin-review todo (PDF 'not generated') for manual handling.
Signature flow was already correctly scoped (UCR is not in DOT_SIGNING).
- handle_process_compliance_service forced the Sales Order workflow_state to
'Review' via set_value, which bypasses ERPNext's allowed transitions and
threw WorkflowPermissionError (Received -> Review) on every run. The Postgres
fulfillment_status is the source of truth; the ERPNext workflow_state is a
cosmetic mirror. Now try the proper apply_workflow action and stay quiet
(debug, not warning) when no valid Review transition exists.
We no longer offer Canadian accountant/accounting setup. Removed all
service-offering content:
- Marketing page (services/telecom/canada-crtc): the 'Set Up Canadian
Accounting (we help)' next-steps card, the '3 hours of complimentary
accounting consultation' deliverable bullet, and the whole 'Accounting
Support' section (assigned accountant, portal chat, $75/hr, 3 complimentary
hours).
- Order page (order/canada-crtc): the '3 hrs Canadian accounting support'
included-feature bullet and the 'Preferred accounting software'
(Xero/QuickBooks) form field + its accounting-hours helper text.
- Fulfillment (canada_crtc.py): dropped the bank-setup email line offering
'3 hours of Canadian accounting consultation'.
Kept factual GST/HST tax advisories and the bank's QuickBooks/Xero
transaction-sync feature (third-party bank capability, not our service).
MCS150UpdateHandler is the catch-all for many admin-assisted DOT services
(UCR, MC authority, audit prep, ETA, name reservation, registered agent,
annual report). My new intake-completeness gate was firing the 'confirm your
MCS-150 details' email for ALL of them -- e.g. a UCR order wrongly emailed the
customer about MCS-150 details. Scope the gate to MCS150_FORM_SLUGS (the
services that actually file an MCS-150: mcs150-update, dot-registration,
usdot-reactivation, dot-full-compliance).
Paul Wilson (Compound Technologies) signed up with synthetic@pipeline.com,
which is a genuine, deliverable EarthLink address (pipeline.com MX ->
earthlink-vadesecure.net; he confirmed receipt by phone). Our code had
hardcoded pipeline.com + the synthetic@ prefix as a 'non-deliverable
FMCSA-census placeholder' and silently suppressed every automated email to
him (checkout provisioning, order-creation validation, intake reminders,
set-password invites). Nothing in the codebase actually generates that
address, so the placeholder rationale was wrong. Removed pipeline.com and the
synthetic@ rule from all four suppression sites; only RFC-reserved
example.com/test.com/invalid remain blocked.
Closes the data gap for orders that bypass the full intake (e.g. the DOT
compliance-remediation pipeline) and for all MCS-150 variants:
- Worker intake-completeness gate (mcs150_update): before filling, check the
customer-required operational fields the FMCSA census cannot supply
(operation classification, cargo, CURRENT annual mileage, email; plus
signer/address for new-registration/reactivation, and states-of-operation
for 150B hazmat). If missing, email the customer a census-pre-filled intake
link and hold the order at fulfillment_status='awaiting_intake' with an admin
todo, instead of fabricating a blank filing. The existing intake PUT endpoint
already re-dispatches the worker on submit, so filing auto-resumes.
- Intake wizard (Wizard.astro): when resuming ?order=CO-xxx for a DOT/MCS order,
seed still-empty fields from the FMCSA census (name/address/fleet/interstate)
so the customer only confirms the operational details.
- /api/v1/dot/census now also returns total_drivers + a normalized
carrier_operation_code for the prefill.
- MCS150Step.astro extended to collect every field the filler needs across all
variants: mailing address, cdl_drivers, primary_vehicle_type,
reason_for_filing, usdot_revoked, cell/fax, hazmat-safety-permit block
(needs_hmsp, operating states, security plan), and intermodal-equipment
provider counts; all prefill from intake_data.
verify_mcs150_variants.py covers 150/150B/150C end-to-end (ALL PASS).
Adds the previously-unmapped fields so every variant fills fully:
- Q25 hazmat C/S/B/NB matrix (HAZMAT_ROW_MAP x HAZMAT_COL_MAP, 156 boxes)
- MCS-150B states-of-operation checkboxes (full name or 2-letter code), HMSP
Hazard/Permit/Security radios, and accident count (32accidentNumber)
- MCS-150C intermodal equipment counts (20owned/leased/serviced) + correct
field renumbering (17dunbrad/18irs/19eMail) + USDOT Button + named-export
Reason/Mailing radios
- Structured fleet via intake['vehicles'] = {vehicle_type: {owned, term_leased,
trip_leased}} across all Q26 vehicle rows; non-CMV count; cell/fax; second
officer
- _set_button now resolves a candidate tuple against each field's actual export
states, so numeric (/0../4) and named (/Yes,/Biennial...) radios both work
verify_mcs150_variants.py exercises all three variants end-to-end: ALL PASS.
certifyBox is the Q29 Passenger Carrier Compliance Certification YES box
(page 3, y=530), not a general perjury checkbox. It was being checked
unconditionally, which wrongly marked freight/property carriers as passenger
carriers. Now only check it when the carrier is a passenger carrier; the
Q31 perjury declaration is made via the signature.
Fixes a batch of missing fields the FMCSA census does not provide and the
filler was mis-mapping:
- Corrected the question->field mapping to match the actual form: Q22 =
COMPANY OPERATIONS (interstate/intrastate, 22xBox), Q23 = OPERATION
CLASSIFICATIONS (for-hire/private/govt, 23xBox). These were swapped, and
the bogus entity-type->23xBox map (no entity-type question exists on this
form revision) was removed.
- Added proper radio-group handling for Reason for Filing (Biennial Update),
Mailing-address (Same as principal vs below), and Q28 USDOT-revoked, with
correct option indices (these are /0../n radios, not /Yes checkboxes; the
old code set them to /Yes and never selected the right option).
- Map interstate/intrastate from the FMCSA census carrierOperationCode, and
populate email/phone/mileage/cargo from intake.
- AcroForm checkbox/radio appearances use a ZapfDingbats glyph that
poppler/Preview fail to render (value set but box looks empty). Now stamp
an explicit X overlay into the page content for every 'on' box so it shows
in every viewer and in the faxed output.
The official MCS-150/150B/150C PDFs ship with 8 (150/150B) or 4 (150C)
FMCSA instruction/example pages before the actual fillable form. We were
generating + faxing/submitting all of them. Trimmed the source templates
down to the FORM pages only:
MCS-150 11 -> 3 pages (289 fields preserved)
MCS-150B 12 -> 4 pages (349 fields preserved)
MCS-150C 6 -> 2 pages (33 fields preserved)
The filler iterates writer.pages (no absolute index) and signature
anchors are derived dynamically via enumerate(reader.pages), so no
page-specific markup needed fixing. Removed one-off diag script.
handle_process_compliance_service assumed handlers return local temp
paths and re-uploaded each to MinIO. The MCS-150 handler uploads itself
and returns the MinIO key, so the re-upload tried to read a nonexistent
local file and logged a 'File not found' error after the order was
already correctly held at the admin gate. Now we skip files that don't
exist locally and keep the returned key as-is.
- When intake lacks signer_name, backfill it from the name the client
typed when signing the perjury certification (that name is exactly what
belongs in the form's print/type-name field, certifyName).
- After a client-approved re-dispatch, re-point the signed esign record at
the freshly filled form and re-stamp the signature, so the signed PDF an
admin reviews reflects the current complete form (not a stale earlier
fill). Field layout (and thus signature anchors) is unchanged across
fills, so the recorded anchor coordinates stay valid.
- fill_mcs150 now uses auto_regenerate=True so pypdf writes appearance
streams for every text field. Preview/Chrome ignore /NeedAppearances and
were showing blank widgets over the values; generated /AP streams make
the text render in all viewers.
- New verify_mcs150.py reads each widget's /AP /N appearance stream (the
literal drawn glyphs) to confirm expected values actually render, since
the container has no OCR/raster tooling. Exits non-zero on any miss.
The MCS-150 biennial update re-confirms the carrier's existing FMCSA
record. Previously the PDF filler only had whatever the intake form
collected; rescued/sparse orders (or orders where the carrier's data
lives in FMCSA, not the intake) produced near-empty forms. Now we pull
the carrier census (legal name, address, EIN, fleet counts) from the
FMCSA carrier API and merge it under any customer-provided intake values
(customer edits win), so the form is pre-filled with the carrier's
current registered data. Refactored the FMCSA fetch into a shared
_fetch_fmcsa_carrier helper used by both enrichment and status check.
Customer saw the MCS-150 looking blank / 'data covered by the form fields': the
values were correctly written to the AcroForm /V, but pypdf left the template's
empty /AP appearance streams in place and NeedAppearances was false, so viewers
rendered the blank widget over the value. Setting AcroForm /NeedAppearances=true
makes viewers regenerate appearances from the values. (The missing signature was
a downstream effect of the separate fobj_put MinIO-upload bug, now fixed -- with
no PDF in MinIO the anchor extraction + signature stamping both failed.)
When an MCS-150/USDOT order hits the pre-submission admin-verification gate, the
Telegram FULFILLMENT NEEDED alert now appends a presigned link to the prepared
PDF (via the public minio.performancewest.net endpoint, IP-allowlisted to admin)
so you can review the document straight from the alert before approving. Added
notify_fulfillment_todo(view_url=...) + a _presigned_view_url helper (public
endpoint + explicit region to avoid the region-probe that 403s from the worker).
The MCS-150/USDOT PDF was generated fine but the MinIO upload threw 'Minio object
has no attribute fobj_put' (wrong method name + signature), so the prepared filing
PDF was never persisted -- nothing for an admin to review at the verification gate,
and the esign-completed re-dispatch failed with 'File not found'. Use the correct
minio fput_object(bucket, key, file_path). Affects every MCS-150/USDOT filing.
ensureComplianceSalesOrder skipped the FMCSA-census placeholder email, but a real
paying customer (Paul Wilson) genuinely uses synthetic@pipeline.com, so his SO
never got created/regenerated. A Sales Order is internal bookkeeping, not an
outbound email, so the placeholder skip is unnecessary here (the email/portal
guard in ensureCompliancePortalUser still protects actual sends).
Mitchell's batch CB-95BA6C90: Stripe correctly charged $450.88 ($437.75 net +
$13.13 surcharge), but the DB + Telegram showed $503.40 with a $65.65 surcharge.
Two bugs:
1) On Stripe session creation, the per-row surcharge UPDATE wrote the FULL batch
surcharge ($13.13) to EVERY row via WHERE batch_id, so anything summing the
per-row field (the Telegram order notification) over-counted Nx (5 x $13.13 =
$65.65). Now the single surcharge is split across the rows so they sum to the
true total. Stripe was always charged correctly (one surcharge line item).
2) ensureComplianceSalesOrder built the ERPNext SO from full line-item prices but
applied NO discount, so the SO grand total over-stated what the customer paid.
Now applies the promo/bundle discount via apply_discount_on=Grand Total +
discount_amount on both the primary and fallback SO create.
Per request: after the customer signs but BEFORE we submit to the government, hold
the order for a human to verify the prepared filing is correct.
- MCS-150 handler (mcs150-update + usdot-reactivation): new admin-verification gate
after the signature gate -- if not admin_approved, set fulfillment_status=
'ready_to_file', create a HIGH-priority 'VERIFY before filing' admin todo, and
STOP (no FMCSA submission). job_server injects admin_approved from the dispatch
payload (mirrors client_approved).
- New admin endpoint POST /api/v1/admin/compliance-orders/:id/approve-submit
(requireAdmin): verifies status=ready_to_file, re-dispatches the worker with
admin_approved=true to proceed to actual submission.
- Durable submission EVIDENCE: the web/fax submitters only wrote confirmation
screenshots to an ephemeral temp dir. Now _upload_submission_evidence copies the
FMCSA confirmation screenshot + attested PDF + fax_log_id to MinIO under
filings/<slug>/<order>/evidence/ and records the keys on the order, so we keep
proof of every government submission.
(state-trucking + the FCC handlers already gate via admin todos / auto_filing.py;
this brings MCS-150 to parity and adds evidence retention.)
The rescue onboarding emails hardcoded a 60-minute expiry -- way too short for a
paid customer who hasn't engaged yet (they may not check email for hours/days),
so Paul's and Mitchell's links expired before they used them. Onboarding links
now last 7 days (ONBOARDING_TTL_MINUTES); the standard security password-RESET
window bumped 30min -> 2h. Re-issued fresh 7-day links to all 3 affected
customers (none had set a password yet) via reissue-onboarding-links.mjs, cc'd.
Paid Jun 1 for MCS-150 (card), no customers row -> couldn't log in -> no intake ->
filing stuck 'NEEDS MANUAL FILING'. Created his customers row + sent login +
intake email (cc justin). All 3 real paying customers now rescued; the underlying
card/PayPal login bug is fixed so new orders won't hit this.
Tracks the rehab pool (rehab02-04 / .91-.93) delivery + bounce + Spamhaus ZEN
DNSBL status in the daily report and alert body. Alerts only if a rehab IP lands
on a DNSBL or rehab delivery drops <40% with real volume (recipient quality
slipped) -- a recovering IP naturally bounces more so the threshold is lenient.
Day-0 batch saw ~45% bounce because 'no listmonk bounce record' is a weak
liveness signal. Now require the recipient's domain to have >=2 enabled
subscribers (a real org, not a one-off typo'd address), which materially lowers
the dead-mailbox bounce rate on the recovering IPs.
The 3 IPs (mta02-04 / .91-.93) retired after the May 30-31 over-volume blast are
NOT on any DNSBL (Spamhaus/Barracuda/SpamCop/SORBS all clean) and have clean PTRs
+ SPF/DKIM/DMARC -- the damage was provider-internal reputation, which recovers
with slow clean sending. scripts/ip_rehab.py sends a tiny ramping trickle
(10/IP/day -> cap 60) of genuine CAN-SPAM-compliant compliance check-in mail to
clean business-domain, never-bounced recipients via dedicated heavily-throttled
postfix transports rehab02/03/04 (30s/msg, bound to .91/.92/.93). Routing uses an
X-PW-Rehab-IP header + header_checks FILTER to override the transport_maps randmap
warmup rotation (verified: mail routes via rehab transports, status=sent). Daily
cron pw-ip-rehab. After ~2-3 weeks of clean sending the IPs can be reallocated.
