justin/new-site
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
new-site/docs/plans/Plan.md

16 KiB
Raw Blame History

Plan

Status: COMPLETE (all phases implemented + validated 2026-06-02)

Phase What Status
1 Hazmat/PHMSA handler + product (hazmat-phmsa, $149) ✅
1.5 Order-form bundle/mutual-exclusion enforcement (server-side) ✅
2 State-trucking intake form (slug-gated) + REQUIRED_FIELDS + admin-todo fields ✅
2.5 BOC-3 authority-aware (active/pending/revoked/none) + upsell-approve follow-ups ✅
2.6 Pipeline activation gating (require_active edges, FMCSA poll, waiting_on_activation) ✅
3 State emissions (non-CA) product state-emissions ($199) ✅
4 Order landing pages for all state/hazmat/emissions slugs (48 pages build) ✅
Adv Prerequisite-aware DOT lookup + state recommendations ✅
5 Campaign builder deficiency segments + LP routing + --list-segments ✅
Val Consistency checker (24/24) + campaign segment test (synthetic) ✅

Remaining ops (not code): create the 6 Listmonk source-campaign templates and set their CAMPAIGN_*_ID envs (CAMPAIGN_FOR_HIRE_ID, CAMPAIGN_IRP_IFTA_ID, CAMPAIGN_INTRASTATE_ID, CAMPAIGN_WEIGHT_TAX_ID, CAMPAIGN_EMISSIONS_ID, CAMPAIGN_HAZMAT_ID). Until set, those segments are reported by --list-segments but skipped by the scheduled run. Optional follow-up: a client-side incompatibility UX hint in the order form (server already enforces).

Goal

Make every trucker deficiency type we flag actually fulfillable: each flagged deficiency must have (1) a service handler that can complete the work, (2) a checkout/order path, and (3) an intake form that collects all information that handler needs before the job runs. Only after fulfillment is complete and verified do we extend the campaign builder to email those deficiency segments.

Scope / affected areas

Deficiency flags in play (live counts): for_hire (19,811), interstate_irp_ifta (19,761), intrastate_authority (14,081), state_emissions (12,424), state_weight_tax (6,289), state_permit (3,418), mcs150_overdue (4,539), hazmat (514), zero_fleet (134).

Files / systems:

  • scripts/workers/services/__init__.py — handler registry
  • scripts/workers/services/state_trucking.py — IRP/IFTA/weight-tax/permit/intrastate handler (admin-todo only today)
  • scripts/workers/services/mcs150_update.py — MCS-150 + reactivation (real FMCSA filing)
  • scripts/workers/services/boc3_filing.py — BOC-3 (Playwright)
  • NEW scripts/workers/services/hazmat_phmsa.py — only fully-missing fulfillment path
  • site/src/lib/intake_manifest.ts — per-service intake steps + pricing/meta
  • site/src/components/intake/steps/DOTIntakeStep.astro — unified DOT intake (no state-trucking sections today)
  • NEW site/src/components/intake/steps/StateTruckingIntakeStep.astro — state filing fields
  • api/src/routes/compliance-orders.ts — COMPLIANCE_SERVICES (products) + REQUIRED_FIELDS (validation; none defined for any DOT/state-trucking slug today)
  • api/src/routes/checkout.ts — slug allowlist
  • api/src/routes/dot-lookup.ts — recommended-services mapping
  • NEW site/src/pages/order/*.astro — landing pages for state-trucking + hazmat (none exist)
  • scripts/build_trucking_campaigns.py — campaign builder (extend last)

Key findings (grounding)

  1. Fulfillment handlers already exist for ~98% of flags. The single fully missing path is hazmat / PHMSA registration (no handler, product, page).
  2. State-trucking intake collects nothing. All 13 state slugs map to ["review"] in intake_manifest.ts with a comment "info collected at checkout" — but checkout collects no per-filing fields. So IRP has no vehicle/weight/jurisdiction data, IFTA has no fleet/base-state, NY HUT has no vehicle list, intrastate-authority has no insurance/authority-type, etc. The handler's admin todo is therefore incomplete and an admin must chase the customer for data.
  3. REQUIRED_FIELDS has zero entries for any DOT or state-trucking slug, so the API performs no intake validation for these orders.
  4. No dedicated order landing pages for IRP/IFTA/state-tax/permit/intrastate or hazmat. Checkout works by slug, but campaign emails have no clean LP to drive conversions.
  5. State emissions flags (NY/CO/MD/NJ/MA/etc., 12,424) only map to a product for CA (CARB via ca-mcp-carb). Non-CA emissions have no product — decide whether to build or fold into existing state DOT/permit service.

Approach (concrete ordered steps)

Phase 1 — Close the hazmat fulfillment gap (only fully-missing path)

  1. Add HazmatPHMSAHandler in scripts/workers/services/hazmat_phmsa.py (admin-assisted, mirrors state_trucking.py: creates admin_todo with PHMSA registration steps, sends status email). Slug hazmat-phmsa.
  2. Register "hazmat-phmsa": HazmatPHMSAHandler in services/__init__.py.
  3. Add product to COMPLIANCE_SERVICES, meta to intake_manifest.ts, slug to checkout allowlist.

Phase 1.5 — Order-form incompatibility enforcement (bundles vs components, dupes)

Today the batch endpoint (compliance-orders.ts POST /batch) only dedupes and hard-codes one case (drop standalone fcc-499a when fcc-499a-499q present). There is no general rule preventing a customer from selecting a bundle plus its own components (e.g. dot-full-compliance + mcs150-update + boc3-filing, or state-trucking-bundle + irp-registration), or other incompatible combos. This double-charges and creates duplicate filings.

Build a single source of truth for service relationships and enforce it:

  • Add BUNDLE_COMPONENTS map (bundle slug -> component slugs) covering dot-full-compliance, state-trucking-bundle, new-carrier-bundle, fcc-full-compliance, plus the DB service_bundles rows.
  • Add INCOMPATIBLE_PAIRS / mutually-exclusive groups (e.g. usdot-reactivation vs carrier-closeout; fcc-499a vs fcc-499a-499q; emergency-temp-authority vs mc-authority where applicable).
  • Server-side (authoritative) in /batch: when a bundle is present, drop any of its components from the cart (keep the bundle), reject hard-incompatible pairs with a clear error, and keep dedup. Generalize the existing 499a special-case into this map.
  • Client-side (order form / cart UI): disable/grey out a component when its parent bundle is selected (and vice-versa: selecting all components suggests the bundle), and prevent selecting mutually-exclusive options, with an inline explanation. Mirror the server map so UX matches enforcement.

Phase 2 — Make state-trucking intake actually collect required data

  1. Build StateTruckingIntakeStep.astro (one shared step, sections shown by slug, mirroring DOTIntakeStep.astro's section-gating pattern):
    • Carrier identity: legal name, DOT#, MC#, base state, contact (prefill from DOT lookup).
    • IRP/IFTA: power units w/ VIN+plate+gross-weight rows, operating jurisdictions, fuel type.
    • Weight-distance (OR/NY/KY/NM/CT): vehicle list + gross weights + (OR) declared combined weight.
    • CA MCP+CARB: fleet engine model-years for CARB Clean Truck Check, CA# if any.
    • Intrastate authority: authority type, insurance carrier + policy#, cargo, BOC-3 on file?
    • State DOT / OSOW: as needed.
  2. Update intake_manifest.ts: replace ["review"] with ["state-trucking", "review"] for the 13 slugs; wire the step into the Wizard.
  3. Add REQUIRED_FIELDS entries in compliance-orders.ts for each state-trucking slug + the DOT slugs (mcs150, ucr, boc3, dot-registration, mc-authority, etc.) so intake is validated server-side. Mirror handler "Intake data needed" docstrings.
  4. Update state_trucking.py handler to read + surface the new intake fields in the admin todo (so admins get vehicle lists, jurisdictions, insurance, etc.).

Phase 2.5 — Make BOC-3 authority-aware (preexisting authority handling)

The BOC-3 attaches to a carrier's operating authority (MC/FF/MX docket). Today boc3_filing.py only reads commonAuthorityStatus/contractAuthorityStatus/ brokerAuthorityStatus to print a status string and otherwise always files a fresh BOC-3. That can be wrong/wasteful depending on the preexisting authority. Add branching off the live FMCSA authority state:

  • Active authority: file/refresh BOC-3 only. (current behavior)
  • Pending authority: file BOC-3 + flag that active insurance must be on file for the authority to activate; create follow-up todo.
  • Revoked/inactive authority: file BOC-3 and flag/upsell reinstatement (OP-1 reinstatement + $80 gov fee, route via usdot-reactivation/mc-authority reinstatement branch). BOC-3 alone does not reinstate.
  • No authority (USDOT only): BOC-3 has nothing to attach to — flag that MC authority (mc-authority) is likely needed first; do not silently file. Implementation: have process()/handle() read full authority status (reuse _check_boc3_status, expanded to return structured fields), select the branch, adjust the admin-todo + customer email, and emit a recommended_followups list the order timeline / upsell can surface. No automatic charge for the follow-up — surface it for the customer/admin to approve.

Phase 2.6 — Prerequisite/activation gating (wait for FMCSA "active", not just "submitted")

There are real FMCSA dependency chains where a downstream filing must wait for an upstream item to be active at the agency, not merely submitted by us. The existing pipeline_orchestrator.py models ordering via wait_for, but a step is treated as satisfied when pipeline_step_status == "completed" (= our handler finished), which is NOT the same as FMCSA showing it active.

True prerequisites to enforce:

  • MC Authority (OP-1) requires an active USDOT.
  • Authority activation requires BOC-3 on file + insurance (BMC-91) on file, then a mandatory ~21-day vetting/protest period before it goes active. BOC-3 + insurance CAN be filed while authority is pending (parallel OK).
  • IRP / IFTA / intrastate-authority / UCR that depend on active authority (or active USDOT) must wait for that activation, not just for the prior order to be submitted.

Implementation:

  • Add an "activation gate" to the orchestrator: for dependency edges flagged require_active: true, poll FMCSA (mobile QC API for USDOT status; L&I for authority/BOC-3/insurance) and only mark the dependency satisfied when the agency reports active. Until then, hold the downstream step in a waiting_on_activation state with a next-poll timestamp.
  • Encode the 21-day authority vetting window as an expected-activation estimate so the timeline/customer comms set correct expectations.
  • Expand PIPELINES edges with require_active flags (USDOT→MC, USDOT→IRP/IFTA/ UCR/intrastate, authority-active→IRP-for-hire/intrastate).
  • Standalone (non-bundle) orders: when a single service is ordered whose prerequisite isn't active yet, surface a clear "blocked until X is active" status + recommended prerequisite order rather than filing prematurely.

Phase 3 — Decide + handle state emissions (non-CA)

  1. Either: (a) add a generic state-emissions service handler+product covering NY/CO/MD/NJ/MA clean-truck/ACT programs, or (b) map those flags to state-dot-registration / advisory-only. (Open question — see below.)

Phase 4 — Order landing pages

  1. Create site/src/pages/order/*.astro for: irp-ifta (combined), state weight taxes (one templated page per state or a single state-aware page), ca-mcp-carb, intrastate-authority, hazmat-phmsa. Reuse existing order-page layout (e.g. boc3-filing.astro, ucr-registration.astro as templates).

Phase 5 — Extend campaign builder (only after 1-4 verified)

  1. Add new campaign segments to build_trucking_campaigns.py keyed off deficiency_flags: for-hire/BOC-3+UCR, IRP/IFTA, intrastate-authority, state weight-tax (per-state), CA MCP/CARB, hazmat. Each links to its LP.
  2. Create the corresponding Listmonk source campaigns (templates) and wire IDs.

Validation (how each part is verified)

  • Handlers: unit-invoke each handler with a synthetic order_data dict in a throwaway script; assert an admin_todos row is created with the expected fields and (dev mode) no live filing fires. Confirm SERVICE_HANDLERS resolves every new slug.
  • Intake completeness (the core ask): write a check that, for every slug, cross-references REQUIRED_FIELDS[slug] against the fields the intake step emits and against the keys the handler reads — fail if a handler-needed field is never collected. This is the verifiable "we collect all needed info" metric.
  • Checkout/products: assert every flagged slug exists in COMPLIANCE_SERVICES, SERVICE_META, checkout allowlist, and SERVICE_HANDLERS (one consistency test).
  • Order pages: build site (astro build / existing build script) and confirm each new /order/<slug> route renders; smoke-load locally.
  • Campaign builder: run build_trucking_campaigns.py --dry-run and assert each new segment selects a nonzero, deduped audience pointing at a valid LP.
  • End-to-end: place a test order per new slug through checkout in dev, verify intake validation blocks missing fields, and the handler produces a complete admin todo.

Open questions / decisions

RESOLVED (per user 2026-06-02):

  1. BOC-3 follow-ups + prerequisite blockers → upsell-approve, advise pre-order where possible. Two layers:
    • Pre-order advisory (preferred): extend the existing DOT Compliance Check tool (site/public/tools/dot-compliance-check/) + dot-lookup recommended services to be prerequisite-aware. When a recommended service needs an active prerequisite (e.g. IRP needs active authority), show the dependency, pre-select the prerequisite, order the cart correctly, and state the ~21-day authority activation expectation — all before payment.
    • Post-order upsell-approve: when a handler discovers a blocker mid-fulfillment (e.g. BOC-3 ordered but authority revoked → needs reinstatement), write a recommended_followups entry on the order and render a one-click, pre-filled "Add this service" card on the order timeline/portal. Customer confirms + pays via the existing checkout. No auto-charge.
    • Standalone checkout guard: if a service is bought directly and its prereq isn't active at FMCSA, warn + offer to add the prereq rather than filing something unfulfillable.
  2. State emissions (non-CA): build a real product. Add a state-emissions service (handler + product + intake + page) covering NY/CO/MD/NJ/MA clean-truck / Advanced Clean Trucks programs (CA stays on ca-mcp-carb).
  3. Pre-order advisory: yes. Covered by 1 above — reuse the compliance-check tool as the advisory surface.

STILL OPEN: 4. Pricing for hazmat-phmsa and state-emissions. DEFAULT: hazmat-phmsa = $149 (admin-assisted PHMSA registration; gov fee $25 placardable-hazmat reg billed at cost). state-emissions = $199 (NY/CO/MD/NJ/MA clean-truck / ACT advisory + registration assist). 5. Vehicle-list intake fidelity. DEFAULT: lightweight up front — collect fleet count + base/operating states

  • fuel type + gross-weight bracket at intake; collect full per-vehicle VIN/plate/weight rows via a post-order follow-up form only when the specific filing requires it (IRP, weight-distance taxes). Keeps conversion high.
  1. Standalone-order prereq guard. DEFAULT: warn + offer to add the prerequisite (pre-selected), allow override ("file anyway"). Never a hard block.
  2. Long-term home for the reviewable plan (docs/plans/ used here; no side_panel tool in this harness).