diff --git a/docs/plans/Plan.md b/docs/plans/Plan.md new file mode 100644 index 0000000..0ea37a4 --- /dev/null +++ b/docs/plans/Plan.md @@ -0,0 +1,230 @@ +# Plan + +## Goal + +Make every trucker deficiency type we flag actually *fulfillable*: each flagged +deficiency must have (1) a service handler that can complete the work, (2) a +checkout/order path, and (3) an intake form that collects **all** information +that handler needs before the job runs. Only after fulfillment is complete and +verified do we extend the campaign builder to email those deficiency segments. + +## Scope / affected areas + +Deficiency flags in play (live counts): +for_hire (19,811), interstate_irp_ifta (19,761), intrastate_authority (14,081), +state_emissions (12,424), state_weight_tax (6,289), state_permit (3,418), +mcs150_overdue (4,539), hazmat (514), zero_fleet (134). + +Files / systems: +- `scripts/workers/services/__init__.py` — handler registry +- `scripts/workers/services/state_trucking.py` — IRP/IFTA/weight-tax/permit/intrastate handler (admin-todo only today) +- `scripts/workers/services/mcs150_update.py` — MCS-150 + reactivation (real FMCSA filing) +- `scripts/workers/services/boc3_filing.py` — BOC-3 (Playwright) +- **NEW** `scripts/workers/services/hazmat_phmsa.py` — only fully-missing fulfillment path +- `site/src/lib/intake_manifest.ts` — per-service intake steps + pricing/meta +- `site/src/components/intake/steps/DOTIntakeStep.astro` — unified DOT intake (no state-trucking sections today) +- **NEW** `site/src/components/intake/steps/StateTruckingIntakeStep.astro` — state filing fields +- `api/src/routes/compliance-orders.ts` — `COMPLIANCE_SERVICES` (products) + `REQUIRED_FIELDS` (validation; **none defined for any DOT/state-trucking slug today**) +- `api/src/routes/checkout.ts` — slug allowlist +- `api/src/routes/dot-lookup.ts` — recommended-services mapping +- **NEW** `site/src/pages/order/*.astro` — landing pages for state-trucking + hazmat (none exist) +- `scripts/build_trucking_campaigns.py` — campaign builder (extend last) + +## Key findings (grounding) + +1. **Fulfillment handlers already exist** for ~98% of flags. The single fully + missing path is **hazmat / PHMSA registration** (no handler, product, page). +2. **State-trucking intake collects nothing.** All 13 state slugs map to + `["review"]` in `intake_manifest.ts` with a comment "info collected at + checkout" — but checkout collects no per-filing fields. So IRP has no + vehicle/weight/jurisdiction data, IFTA has no fleet/base-state, NY HUT has no + vehicle list, intrastate-authority has no insurance/authority-type, etc. The + handler's admin todo is therefore incomplete and an admin must chase the + customer for data. +3. **`REQUIRED_FIELDS` has zero entries** for any DOT or state-trucking slug, so + the API performs no intake validation for these orders. +4. **No dedicated order landing pages** for IRP/IFTA/state-tax/permit/intrastate + or hazmat. Checkout works by slug, but campaign emails have no clean LP to + drive conversions. +5. **State emissions** flags (NY/CO/MD/NJ/MA/etc., 12,424) only map to a product + for CA (CARB via `ca-mcp-carb`). Non-CA emissions have no product — decide + whether to build or fold into existing state DOT/permit service. + +## Approach (concrete ordered steps) + +### Phase 1 — Close the hazmat fulfillment gap (only fully-missing path) +1. Add `HazmatPHMSAHandler` in `scripts/workers/services/hazmat_phmsa.py` + (admin-assisted, mirrors `state_trucking.py`: creates admin_todo with PHMSA + registration steps, sends status email). Slug `hazmat-phmsa`. +2. Register `"hazmat-phmsa": HazmatPHMSAHandler` in `services/__init__.py`. +3. Add product to `COMPLIANCE_SERVICES`, meta to `intake_manifest.ts`, slug to + checkout allowlist. + +### Phase 1.5 — Order-form incompatibility enforcement (bundles vs components, dupes) +Today the batch endpoint (`compliance-orders.ts` POST `/batch`) only dedupes and +hard-codes one case (drop standalone `fcc-499a` when `fcc-499a-499q` present). +There is no general rule preventing a customer from selecting a **bundle plus its +own components** (e.g. `dot-full-compliance` + `mcs150-update` + `boc3-filing`, or +`state-trucking-bundle` + `irp-registration`), or other incompatible combos. This +double-charges and creates duplicate filings. + +Build a single source of truth for service relationships and enforce it: +- Add `BUNDLE_COMPONENTS` map (bundle slug -> component slugs) covering + `dot-full-compliance`, `state-trucking-bundle`, `new-carrier-bundle`, + `fcc-full-compliance`, plus the DB `service_bundles` rows. +- Add `INCOMPATIBLE_PAIRS` / mutually-exclusive groups (e.g. `usdot-reactivation` + vs `carrier-closeout`; `fcc-499a` vs `fcc-499a-499q`; emergency-temp-authority + vs mc-authority where applicable). +- Server-side (authoritative) in `/batch`: when a bundle is present, **drop any of + its components** from the cart (keep the bundle), reject hard-incompatible pairs + with a clear error, and keep dedup. Generalize the existing 499a special-case + into this map. +- Client-side (order form / cart UI): disable/grey out a component when its parent + bundle is selected (and vice-versa: selecting all components suggests the bundle), + and prevent selecting mutually-exclusive options, with an inline explanation. + Mirror the server map so UX matches enforcement. + +### Phase 2 — Make state-trucking intake actually collect required data +4. Build `StateTruckingIntakeStep.astro` (one shared step, sections shown by + slug, mirroring `DOTIntakeStep.astro`'s section-gating pattern): + - Carrier identity: legal name, DOT#, MC#, base state, contact (prefill from DOT lookup). + - IRP/IFTA: power units w/ VIN+plate+gross-weight rows, operating jurisdictions, fuel type. + - Weight-distance (OR/NY/KY/NM/CT): vehicle list + gross weights + (OR) declared combined weight. + - CA MCP+CARB: fleet engine model-years for CARB Clean Truck Check, CA# if any. + - Intrastate authority: authority type, insurance carrier + policy#, cargo, BOC-3 on file? + - State DOT / OSOW: as needed. +5. Update `intake_manifest.ts`: replace `["review"]` with + `["state-trucking", "review"]` for the 13 slugs; wire the step into the Wizard. +6. Add `REQUIRED_FIELDS` entries in `compliance-orders.ts` for each state-trucking + slug + the DOT slugs (mcs150, ucr, boc3, dot-registration, mc-authority, etc.) + so intake is validated server-side. Mirror handler "Intake data needed" docstrings. +7. Update `state_trucking.py` handler to read + surface the new intake fields in + the admin todo (so admins get vehicle lists, jurisdictions, insurance, etc.). + +### Phase 2.5 — Make BOC-3 authority-aware (preexisting authority handling) +The BOC-3 attaches to a carrier's operating authority (MC/FF/MX docket). Today +`boc3_filing.py` only reads `commonAuthorityStatus`/`contractAuthorityStatus`/ +`brokerAuthorityStatus` to print a status string and otherwise always files a +fresh BOC-3. That can be wrong/wasteful depending on the preexisting authority. +Add branching off the live FMCSA authority state: +- **Active authority:** file/refresh BOC-3 only. (current behavior) +- **Pending authority:** file BOC-3 + flag that active insurance must be on file + for the authority to activate; create follow-up todo. +- **Revoked/inactive authority:** file BOC-3 **and** flag/upsell reinstatement + (OP-1 reinstatement + $80 gov fee, route via `usdot-reactivation`/`mc-authority` + reinstatement branch). BOC-3 alone does not reinstate. +- **No authority (USDOT only):** BOC-3 has nothing to attach to — flag that MC + authority (`mc-authority`) is likely needed first; do not silently file. +Implementation: have `process()/handle()` read full authority status (reuse +`_check_boc3_status`, expanded to return structured fields), select the branch, +adjust the admin-todo + customer email, and emit a `recommended_followups` list +the order timeline / upsell can surface. No automatic charge for the follow-up — +surface it for the customer/admin to approve. + +### Phase 2.6 — Prerequisite/activation gating (wait for FMCSA "active", not just "submitted") +There are real FMCSA dependency chains where a downstream filing must wait for an +upstream item to be **active at the agency**, not merely submitted by us. The +existing `pipeline_orchestrator.py` models ordering via `wait_for`, but a step is +treated as satisfied when `pipeline_step_status == "completed"` (= our handler +finished), which is NOT the same as FMCSA showing it active. + +True prerequisites to enforce: +- **MC Authority (OP-1)** requires an **active USDOT**. +- **Authority activation** requires **BOC-3 on file + insurance (BMC-91) on file**, + then a **mandatory ~21-day vetting/protest period** before it goes active. + BOC-3 + insurance CAN be filed while authority is pending (parallel OK). +- **IRP / IFTA / intrastate-authority / UCR** that depend on *active* authority + (or active USDOT) must wait for that activation, not just for the prior order to + be submitted. + +Implementation: +- Add an "activation gate" to the orchestrator: for dependency edges flagged + `require_active: true`, poll FMCSA (mobile QC API for USDOT status; L&I for + authority/BOC-3/insurance) and only mark the dependency satisfied when the + agency reports active. Until then, hold the downstream step in a + `waiting_on_activation` state with a next-poll timestamp. +- Encode the 21-day authority vetting window as an expected-activation estimate so + the timeline/customer comms set correct expectations. +- Expand `PIPELINES` edges with `require_active` flags (USDOT→MC, USDOT→IRP/IFTA/ + UCR/intrastate, authority-active→IRP-for-hire/intrastate). +- Standalone (non-bundle) orders: when a single service is ordered whose + prerequisite isn't active yet, surface a clear "blocked until X is active" + status + recommended prerequisite order rather than filing prematurely. + +### Phase 3 — Decide + handle state emissions (non-CA) +8. Either: (a) add a generic `state-emissions` service handler+product covering + NY/CO/MD/NJ/MA clean-truck/ACT programs, or (b) map those flags to + `state-dot-registration` / advisory-only. (Open question — see below.) + +### Phase 4 — Order landing pages +9. Create `site/src/pages/order/*.astro` for: irp-ifta (combined), state weight + taxes (one templated page per state or a single state-aware page), + ca-mcp-carb, intrastate-authority, hazmat-phmsa. Reuse existing order-page + layout (e.g. `boc3-filing.astro`, `ucr-registration.astro` as templates). + +### Phase 5 — Extend campaign builder (only after 1-4 verified) +10. Add new campaign segments to `build_trucking_campaigns.py` keyed off + `deficiency_flags`: for-hire/BOC-3+UCR, IRP/IFTA, intrastate-authority, + state weight-tax (per-state), CA MCP/CARB, hazmat. Each links to its LP. +11. Create the corresponding Listmonk source campaigns (templates) and wire IDs. + +## Validation (how each part is verified) + +- **Handlers:** unit-invoke each handler with a synthetic `order_data` dict in a + throwaway script; assert an `admin_todos` row is created with the expected + fields and (dev mode) no live filing fires. Confirm `SERVICE_HANDLERS` resolves + every new slug. +- **Intake completeness (the core ask):** write a check that, for every slug, + cross-references `REQUIRED_FIELDS[slug]` against the fields the intake step + emits and against the keys the handler reads — fail if a handler-needed field + is never collected. This is the verifiable "we collect all needed info" metric. +- **Checkout/products:** assert every flagged slug exists in `COMPLIANCE_SERVICES`, + `SERVICE_META`, checkout allowlist, and `SERVICE_HANDLERS` (one consistency test). +- **Order pages:** build site (`astro build` / existing build script) and confirm + each new `/order/` route renders; smoke-load locally. +- **Campaign builder:** run `build_trucking_campaigns.py --dry-run` and assert + each new segment selects a nonzero, deduped audience pointing at a valid LP. +- **End-to-end:** place a test order per new slug through checkout in dev, verify + intake validation blocks missing fields, and the handler produces a complete + admin todo. + +## Open questions / decisions + +**RESOLVED (per user 2026-06-02):** +1. **BOC-3 follow-ups + prerequisite blockers → upsell-approve, advise pre-order + where possible.** Two layers: + - *Pre-order advisory (preferred):* extend the existing DOT Compliance Check + tool (`site/public/tools/dot-compliance-check/`) + `dot-lookup` recommended + services to be **prerequisite-aware**. When a recommended service needs an + active prerequisite (e.g. IRP needs active authority), show the dependency, + pre-select the prerequisite, order the cart correctly, and state the ~21-day + authority activation expectation — all before payment. + - *Post-order upsell-approve:* when a handler discovers a blocker mid-fulfillment + (e.g. BOC-3 ordered but authority revoked → needs reinstatement), write a + `recommended_followups` entry on the order and render a one-click, pre-filled + "Add this service" card on the order timeline/portal. Customer confirms + pays + via the existing checkout. **No auto-charge.** + - *Standalone checkout guard:* if a service is bought directly and its prereq + isn't active at FMCSA, warn + offer to add the prereq rather than filing + something unfulfillable. +2. **State emissions (non-CA): build a real product.** Add a `state-emissions` + service (handler + product + intake + page) covering NY/CO/MD/NJ/MA clean-truck + / Advanced Clean Trucks programs (CA stays on `ca-mcp-carb`). +3. **Pre-order advisory: yes.** Covered by 1 above — reuse the compliance-check + tool as the advisory surface. + +**STILL OPEN:** +4. Pricing for `hazmat-phmsa` and `state-emissions`. + **DEFAULT:** `hazmat-phmsa` = $149 (admin-assisted PHMSA registration; gov fee + $25 placardable-hazmat reg billed at cost). `state-emissions` = $199 + (NY/CO/MD/NJ/MA clean-truck / ACT advisory + registration assist). +5. Vehicle-list intake fidelity. + **DEFAULT:** lightweight up front — collect fleet count + base/operating states + + fuel type + gross-weight bracket at intake; collect full per-vehicle + VIN/plate/weight rows via a post-order follow-up form only when the specific + filing requires it (IRP, weight-distance taxes). Keeps conversion high. +6. Standalone-order prereq guard. + **DEFAULT:** warn + offer to add the prerequisite (pre-selected), allow override + ("file anyway"). Never a hard block. +7. Long-term home for the reviewable plan (`docs/plans/` used here; no + `side_panel` tool in this harness). diff --git a/scripts/workers/services/hazmat_phmsa.py b/scripts/workers/services/hazmat_phmsa.py new file mode 100644 index 0000000..52e4ea3 --- /dev/null +++ b/scripts/workers/services/hazmat_phmsa.py @@ -0,0 +1,177 @@ +""" +Hazmat / PHMSA Registration Service Handler. + +Carriers that transport placardable quantities of hazardous materials must +register annually with PHMSA (Pipeline and Hazardous Materials Safety +Administration) under 49 CFR Part 107 Subpart G. This is separate from the +USDOT/FMCSA registration and from any HM safety permit. + +Service slug: hazmat-phmsa +Price: $149 (admin-assisted) +Gov fee: PHMSA registration fee (varies by carrier size; ~$25 + $250-$3,000 + processing fee depending on revenue/size) — billed at cost. + +This is admin-assisted: we collect the carrier's hazmat profile via the intake +form, then file the PHMSA registration (Form via https://hazmatonline.phmsa.dot.gov) +on the carrier's behalf and send the registration certificate. + +Intake data needed: + - DOT number + - Legal name / DBA + - Business address + contact + - EIN + - Hazmat classes / divisions transported + - Whether they transport in bulk packaging + - Estimated annual gross revenue (drives the PHMSA fee bracket) + - Number of employees + - Whether a small business (SBA size standard) +""" + +from __future__ import annotations + +import json +import logging +import os +from datetime import datetime + +LOG = logging.getLogger("workers.services.hazmat_phmsa") + +# PHMSA registration fee brackets (49 CFR 107.612). The processing fee depends on +# whether the registrant qualifies as a small business / not-for-profit. +PHMSA_FEE_INFO = { + "small_business": {"registration_fee_cents": 2500, "processing_fee_cents": 25000}, # $25 + $250 + "not_small": {"registration_fee_cents": 2500, "processing_fee_cents": 300000}, # $25 + $3,000 + "portal": "https://hazmatonline.phmsa.dot.gov", + "regulation": "49 CFR Part 107 Subpart G", +} + + +class HazmatPHMSAHandler: + """Handle PHMSA hazmat registration orders (admin-assisted).""" + + SERVICE_SLUG = "hazmat-phmsa" + SERVICE_NAME = "PHMSA Hazmat Registration" + + async def process(self, order_data: dict) -> list[str]: + """Entry point called by job_server. Delegates to handle().""" + order_number = order_data.get("order_number", order_data.get("name", "")) + return self.handle(order_data, order_number) + + def handle(self, order_data: dict, order_number: str) -> list[str]: + """Process a PHMSA hazmat registration order.""" + LOG.info("[%s] Processing PHMSA hazmat registration order", order_number) + + intake = order_data.get("intake_data") or {} + if isinstance(intake, str): + intake = json.loads(intake) + + dot_number = intake.get("dot_number", "") + entity_name = intake.get("entity_name", intake.get("legal_name", + order_data.get("customer_name", ""))) + customer_email = order_data.get("customer_email", "") + + # Determine fee bracket from small-business flag. + is_small = bool(intake.get("small_business")) + fee = PHMSA_FEE_INFO["small_business" if is_small else "not_small"] + + hazmat_classes = intake.get("hazmat_classes", []) + bulk = bool(intake.get("bulk_packaging")) + + steps = [ + f"1. Log into PHMSA portal: {PHMSA_FEE_INFO['portal']}", + "2. Start a new registration (or renewal) under 49 CFR Part 107 Subpart G", + "3. Enter carrier identity (legal name, DOT#, EIN, address)", + f"4. Enter hazmat classes/divisions: {', '.join(hazmat_classes) if hazmat_classes else 'PER INTAKE'}", + f"5. Indicate bulk packaging: {'YES' if bulk else 'NO'}", + f"6. Select fee bracket: {'small business ($25 + $250)' if is_small else 'standard ($25 + $3,000)'}", + "7. Pay the PHMSA registration + processing fee (billed to client at cost)", + "8. Download the Certificate of Registration", + "9. Send certificate + registration number to client; set renewal reminder (annual)", + ] + + todo_data = { + "order_number": order_number, + "service": self.SERVICE_NAME, + "service_slug": self.SERVICE_SLUG, + "dot_number": dot_number, + "entity_name": entity_name, + "customer_email": customer_email, + "hazmat_classes": hazmat_classes, + "bulk_packaging": bulk, + "small_business": is_small, + "estimated_gov_fee_cents": fee["registration_fee_cents"] + fee["processing_fee_cents"], + "intake_data": intake, + "steps": steps, + } + + try: + import psycopg2 + conn = psycopg2.connect(os.environ.get("DATABASE_URL", "")) + try: + with conn.cursor() as cur: + cur.execute(""" + INSERT INTO admin_todos ( + title, category, priority, order_number, service_slug, + description, data, status + ) VALUES (%s, %s, %s, %s, %s, %s, %s, 'pending') + """, ( + f"PHMSA Hazmat Registration — {entity_name} (DOT {dot_number})" + if dot_number else f"PHMSA Hazmat Registration — {entity_name}", + "filing", + "high", + order_number, + self.SERVICE_SLUG, + f"Service: {self.SERVICE_NAME}\n" + f"DOT: {dot_number}\n" + f"Hazmat classes: {', '.join(hazmat_classes) if hazmat_classes else 'see intake'}\n" + f"Bulk packaging: {'Yes' if bulk else 'No'}\n" + f"Small business: {'Yes' if is_small else 'No'}\n" + f"Est. gov fee: ${(fee['registration_fee_cents'] + fee['processing_fee_cents']) / 100:,.2f}\n" + f"Customer: {customer_email}\n\n" + f"Steps:\n" + "\n".join(steps), + json.dumps(todo_data), + )) + conn.commit() + finally: + conn.close() + LOG.info("[%s] Admin todo created for PHMSA hazmat registration", order_number) + except Exception as exc: + LOG.error("[%s] Failed to create admin todo: %s", order_number, exc) + + self._send_status_email(order_number, entity_name, dot_number, customer_email) + return [] + + def _send_status_email(self, order_number, entity_name, dot_number, customer_email): + """Send the client a status email.""" + if not customer_email: + return + try: + import smtplib + from email.mime.text import MIMEText + + dot_line = f" (DOT# {dot_number})" if dot_number else "" + body = ( + f"Hi,\n\n" + f"We've received your PHMSA Hazmat Registration order for " + f"{entity_name}{dot_line}.\n\n" + f"Order: {order_number}\n\n" + f"Our team will prepare and file your PHMSA registration " + f"(49 CFR Part 107) and send you the Certificate of Registration " + f"once complete, typically within 1-2 business days. The PHMSA " + f"government fee is billed at cost and depends on your business size.\n\n" + f"Questions? Reply to this email or call (888) 411-0383.\n\n" + f"Performance West Inc.\n" + f"DOT Compliance Services\n" + ) + + msg = MIMEText(body) + msg["Subject"] = f"PHMSA Hazmat Registration In Progress — {entity_name}{dot_line}" + msg["From"] = "noreply@performancewest.net" + msg["To"] = customer_email + + with smtplib.SMTP("localhost", 25) as s: + s.sendmail(msg["From"], [customer_email], msg.as_string()) + + LOG.info("[%s] Status email sent to %s", order_number, customer_email) + except Exception as exc: + LOG.warning("[%s] Failed to send status email: %s", order_number, exc) diff --git a/site/src/components/intake/steps/StateTruckingIntakeStep.astro b/site/src/components/intake/steps/StateTruckingIntakeStep.astro new file mode 100644 index 0000000..5780cca --- /dev/null +++ b/site/src/components/intake/steps/StateTruckingIntakeStep.astro @@ -0,0 +1,296 @@ +--- +// State-level trucking + hazmat/emissions intake. Slug-gated sections, mirroring +// DOTIntakeStep.astro. Collects the carrier identity plus the specific data each +// state filing / hazmat / emissions handler needs. +--- + +
+

Filing Information

+

+ Provide your carrier details so we can prepare your state filing(s). We file on your behalf. +

+ +
+ +

Carrier Information

+
+ +
+
+ + + +
+
+ + +
+ + + + + + + + + + + + + + + +
+ + +
+ + + +