justin/new-site
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
new-site/docs/trucking-system.md
justin 899b880e7f trucking: weekly FMCSA source refresh so new non-compliant carriers are caught 
The FMCSA census was a one-time snapshot (last loaded ~May 30) with NO refresh
timer -- carriers newly falling out of MCS-150/UCR compliance were never picked
up. New scripts/workers/fmcsa_source_refresh.py orchestrates the full pipeline
(census download -> enrichment -> deficiency flag -> verify new emails ->
MX-tag new) and runs weekly via cron pw-fmcsa-refresh (Sun 09:00 UTC), codified
in the mail-pipeline Ansible role.

Idempotent + incremental: the census upsert preserves email_verified /
listmonk_sent_at / deficiency_flags, so existing carriers keep their send state
and only census fields refresh; new DOTs flow into verification then campaigns.
A carrier who refiled gets a fresh mcs150_parsed, so the builder's overdue
WHERE clause stops targeting them automatically. Verify is capped per run
(20k) so it never stalls on millions of rows.

(Healthcare already auto-catches newly-revalidation-overdue providers within
its 63k institutional pool via pw-hc-refresh Mon/Wed/Fri.)
2026-06-17 20:44:54 -05:00

214 lines
9.5 KiB
Markdown
Raw Blame History

# Trucking / DOT Compliance System
**Last updated:** 2026-05-29
See also:
- [Trucking Architecture Diagram](trucking-architecture.svg)
- [FMCSA Product Plan](fmcsa-trucking-plan.md)
- [System Architecture](architecture.md)
## Overview
Performance West's trucking vertical provides DOT/FMCSA compliance services to 2M+ US motor carriers. The system covers federal FMCSA requirements and state-level obligations across all 50 states + DC.
## Data Pipeline
```
FMCSA Socrata API (data.transportation.gov)
|
v
fmcsa_census_downloader.py ──> fmcsa_carriers table (2M+ records)
| - dot_number, legal_name, email
| - phy_state, fleet size, MCS-150 date
| - carrier_operation (A=Interstate, B=Intra HM, C=Intra)
v
fmcsa_deficiency_flagger.py ──> deficiency_flags, campaign_eligible
| - MCS-150 overdue/severe
| - For-hire, hazmat, zero fleet
| - Interstate IRP/IFTA needs
| - State weight tax, carrier permit, emissions
| - Intrastate authority
v
Listmonk lists ──> Email campaigns (state-segmented)
- FMCSA MCS-150 Overdue Carriers
- FMCSA State - California (MCP/CARB)
- FMCSA State - Oregon (Weight-Mile Tax)
- FMCSA State - New York (HUT)
- FMCSA State - Kentucky (KYU)
- FMCSA State - Texas
- FMCSA State - Florida
- FMCSA State - Connecticut (HUF)
- FMCSA State - New Mexico (WDT)
```
## Compliance Checker
**Endpoint:** `GET /api/v1/dot/lookup?dot=XXXXXX`
### Federal Checks (1-7) — from FMCSA QCMobile API
| # | Check | Source | Colors |
|---|-------|--------|--------|
| 1 | Operating Status | FMCSA API `allowedToOperate` | Green/Red |
| 2 | MCS-150 Biennial Update | Census + API `mcs150Outdated` | Green/Yellow/Red |
| 3 | Insurance (BIPD/Cargo/Bond) | FMCSA API `*InsuranceOnFile` | Green/Red per type |
| 4 | Safety Rating | FMCSA API `safetyRating` (S/C/U) | Green/Yellow/Red |
| 5 | Operating Authority | FMCSA API `*AuthorityStatus` | Green/Red |
| 6 | Out-of-Service Rate | FMCSA API `*OosRate` vs national avg | Green/Yellow/Red |
| 7 | Crash Record | FMCSA API `crashTotal`/`fatalCrash` | Green/Yellow/Red |
### State Checks (8-14) — from state_trucking_requirements table
| # | Check | Trigger | Color |
|---|-------|---------|-------|
| 8 | IRP Registration | Interstate (carrier_op=A) + IRP state | Yellow |
| 9 | IFTA Fuel Tax | Interstate + IFTA state | Yellow |
| 10 | Weight/Distance Tax | phy_state in OR/NY/KY/NM/CT | Yellow |
| 11 | State Carrier Permit | State requires (CA MCP, etc.) | Yellow |
| 12 | Emissions (CARB/ACT) | State has emissions program | Yellow |
| 13 | Intrastate Authority | For-hire + state requires | Yellow |
| 14 | State DOT Number | State requires (CA, TX) | Yellow |
**State Requirements API:** `GET /api/v1/dot/state-requirements?base_state=CA&operating_states=CA,OR,NV`
Returns structured requirements per state + recommended services with pricing.
## Services Catalog
### Federal DOT Services
| Slug | Service | Price | Handler |
|------|---------|-------|---------|
| `mcs150-update` | MCS-150 Biennial Update | $69 | MCS150UpdateHandler (admin todo) |
| `boc3-filing` | BOC-3 Process Agent Filing | $89 | BOC3FilingHandler (Playwright → processagent.com) |
| `ucr-registration` | UCR Annual Registration | $69 + gov | MCS150UpdateHandler |
| `dot-registration` | New USDOT Number | $89 | MCS150UpdateHandler |
| `mc-authority` | MC Operating Authority | $349 + $300 | MCS150UpdateHandler |
| `dot-drug-alcohol` | D&A Compliance Program | $149/yr | MCS150UpdateHandler |
| `dot-audit-prep` | Safety Audit Prep | $399 | MCS150UpdateHandler |
| `dot-full-compliance` | DOT Full Bundle | $499 | MCS150UpdateHandler |
### State Trucking Services
| Slug | Service | Price | Handler |
|------|---------|-------|---------|
| `irp-registration` | IRP Registration | $199 | StateTruckingHandler |
| `ifta-application` | IFTA Application + Decals | $149 | StateTruckingHandler |
| `ifta-quarterly` | IFTA Quarterly Filing | $99 | StateTruckingHandler |
| `or-weight-mile-tax` | Oregon Weight-Mile Tax | $199 | StateTruckingHandler |
| `ny-hut-registration` | NY Highway Use Tax | $199 | StateTruckingHandler |
| `ky-kyu-registration` | KY Weight-Distance Tax | $199 | StateTruckingHandler |
| `nm-weight-distance` | NM Weight-Distance Tax | $199 | StateTruckingHandler |
| `ct-highway-use-fee` | CT Highway Use Fee | $199 | StateTruckingHandler |
| `ca-mcp-carb` | California MCP + CARB | $349 | StateTruckingHandler |
| `state-dot-registration` | State DOT Registration | $149 | StateTruckingHandler |
| `intrastate-authority` | Intrastate Authority | $249 | StateTruckingHandler |
| `osow-permit` | Oversize/Overweight Permit | $99 | StateTruckingHandler |
| `state-trucking-bundle` | State Compliance Bundle | $599 | StateTruckingHandler |
## Vendors
| Vendor | Service | Cost | Integration |
|--------|---------|------|-------------|
| **Process Agent LLC** (Registered Agents Inc) | BOC-3 blanket filing | $25/yr | Playwright on processagent.com |
| **Northwest Registered Agent** (CorpTools API) | DC registered agent, formations | Varies | REST API (JWT auth) |
| **National Drug Screening** | D&A testing programs | TBD | Reseller program (pending) |
| **Tivly** | Insurance referrals | Commission | Affiliate (pending) |
## Database Tables
| Table | Migration | Purpose |
|-------|-----------|---------|
| `fmcsa_carriers` | 078 | 2M+ carrier census records |
| `state_trucking_requirements` | 079 | 51-jurisdiction state requirements |
| `carrier_operating_states` | 080 | Per-carrier operating state tracking |
| `compliance_orders` | (existing) | All DOT + state orders |
| `admin_todos` | (existing) | Admin filing tasks |
### Key Indexes on fmcsa_carriers
- `dot_number` (PK)
- `email_address`
- `phy_state`
- `mcs150_parsed`
- `authorized_for_hire`
- `campaign_eligible`
## Frontend Pages
| Page | Path | Type |
|------|------|------|
| DOT Compliance Check | `/tools/dot-compliance-check/` | Static HTML |
| DOT Order Page | `/order/dot-compliance/` | Static HTML |
| Trucking Services | `/services/trucking/` | Static HTML |
| California Compliance | `/services/trucking/california/` | Static HTML |
| Texas Compliance | `/services/trucking/texas/` | Static HTML |
| Oregon Compliance | `/services/trucking/oregon/` | Static HTML |
| New York Compliance | `/services/trucking/new-york/` | Static HTML |
| Florida Compliance | `/services/trucking/florida/` | Static HTML |
| Pricing (Trucking section) | `/pricing/` | Static HTML |
| 8 DOT intake pages | `/order/{slug}/` | Astro (review-only) |
## Order Flow
```
1. Carrier visits /tools/dot-compliance-check
|
2. Enters DOT# → API returns 14 checks (7 federal + 7 state)
|
3. CTA: "Fix My DOT Filings" → /order/dot-compliance?dot=X&services=Y&state=Z
|
4. Selects services (federal + state pre-selected from checker)
|
5. Stripe checkout ($0 bypass if promo covers total)
|
6. Confirmation email: "Order Confirmed — we're on it"
|
7. Worker dispatches to handler:
├── BOC3FilingHandler → Playwright on processagent.com ($25)
├── MCS150UpdateHandler → admin todo (Login.gov MFA)
└── StateTruckingHandler → admin todo (state-specific steps)
|
8. Admin completes filing → sends confirmation to customer
```
## Campaign Segmentation
| Segment | Count | Criteria |
|---------|-------|----------|
| All with email | 49,186 | email_address IS NOT NULL |
| Campaign eligible | 29,792 | Has actionable flags, not stale |
| Interstate (IRP/IFTA) | 19,761 | carrier_operation = 'A' |
| MCS-150 overdue | 19,176 | mcs150_parsed < 2 years ago |
| For-hire carriers | 19,811 | authorized_for_hire = TRUE |
| Intrastate authority needed | 14,081 | For-hire in 29 states |
| Emissions states | 12,424 | CARB/ACT states |
| Weight-tax states | 6,289 | OR/NY/KY/NM/CT |
| CA MCP/CARB needed | 3,418 | phy_state = CA |
Note: Census currently has 100K of 2M records. Full download will 5-10x these numbers.
## Key Files
```
api/
src/routes/dot-lookup.ts — compliance checker API (14 checks + state reqs)
src/routes/compliance-orders.ts — 21 DOT service slugs in COMPLIANCE_SERVICES
src/routes/checkout.ts — DOT_SLUGS set for "Order Confirmed" email
migrations/078_fmcsa_carriers.sql
migrations/079_state_trucking_requirements.sql
migrations/080_carrier_operating_states.sql
scripts/workers/
fmcsa_census_downloader.py — Socrata API bulk download
fmcsa_source_refresh.py — WEEKLY refresh orchestrator (census ->
enrichment -> flag -> verify -> mx-tag);
cron pw-fmcsa-refresh (Sun 09:00 UTC), so
newly-non-compliant carriers are auto-caught
fmcsa_deficiency_flagger.py — flag carriers + populate Listmonk lists
services/mcs150_update.py — MCS-150 handler (admin todo)
services/boc3_filing.py — BOC-3 handler (Playwright + admin fallback)
services/boc3_playwright.py — processagent.com Playwright automation
services/state_trucking.py — all state services (admin todo)
site/
public/tools/dot-compliance-check/ — compliance checker frontend
public/order/dot-compliance/ — batch order page
public/services/trucking/ — landing pages (main + 5 states)
src/lib/intake_manifest.ts — DOT + state service manifests
```
Reference in a new issue View git blame Copy permalink