b48d0cb799
Companion to the worker MinIO-retry fix. Makes the worker auto-recover from process death (crash, manual kill, missed boot trigger), not just MinIO outages. - start_worker.bat: propagate Python's exit code (exit /b %rc%) so Task Scheduler can actually detect a failed run (it previously always exited 0). - reconfigure_task.ps1 (new): re-registers PW-DocserverWorker with RestartCount=99 / 1-min interval, StartWhenAvailable, and two triggers — AtStartup plus a 5-min repeating trigger with MultipleInstances=IgnoreNew, so a dead worker relaunches within ~5 min and never double-runs. Idempotent. - install.ps1: same self-healing settings for fresh installs. - Verified on the box: killed the worker -> task relaunched it; firing again while running stayed at one instance. Docs updated to match reality: - docserver/README.md: new 'Reliability / self-healing' section. - document-generation.md: corrected the stale 'Flask DocServer :5050 / HTTP' description to the actual MinIO outbound-only transport. - e2e-test-plan.md: removed the outdated 'Word COM fails under SYSTEM / requires RDP after every reboot' limitation; now self-healing under SYSTEM session 0. - infrastructure.md: fixed VM spec (Win Server 2019, Word 16.0, Python 3.13, SSH port 22422) + self-healing note. - architecture.md / formation-system.md: trigger + self-healing details.
342 lines
17 KiB
Markdown
342 lines
17 KiB
Markdown
# Performance West — 52-Jurisdiction Business Formation System
|
||
|
||
**Last updated:** 2026-04-05
|
||
|
||
50 US states + District of Columbia + British Columbia (Canada) = 52 jurisdictions.
|
||
|
||
## Architecture
|
||
|
||
```
|
||
Customer Website API Database
|
||
| | | |
|
||
+- Select state ----------->| | |
|
||
+- Search name ------------>+- GET /states/:code/ ----->+- (proxy to Python) -->|
|
||
| <-- available/taken ---- | name-search | |
|
||
+- Fill details ----------->| | |
|
||
+- Submit order ----------->+- POST /formations ------->+- INSERT ------------->|
|
||
| <-- order number ------ | | formation_orders |
|
||
| | | |
|
||
| ERPNext (BPM Orchestrator) | |
|
||
| | | |
|
||
| +------+------+ | |
|
||
| | Webhook |<-- order.created --| |
|
||
| | triggers | (not polling) | |
|
||
| | worker | | |
|
||
| +------+------+ | |
|
||
| | | |
|
||
| +------+------+ | |
|
||
| | Load state | | |
|
||
| | adapter | | |
|
||
| +------+------+ | |
|
||
| | | |
|
||
| +------+------+ | |
|
||
| | Playwright |--> State SOS | |
|
||
| | automation | Portal | |
|
||
| +------+------+ | |
|
||
| | | |
|
||
| +------+------+ | |
|
||
| | Update DB |---- UPDATE ------->| |
|
||
| | with result | status=filed | |
|
||
| +------+------+ | |
|
||
| | | |
|
||
| <-- email docs --------- | | |
|
||
```
|
||
|
||
**ERPNext as BPM orchestrator:** ERPNext webhooks trigger formation workers on order events (e.g., `order.created`, `payment.confirmed`). Workers are event-driven, not polling-based.
|
||
|
||
## Directory Structure
|
||
|
||
```
|
||
scripts/formation/
|
||
+-- __init__.py
|
||
+-- base.py # StatePortal base class + data models
|
||
+-- name_search.py # Multi-state name search coordinator
|
||
+-- formation_worker.py # Event-driven queue processor (ERPNext webhook triggers)
|
||
+-- operating_agreement.py # Template-based .docx/.pdf generator
|
||
+-- ein_worker.py # IRS EIN online application (Playwright)
|
||
+-- document_delivery.py # SMTP email with attached formation docs
|
||
+-- states/
|
||
+-- __init__.py # Registry: get_adapter(), get_config(), STATES dict
|
||
+-- al/ # Alabama
|
||
| +-- __init__.py
|
||
| +-- config.py # Portal URLs, NW RA address, fees, CSS selectors
|
||
| +-- adapter.py # ALPortal(StatePortal) -- search + filing automation
|
||
+-- ak/ # Alaska
|
||
+-- az/ # Arizona
|
||
... (51 US: 50 states + DC)
|
||
+-- dc/ # District of Columbia
|
||
+-- bc/ # British Columbia (Canada) -- CRTC Carrier Package
|
||
```
|
||
|
||
## BC (Canada) — CRTC Carrier Package
|
||
|
||
British Columbia uses a completely different workflow from US states:
|
||
|
||
- **Not a state SOS filing** — BC adapter handles the CRTC Carrier Package registration
|
||
- **Different portal:** BC Corporate Online (bcregistryservices.gov.bc.ca) — anonymous, no login required
|
||
- **CRTC requirements:** Carrier registration with the Canadian Radio-television and Telecommunications Commission
|
||
- **Separate API endpoint:** `/api/v1/canada-crtc` for BC-specific orders
|
||
- **Separate DB table:** `canada_crtc_orders` tracks BC orders independently
|
||
- **14-step pipeline** managed by `services/canada_crtc.py`
|
||
|
||
The BC adapter implements `BCPortal(StatePortal)` but overrides both name search and filing methods for Canadian requirements. BC government fees are passed through at cost.
|
||
|
||
### BC Config (`scripts/formation/states/bc/config.py`)
|
||
|
||
The BC config contains configuration blocks for all Canadian-specific services:
|
||
|
||
| Config Block | Purpose |
|
||
|-------------|---------|
|
||
| `bits` | BITS (international telecom) registration — filing URL, required documents |
|
||
| `ccts` | CCTS (complaint commission) membership — application URL, required info |
|
||
| `gckey` | GCKey signup wizard — 5-step URLs, CSS selectors, hCaptcha sitekeys, password rules |
|
||
| `ats` | Annual Telecommunications Survey — 6 survey forms (REP-T/T1, REP-U, 802a, 802j, Facilities, Pricing) with deadlines and revenue thresholds |
|
||
| `corporate_obligations` | BC corporate tax/filing obligations — T2, GST/HST, T4/T4A, BC PST, WorkSafeBC, CRTC update |
|
||
|
||
### GCKey Provisioning
|
||
|
||
Each carrier needs its own GCKey account (Government of Canada credential) for filing
|
||
annual telecommunications surveys via My CRTC Account.
|
||
|
||
- **Username format:** `pw-{bc_number}`
|
||
- **Recovery email:** `regulatory@domain.ca` (we control the mailbox)
|
||
- **Automation:** `gckey_provisioner.py` — Playwright-based 5-step signup wizard
|
||
- **hCaptcha:** Invisible challenge on Step 2 (sitekey `99871bd1-7b22-417a-b6cc-7ef645e5147a`)
|
||
- **Credentials stored** in ERPNext Sensitive ID (encrypted)
|
||
- **Steps 3-5** selectors inferred from Step 1-2 recon — need verification on first live run
|
||
|
||
### Compliance Calendar (17 Entries Per Carrier)
|
||
|
||
Created at pipeline Step 13. Entries span regulatory, corporate tax, and survey obligations:
|
||
|
||
**Regulatory (7):** BC Annual Report, CRTC Maintenance, Mailbox Renewal, Domain Renewal,
|
||
DID Renewal, CCTS Renewal, CRTC Registration Update
|
||
|
||
**Corporate Tax (6):** T2 Return (Jun 30), Tax Payment (Mar 31), GST/HST (Mar 31),
|
||
T4/T4A Slips (Feb 28), BC PST (volume-based), WorkSafeBC (Mar 1 — if employees)
|
||
|
||
**Annual Telecom Surveys (4+):** REP-T/T1 (mandatory for ALL carriers, Mar 1),
|
||
REP-U/802a/802j/Facilities/Pricing (only if >$10M CAD revenue)
|
||
|
||
## State Adapter Status
|
||
|
||
### Name Search Methods
|
||
|
||
| Method | States | How It Works |
|
||
|--------|--------|-------------|
|
||
| **Socrata REST API** | CO, IA, IL, MI, NY, OR, PA, VT, WA | Free JSON API via state open data portals. No browser needed. |
|
||
| **SFTP Bulk Download** | FL | Free SFTP (public credentials) with daily entity dumps. Pre-load into local DB. |
|
||
| **Playwright** | All remaining ~40 US states | Browser automation against state SOS web portals. |
|
||
| **BC Corporate Online** | BC (Canada) | Browser automation against BC Registry Services. |
|
||
|
||
### Filing Method
|
||
|
||
**All 51 US jurisdictions use Playwright** — no state offers a filing API.
|
||
**BC (Canada)** uses Playwright against BC Corporate Online + CRTC portal.
|
||
|
||
### Implementation Status
|
||
|
||
| State | Name Search | LLC Filing | Corp Filing | Selectors Verified |
|
||
|-------|-------------|-----------|-------------|-------------------|
|
||
| **WY** | Working | Stub (needs filing walkthrough) | Stub | Name search: YES |
|
||
| **CO** | Socrata API (working) | Stub | Stub | N/A (API) |
|
||
| **BC** | Stub (BC Corporate Online) | N/A (CRTC Carrier Package) | Stub | Need portal inspection |
|
||
| All others | Stub (structure ready) | Stub | Stub | Need portal inspection |
|
||
|
||
~12 states have real implementations, ~40 still stubbed.
|
||
|
||
### Per-State Work Required
|
||
|
||
For each state, the following work is needed to go live:
|
||
|
||
1. **Visit the state's name search URL** in a browser
|
||
2. **Inspect form elements** — record CSS selectors for input fields, buttons, result areas
|
||
3. **Update `config.py`** with verified selectors
|
||
4. **Test name search** with known entities
|
||
5. **Visit the filing URL** and walk through the LLC formation form
|
||
6. **Record all filing form selectors** — name, agent, address, member, payment fields
|
||
7. **Implement the `file_llc()` method** with the state-specific workflow
|
||
8. **Test with a real formation** (or test/sandbox environment if available)
|
||
9. **Repeat for `file_corporation()`**
|
||
|
||
Estimated time: **1-3 hours per state** depending on portal complexity.
|
||
|
||
## API Endpoints
|
||
|
||
| Method | Path | Description |
|
||
|--------|------|-------------|
|
||
| `GET` | `/api/v1/states` | All 52 jurisdictions with fees |
|
||
| `GET` | `/api/v1/states/:code/name-search?name=...` | Name availability check |
|
||
| `POST` | `/api/v1/formations` | Create formation order |
|
||
| `GET` | `/api/v1/formations/:orderNumber` | Check order status |
|
||
| `GET` | `/api/v1/bundles` | List available service bundles |
|
||
| `GET` | `/api/v1/payment-methods` | Available payment methods + surcharges |
|
||
| `POST` | `/api/v1/agents` | Sales agent referral tracking |
|
||
| `POST` | `/api/v1/canada-crtc` | BC (Canada) CRTC Carrier Package order |
|
||
|
||
## Database Tables
|
||
|
||
### `state_filing_fees` — 52 rows (populated via migration 002)
|
||
All state/jurisdiction filing fees, portal URLs, special requirements (publication, franchise tax, etc.)
|
||
|
||
### `formation_orders` — Customer orders
|
||
Full order lifecycle: received -> processing -> submitted -> filed -> delivered.
|
||
Includes `erpnext_invoice_name` and `erpnext_payment_request` columns for ERPNext integration.
|
||
|
||
### `nwra_wholesale_pricing` — NW RA costs per state
|
||
Our wholesale cost for registered agent service (TBD from portal exploration).
|
||
|
||
### `canada_crtc_orders` — BC Canada orders
|
||
Separate table for CRTC Carrier Package orders with Canadian-specific fields.
|
||
Includes `erpnext_invoice_name` and `erpnext_payment_request` columns.
|
||
|
||
### `service_bundles` — Pre-configured service bundles
|
||
Bundle definitions (e.g., "Complete Formation + RA + EIN") with 20% discount applied.
|
||
|
||
### `bundle_orders` — Bundle order tracking
|
||
Links bundle purchases to individual service orders.
|
||
|
||
### `sales_agents` — Referral agent accounts
|
||
Agent ID (REF-XXXXX), contact info, commission tier, payout details.
|
||
|
||
### `commission_ledger` — Agent commission tracking
|
||
Per-order commission records: agent_id, order_id, service_type, flat_commission_amount, status (pending/paid).
|
||
|
||
### `payment_surcharges` — Payment method surcharge rates
|
||
ACH 0%, Card 3%, Klarna 5%, CashApp 3%, AmazonPay 3%, Crypto 0%.
|
||
|
||
### `accounting_advisors` — Advisor profiles
|
||
Accounting advisor details for post-formation consulting referrals.
|
||
|
||
### `accounting_support_accounts` — Client accounting support
|
||
Links clients to assigned accounting advisors.
|
||
|
||
### `conversation_flags` — Support conversation metadata
|
||
Flags for support conversations (escalation, priority, category).
|
||
|
||
## Formation Worker
|
||
|
||
`scripts/formation/formation_worker.py` is triggered by ERPNext webhooks (event-driven):
|
||
|
||
1. ERPNext webhook fires on `order.created` or `payment.confirmed`
|
||
2. Worker loads the appropriate state adapter
|
||
3. Verifies name availability
|
||
4. Submits the filing via Playwright
|
||
5. Updates DB with filing number and status
|
||
6. Generates operating agreement (if ordered)
|
||
7. Obtains EIN (if ordered)
|
||
8. Emails all documents to customer
|
||
|
||
**Human-paced delays:** 30-120 minutes between orders (configurable via env vars) to avoid appearing automated.
|
||
|
||
**Single-instance locking:** `fcntl.flock` on `/tmp/formation-worker.lock` prevents concurrent runs.
|
||
|
||
## Document Generation — PDF Conversion
|
||
|
||
- **Primary:** Windows DocServer (108.181.102.34, port 22422) — Office 365 Word COM automation via MinIO transport
|
||
- `docserver_worker.py` polls `to-convert/` bucket every 12 seconds
|
||
- Converts via Word COM, drops PDF in `converted/` bucket
|
||
- Heartbeat file at `minio://performancewest/docserver-heartbeat.json` (60s interval)
|
||
- Atomic uploads via `.tmp_` prefix + `copy_object` rename
|
||
- Task Scheduler: `PW-DocserverWorker` — self-healing: restarts on failure (99×/1 min) + AtStartup and a 5-min repeating trigger (relaunches within ~5 min if the process dies). The worker also retries MinIO on outage instead of exiting.
|
||
- **Fallback:** LibreOffice headless (`soffice --headless --convert-to pdf`) auto-activates when DocServer heartbeat stale (>5 min)
|
||
- **E2E tested:** 36KB DOCX → 82KB PDF in 12 seconds total round-trip
|
||
|
||
Operating agreements, CRTC letters, and other generated documents go through DocServer first. If the Windows VM is down, the system falls back to LibreOffice automatically.
|
||
|
||
## Northwest Registered Agent Integration
|
||
|
||
NW RA is used **only for registered agent service**, not for filings.
|
||
- Wholesale portal at accounts.northwestregisteredagent.com
|
||
- We order RA service for the customer's state after filing completes
|
||
- NW RA address for each state is stored in the state's `config.py`
|
||
|
||
## Operating Agreement Generator
|
||
|
||
Template-based using `python-docx`:
|
||
- Standard 10-article LLC operating agreement
|
||
- Variables filled from order data (entity name, state, members, etc.)
|
||
- Outputs `.docx` + `.pdf` (via Windows DocServer, LibreOffice fallback)
|
||
- Disclaimer: "This is not legal advice"
|
||
|
||
## EIN Obtainment
|
||
|
||
Playwright automation against IRS EIN Assistant:
|
||
- Available Mon-Fri 7am-10pm ET only
|
||
- Fills SS-4 equivalent online form
|
||
- Extracts EIN from confirmation page
|
||
- Saves PDF confirmation
|
||
|
||
## Pricing
|
||
|
||
| Component | Customer Pays | Our Cost |
|
||
|-----------|--------------|----------|
|
||
| State filing fee | Pass-through (varies $35-$725) | Same |
|
||
| Service fee (basic) | $179 | $0 (our margin) |
|
||
| Service fee (complete) | $399 | $0 (our margin) |
|
||
| RA service | $99/year (WY: $49/year) | ~$80/year (NW RA wholesale, TBD) |
|
||
| EIN | $49 | $0 (IRS is free) |
|
||
| Operating agreement | $99 | $0 (template-generated) |
|
||
| Expedited | Pass-through (varies by state) | Same |
|
||
|
||
## Service Bundles (20% Off)
|
||
|
||
Customers can purchase pre-configured bundles at a 20% discount off individual pricing:
|
||
|
||
| Bundle | Includes | Individual Total | Bundle Price |
|
||
|--------|----------|-----------------|-------------|
|
||
| **Starter** | Basic Formation + EIN | $198 | $158 |
|
||
| **Professional** | Complete Formation + EIN + RA (1yr) | $573 | $458 |
|
||
| **Full Package** | Complete Formation + EIN + RA (1yr) + Operating Agreement | $672 | $538 |
|
||
|
||
Bundle orders are tracked in `service_bundles` and `bundle_orders` tables.
|
||
|
||
## Payment Methods & Surcharges
|
||
|
||
| Method | Surcharge | Notes |
|
||
|--------|-----------|-------|
|
||
| **ACH Direct Debit** | 0% | Recommended — lowest cost |
|
||
| **Credit/Debit Card** | 3% | Via Adyen (Visa/MC/Amex + Apple Pay + Google Pay) |
|
||
| **Klarna (Pay in 4)** | 5% | Via Adyen — installment payments |
|
||
| **Cash App Pay** | 3% | Via Adyen |
|
||
| **Amazon Pay** | 3% | Via Adyen |
|
||
| **Crypto (BTC/ETH/USDC/USDT/MATIC/TRX/BNB/LTC/DOGE)** | 0% | Via SHKeeper (pay.performancewest.net) |
|
||
|
||
Surcharges are displayed at checkout before payment confirmation. Stored in `payment_surcharges` table. Surcharge injection is handled by the `performancewest_erpnext` Frappe app hook.
|
||
|
||
**SHKeeper** runs as a self-hosted instance in k3s (Kubernetes) at `pay.performancewest.net` for crypto payments. Supports any ERC-20/TRC-20/BEP-20 token. Webhook callbacks confirm payment and trigger formation workflow via ERPNext.
|
||
|
||
## Sales Agent Commission System
|
||
|
||
Sales agents refer clients using a unique referral code (format: `REF-XXXXX`).
|
||
|
||
### How It Works
|
||
|
||
1. Agent shares their referral link: `performancewest.net/form?ref=REF-XXXXX`
|
||
2. Client receives a **5% discount** on service fees
|
||
3. Agent earns a **flat commission per service** (not percentage-based)
|
||
|
||
### Commission Schedule
|
||
|
||
| Service | Agent Commission |
|
||
|---------|-----------------|
|
||
| Canada CRTC Package | $300 |
|
||
| Basic Formation | $50 |
|
||
| Bundle Purchase | $100 |
|
||
|
||
Commissions are tracked in ERPNext Commission Ledger DocType (with PostgreSQL backup in `commission_ledger`). Paid 14 days after order delivery via Relay ACH.
|
||
|
||
## Discount Code System
|
||
|
||
Discount codes can be created in ERPNext and applied at checkout:
|
||
|
||
- **Percentage-based** (e.g., 10% off)
|
||
- **Fixed amount** (e.g., $25 off)
|
||
- **Usage limits** (single-use, multi-use, or unlimited)
|
||
- **Expiration dates**
|
||
- Discount codes stack with agent referral discounts (capped at 15% total)
|
||
- Discounts apply to **service fees only** — never to state filing fees, government fees, or expedited fees
|
||
|
||
## Employment Pages
|
||
|
||
Employment/careers pages exist in the codebase but are **hidden in production** (not linked from navigation, not indexed by search engines). Gated behind `import.meta.env.DEV` flag. Reserved for future use.
|