diff --git a/docs/architecture.md b/docs/architecture.md index 6c03d31..af5724c 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,6 +1,11 @@ # System Architecture -**Last updated:** 2026-04-17 (15 Docker containers + k3s SHKeeper pods + Windows DocServer VM + dev stack + crypto treasury + foreign qualification + compliance check tool) +**Last updated:** 2026-05-07 (15 Docker containers + k3s SHKeeper pods + Windows DocServer + Postfix/OpenDKIM + bounce watcher + dev stack) + +See also: +- [Business Flow Diagram](business-flow.svg) +- [Technical Architecture Diagram](technical-architecture.svg) +- [Order Processing Flowchart](order-flow.svg) ## Overview diff --git a/docs/business-flow.svg b/docs/business-flow.svg new file mode 100644 index 0000000..e9aeb8c --- /dev/null +++ b/docs/business-flow.svg @@ -0,0 +1,215 @@ + + + + + + + + + + + + + + + Performance West — Business Flow + Customer acquisition → compliance check → order → filing → delivery + + + + Acquisition Channels + 📧 Listmonk email campaigns + 🔍 Google / Bing organic + 💬 Reddit / forum monitoring + 📞 Referrals / word of mouth + ⚖️ Attorney referral program + 📊 FCC deficiency alerts (2,718) + Direct SMTP via Postfix + DKIM + + + + + + + Free Compliance Check + performancewest.net/tools/ + fcc-compliance-check + • Enter FRN → auto-run from email + • Queries RMD, CPNI, USAC, BDC + • Shows issues with severity + • CTA: "Fix These Issues" + Logged to compliance_check_log + + + + Order + + + + Checkout & Payment + Batch or single service order + 💳 Stripe (card, ACH, Klarna) + 🅿️ PayPal + ₿ SHKeeper (BTC, ETH, USDC) + → PG compliance_orders + → ERPNext Sales Order + → Telegram notification + + + + Dispatch + + + + Worker Processing + Python job server (port 8090) + 1. Generate documents (DOCX→PDF) + 2. Upload to MinIO + 3. Send for eSign (portal link) + 4. Client signs → resume pipeline + 5. File with FCC/USAC (Playwright) + 6. Capture confirmation + 7. Email docs to client + AUTO_FILING=false → admin review + + + + + + + Delivery & Portal + 📄 Documents emailed to client + 📋 FCC confirmation number + 🌐 Portal order tracking + 📊 ERPNext Sales Invoice (paid) + 📅 Compliance calendar created + 🔄 Annual renewal auto-scheduled + + + + Service Catalog — 20 Telecom Services + Corporate + Privacy + TCPA + + + + FCC Carrier Reg + $1,299 + Bundle: 6 services + + + 499-A Filing + $499 + Annual revenue + + + CPNI Cert + $199 + Annual cert + + + RMD Filing + $249 + +$100 FCC fee + + + CALEA SSI + $799 + SSI plan + + + BDC Filing + $199-349 + Voice / BB / both + + + STIR/SHAKEN + $499 + Implementation + + + State PUC + $399 + Per state + fees + + + NECA OCN + $2,650 + Incl. NECA fees + + + Compliance Check + FREE + Lead generation tool + + + + eSign Portal — Officer Signature Required Before Filing + + + Handler generates + document + + + + Upload PDF to + MinIO + + + + Email JWT link + to officer + + + + Client draws/ + types signature + + + + Resume + pipeline + + Applies to: RMD (perjury), CPNI, CALEA SSI, 499-A engagement letter, discontinuance + + + + Email Infrastructure — Direct SMTP + + Campaign sending: + Listmonk → Postfix (207.174.124.71) → DKIM signed → direct delivery + 200/day warmup → 2,718 FCC deficiency subscribers on List 9 + + Transactional email: + Carbonio (co.carrierone.com) — order confirmations, intake links, eSign links + + Auth: SPF (-all) + DKIM (2048-bit) + DMARC (quarantine) + PTR (FCrDNS) + + + + Analytics & Tracking + 📊 Umami (analytics.performancewest.net) — page views, events, funnels + 🎯 Custom events: compliance-check-start/complete, order-cta-click, checkout-start + 📈 Server-side: payment-complete event, compliance_check_log table + 📧 Campaign tracking: opens (TrackView pixel), clicks (auto link wrapping) + 🔄 UTM attribution: listmonk campaigns → compliance checker → checkout + + + + Monitoring & Alerts + 📡 Prometheus + Grafana — 10 service probes, container health + 🔔 Alertmanager → Telegram (critical 1h, warning 6h repeat) + 🤖 Playwright selector health checks (daily cron) + 📨 Bounce watcher daemon → Listmonk webhook (blocklist after 2) + 🔐 Nightly container security updates (base image patches) + + + + Backup & Data + 🗄️ PostgreSQL: 4x/day dumps → MinIO → off-site rsync + 🗄️ MariaDB (ERPNext): daily dump → off-site + 📦 MinIO mirror: daily bucket sync + 📝 Forgejo: daily repo dump + 💾 Off-site: rsync to appbackups LXC (207.174.124.50) + + + Performance West Inc. — Cheyenne, WY — Updated 2026-05-07 + diff --git a/docs/infrastructure.md b/docs/infrastructure.md index 70448f3..9da43f2 100644 --- a/docs/infrastructure.md +++ b/docs/infrastructure.md @@ -1,6 +1,6 @@ # Infrastructure -**Last updated:** 2026-04-06 +**Last updated:** 2026-05-07 ## Production Server — Linux VM @@ -14,7 +14,7 @@ | Disk | 232 GB SSD | | Network | Bridged, static IP | -## Proxmox VM — Windows (DocServer) — NOT YET PROVISIONED +## Windows DocServer VM | Resource | Spec | |----------|------| @@ -22,10 +22,29 @@ | vCPU | 2 | | RAM | 4 GB | | Disk | 40 GB SSD | -| Software | Microsoft Office 2021 | -| Service | DocServer on port 5050 | +| Software | Microsoft Office 2021 + Python 3.12 | +| Service | docserver_worker.py (polls MinIO, converts via Word COM) | -The Windows VM will provide high-fidelity DOCX-to-PDF conversion via Office 2021. DocServer exposes a REST API on port 5050. LibreOffice on the Linux VM serves as a fallback. +Pixel-perfect DOCX→PDF conversion via Microsoft Word. Worker polls MinIO `to-convert/` bucket, converts via Word COM, uploads PDF to `converted/`. No HTTP server needed — MinIO is the transport. Requires RDP login after reboot (Word COM needs interactive session). LibreOffice headless is the automatic fallback. + +## Email Servers + +| IP | Hostname | PTR | Role | +|----|----------|-----|------| +| 207.174.124.15 | co.carrierone.com | co.carrierone.com | Carbonio — transactional email, mailboxes | +| 207.174.124.22 | cp.carrierone.com | cp.carrierone.com | HestiaCP — DNS, Exim4 MTA, .ca domain provisioning | +| 207.174.124.71 | perfwest.performancewest.net | perfwest.performancewest.net | Postfix + OpenDKIM — Listmonk campaign sending | + +All three have DKIM (2048-bit RSA), SPF (`-all` hard fail), and DMARC (`p=quarantine`). + +## Backup Server + +| Resource | Spec | +|----------|------| +| IP | 207.174.124.50 | +| Type | LXC container | +| Role | Off-site backup destination | +| Schedule | PG 4x/day, MariaDB daily, MinIO daily, Forgejo daily | ## External Infrastructure Dependencies diff --git a/docs/order-flow.svg b/docs/order-flow.svg new file mode 100644 index 0000000..84cac96 --- /dev/null +++ b/docs/order-flow.svg @@ -0,0 +1,166 @@ + + + + + + + + + + + + + + FCC Compliance Order — Processing Flow + + + + Email Campaign + Listmonk → Postfix + + + + + Free Check + ?frn= auto-run + + + + + Results + CTA + "Fix N Issues" + + + + + Stripe Checkout + Card / ACH / PayPal + + paid + + + + Order Created + PG + ERPNext SO + + + + dispatch + + + + Worker Processing Pipeline (Python job server) + + + + 1. Generate Documents + • Template selected by carrier type + • DOCX generated (python-docx) + • Convert to PDF (Word VM / LibreOffice) + _styles.py shared across 26 generators + + + + + + 2. Upload to MinIO + • PDF stored at + compliance/{order}/ + • Presigned URL for portal + + + + + + 3. eSign Gate + • Insert esign_records row + • Email JWT signing link + • PAUSE — wait for signature + Handler returns [] (no delivery yet) + + + + client signs + + + + 3b. Signature Received + esign_completed → re-dispatch + handler with client_approved=true + + + + + + 4. Auto-Filing Check + AUTO_FILING_ENABLED=true? + YES → proceed to filing + NO → admin review todo + + + + + + 5. File with FCC/USAC + • Playwright browser automation + • ECFS, RMD portal, USAC E-File + • Screenshot confirmation capture + Failure → Telegram alert + admin todo + + + + + + 6. Record & Deliver + • filing_state recorded in PG + • ERPNext SO → "Filed" workflow + • Confirmation email to client + Compliance Deadline DocType created + + + + + + 7. Client Delivery + • Documents emailed (PDF) + • Confirmation # in email + • Portal shows "Filed" status + Invoice marked Paid in ERPNext + + + + + + 8. Renewal Cycle + • Compliance Calendar entry created + • Reminder emails at 30/14/7 days + • Auto-invoice via renewal_worker + Annual: RMD Mar 1, CPNI Mar 1, 499-A Oct 1 + + + + Service Handlers (scripts/workers/services/) + + Playwright auto-filing: + rmd_filing, cpni_certification, form_499a, form_499q, cores_frn_registration, bdc_filing, foreign_carrier_affiliation, form_499_initial + + Admin-driven (todo-based): + fcc_carrier_registration (8-step), state_puc, ocn_registration, calea_ssi, form_499a_discontinuance, dc_agent, cdr_analysis, canada_crtc (14-step) + + + + eSign Required Before Filing + RMD (perjury 47 CFR § 1.16) | CPNI (officer attestation) | CALEA SSI (plan signature) | 499-A engagement letter | Discontinuance (officer signs deactivation letter) + Generic portal: /portal/esign/ | esign_records table | JWT 72h tokens | Draw or type signature | Webhook resumes pipeline + + + + Data stores: + PG compliance_orders → ERPNext Sales Order → ERPNext Sales Invoice → ERPNext Payment Entry + Documents: + MinIO compliance/{order}/ → eSign portal → Postfix email delivery + Notifications: + Telegram (new order + filing failures) | Email (intake, eSign, confirmation, renewal reminders) + + Performance West Inc. — Updated 2026-05-07 + diff --git a/docs/technical-architecture.svg b/docs/technical-architecture.svg new file mode 100644 index 0000000..e7e0321 --- /dev/null +++ b/docs/technical-architecture.svg @@ -0,0 +1,208 @@ + + + + + + + + Performance West — Technical Architecture + 207.174.124.71 (Debian 13) — Docker Compose multi-service stack — Updated 2026-05-07 + + + + Internet — Clients & External APIs + Browsers + Email recipients + Stripe / PayPal + FCC CORES / ECFS / USAC + ERPNext portal users + Umami / Grafana + + + + + + + + + nginx Reverse Proxy — TLS termination, sub_filter branding, rate limiting + :443 → site:4322 + :443 → api:3001 + portal → erpnext:8000 + lists → listmonk:9100 + analytics → umami:3100 + grafana:3000 + + + Application Tier + + + + site (Astro → nginx) + Static site + public HTML + Compliance checker, order pages + eSign portal, pricing + Port 4322 | nginx:alpine + + + + api (Express.js) + REST API, checkout, webhooks + FCC lookup, compliance orders + Portal auth (JWT 72h) + Port 3001 | node:22-slim + + + + workers (Python) + Job server, 27 service handlers + Playwright FCC/USAC automation + DOCX gen → PDF (Word VM / LO) + Port 8090 | python:3.12-slim + + + + erpnext (Frappe) + CRM, Sales Orders, Invoices + Customer portal (/orders) + Custom: PW apps + payments + Port 8080 | frappe/erpnext:v15 + + + + listmonk + Email campaigns (110+) + Subscriber lists (9) + Bounce webhook API + Port 9100 | listmonk:latest + + + + Monitoring Stack + Prometheus + Alertmanager → Telegram + Grafana dashboards (3000) + Umami analytics (3100) + cAdvisor, node-exporter, blackbox + + + Data Tier + + + + PostgreSQL + compliance_orders, telecom_entities + esign_records, check_log + 77 migrations applied + Port 5432 | 4x/day backup + + + + MariaDB (ERPNext) + Sales Orders, Invoices, Customers + Compliance Calendar/Deadline + 7 custom DocTypes + Port 3306 | daily backup + + + + MinIO (S3) + Document storage (DOCX, PDF) + Compliance packets, binders + DocServer DOCX↔PDF queue + Port 9000/9001 | daily mirror + + + + Redis + ERPNext cache + Queue, SocketIO + Port 6379 + + + + Ollama (LLM) + qwen2.5:7b + Doc generation + Port 11434 + + + + Forgejo (Git) + git.performancewest.net + Source code repo + Port 2222 (SSH) / 3033 + + + + k3s (SHKeeper) + Crypto payments + BTC, ETH, USDC + crypto.performancewest.net + + + + Umami PG + Analytics events + Sessions, page views + Internal only + + + External Services & Servers + + + Carbonio Mail + 207.174.124.15 + co.carrierone.com + DKIM + transactional email + PTR: co.carrierone.com + + + HestiaCP + 207.174.124.22 + cp.carrierone.com + DNS, Exim4 MTA, .ca domains + PTR: cp.carrierone.com + + + Postfix (local) + 207.174.124.71 + Campaign SMTP + OpenDKIM + Bounce watcher daemon + PTR: perfwest.performancewest.net + + + Windows DocServer + Word COM → PDF + Polls MinIO to-convert/ + Fallback: LibreOffice + Requires RDP login after reboot + + + Backup Server + 207.174.124.50 + appbackups LXC + PG, MariaDB, MinIO, Forgejo + Off-site rsync daily + + + DNS (HE + HestiaCP) + ns0.cp.carrierone.com + ns1-5.he.net (Hurricane Electric) + DNSSEC: not enabled + Reverse zone: 124.174.207.in-addr.arpa + + + ARIN + IP block: 207.174.124.0/23 + Owner: Carrier One (COTN) + rDNS delegated to HestiaCP + + + + Scheduled Jobs (systemd timers + cron) + PG backup 4x/day | MariaDB daily | MinIO mirror 3:30 AM | Forgejo dump 4 AM | Entity scraper 3 AM | Entity cache 2 AM | RMD scraper 4 AM | Renewal check 11 PM + Campaign warmup 8:15 AM M-F | Payment reminder every 5m | AMB scraper 6 AM | CDR retention midnight | Container security 3:30 AM | Cold wallet sweep 10 PM | Bounce watcher (daemon) + + + Performance West Inc. — 207.174.124.71 — Debian 13 — Docker Compose — Updated 2026-05-07 +