justin/new-site
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
new-site/scripts/workers
History
justin 899b880e7f trucking: weekly FMCSA source refresh so new non-compliant carriers are caught 
The FMCSA census was a one-time snapshot (last loaded ~May 30) with NO refresh
timer -- carriers newly falling out of MCS-150/UCR compliance were never picked
up. New scripts/workers/fmcsa_source_refresh.py orchestrates the full pipeline
(census download -> enrichment -> deficiency flag -> verify new emails ->
MX-tag new) and runs weekly via cron pw-fmcsa-refresh (Sun 09:00 UTC), codified
in the mail-pipeline Ansible role.

Idempotent + incremental: the census upsert preserves email_verified /
listmonk_sent_at / deficiency_flags, so existing carriers keep their send state
and only census fields refresh; new DOTs flow into verification then campaigns.
A carrier who refiled gets a fresh mcs150_parsed, so the builder's overdue
WHERE clause stops targeting them automatically. Verify is capped per run
(20k) so it never stalls on millions of rows.

(Healthcare already auto-catches newly-revalidation-overdue providers within
its 63k institutional pool via pw-hc-refresh Mon/Wed/Fri.)
2026-06-17 20:44:54 -05:00
..
cdr_adapters Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_presets Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_transports Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
crypto_offramp Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
icc_adapters Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
services fix(mcs150): point intake email to per-slug wizard (not sales page) + add Trailers field 2026-06-16 16:21:57 -05:00
__init__.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
access_stim_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
amb_location_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
base_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
binder_compiler.py Add Canadian Wholesale Vendor Reference Guide to CRTC binder 2026-05-27 11:11:47 -05:00
business_days.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_classifier.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_ingester.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_npa_importer.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_puller.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_retention_sweeper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_sftpgo_provisioner.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_unlock_nudge.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
client_email_processor.py add FMCSA confirmation email monitor: IMAP watcher links confirmations to orders, Telegram notify 2026-05-30 19:01:42 -05:00
cold_wallet_sweeper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
commission_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
completion_emails.py fix: worker emails (localhost:25 -> SMTP relay) + create ERPNext SO on webhook payment 2026-06-09 14:40:46 -05:00
compliance_alert_list.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
conversation_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
convert_guides.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
create_campaigns.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
create_campaigns_p2.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
create_compliance_campaign.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
crypto_payment_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
daily_paper_batch.py healthcare: verify CMS-10114 update path, correct NPI Enumerator address, build CMS-10114 filler 2026-06-07 02:04:41 -05:00
delivery_worker.py fix: stop suppressing synthetic@pipeline.com (real customer address) 2026-06-10 14:41:19 -05:00
deminimis_factor_check.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
email_verifier.py fix(verifier): mx_unreachable was mislabeling live big-ISP mailboxes 2026-06-17 05:48:08 -05:00
entity_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
erpnext_client.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fax_sender.py add dev mode guards: skip FMCSA submission, fax, and web filing in non-production environments 2026-05-30 22:28:59 -05:00
fcc_499_filer_import.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_499_filer_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_cores_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_rmd_auditor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_rmd_removed_scraper.py Fix long-running PG transactions in RMD scrapers 2026-05-03 04:13:33 -05:00
fcc_rmd_scraper.py Fix long-running PG transactions in RMD scrapers 2026-05-03 04:13:33 -05:00
fix_349.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fix_campaigns.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fl_entity_downloader.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
flag_outside_counsel.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fmcsa_census_downloader.py Fix census download crash at 100K: integer out of range 2026-05-30 13:41:24 -05:00
fmcsa_deficiency_flagger.py Fix state list import: use individual subscriber creates instead of bulk 2026-05-29 14:26:44 -05:00
fmcsa_enrichment.py FMCSA enrichment: OOS orders bulk download + authority/insurance API lookup 2026-05-30 14:46:40 -05:00
fmcsa_source_refresh.py trucking: weekly FMCSA source refresh so new non-compliant carriers are caught 2026-06-17 20:44:54 -05:00
gckey_provisioner.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
hestia_provisioner.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
icc_ingester.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
ink_signature_cli.py Add portable Line-us pen-arm support to ink-signature pipeline 2026-06-07 03:45:46 -05:00
intake_reminder.py intake-reminder: weekly fallback so capped paid orders aren't abandoned 2026-06-15 22:13:27 -05:00
irp_invoice_poller.py feat(govfee): exact fees + agency processing fees; IRP email/invoice reconciliation 2026-06-16 04:58:14 -05:00
job_server.py mcs150/workers: don't fill MCS-150 for non-form services; quiet ERPNext workflow advance 2026-06-10 17:22:38 -05:00
listmonk_import.py legal: permanent do-not-contact for dataspindle.com + close re-import gap 2026-06-11 13:24:10 -05:00
mac_routing.py healthcare: verify CMS-10114 update path, correct NPI Enumerator address, build CMS-10114 filler 2026-06-07 02:04:41 -05:00
payment_reminder.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
pipeline_orchestrator.py fix(checkout): route dot-new-carrier-bundle on success page + worker pipeline 2026-06-08 23:48:56 -05:00
quarterly_499q_notify.py Implement 499-Q quarterly filing lifecycle 2026-05-03 02:28:04 -05:00
relay_deposit_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
relay_integration.py fix(relay): logging.getenv -> crash on import (card loading was broken) 2026-06-16 03:30:40 -05:00
remove_adversary_subscribers.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
renewal_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
rmd_deficiency_campaign.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
swap_discount_to_4pay.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
telegram_notify.py feat(telegram): include a presigned PDF view link in the admin-review alert 2026-06-10 12:13:43 -05:00
update_campaign_ctas.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
usac_audit_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
usac_import.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
usf_factor_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
worker_email.py fix: worker emails (localhost:25 -> SMTP relay) + create ERPNext SO on webhook payment 2026-06-09 14:40:46 -05:00
wy_entity_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00