new-site/scripts/workers
justin 78c04b8bc3 Add Playwright failure monitoring: Telegram alerts + screenshots + health check
When any Playwright submission fails (selector not found, timeout, etc.):
1. Full-page screenshot captured and uploaded to MinIO
2. Telegram alert sent immediately with error details + screenshot link
3. Email alert to ops with same info
4. Admin todo includes screenshot MinIO path for debugging
5. Client order stays pending for manual completion

Proactive selector health check (daily 7am CT cron):
- Navigates to each portal (FCC RMD, USAC E-File, FCC CPNI/ECFS)
- Verifies all critical selectors are still present in the DOM
- If selectors are missing (UI changed): alerts via Telegram + email
  BEFORE any real client order fails
- Reports which service slugs are affected

Integrated into:
- RMD filing handler (fccprod.servicenowservices.com)
- Form 499-A handler (forms.universalservice.org)
- Form 499-Q handler (already had error handling)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-04 02:44:02 -05:00
..
cdr_adapters Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_presets Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_transports Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
crypto_offramp Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
icc_adapters Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
services Add Playwright failure monitoring: Telegram alerts + screenshots + health check 2026-05-04 02:44:02 -05:00
__init__.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
access_stim_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
amb_location_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
base_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
binder_compiler.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
business_days.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_classifier.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_ingester.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_npa_importer.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_puller.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_retention_sweeper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_sftpgo_provisioner.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cdr_unlock_nudge.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
client_email_processor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
cold_wallet_sweeper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
commission_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
compliance_alert_list.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
conversation_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
convert_guides.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
create_campaigns.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
create_campaigns_p2.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
create_compliance_campaign.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
crypto_payment_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
delivery_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
deminimis_factor_check.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
entity_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
erpnext_client.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_499_filer_import.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_499_filer_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_cores_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_rmd_auditor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fcc_rmd_removed_scraper.py Fix long-running PG transactions in RMD scrapers 2026-05-03 04:13:33 -05:00
fcc_rmd_scraper.py Fix long-running PG transactions in RMD scrapers 2026-05-03 04:13:33 -05:00
fix_349.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fix_campaigns.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
fl_entity_downloader.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
flag_outside_counsel.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
gckey_provisioner.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
hestia_provisioner.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
icc_ingester.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
job_server.py Add FCC Carrier/ISP Registration: API, checkout, handler, dispatch 2026-04-29 08:48:36 -05:00
listmonk_import.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
payment_reminder.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
quarterly_499q_notify.py Implement 499-Q quarterly filing lifecycle 2026-05-03 02:28:04 -05:00
relay_deposit_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
relay_integration.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
remove_adversary_subscribers.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
renewal_worker.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
rmd_deficiency_campaign.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
swap_discount_to_4pay.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
update_campaign_ctas.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
usac_audit_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
usac_import.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
usf_factor_monitor.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00
wy_entity_scraper.py Initial commit — Performance West telecom compliance platform 2026-04-27 06:54:22 -05:00