feat(healthcare): route NPPES/PECOS Playwright flows through residential SOCKS proxy
CMS healthcare portals (NPPES, PECOS, I&A) block datacenter IPs, so the healthcare browser automation needs to egress via the residential proxy on hg409y7ez04.sn.mynetname.net (username 'performancewest'). - undetected_browser: use_proxy now accepts an env-var name, so callers can select a domain-specific proxy. _proxy_config(proxy_env) reads it and falls back to UNDETECTED_PROXY_URL. Healthcare uses 'HEALTHCARE_PROXY_URL'. - probe_npi_undetected: launches with use_proxy='HEALTHCARE_PROXY_URL' when set. - npi_provider: documents that the (future) automated NPPES/PECOS flows must use the healthcare proxy. - Plumb HEALTHCARE_PROXY_URL (+ UNDETECTED_PROXY_URL fallback) through the ansible env template and docker-compose workers env. The credential itself is NOT in the repo. Set the full URL in the ansible vault as vault_healthcare_proxy_url: socks5://performancewest:<password>@hg409y7ez04.sn.mynetname.net:<port> Verified parsing + Playwright proxy-dict wiring with a unit test.
This commit is contained in:
parent
bd9a70607f
commit
17318f6e7d
5 changed files with 70 additions and 10 deletions
|
|
@ -5,6 +5,7 @@ real endpoints and a fingerprint-detection page and prints what it sees.
|
|||
Run: python3 scripts/probe_npi_undetected.py
|
||||
"""
|
||||
import asyncio
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, "scripts")
|
||||
|
|
@ -12,6 +13,12 @@ from workers.services.telecom.undetected_browser import ( # noqa: E402
|
|||
undetected_browser, is_using_patchright,
|
||||
)
|
||||
|
||||
# Route healthcare (NPPES/PECOS/I&A) traffic through the residential SOCKS
|
||||
# proxy (username "performancewest"). Set HEALTHCARE_PROXY_URL=1 (or any
|
||||
# truthy value) to force it; the proxy is also used automatically whenever
|
||||
# HEALTHCARE_PROXY_URL is configured with a real URL.
|
||||
USE_HEALTHCARE_PROXY = bool(os.environ.get("HEALTHCARE_PROXY_URL", "").strip())
|
||||
|
||||
TARGETS = [
|
||||
# NPPES public registry UI (where NPI lookups/updates happen)
|
||||
("NPPES registry", "https://npiregistry.cms.hhs.gov/"),
|
||||
|
|
@ -28,7 +35,10 @@ SANNYSOFT = "https://bot.sannysoft.com/"
|
|||
|
||||
async def probe(headless: bool):
|
||||
print(f"\n{'='*60}\nbackend = {'patchright' if is_using_patchright() else 'vanilla-playwright'} | headless={headless}\n{'='*60}")
|
||||
async with undetected_browser(headless=headless) as (ctx, page):
|
||||
async with undetected_browser(
|
||||
headless=headless,
|
||||
use_proxy="HEALTHCARE_PROXY_URL" if USE_HEALTHCARE_PROXY else False,
|
||||
) as (ctx, page):
|
||||
# 1. navigator.webdriver + a couple of fingerprint signals
|
||||
try:
|
||||
await page.goto("about:blank")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue