justin/new-site
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
96 commits 2 branches 0 tags 96 MiB
Commit graph

3 commits

Author SHA1 Message Date
justin
a4a5500bfc Add Prometheus + Grafana + Alertmanager monitoring stack 
Full observability stack with Telegram alerting:

Components:
- Prometheus: metrics collection, 90-day retention
- Grafana: dashboards at monitoring.performancewest.net
- Alertmanager: routes alerts to Telegram bot
- node-exporter: OS metrics (CPU, RAM, disk, network)
- cAdvisor: container metrics (CPU, memory, restarts)
- postgres-exporter: PostgreSQL connection/query metrics
- nginx-exporter: request rate, 5xx errors, connections
- blackbox-exporter: HTTP/TCP endpoint probing + SSL cert checks

Alert rules:
- Service down (HTTP probe, TCP port, container missing)
- Container restart loops
- High CPU/memory/disk/load
- PostgreSQL down or high connections
- SSL cert expiring (14d warning, 3d critical)
- Slow HTTP responses, high 5xx rate

Blackbox probes all public endpoints:
  performancewest.net, api, dev, crm, lists, analytics,
  minio, crypto, pay

Telegram alerts: critical=1h repeat, warning=6h repeat,
  auto-resolve notifications

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-05-01 02:08:39 -05:00
justin
97e8664cbf Add security-updates Ansible role for automated patching 
Comprehensive security update automation:

1. Debian OS (unattended-upgrades) — tightened to security-only:
   - Removed general Debian updates (prevents feature/breaking changes)
   - Only Debian-Security origins auto-installed
   - Email admin on every upgrade via ops@performancewest.net
   - Auto-reboot at 4 AM if kernel update requires it
   - needrestart auto-restarts services after library updates

2. Docker CE — major version guard:
   - Patch updates within pinned major version auto-applied
   - Major version jumps held + admin alerted for manual review
   - docker-ce, docker-ce-cli, containerd.io all version-guarded

3. Container base images — daily at 3:30 AM:
   - Pulls latest base images for all docker-compose services
   - Compares image digests — only rebuilds if changed
   - Restarts only affected services (not full stack)
   - Alerts admin on rebuild failures requiring manual intervention
   - Covers both prod and dev compose projects

4. k3s — weekly Sunday at 3:45 AM:
   - Patch updates within current minor auto-applied
   - Minor/major upgrades alert admin for manual review
   - Verifies node Ready status after update
   - Alerts on failures with investigation instructions

5. Admin notifications via SMTP:
   - [INFO] for successful patches
   - [WARNING] for available major upgrades needing review
   - [CRITICAL] for failures requiring immediate intervention
   - Falls back to syslog if SMTP unavailable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-30 01:24:57 -05:00
justin
f8cd37ac8c Initial commit — Performance West telecom compliance platform 
Includes: API (Express/TypeScript), Astro site, Python workers,
document generators, FCC compliance tools, Canada CRTC formation,
Ansible infrastructure, and deployment scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-04-27 06:54:22 -05:00