fix(erpnext): self-heal outgoing Email Account password from SMTP_* env

Root cause of recurring 'Password not found for Email Account Performance West
Outgoing': the account was shipped as a fixture with awaiting_password=1 and no
password. Email Account SMTP passwords are encrypted per-site and cannot live in
a fixture, so every `bench migrate` reimported the fixture and re-broke
outgoing mail (login notifications, password resets, welcome emails).

- Remove the Email Account fixture (it cannot carry the encrypted secret).
- Add email_account_sync.sync_outgoing_password: idempotent, exception-safe
  upsert that reconciles the account + password from SMTP_* env and clears
  awaiting_password.
- Wire it to after_migrate (repairs at end of every deploy/migrate, right after
  fixtures import) and the daily scheduler (heals out-of-band restore/restart
  drift).
- Pass SMTP_* into the erpnext + erpnext-scheduler containers so the sync has
  the secret (they previously had no SMTP env).
This commit is contained in:
justin 2026-06-17 09:48:16 -05:00
parent 1eb29f80be
commit 557b45f65d
4 changed files with 161 additions and 21 deletions

View file

@ -183,6 +183,14 @@ services:
# shows "Link invalid" and the portal Compliance section is empty. # shows "Link invalid" and the portal Compliance section is empty.
- CUSTOMER_JWT_SECRET=${CUSTOMER_JWT_SECRET} - CUSTOMER_JWT_SECRET=${CUSTOMER_JWT_SECRET}
- DATABASE_URL=postgresql://pw:${DB_PASSWORD:-pw_dev_2026}@api-postgres:5432/performancewest - DATABASE_URL=postgresql://pw:${DB_PASSWORD:-pw_dev_2026}@api-postgres:5432/performancewest
# Outgoing mail: the "Performance West Outgoing" Email Account password is
# reconciled from these on `bench migrate` (after_migrate hook), so the
# account can never be left with awaiting_password=1 / empty password.
- SMTP_HOST=${SMTP_HOST}
- SMTP_PORT=${SMTP_PORT}
- SMTP_USER=${SMTP_USER}
- SMTP_PASS=${SMTP_PASS}
- SMTP_FROM=${SMTP_FROM}
volumes: volumes:
- erpnext-frappe-public:/home/frappe/frappe-bench/apps/frappe/frappe/public - erpnext-frappe-public:/home/frappe/frappe-bench/apps/frappe/frappe/public
- erpnext-erpnext-public:/home/frappe/frappe-bench/apps/erpnext/erpnext/public - erpnext-erpnext-public:/home/frappe/frappe-bench/apps/erpnext/erpnext/public
@ -219,6 +227,14 @@ services:
- REDIS_CACHE=redis://erpnext-redis:6379/0 - REDIS_CACHE=redis://erpnext-redis:6379/0
- REDIS_QUEUE=redis://erpnext-redis:6379/1 - REDIS_QUEUE=redis://erpnext-redis:6379/1
- REDIS_SOCKETIO=redis://erpnext-redis:6379/2 - REDIS_SOCKETIO=redis://erpnext-redis:6379/2
# Daily scheduler self-heals the outgoing Email Account password from these
# (email_account_sync.sync_outgoing_password), covering drift from
# out-of-band restarts / DB restores.
- SMTP_HOST=${SMTP_HOST}
- SMTP_PORT=${SMTP_PORT}
- SMTP_USER=${SMTP_USER}
- SMTP_PASS=${SMTP_PASS}
- SMTP_FROM=${SMTP_FROM}
depends_on: depends_on:
- erpnext-mariadb - erpnext-mariadb
- erpnext-redis - erpnext-redis

View file

@ -0,0 +1,122 @@
"""
Self-healing sync for the outgoing Email Account password.
Background
----------
ERPNext stores the SMTP password for an Email Account encrypted in the
``__Auth`` table, keyed by the site's ``encryption_key``. That secret is NOT
portable: it cannot live in a fixture, in git, or in a DB dump that gets
restored under a different key. As a result the "Performance West Outgoing"
account would periodically end up with ``awaiting_password = 1`` and an empty
password, after which every outgoing send (login notifications, password
resets, welcome emails) throws::
Password not found for Email Account Performance West Outgoing
The recurrence vectors we have actually hit:
* ``bench migrate`` reimporting an Email Account *fixture* (fixtures can't
carry the encrypted password, and the fixture set ``awaiting_password=1``).
* a site DB restore from a backup taken under a different encryption key.
Fix
---
Treat the *environment* (``SMTP_*`` vars, same ones the API/workers use) as the
single source of truth and reconcile the Email Account doc to it. This runs:
* ``after_migrate`` repairs the account at the end of every deploy/migrate,
immediately after fixtures import, so migrate can never leave it broken.
* daily via the scheduler heals drift from out-of-band restarts / restores.
The function is idempotent and never raises into the migrate/scheduler flow:
a misconfigured or missing secret must not abort a deploy.
"""
from __future__ import annotations
import os
import frappe
# The single outgoing account this app manages. Matches the value formerly
# shipped as a fixture (email_account_name / name).
ACCOUNT_NAME = "Performance West Outgoing"
def _env(*names: str, default: str = "") -> str:
"""First non-empty value among the given environment variable names."""
for n in names:
v = os.environ.get(n)
if v:
return v
return default
def sync_outgoing_password() -> None:
"""Reconcile the outgoing Email Account to the SMTP_* environment.
Idempotent and exception-safe logs and returns on any problem so it can
be wired into ``after_migrate`` and ``scheduler_events`` without risk of
aborting a deploy or a scheduler tick.
"""
try:
host = _env("SMTP_HOST")
user = _env("SMTP_USER", "SMTP_FROM")
password = _env("SMTP_PASS")
port = _env("SMTP_PORT", default="587")
# email_id is the From address; fall back to the login user.
email_id = _env("SMTP_FROM_EMAIL", "SMTP_USER") or user
if not (host and user and password):
# No secret available in this environment — leave the account as-is
# rather than wiping a working password. (e.g. dev box with no SMTP.)
frappe.logger("pw_email_sync").info(
"SMTP_HOST/USER/PASS not all set; skipping outgoing password sync"
)
return
try:
port_int = int(str(port).strip())
except (TypeError, ValueError):
port_int = 587
exists = frappe.db.exists("Email Account", ACCOUNT_NAME)
if exists:
doc = frappe.get_doc("Email Account", ACCOUNT_NAME)
else:
doc = frappe.new_doc("Email Account")
doc.email_account_name = ACCOUNT_NAME
# `name` is set from email_account_name on insert; set explicitly so
# downstream references (notifications, the fixture name) line up.
doc.name = ACCOUNT_NAME
# Reconcile connection settings + the password from the environment.
doc.email_id = email_id
doc.login_id = user
doc.login_id_is_different = 1 if (user and user != email_id) else 0
doc.smtp_server = host
doc.smtp_port = port_int
doc.use_tls = 1
doc.enable_outgoing = 1
doc.default_outgoing = 1
doc.password = password
# The whole point: clear the flag that makes ERPNext refuse to send.
doc.awaiting_password = 0
# Skip the live SMTP round-trip validation on save (deploys run offline
# / behind egress controls); a separate connectivity check covers that.
doc.flags.ignore_validate = True
if exists:
doc.save(ignore_permissions=True)
else:
doc.insert(ignore_permissions=True)
frappe.db.commit()
frappe.logger("pw_email_sync").info(
f"Synced outgoing Email Account '{ACCOUNT_NAME}' "
f"({user}@{host}:{port_int}); awaiting_password cleared"
)
except Exception:
# Never abort migrate / scheduler. Log full traceback for diagnosis.
frappe.log_error(
title="pw outgoing email account sync failed",
message=frappe.get_traceback(),
)

View file

@ -1,20 +0,0 @@
[
{
"name": "Performance West Outgoing",
"doctype": "Email Account",
"email_id": "noreply@performancewest.net",
"email_account_name": "Performance West Outgoing",
"smtp_server": "co.carrierone.com",
"smtp_port": 587,
"use_tls": 1,
"login_id": "noreply@performancewest.net",
"enable_outgoing": 1,
"enable_incoming": 0,
"default_outgoing": 1,
"awaiting_password": 1,
"used_for_outgoing_emails_from": "System Notifications",
"add_signature": 0,
"auto_follow_threads": 0,
"enable_auto_reply": 0
}
]

View file

@ -10,7 +10,13 @@ fixtures = [
{"dt": "Custom Field", "filters": [["dt", "in", ["Sales Order", "Sales Invoice", "Payment Request"]]]}, {"dt": "Custom Field", "filters": [["dt", "in", ["Sales Order", "Sales Invoice", "Payment Request"]]]},
{"dt": "Notification", "filters": [["name", "like", "CRTC%"]]}, {"dt": "Notification", "filters": [["name", "like", "CRTC%"]]},
{"dt": "Notification", "filters": [["name", "like", "Admin %"]]}, {"dt": "Notification", "filters": [["name", "like", "Admin %"]]},
{"dt": "Email Account", "filters": [["email_account_name", "=", "Performance West Outgoing"]]}, # NB: the "Performance West Outgoing" Email Account is intentionally NOT a
# fixture. Its SMTP password is stored encrypted (per-site encryption_key)
# and cannot be carried in a fixture, so reimporting it on `bench migrate`
# would set awaiting_password=1 and an empty password — breaking all
# outgoing mail ("Password not found for Email Account ..."). The account is
# created/repaired idempotently from the SMTP_* env via
# email_account_sync.sync_outgoing_password (after_migrate + scheduler).
# Subscription plans for recurring renewals (RA, annual report, CRTC maintenance, # Subscription plans for recurring renewals (RA, annual report, CRTC maintenance,
# formation maintenance bundles). Pricing updated per go-live-todo.md:37, 261. # formation maintenance bundles). Pricing updated per go-live-todo.md:37, 261.
{"dt": "Subscription Plan"}, {"dt": "Subscription Plan"},
@ -44,3 +50,19 @@ doc_events = {
csrf_ignore_methods = [ csrf_ignore_methods = [
"performancewest_erpnext.api.stripe_webhook", "performancewest_erpnext.api.stripe_webhook",
] ]
# Reconcile the outgoing Email Account password from the SMTP_* environment at
# the end of every migrate (fixtures import just before this, and cannot carry
# the encrypted password). Keeps "Password not found for Email Account ..." from
# recurring after a deploy.
after_migrate = [
"performancewest_erpnext.email_account_sync.sync_outgoing_password",
]
# Daily self-heal: catches drift from out-of-band restarts / DB restores that
# wipe the encrypted password without going through migrate.
scheduler_events = {
"daily": [
"performancewest_erpnext.email_account_sync.sync_outgoing_password",
],
}