ENG-3566: Refactor engine creation to use SQLAlchemy creator pattern

[erosselli]

Part of ENG-3566

Description Of Changes

Refactors all 6 database engines to use SQLAlchemy's creator pattern instead of baking credentials into connection URIs. The creator callable is invoked by the pool on every new connection, resolving credentials at connect time rather than engine construction time. This is the structural seam required for dynamic credential rotation via AWS Secrets Manager (design doc: #8016).

Today the creators read from static config (CONFIG.database). In follow-up work, the credential helpers will be swapped to call provider.get_secret() — no further engine changes needed.

Also updates the design doc to remove the lazy factory requirement, since the creator closure captures a provider reference (not credentials), making it independent of when the engine is constructed.

Code Changes

• database_settings.py — Add raw_password / raw_readonly_password properties that reverse quote_plus escaping for direct psycopg2.connect() / asyncpg.connect() use
• engine_creators.py (new) — make_sync_creator / make_async_creator factories that capture per-engine config (keepalives, SSL) in closures while resolving credentials per-connection. Includes credential helpers and asyncpg param/SSL conversion
• session.py — Add optional creator parameter to get_db_engine(). When provided, uses dialect-only URL and skips connect_args (creator handles them)
• ctl_session.py — Switch all 3 engines (async, readonly async, sync) from URI-based to creator-based. SSL context and asyncpg param handling moved into engine_creators.py
• session_management.py — Switch API and readonly sync engines to use make_sync_creator with keepalive connect_args
• tasks/__init__.py — Switch Celery task engine to use make_sync_creator with keepalive connect_args
• dynamic-database-credentials.md — Remove lazy factory requirement from design doc

Steps to Confirm

1. Verify all existing database connections work: hit GET /health/database and confirm all pools are healthy
2. Run existing test suites to confirm no regressions (public API surface is unchanged)
3. Verify that the async creator correctly wraps connections using SA internals (AsyncAdapt_asyncpg_connection)

Pre-Merge Checklist

• Issue requirements met
• All CI pipelines succeeded
• CHANGELOG.md updated
  • Add a db-migration This indicates that a change includes a database migration label to the entry if your change includes a DB migration
  • Add a high-risk This issue suggests changes that have a high-probability of breaking existing code label to the entry if your change includes a high-risk change (i.e. potential for performance impact or unexpected regression) that should be flagged
  • Updates unreleased work already in Changelog, no new entry necessary
• UX feedback:
  • No UX review needed
• Followup issues:
  • Followup issues created
• Database migrations:
  • No migrations
• Documentation:
  • No documentation updates required

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored	Preview	May 8, 2026 8:08pm
fides-privacy-center	Ignored		May 8, 2026 8:08pm

[codecov]

Codecov Report

❌ Patch coverage is 71.91011% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.21%. Comparing base (292d8b2) to head (f07c00d).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
src/fides/common/engine_creators.py	77.19%	8 Missing and 5 partials ⚠️
src/fides/api/db/session.py	55.00%	5 Missing and 4 partials ⚠️
src/fides/config/database_settings.py	66.66%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8148      +/-   ##
==========================================
- Coverage   85.23%   85.21%   -0.02%     
==========================================
  Files         638      639       +1     
  Lines       42011    42070      +59     
  Branches     4937     4945       +8     
==========================================
+ Hits        35807    35852      +45     
- Misses       5096     5103       +7     
- Partials     1108     1115       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.