fix(proxy): narrow budget-safe gate to primary usage by Komzpa · Pull Request #561 · Soju06/codex-lb

Komzpa · 2026-05-06T20:53:59Z

Summary

narrows the budget-safe Responses routing hard gate to primary-window usage only
keeps secondary/weekly usage as a routing strategy signal instead of a hard exclusion signal
makes the degraded all-primary-pressured usage_weighted fallback prefer lower primary pressure before lower weekly usage
adds OpenSpec coverage and regression tests for the [follow-up] Narrow _state_above_budget_threshold primary-only gating #446 edge cases

Closes #446.

Verification

openspec validate narrow-budget-safe-primary-gate --strict
UV_PROJECT_ENVIRONMENT=/home/kom/.venvs/codex-lb-fix-446 uv run pytest tests/unit/test_proxy_load_balancer_refresh.py -k 'budget_safe or reads_cached_usage_once' -q\n- UV_PROJECT_ENVIRONMENT=/home/kom/.venvs/codex-lb-fix-446 uv run ruff check app/modules/proxy/load_balancer.py tests/unit/test_proxy_load_balancer_refresh.py\n- UV_PROJECT_ENVIRONMENT=/home/kom/.venvs/codex-lb-fix-446 uv run ruff format --check app/modules/proxy/load_balancer.py tests/unit/test_proxy_load_balancer_refresh.py\n- UV_PROJECT_ENVIRONMENT=/home/kom/.venvs/codex-lb-fix-446 uv run ty check app/modules/proxy/load_balancer.py tests/unit/test_proxy_load_balancer_refresh.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 963e61a60f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-06T20:57:05Z

+        and state_list
+        and all(_state_above_budget_threshold(state, budget_threshold_pct) for state in state_list)
+    ):
+        return SelectionResult(min(state_list, key=_primary_budget_pressure_sort_key), None)


Route all-pressured fallback through availability checks

When every candidate is above the primary threshold, this branch returns min(state_list, ...) directly instead of delegating to select_account. That bypasses the availability filtering in app/core/balancer/logic.py (RATE_LIMITED/QUOTA_EXCEEDED reset checks, cooldown suppression, and error-backoff handling), so under usage_weighted the balancer can pick an account that is currently unavailable just because it has lower primary usage. This regression appears whenever all primary usages are > threshold and at least one candidate is blocked/unrecoverable at selection time.

Useful? React with 👍 / 👎.

Soju06 · 2026-05-09T10:35:20Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b10cabe0f9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-09T10:38:45Z

+        for state in sorted(state_list, key=_primary_budget_pressure_sort_key):
+            candidate = select_account(
+                [state],


Preserve health-tier preference in all-pressured fallback

This loop evaluates each account in isolation, which bypasses select_account’s pool-level health-tier logic (healthy > probing > draining). In the usage_weighted + all-primary-pressured path, a draining account with slightly lower primary usage can now be selected before a healthy account, even though the normal selector intentionally prefers healthy capacity first. That is a behavioral regression in failover safety and can route traffic to degraded accounts when healthy ones are available.

Useful? React with 👍 / 👎.

Soju06 · 2026-05-09T10:43:56Z

Thanks @Komzpa -- the #446 narrowing is exactly right. Codex review just landed two P1 findings on b10cabe that I think are real and worth addressing before merge:

P1 -- bypassing availability filtering

The new all-primary-pressured branch iterates sorted(state_list, key=_primary_budget_pressure_sort_key) and calls select_account([state], ...) per single-state list. select_account's availability filtering (RATE_LIMITED / QUOTA_EXCEEDED reset, cooldown suppression, error-backoff) does run on each single-state call, but the ordering of which states get tried first comes from the outer sort, not from the selector. So under usage_weighted with all candidates primary-pressured, an account that is currently unavailable can win the iteration just because its primary usage is the lowest.

P1 -- bypassing health-tier preference

Same shape: select_account([state]) cannot reason about healthy vs probing vs draining when there's only one state in the list, so the pool-level healthy > probing > draining preference that select_account would normally apply is silently lost. A draining account with slightly lower primary usage can now win over a healthy account in the all-pressured fallback.

Suggested shape

Keep the gate-narrowing change as-is (that's the right fix for #446), but rework the all-pressured fallback so it stays inside one select_account call:

Either: include health tier and availability flags in _primary_budget_pressure_sort_key (e.g. (health_tier_rank, availability_blocked, primary_used, secondary_used, last_selected, account_id)) and let min(states, key=...) produce the final ordering before calling select_account([sorted_state_list], ...) once.
Or, push the primary-first rule into the existing usage_weighted ranking inside select_account so it applies naturally instead of being a separate fallback path. That keeps health/availability/cooldown logic in one place.

The second option is cleaner architecturally, the first is less invasive. Either is fine.

Could you also add a regression test for each P1:

def test_all_primary_pressured_fallback_skips_unavailable_account()
def test_all_primary_pressured_fallback_prefers_healthy_over_draining()

Tests can stub AccountState.cooldown_until / error_backoff_until / status RATE_LIMITED for the unavailability case, and health_tier=draining vs healthy for the second.

The OpenSpec proposal copy is correct -- both scenarios (secondary-only pressure stays eligible and primary-first ordering in the fallback) hold under either fix shape, so no spec change required.

Ping me with the new commit and I'll re-run @codex review.

chatgpt-codex-connector · 2026-05-09T10:44:08Z

To use Codex here, create an environment for this repo.

Soju06 · 2026-05-10T12:39:02Z

@codex review

chatgpt-codex-connector · 2026-05-10T12:42:54Z

Codex Review: Didn't find any major issues. Another round soon, please!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Soju06 · 2026-05-10T13:08:28Z

@all-contributors please add @Komzpa for code, test

allcontributors · 2026-05-10T13:08:30Z

@Soju06

@Komzpa already contributed before to code, test

…l fetch timeouts, image rewrite) Upstream batch v1.16.0 highlights: - proxy: expose drain status for graceful deploys (Soju06#564) - proxy: narrow budget-safe gate to primary usage (Soju06#561) - proxy: handle model fetch timeouts (Soju06#541) - proxy: inline-rewrite input_image file references (Soju06#571) - proxy: retry transient stream timeouts (Soju06#551) - proxy: use DEFAULT_HOME_DIR for oversized response.create dumps (Soju06#575) - accounts: split compact quota row + recover quota status (Soju06#562, Soju06#559) - db: size background pool for burst traffic (Soju06#563) - upstream: drop top_p for gpt-5 (Soju06#538) Conflict resolved: - app/modules/proxy/service.py: kept fork's multi-line import + FailurePhase (used by deterministic failover at line 8294). Merge cleanup: - app/main.py: removed pre-existing duplicate InFlightMiddleware class + duplicate _is_benign_metrics_bind_failure helper that shadowed canonical versions and broke new upstream test (test_in_flight_middleware_does_not_count_drain_status). Verification: - pytest tests/unit: 1459 passed, 17 failed (all pre-existing on backup branch backup/pre-upstream-merge-20260511 — date-bound API key tests, test_db_migrate.py SQLite env, settings duplicate validators). - openspec validate --specs: 19/19 passed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Komzpa mentioned this pull request May 6, 2026

Tested live aggregate stack and suggested merge order #554

Open

chatgpt-codex-connector Bot reviewed May 6, 2026

View reviewed changes

fix(proxy): narrow budget-safe gate to primary usage

b10cabe

Komzpa force-pushed the fix/primary-budget-threshold branch from 963e61a to b10cabe Compare May 6, 2026 21:32

chatgpt-codex-connector Bot reviewed May 9, 2026

View reviewed changes

Komzpa added 3 commits May 9, 2026 14:56

fix(proxy): preserve selector gates in primary-pressure fallback

cd359aa

fix(proxy): apply primary fallback after availability filters

8bf588b

fix(proxy): preserve budget-safe routing semantics

f3f12d9

Komzpa force-pushed the fix/primary-budget-threshold branch from 2b22cf6 to f3f12d9 Compare May 9, 2026 15:30

style(proxy): format budget threshold helper

a8bd82d

Soju06 merged commit 3ed7834 into Soju06:main May 10, 2026
18 checks passed

This was referenced May 10, 2026

chore(main): release 1.17.0 #576

Merged

[design] Reconsider budget-safe routing gate vs health-tier overlap and per-window thresholds #578

Open

Komzpa mentioned this pull request May 10, 2026

feat(proxy): add opportunistic traffic class #567

Open

This was referenced May 11, 2026

fix: remove codex_spark quota gating causing 503 for plus plan accounts #513

Open

The requested account is always the same after v1.15.0 #589

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(proxy): narrow budget-safe gate to primary usage#561

fix(proxy): narrow budget-safe gate to primary usage#561
Soju06 merged 5 commits into
Soju06:mainfrom
Komzpa:fix/primary-budget-threshold

Komzpa commented May 6, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot May 6, 2026

Uh oh!

Soju06 commented May 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot May 9, 2026

Uh oh!

Soju06 commented May 9, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 9, 2026

Uh oh!

Soju06 commented May 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 10, 2026

Uh oh!

Uh oh!

Soju06 commented May 10, 2026

Uh oh!

allcontributors Bot commented May 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Komzpa commented May 6, 2026

Summary

Verification

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 6, 2026

Choose a reason for hiding this comment

Uh oh!

Soju06 commented May 9, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 9, 2026

Choose a reason for hiding this comment

Uh oh!

Soju06 commented May 9, 2026

P1 -- bypassing availability filtering

P1 -- bypassing health-tier preference

Suggested shape

Uh oh!

chatgpt-codex-connector Bot commented May 9, 2026

Uh oh!

Soju06 commented May 10, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 10, 2026

Uh oh!

Uh oh!

Soju06 commented May 10, 2026

Uh oh!

allcontributors Bot commented May 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants