Skip to content

Log serverless request receipt#2303

Open
hansent wants to merge 4 commits into
mainfrom
codex/serverless-request-received-log
Open

Log serverless request receipt#2303
hansent wants to merge 4 commits into
mainfrom
codex/serverless-request-received-log

Conversation

@hansent
Copy link
Copy Markdown
Collaborator

@hansent hansent commented Apr 30, 2026

Summary

Adds an early Request received log in the GCP serverless authorization middleware immediately after the existing observability context is prepared.

Why

Some model requests can crash or terminate the inference server before the normal access log is emitted. Because the existing access log is written after request handling completes, those crash-triggering requests can disappear from the logs entirely. Logging request receipt as soon as the serverless middleware sees the request gives us a durable breadcrumb with the route that was hit before any model loading or inference work begins.

Details

  • Reuses the existing serverless observability setup instead of adding a separate outer middleware.
  • Logs only when API_LOGGING_ENABLED is enabled.
  • Includes method, path, request_id, and execution_id when available from _prepare_serverless_observability_context.
  • Leaves the existing after-response structured access log unchanged.

Validation

  • python -m py_compile inference/core/interfaces/http/http_api.py tests/inference/unit_tests/core/interfaces/http/test_http_api.py
  • git diff --cached --check

I could not run pytest in this local shell because the environment is missing pytest, fastapi, and starlette.

hansent added 3 commits April 30, 2026 17:17
…est-received-log

# Conflicts:
#	inference/core/interfaces/http/http_api.py
#	tests/inference/unit_tests/core/interfaces/http/test_http_api.py
@hansent hansent marked this pull request as ready for review May 11, 2026 14:18
@hansent hansent changed the title [codex] Log serverless request receipt Log serverless request receipt May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant