Skip to content

perf(client): exponential backoff in UploadRenderedProfile — ΔT≤5s vs const 5s sleep#98

Open
KorsarOfficial wants to merge 1 commit intoyandex:mainfrom
KorsarOfficial:perf/exponential-backoff
Open

perf(client): exponential backoff in UploadRenderedProfile — ΔT≤5s vs const 5s sleep#98
KorsarOfficial wants to merge 1 commit intoyandex:mainfrom
KorsarOfficial:perf/exponential-backoff

Conversation

@KorsarOfficial
Copy link
Copy Markdown

@KorsarOfficial KorsarOfficial commented Mar 18, 2026

Closes #93

Problem

UploadRenderedProfile contained an unconditional time.Sleep(5s) with a // FIXME comment. ClickHouse async_insert makes the uploaded profile temporarily invisible, but the fixed sleep adds a constant 5-second penalty to every call, including the common case where data is available in < 100 ms.

Complexity / latency analysis

Let T_vis = actual ClickHouse visibility latency (random variable).

Strategy E[latency] Worst case
Sleep(5s) 5 s 5 s
Exponential backoff (this PR) E[min(T_vis, 5s)] + O(log(5s/100ms)) overhead 5 s

Backoff sequence: 100 ms, 200 ms, 400 ms, 800 ms, 1600 ms, then capped at remaining.
Number of poll attempts: at most ⌈log₂(5000/100)⌉ + 1 = 7.

For T_vis ~ Uniform(0, 200 ms): E[latency] ≈ 100 ms vs 5000 ms50× improvement.

Implementation

const maxWait = 5 * time.Second
delay := 100 * time.Millisecond
deadline := time.Now().Add(maxWait)
for {
    taskID, _, err = c.MergeProfilesProto(ctx, req, taskAnnotation)
    if err == nil { break }
    remaining := time.Until(deadline)
    if remaining <= 0 { return "", "", fmt.Errorf("failed to render uploaded profile: %w", err) }
    if delay > remaining { delay = remaining }  // clamp to deadline
    select {
    case <-ctx.Done(): return "", "", ctx.Err()
    case <-time.After(delay):
    }
    delay *= 2
}

Invariants:

  • Total wait ≤ maxWait = 5s (unchanged worst-case reliability)
  • Context cancellation respected — no goroutine leak
  • delay is clamped to remaining before the last sleep, preventing overshoot

@KorsarOfficial
Copy link
Copy Markdown
Author

📄 Full analysis report (PDF): 08-perforator-optimizations.pdf

Covers complexity analysis, concurrency audit, and verification for all 7 optimizations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(symbolizer): replace unconditional 5s sleep with exponential backoff in UploadRenderedProfile

1 participant