Skip to content

fix: support distance-based score filtering in retrieve tool#443

Open
kaiisfree wants to merge 1 commit into
strands-agents:mainfrom
kaiisfree:fix/distance-metric-filter
Open

fix: support distance-based score filtering in retrieve tool#443
kaiisfree wants to merge 1 commit into
strands-agents:mainfrom
kaiisfree:fix/distance-metric-filter

Conversation

@kaiisfree
Copy link
Copy Markdown

Summary

  • Adds score_metric parameter ("similarity" | "distance", default "similarity") to the retrieve tool
  • When score_metric="distance", inverts the filter comparison (<= instead of >=) so lower scores (closer matches) are kept
  • Backward compatible: default behavior unchanged

Fixes #417

Root Cause

filter_results_by_score unconditionally uses >= min_score, which inverts filtering when the backing vector store returns cosine distance (0 = exact match) instead of cosine similarity (1 = exact match).

Changes

  • Added score_metric enum parameter to TOOL_SPEC
  • Modified filter_results_by_score to respect metric type
  • Added tests for both similarity and distance filtering modes

Test plan

  • Existing similarity-based tests pass (22/22)
  • New distance-based test cases verify correct filtering (7 new tests)
  • Default behavior unchanged (backward compatible)

🤖 Generated with Claude Code

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

The filter_results_by_score function assumed scores are similarity metrics
(higher = more relevant), but vector stores using cosine distance return
lower scores for better matches. This adds a score_metric parameter to
control filtering direction.

Fixes strands-agents#417

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@starJammer
Copy link
Copy Markdown

@kaiisfree thanks for the PR and for looking at this. The only issue I see with your solution is that it relies on the LLM reasoning about whether it's a similarity or not since the similarity parameter is exposed in the tool description.

I would prefer it if the similarity/distance could be determined programmatically beforehand by the developer so that the LLM doesn't have to "understand" it as much and will either get a similarity or distance version of the tool.

What do you think? I'm not sure how to achieve this, honestly.
My naive approach would be to create two entirely separate tools, one for similarity and one for distance. The engineer can select which is appropriate and the documentation can include sane suggestions about which retrieve to use with which vector store.

@poshinchen
Copy link
Copy Markdown
Contributor

/strands

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants