Commit 15d8640
Foundry Evals integration for Python
Merged and refactored eval module per Eduard's PR review:
- Merge _eval.py + _local_eval.py into single _evaluation.py
- Convert EvalItem from dataclass to regular class
- Rename to_dict() to to_eval_data()
- Convert _AgentEvalData to TypedDict
- Simplify check system: unified async pattern with isawaitable
- Parallelize checks and evaluators with asyncio.gather
- Add all/any mode to tool_called_check
- Fix bool(passed) truthy bug in _coerce_result
- Remove deprecated function_evaluator/async_function_evaluator aliases
- Remove _MinimalAgent, tighten evaluate_agent signature
- Set self.name in __init__ (LocalEvaluator, FoundryEvals)
- Limit FoundryEvals to AsyncOpenAI only
- Type project_client as AIProjectClient
- Remove NotImplementedError continuous eval code
- Add evaluation samples in 02-agents/ and 03-workflows/
- Update all imports and tests (167 passing)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 7c85f98 commit 15d8640
21 files changed
Lines changed: 7105 additions & 75 deletions
File tree
- python
- packages
- azure-ai
- agent_framework_azure_ai
- tests
- core
- agent_framework
- _workflows
- tests
- core
- workflow
- samples
- 02-agents/evaluation
- 03-workflows/evaluation
- 05-end-to-end/evaluation/foundry_evals
Lines changed: 8 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
11 | 11 | | |
12 | 12 | | |
13 | 13 | | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
14 | 19 | | |
15 | 20 | | |
16 | 21 | | |
| |||
31 | 36 | | |
32 | 37 | | |
33 | 38 | | |
| 39 | + | |
34 | 40 | | |
35 | 41 | | |
36 | 42 | | |
37 | 43 | | |
| 44 | + | |
| 45 | + | |
38 | 46 | | |
0 commit comments