Per-tool concurrency groups were proposed in #7881 so related stateful tools can execute sequentially while unrelated tools remain concurrent. We should defer that portion of #7881 and design it independently across SDKs rather than introducing a Python-only API before its semantics and integration points are settled.
Questions the design should answer include:
- Should per-tool concurrency controls be introduced in both .NET and Python at the same time, with aligned naming and semantics?
- How should developers configure discovered MCP tools: one group per MCP server, mappings of tool names to groups, post-discovery mutation, or MCP metadata/annotations?
- How should locally executed MCP tools differ from provider-hosted MCP tools, which bypass the local function-invocation scheduler?
- Are groups scoped only to calls in one model-response batch, or should they coordinate across concurrent runs, agents, sessions, or shared tool instances?
- What are the required middleware semantics when a call fails closed or requests termination? No later queued side-effecting call should start after the batch has been stopped.
- How should approval pauses and replay preserve original model call order?
- What cancellation guarantees can be made for async functions versus synchronous functions already running in worker threads?
- How should user-defined group identifiers be represented without colliding with scheduler-internal keys?
- How should dynamically added tools, MCP reloads, and progressive disclosure inherit or retain concurrency policy?
- What result-ordering and observability guarantees should be shared across SDKs?
This issue is specifically about selective, tool-level controls. Batch-wide execution policy should remain separate: Python can add allow_concurrent_invocation to FunctionInvocationConfiguration, mirroring .NET, while retaining Python's existing parallel default.
Roger Barreto (@rogerbarreto), could you advise whether this should be designed and implemented for .NET and Python simultaneously?
Per-tool concurrency groups were proposed in #7881 so related stateful tools can execute sequentially while unrelated tools remain concurrent. We should defer that portion of #7881 and design it independently across SDKs rather than introducing a Python-only API before its semantics and integration points are settled.
Questions the design should answer include:
This issue is specifically about selective, tool-level controls. Batch-wide execution policy should remain separate: Python can add
allow_concurrent_invocationtoFunctionInvocationConfiguration, mirroring .NET, while retaining Python's existing parallel default.Roger Barreto (@rogerbarreto), could you advise whether this should be designed and implemented for .NET and Python simultaneously?