[FAQ] How do I adapt the handwritten agent loop in Module 1 Lesson 14 for a non-OpenAI provider using Chat Completions?

[andrew-chung-au]

Course

llm-zoomcamp

Question

How do I adapt the handwritten agent loop in Module 1 Lesson 14 for a non-OpenAI provider using Chat Completions?

Answer

If your provider supports OpenAI-compatible Chat Completions tool calling, replace the Responses API loop with a loop that reads tool calls from response.choices[0].message.tool_calls.

The important differences are:

• Use client.chat.completions.create(..., messages=messages) instead of client.responses.create(..., input=messages).
• Read tool calls from response.choices[0].message.tool_calls, not response.output.
• Append the complete assistant message to the conversation history before adding tool results.
• Add every tool result as a role="tool" message with the matching tool_call_id.
• Continue calling the model until it returns an assistant message with no tool calls.

After running the Lesson 14 cells that define your search function and search_tool schema, use the following code:

import json


def make_tool_result(call, tool_handlers):
    """Run one tool call and format its result for Chat Completions."""
    tool_name = call.function.name
    tool_args = json.loads(call.function.arguments)

    if tool_name not in tool_handlers:
        result = {"error": f"Unknown tool requested: {tool_name}"}
    else:
        result = tool_handlers[tool_name](**tool_args)

    return {
        "role": "tool",
        "tool_call_id": call.id,
        "content": json.dumps(result, indent=2),
    }


def agent_loop(
    client,
    model,
    instructions,
    question,
    tools,
    tool_handlers,
    max_iterations=5,
):
    messages = [
        {"role": "developer", "content": instructions},
        {"role": "user", "content": question},
    ]

    for iteration in range(1, max_iterations + 1):
        print(f"Iteration {iteration}...")

        response = client.chat.completions.create(
            model=model,
            messages=messages,
            tools=tools,
        )

        message = response.choices.message

        # Preserve the assistant message, including its tool calls.
        messages.append(message)

        tool_calls = getattr(message, "tool_calls", None) or []

        # No tool calls means the model has returned its final answer.
        if not tool_calls:
            answer = message.content or ""
            print("\nASSISTANT:\n")
            print(answer)
            return answer

        # A model can request more than one tool in one response.
        for call in tool_calls:
            print(
                "Function call:",
                call.function.name,
                call.function.arguments,
            )

            tool_result = make_tool_result(call, tool_handlers)
            messages.append(tool_result)

    raise RuntimeError(
        f"Agent exceeded the maximum of {max_iterations} iterations."
    )

For example, call the loop using the Lesson 14 FAQ search tool:

answer = agent_loop(
    client=openai_client,
    model=MODEL_ID,
    instructions=instructions,
    question="How do I run Ollama locally?",
    tools=[search_tool],
    tool_handlers={"search": search},
)

answer

For Gemini through Google's OpenAI-compatible endpoint, configure the client like this:

import os

from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()

openai_client = OpenAI(
    api_key=os.environ["GEMINI_API_KEY"],
    base_url="https://generativelanguage.googleapis.com/v1beta/openai/",
)

MODEL_ID = "gemini-3.1-flash-lite"

This pattern was tested with Gemini: on the first iteration the model requested the search tool, the loop added its result as a role="tool" message linked by tool_call_id, and on the second iteration the model returned the final answer.

This approach applies only to providers that support the OpenAI-compatible Chat Completions tool-calling format. Check your provider's documentation for supported models and any provider-specific tool-schema differences.

For the underlying protocol, see OpenAI's function-calling guide and Chat Completions function-calling example.

Checklist

• I have searched existing FAQs and this question is not already answered
• The answer provides accurate, helpful information
• I have included any relevant code examples or links

[github-actions]

✅ FAQ NEW proposal created in PR #350

Please review and approve the changes.