Implementing a Tool-Using Agent

A tool-using agent can be implemented with a surprisingly small amount of code. The core loop is simple:

1. Send messages and tool definitions to the model.
2. If the model returns a tool call, execute it.
3. Add the tool result to the conversation.
4. Send the updated conversation back to the model.
5. Repeat until the model returns a final answer.

Frameworks can help with this loop, but understanding the mechanics is important. If you know what the framework is doing, you can debug failures, design better tools, and decide when to customize the orchestration.

A minimal agent loop

The following Python-style example shows a simplified tool-using loop. It is not tied to one specific vendor, but it reflects the common structure used by modern LLM APIs.

import json

TOOLS = [
    {
        "type": "function",
        "function": {
            "name": "calculator",
            "description": "Evaluate a basic arithmetic expression.",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": "A basic arithmetic expression, such as '12 * 4 + 3'."
                    }
                },
                "required": ["expression"]
            }
        }
    }
]

def calculator(expression: str) -> dict:
    # In production, use a safe parser. Do not eval untrusted input.
    allowed = set("0123456789+-*/(). ")
    if not set(expression).issubset(allowed):
        return {"success": False, "error": "Invalid characters in expression."}
    try:
        return {"success": True, "result": eval(expression, {"__builtins__": {}})}
    except Exception as e:
        return {"success": False, "error": str(e)}

def execute_tool(name: str, arguments: dict) -> dict:
    if name == "calculator":
        return calculator(arguments["expression"])
    return {"success": False, "error": f"Unknown tool: {name}"}

The agent loop might look like this:

def run_agent(client, user_message: str) -> str:
    messages = [
        {"role": "system", "content": "You are a precise assistant. Use tools for arithmetic."},
        {"role": "user", "content": user_message}
    ]

    for _ in range(5):
        response = client.chat(messages=messages, tools=TOOLS)
        assistant_message = response.message
        messages.append(assistant_message)

        tool_calls = assistant_message.get("tool_calls", [])
        if not tool_calls:
            return assistant_message["content"]

        for call in tool_calls:
            name = call["function"]["name"]
            arguments = json.loads(call["function"]["arguments"])
            result = execute_tool(name, arguments)

            messages.append({
                "role": "tool",
                "tool_call_id": call["id"],
                "content": json.dumps(result)
            })

    return "I could not complete the task within the tool-call limit."

This loop includes an important safety feature: a maximum number of iterations. Without that, an agent could get stuck repeatedly calling tools.

Avoid unsafe execution

The calculator example includes a warning: do not use eval on untrusted input in production. Tool implementations must be secure. A production calculator should use a safe expression parser or a dedicated math library.

This illustrates a broader principle: tool schemas constrain the model, but tool implementations must still defend themselves. Never assume the model will always pass safe arguments.

Implementing a search-and-fetch agent

A more realistic agent might use two tools: search and fetch.

[
  {
    "type": "function",
    "function": {
      "name": "search_docs",
      "description": "Search the product documentation for relevant pages.",
      "parameters": {
        "type": "object",
        "properties": {
          "query": {
            "type": "string",
            "description": "A concise documentation search query."
          }
        },
        "required": ["query"]
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "fetch_doc_page",
      "description": "Fetch the full text of a documentation page by URL.",
      "parameters": {
        "type": "object",
        "properties": {
          "url": {
            "type": "string",
            "description": "The documentation page URL returned by search_docs."
          }
        },
        "required": ["url"]
      }
    }
  }
]

This gives the model a natural sequence: search first, fetch second, answer third.

Frameworks and abstractions

Frameworks such as LangChain, LlamaIndex, and Semantic Kernel provide abstractions for tools, agents, memory, retrieval, and orchestration.

LangChain often represents tools as callable functions with metadata and provides agent executors that manage the loop. LlamaIndex is especially strong for document retrieval and knowledge-base workflows. Semantic Kernel organizes tools as plugins and functions that can be composed into plans.

Although the APIs differ, the underlying pattern is similar:

Tool definition
→ model selects tool
→ framework executes tool
→ result returned to model
→ loop continues

Frameworks are helpful when you need integrations, tracing, memory, retrieval, or standard orchestration patterns. But for simple applications, a custom loop may be clearer and easier to control.

Logging and observability

Every tool-using agent should log its tool calls. Logs should include:

• User request ID
• Tool name
• Arguments after validation
• Execution time
• Success or failure
• Error code if failed
• Result summary
• Final model response

Example log record:

{
  "request_id": "req_812",
  "tool_name": "search_docs",
  "arguments": { "query": "API key rotation" },
  "duration_ms": 184,
  "success": true,
  "result_count": 5
}

Logs make debugging possible. If users report bad answers, you need to see whether the model chose the wrong tool, passed bad arguments, received a bad result, or misinterpreted the result.

Human approval steps

For action tools, add human approval. The agent can prepare an action but not execute it until confirmed.

Example:

Agent: I found that this customer appears eligible for a refund under the late-delivery policy. I can create a refund request draft for $42.18. Would you like me to prepare it?

Or:

{
  "action": "create_refund_request",
  "requires_confirmation": true,
  "summary": "Refund request for order ORD-7711 due to late delivery.",
  "amount": 42.18
}

This pattern preserves user control while still allowing the agent to do useful preparation.

Practical takeaway

Implementing a tool-using agent is mostly about controlling a loop. The model proposes tool calls. The application validates and executes them. Results are returned as context. The process repeats until the task is complete.

Start simple. Build one or two tools. Add logging. Set a tool-call limit. Validate every argument. Require confirmation for meaningful actions. Once the loop is reliable, add more sophisticated orchestration or framework support.