Beru Learning

A large language model (LLM) is incredibly powerful, but it has a fundamental limitation: its knowledge is static. It only knows the information it was trained on and cannot access real-time data, browse the web, or interact with other applications. Function calling (also known as tool use) is the technology that breaks the LLM out of this box.

This guide explains in simple terms what function calling is, how it works, and why it's a critical step toward creating more capable and useful AI agents that can interact with the world.

The Problem: LLMs Are Stuck in a Box

Imagine asking a standard LLM, "What's the weather like in New York right now?" It can't give you a real answer. It might say something like, "As an AI, I don't have access to real-time information..." or it might give you the average weather for this time of year. It cannot access a live weather service.

Similarly, if you ask it to "add a meeting to my calendar for 3 PM," it can generate the text for an invitation, but it can't actually access your calendar application to create the event. Function calling solves this by giving the LLM a list of approved "tools" it can ask a human or another program to use on its behalf.

How Function Calling Works: A 3-Step Process

Function calling is a conversation between the LLM and the code that is running it. It's not the LLM executing code itself; it's the LLM deciding when code needs to be run and with what inputs.

You Provide a List of Tools: In your API call to the LLM, you not only provide your prompt but also a description of the functions the LLM is allowed to use. For each function, you define its name, what it does, and what parameters it needs (e.g., a function called get_weather needs a location parameter).
The LLM Decides to Use a Tool: When you ask a question like "What's the weather in New York?", the LLM analyzes your prompt. It recognizes that this question matches the description of the get_weather tool and that "New York" is the required location parameter. Instead of answering you, the LLM's response is a small piece of JSON saying, "Please run the function get_weather with the parameter location: 'New York'."
Your Code Executes the Function and Returns the Result: Your application receives this special response, runs your actual get_weather code (which calls a real weather API), gets the result (e.g., "75 degrees and sunny"), and sends that result back to the LLM in a new API call. The LLM then uses this new information to give you a final, natural language answer: "The weather in New York is currently 75 degrees and sunny."

A Practical Example: The Weather Bot

Let's walk through the process for a developer building a weather bot.

User Prompt: "How is the weather in Tokyo?"

Step 1: The Developer's API Call
The developer sends the user's prompt to the LLM, but also includes the definition of a tool:

tools: [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get the current weather in a given location", "parameters": { ... } } } ]

Step 2: The LLM's Response
The LLM sees that the user's prompt matches the tool's description. It doesn't answer the user. Instead, it responds with a request to call the function:

tool_calls: [ { "function": { "name": "get_current_weather", "arguments": "{\"location\": \"Tokyo\"}" } } ]

Step 3: Execution and Final Answer
The developer's code receives this, calls a weather API for Tokyo, gets back "15°C, cloudy," and sends this information back to the LLM. The LLM then formulates the final answer for the user: "The current weather in Tokyo is 15°C and cloudy."

Why This is a Game-Changer for AI

Function calling is a simple concept with profound implications. It allows LLMs to:

Access Real-Time Information: Connect to any API for live data on stocks, news, weather, and more.
Perform Actions: Interact with external systems to send emails, book appointments, or control smart home devices.
Query Databases: Convert natural language questions into structured database queries (like SQL).
Create More Reliable Agents: By grounding the LLM's responses in real data from external tools, its outputs become more factual and less prone to hallucination.

Frequently Asked Questions

Is the AI running code on my computer?

No, and this is a critical safety point. The LLM is only generating a structured text request (JSON) asking your code to run a function. Your application code is always in control of what actually gets executed.

What's the difference between function calling and plugins?

They are very similar concepts. Plugins (like those in ChatGPT) are essentially a user-friendly implementation of function calling, where the tools are pre-defined third-party applications.

Do all LLMs support function calling?

Most major, modern LLMs from providers like OpenAI, Google, and Anthropic have robust function calling / tool use capabilities built into their APIs.

Key Takeaways

Function calling allows an LLM to access live data and interact with external systems.
The process involves the LLM deciding to use a tool and requesting its execution, rather than running code itself.
A developer defines a list of available functions, and the LLM determines which one to call based on the user's prompt.
This technology is the foundation for building powerful AI agents that can perform actions in the real world.
Your code remains in control, providing a secure way to connect LLMs to your own tools and APIs.

What Is Function Calling? How LLMs Use Tools to Get Live Data