How to add tools to chatbots

Prerequisites

This guide assumes familiarity with the following concepts:

This section will cover how to create conversational agents: chatbots that can interact with other systems and APIs using tools.

note

This how-to guide previously built a chatbot using RunnableWithMessageHistory. You can access this version of the guide in the v0.2 docs.

As of the v0.3 release of LangChain, we recommend that LangChain users take advantage of LangGraph persistence to incorporate memory into new LangChain applications.

If your code is already relying on RunnableWithMessageHistory or BaseChatMessageHistory, you do not need to make any changes. We do not plan on deprecating this functionality in the near future as it works for simple chat applications and any code that uses RunnableWithMessageHistory will continue to work as expected.

Please see How to migrate to LangGraph Memory for more details.

Setup

For this guide, we'll be using a tool calling agent with a single tool for searching the web. The default will be powered by Tavily, but you can switch it out for any similar tool. The rest of this section will assume you're using Tavily.

You'll need to sign up for an account on the Tavily website, and install the following packages:

%pip install --upgrade --quiet langchain-openai tavily-python langgraph

import getpass
import os

if not os.environ.get("OPENAI_API_KEY"):
    os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")

if not os.environ.get("TAVILY_API_KEY"):
    os.environ["TAVILY_API_KEY"] = getpass.getpass("Tavily API Key:")

You will also need your OpenAI key set as OPENAI_API_KEY and your Tavily API key set as TAVILY_API_KEY.

Creating an agent

Our end goal is to create an agent that can respond conversationally to user questions while looking up information as needed.

First, let's initialize Tavily and an OpenAI chat model capable of tool calling:

from langchain_openai import ChatOpenAI
from langchain_tavily import TavilySearch

tools = [TavilySearch(max_results=10, topic="general")]

# Choose the LLM that will drive the agent
# Only certain models support this
model = ChatOpenAI(model="gpt-4o-mini")

API Reference:ChatOpenAI

To make our agent conversational, we can also specify a prompt. Here's an example:

prompt = (
    "You are a helpful assistant. "
    "You may not need to use tools for every query - the user may just want to chat!"
)

Great! Now let's assemble our agent using LangGraph's prebuilt create_react_agent, which allows you to create a tool-calling agent:

from langgraph.prebuilt import create_react_agent

# prompt allows you to preprocess the inputs to the model inside ReAct agent
# in this case, since we're passing a prompt string, we'll just always add a SystemMessage
# with this prompt string before any other messages sent to the model
agent = create_react_agent(model, tools, prompt=prompt)

API Reference:create_react_agent

Running the agent

Now that we've set up our agent, let's try interacting with it! It can handle both trivial queries that require no lookup:

from langchain_core.messages import HumanMessage

agent.invoke({"messages": [HumanMessage(content="I'm Nemo!")]})

API Reference:HumanMessage

{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='40b60204-1af1-40d4-b6a7-b845a2281dd6'),
  AIMessage(content='Hi Nemo! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 11, 'prompt_tokens': 795, 'total_tokens': 806, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsUwqprT2mVdjqu1aaSm1jVVWYVz', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--08282ec6-6d3e-4495-b004-b3b08f3879c3-0', usage_metadata={'input_tokens': 795, 'output_tokens': 11, 'total_tokens': 806, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}

Or, it can use of the passed search tool to get up to date information if needed:

agent.invoke(
    {
        "messages": [
            HumanMessage(
                content="What is the current conservation status of the Great Barrier Reef?"
            )
        ],
    }
)

{'messages': [HumanMessage(content='What is the current conservation status of the Great Barrier Reef?', additional_kwargs={}, response_metadata={}, id='5240955c-d842-408d-af3d-4ee74db29dbd'),
  AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_W37BFkNuZlJu9US1Tl71xpiX', 'function': {'arguments': '{"query":"current conservation status of the Great Barrier Reef","time_range":"year","topic":"general"}', 'name': 'tavily_search'}, 'type': 'function'}], 'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 32, 'prompt_tokens': 804, 'total_tokens': 836, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsV6EJ7F1vDipoG4dpEiBRZvuTLo', 'service_tier': 'default', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run--5f5b32d7-fb80-4913-a7ec-ca9c5acaa101-0', tool_calls=[{'name': 'tavily_search', 'args': {'query': 'current conservation status of the Great Barrier Reef', 'time_range': 'year', 'topic': 'general'}, 'id': 'call_W37BFkNuZlJu9US1Tl71xpiX', 'type': 'tool_call'}], usage_metadata={'input_tokens': 804, 'output_tokens': 32, 'total_tokens': 836, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}),
  ToolMessage(content='{"query": "current conservation status of the Great Barrier Reef", "follow_up_questions": null, "answer": null, "images": [], "results": [{"title": "The Great Barrier Reef: Current Conservation Efforts and Future Outlook", "url": "https://discoverwildscience.com/the-great-barrier-reef-current-conservation-efforts-and-future-outlook-1-279446/", "content": "The Great Barrier Reef, a mesmerizing marvel of nature, stretches over 2,300 kilometers along the northeast coast of Australia. As the largest coral reef system in the world, it is home to an incredible diversity of marine life, including more than 1,500 species of fish and 411 types of hard coral.", "score": 0.6353361, "raw_content": null}, {"title": "Monitoring progress - Protecting the Great Barrier Reef", "url": "https://www.detsi.qld.gov.au/great-barrier-reef/monitoring-progress", "content": "Stay informed about the current state of the Great Barrier Reef through comprehensive monitoring reports and reef report cards. Delve into the scientific research and advancements contributing to reef conservation. Learn about ongoing efforts to track progress and ensure the reef\'s long-term health.", "score": 0.6347929, "raw_content": null}, {"title": "Great Barrier Reef Outlook Report shows that the reef is in serious ...", "url": "https://biodiversitycouncil.org.au/news/great-barrier-reef-outlook-report-shows-that-the-reef-is-in-serious-trouble", "content": "The Great Barrier Reef is in very serious trouble. Climate change is the biggest threat to the reef. Catchment restoration activities that reduce sediment flowing to the reef will aid the health of the reef but cannot match the scale of destruction occurring due to marine heatwaves caused by climate change.", "score": 0.5183761, "raw_content": null}, {"title": "Water pollution threatens Great Barrier Reef\'s survival: new report ...", "url": "https://www.marineconservation.org.au/water-pollution-threatens-great-barrier-reefs-survival-new-report-highlights-funding-need/", "content": "While this investment has supported critical work across the Great Barrier Reef catchments, more funding is needed. At current rates, the target to cut fine sediment by 25% on 2009 levels will not be met until 2047, while the target to reduce dissolved inorganic nitrogen by 60% is not expected to be achieved until 2114.", "score": 0.51383984, "raw_content": null}, {"title": "What is the state of the Great Barrier Reef? - Tangaroa Blue", "url": "https://tangaroablue.org/the-state-of-the-great-barrier-reef/", "content": "The Great Barrier Reef Outlook Report 2024, prepared every five years by the Great Barrier Reef Marine Park Authority, summarises the Reef\'s long-term outlook based on its use, management, and risks.This year\'s report uses data from the Australian Marine Debris Initiative Database to analyse the risks and impacts of marine debris on the Great Barrier Reef and help identify areas for", "score": 0.47489962, "raw_content": null}, {"title": "New report on Great Barrier Reef shows coral cover increases before ...", "url": "https://www.aims.gov.au/information-centre/news-and-stories/new-report-great-barrier-reef-shows-coral-cover-increases-onset-serious-bleaching-cyclones", "content": "Coral cover has increased in all three regions on the Great Barrier Reef and is at regional highs in two of the three regions. But the results come with a note of caution. ... trained scientists during manta tow surveys and is a metric which allows AIMS scientists to provide an overview of the Great Barrier Reef\'s status and keep policy", "score": 0.40330887, "raw_content": null}, {"title": "Cycle of coral bleaching on the Great Barrier Reef now at \'catastrophic ...", "url": "https://www.sydney.edu.au/news-opinion/news/2025/01/21/coral-bleaching-2024-great-barrier-reef-one-tree-island.html", "content": "As the Great Barrier Reef faces increasing threats from climate change, the study calls for a collaborative approach to conservation that involves local communities, scientists and policymakers. Dr Shawna Foo , a Sydney Horizon Fellow and co-author of the study, said: \\"Seeing the impacts on a reef that has largely avoided mass bleaching until", "score": 0.3759361, "raw_content": null}, {"title": "Great Barrier Reef Outlook Report 2024: An ecosystem under pressure", "url": "https://icriforum.org/gbr-outlook-report-2024/", "content": "The 2024 Great Barrier Reef Outlook Report is the fourth in a series of comprehensive five-yearly reports on the Reef\'s health, pressures, management, and potential future. It found climate-driven threats such as warming oceans and severe cyclones have been compounding other impacts from crown-of-thorns starfish outbreaks, poor water quality", "score": 0.34634283, "raw_content": null}, {"title": "UNESCO expresses \'utmost concern\' at the state of the Great Barrier Reef", "url": "https://theconversation.com/unesco-expresses-utmost-concern-at-the-state-of-the-great-barrier-reef-257638", "content": "This 2017 photo from Ribbon Reef, near Cairns, shows what a healthy reef looks like. J Summerling/AP Poor water quality persists. Poor water quality is a major issue on the Great Barrier Reef.", "score": 0.31069487, "raw_content": null}, {"title": "Reef health updates | Reef Authority - gbrmpa", "url": "https://www2.gbrmpa.gov.au/learn/reef-health/reef-health-updates", "content": "As the lead managers of the Great Barrier Reef, the Reef Authority keeps an eye on the Reef year-round — with efforts stepped up over summer, a typically high-risk period from extreme weather. The Reef Authority releases updates on the health of Reef which includes; sea surface temperatures, rainfall and floods, cyclones, crown-of-thorns", "score": 0.18051112, "raw_content": null}], "response_time": 2.07}', name='tavily_search', id='cbf7ae84-1df7-4ead-b00d-f8fba2152720', tool_call_id='call_W37BFkNuZlJu9US1Tl71xpiX'),
  AIMessage(content='The current conservation status of the Great Barrier Reef is concerning. The reef is facing significant threats primarily due to climate change, which is causing marine heatwaves and coral bleaching. A report highlights that while there have been some local efforts in conservation, such as catchment restoration to reduce sediment flow, these cannot keep pace with the destruction caused by climate impacts. Recent findings from the 2024 Great Barrier Reef Outlook Report indicate that climate-driven phenomena like warming oceans and severe cyclones are exacerbating other pressures, such as crown-of-thorns starfish outbreaks and poor water quality.\n\nSome reports have indicated that coral cover has increased in certain regions of the reef, but overall, the health of the reef remains in serious decline. There’s an urgent call for more funding and collaborative efforts between local communities, scientists, and policymakers to enhance conservation measures.\n\nFor more detailed information, you can refer to these articles:\n- [The Great Barrier Reef: Current Conservation Efforts and Future Outlook](https://discoverwildscience.com/the-great-barrier-reef-current-conservation-efforts-and-future-outlook-1-279446/)\n- [Great Barrier Reef Outlook Report 2024: An ecosystem under pressure](https://icriforum.org/gbr-outlook-report-2024/)', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 255, 'prompt_tokens': 2208, 'total_tokens': 2463, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVAxeGL7PKGVkb2DieFPE0ZPgor', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--71441b27-81a0-427f-8784-b2ea674bebd4-0', usage_metadata={'input_tokens': 2208, 'output_tokens': 255, 'total_tokens': 2463, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}

Conversational responses

Because our prompt contains a placeholder for chat history messages, our agent can also take previous interactions into account and respond conversationally like a standard chatbot:

from langchain_core.messages import AIMessage, HumanMessage

agent.invoke(
    {
        "messages": [
            HumanMessage(content="I'm Nemo!"),
            AIMessage(content="Hello Nemo! How can I assist you today?"),
            HumanMessage(content="What is my name?"),
        ],
    }
)

API Reference:AIMessage | HumanMessage

{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='8a67dea0-acd8-40f9-8c28-292c5f81c05f'),
  AIMessage(content='Hello Nemo! How can I assist you today?', additional_kwargs={}, response_metadata={}, id='92a2533e-5c62-4cbe-80f1-302f5f1caf28'),
  HumanMessage(content='What is my name?', additional_kwargs={}, response_metadata={}, id='efa8c3d3-86d7-428f-985e-a3aadd6504bc'),
  AIMessage(content='Your name is Nemo!', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 818, 'total_tokens': 824, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVIf5MX5jXUEjYCorT5bWYzc7iu', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--a1a32c7d-8066-4954-86f9-3a8f43fcb48d-0', usage_metadata={'input_tokens': 818, 'output_tokens': 6, 'total_tokens': 824, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}

If preferred, you can also add memory to the LangGraph agent to manage the history of messages. Let's redeclare it this way:

from langgraph.checkpoint.memory import MemorySaver

memory = MemorySaver()
agent = create_react_agent(model, tools, prompt=prompt, checkpointer=memory)

API Reference:MemorySaver

agent.invoke(
    {"messages": [HumanMessage("I'm Nemo!")]},
    config={"configurable": {"thread_id": "1"}},
)

{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='31c2249a-13eb-4040-b56d-0c8746fa158e'),
  AIMessage(content='Hello, Nemo! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 12, 'prompt_tokens': 795, 'total_tokens': 807, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVRB0FItvtPawTTIAjNwgmlQFFw', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--a9703ca1-de4c-4f76-b622-9683d86ca777-0', usage_metadata={'input_tokens': 795, 'output_tokens': 12, 'total_tokens': 807, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}

And then if we rerun our wrapped agent executor:

agent.invoke(
    {"messages": [HumanMessage("What is my name?")]},
    config={"configurable": {"thread_id": "1"}},
)

{'messages': [HumanMessage(content="I'm Nemo!", additional_kwargs={}, response_metadata={}, id='31c2249a-13eb-4040-b56d-0c8746fa158e'),
  AIMessage(content='Hello, Nemo! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 12, 'prompt_tokens': 795, 'total_tokens': 807, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVRB0FItvtPawTTIAjNwgmlQFFw', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--a9703ca1-de4c-4f76-b622-9683d86ca777-0', usage_metadata={'input_tokens': 795, 'output_tokens': 12, 'total_tokens': 807, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}}),
  HumanMessage(content='What is my name?', additional_kwargs={}, response_metadata={}, id='0cde6457-8d4d-45d5-b175-ad846018c4d2'),
  AIMessage(content='Your name is Nemo! How can I help you today, Nemo?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 15, 'prompt_tokens': 819, 'total_tokens': 834, 'completion_tokens_details': {'accepted_prediction_tokens': 0, 'audio_tokens': 0, 'reasoning_tokens': 0, 'rejected_prediction_tokens': 0}, 'prompt_tokens_details': {'audio_tokens': 0, 'cached_tokens': 0}}, 'model_name': 'gpt-4o-mini-2024-07-18', 'system_fingerprint': 'fp_34a54ae93c', 'id': 'chatcmpl-BjsVTa1plxGPNitbOcw7YVTFdmz1e', 'service_tier': 'default', 'finish_reason': 'stop', 'logprobs': None}, id='run--1d742bc1-5839-4837-b6f4-9a6b92fa6897-0', usage_metadata={'input_tokens': 819, 'output_tokens': 15, 'total_tokens': 834, 'input_token_details': {'audio': 0, 'cache_read': 0}, 'output_token_details': {'audio': 0, 'reasoning': 0}})]}

This LangSmith trace shows what's going on under the hood.

How to add tools to chatbots

Setup

Creating an agent

Running the agent

Conversational responses

Further reading

Was this page helpful?

Setup​

Creating an agent​

Running the agent​

Conversational responses​

Further reading​

Was this page helpful?

Setup

Creating an agent

Running the agent

Conversational responses

Further reading