CoT Based Smart Web Search

Open in Colabarrow-up-rightOpen in GitHubarrow-up-right

Overview

This tutorial demonstrates a chain-of-thought (CoT) based smart web search approach designed to build a plan-and-execute style QA chatbot. The system breaks down complex user queries into a sequence of actionable steps such as search, extraction, reasoning, and response generation. It leverages large language models to perform each task, ensuring that every phase of the process is handled efficiently and accurately.

Throughout the tutorial, you will learn how to set up your environment, install the necessary packages, and define key data models using Pydantic. These models enforce a structured workflow by representing each task step (e.g., search, think, and respond) and by specifying how references and final outputs should be formatted in Markdown.

The workflow is composed of multiple nodes, each responsible for a specific phase of the processing pipeline. You will see how the system:

  1. Generates a plan from a user query

  2. Retrieves raw information from the web

  3. Extracts and summarizes the relevant data

  4. Optionally refines the results through additional reasoning

  5. Compiles a comprehensive final answer

The nodes are dynamically connected through a routing function that adapts to the current state of the process.

A key aspect of this tutorial is the integration of asynchronous execution, which allows the entire workflow to run in real time. This not only improves performance but also provides immediate feedback during long-running tasks, making it easier to monitor and debug the process.

Key Features

  • Chain-of-Thought Query Expansion: Breaks down complex queries into manageable steps.

  • Modular Workflow: Separates the process into distinct nodes (planning, searching, extracting, reasoning, responding).

  • Asynchronous Execution: Enables real-time monitoring and concurrency for improved performance.

  • Dynamic Task Routing: Adapts the processing sequence based on the current state, ensuring each node is executed as needed.

  • Robust Data Modeling: Uses Pydantic to ensure consistent data structures and type safety.

Table of Contents

References


Environment Setup

Setting up your environment is the first step. See the Environment Setuparrow-up-right guide for more details.

[Note]

The langchain-opentutorial is a package of easy-to-use environment setup guidance, useful functions and utilities for tutorials. Check out the langchain-opentutorialarrow-up-right for more details.

You can alternatively set API keys such as OPENAI_API_KEY in a .env file and load them.

[Note] This is not necessary if you've already set the required API keys in previous steps.

Data Models and Type Definitions

This section establishes the foundations for the entire tutorial by defining the key data structures using Pydantic. Our workflow relies on these models to ensure consistent information flow:

  • TaskStep tracks a single step (either search, think, or respond).

  • TaskSteps holds a list of such steps.

  • SearchQuries, KeyInformation, and SearchResultData model how we handle web search queries and their results.

  • Reference and FinalResponse describe how we finalize and format the output response, including references.

Using these data models enforces structure, making the entire process easier to debug and maintain.

Define Agent and Worker State

In this part, two distinct state models are defined: one for the overall agent and one for individual worker tasks.

We use two different states:

  1. AgentState : Represents the overall flow of the conversation, including steps, search results, think results, and the final answer.

  2. WorkerState : Manages the current task at a more granular level (e.g., one node’s execution).

This separation improves modularity and makes it simpler to track local vs. global information during a workflow run.

Initialize Chat Models and Search Tool

We initialize two ChatOpenAI models:

  • smart model: A more capable model (e.g., gpt-4o) used for planning or complex reasoning.

  • fast model: A lighter model (e.g., gpt-4o-mini) used for simpler tasks or when we prioritize speed.

This allows you to balance quality versus cost and speed. For instance, plan generation might use smart model, while quick lookups or shorter tasks might use fast model.

Define Nodes for the Workflow

A node in this workflow is a single unit of functionalityβ€”such as β€œplan steps”, β€œsearch the web”, β€œthink about results”, or β€œrespond to user”. Splitting the entire chain-of-thought (CoT) approach into nodes makes the system easier to extend and debug.

Plan Steps Node

The plan steps node is the entry point of the workflow. It takes the user's query and generates a structured plan outlining the necessary steps to answer it. These steps include creating search tasks, optional think tasks for deeper analysis, and a final respond task to craft the answer. A well-defined plan here is crucial for effective execution in later nodes.

Search Node

This node handles external web search. It receives the step’s description, produces multiple queries, and calls the TavilySearchResults tool to get results. Each result is then summarized or filtered, before being passed on to the rest of the pipeline.

  1. Generate multiple queries (to cover different angles).

  2. Perform the searches asynchronously.

  3. Store aggregated information for the next step.

Collect Search Results Node

This optional node simply waits for all concurrent searches to finish and then aggregates the results. It does nothing beyond returning the existing state, but it ensures we have a single synchronization point before moving forward (useful in parallel β€œmap” scenarios).

Think Node

We optionally refine and analyze the data gathered from the search. If the user’s query is more complex, we insert a think step that leverages fast_model (or smart_model) to process the intermediate data. This can produce more concise or better-structured information to pass on to the final respond phase.

Response Node

Finally, we combine all steps, including the user’s query, search results, and optional think results, into a coherent answer. The answer is generated in Markdown, with inline citations and references appended at the end.

This step demonstrates how we can produce a well-formatted, easily readable response to the end user.

Route Tasks and Compile Workflow

This portion of the tutorial shows how to:

  1. Decide which step should execute next (the task router).

  2. Build our entire workflow as a graph, using the StateGraph class.

  3. Visualize the graph to confirm all nodes connect properly.

Combining these elements provides a clear flow from plan β†’ search β†’ optional think β†’ respond, with conditionals for concurrency.

Task Router Function

The router examines all pending steps. If we find one or more search steps (marked pending), we dispatch them concurrently using the Send object. If no search steps remain, we look for think or respond steps. When none remain, we conclude by returning END.

This ensures the system adapts dynamically: if a step is done, it moves on to the next relevant step without manual intervention.

Build Workflow Graph

We add nodes to the workflow using .add_node(...), specify the start node (plan_steps_node), and link them with .add_edge(...) or .add_conditional_edges(...). Then we compile with a MemorySaver checkpoint mechanism (optional) to track states.

Graph-based logic clarifies how each node transitions to the next, whether conditionally or directly.

Visualize Graph

For debugging or documentation, we can generate a diagram of our graph’s nodes and edges. This makes it easy to see the plan from start to finish, identify concurrency, and check for mistakes (like a node missing an edge to the final stage).

png

Run Workflow

Here, we demonstrate the end-to-end execution:

  1. Configure the run parameters (e.g., thread IDs or optional debugging).

  2. Initialize the graph with the user’s query.

  3. Execute asynchronously, streaming real-time results.

Finally, we show how to examine the final stateβ€”looking at the produced Markdown answer and references.

Streaming Graph Execution

We define astream_graph(...) to retrieve and print updates during the run. Each node’s outputs appear in real time, letting you monitor progress and debug problems such as missing data or infinite loops.

Configure and Run Workflow Graph

We create a config, store it in memory if needed, and supply the user’s query. The asynchronous execution triggers each node in the correct order.

Impact of AI on Global Trade Efficiency During the Pandemic

Overview

The COVID-19 pandemic significantly disrupted global trade and supply chains, highlighting vulnerabilities across industries. Artificial Intelligence (AI) emerged as a critical tool for enhancing supply chain efficiency and resilience during this period.

Key Findings on AI's Impact During the Pandemic

  1. Enhancing Resilience:

    • AI improved supply chain resilience by providing enhanced transparency and real-time insights. This allowed companies to adapt quickly to disruptions, ensuring last-mile delivery and minimizing the impact of supply chain interruptions. AI facilitated agile procurement strategies, which were essential during the pandemic (sourcearrow-up-right).

    • Research indicated that AI could enhance supply chain management efficiency by up to 40% through effective data processing and trend prediction ([source](https://hypersonix.ai/blo

...output cut to prevent excessive output...

Impact of AI on Global Trade Efficiency During the Pandemic

Overview

The COVID-19 pandemic significantly disrupted global trade and supply chains, revealing vulnerabilities across industries. Artificial Intelligence (AI) emerged as a crucial tool for enhancing supply chain efficiency and resilience during this period.

Key Findings on AI's Impact During the Pandemic

  1. Enhancing Resilience: AI improved supply chain resilience by providing enhanced transparency and real-time insights, allowing companies to adapt quickly to disruptions, ensuring last-mile delivery, and minimizing the impact of supply chain interruptions. AI facilitated agile procurement strategies essential during the pandemic[1][2].

  2. Data-Driven Decision Making: Companies shifted from cost-driven approaches to data-driven strategies, utilizing AI to analyze diverse data sources for predicting disruptions and developing resilience, thereby gaining a competitive advantage post-pandemic[3]. AI's role in synchronizing manufacturing and inventory planning was crucial for mitigating pandemic disruptions[4].

  3. Transformative Automation: The integration of AI and automation transformed logistics and supply chain operations, enabling fully automated, transparent supply chains vital for future global trade[5].

  1. AI Revolutionizing Operations: By 2025, AI is expected to continue revolutionizing supply chains, focusing on enhancing efficiency and resilience. Key trends include the utilization of generative AI for improved decision-making and demand prediction[6]. Supply chain leaders must navigate geopolitical uncertainties and technological changes while embracing predictive technologies[7].

  2. Focus on Sustainability: AI is poised to drive sustainability in supply chains through optimized route planning and predictive capabilities, leading to cost reductions and improved operational efficiency[8].

  3. Dynamic Inventory Management: AI will play a crucial role in addressing challenges such as geopolitical conflicts and resource scarcity by enabling dynamic inventory management and maximizing productivity[9].

Future Projections for AI in Supply Chain Management (2035)

  1. Hyperconnected Supply Chains: By 2035, supply chains are expected to be directed by artificial superintelligence, utilizing deep learning and swarm coordination algorithms. This will lead to decentralized networks supported by blockchain protocols, enhancing tracking and coordination[10].

  2. Transforming Business Models: Companies will convert their supply chains into value networks to drive growth, leveraging AI and machine learning as key enablers for competitive advantage[11].

  3. Mitigating Risks and Enhancing Resilience: Advanced technologies, including AI, will be essential in enhancing supply chain visibility and resilience, enabling organizations to respond quickly to disruptions and minimize inefficiencies[12].

Conclusion

The pandemic underscored the importance of AI in enhancing global trade efficiency, with trends indicating a continued reliance on AI technologies to bolster supply chain resilience, operational efficiency, and sustainability. As we look toward 2025 and beyond, businesses must adapt to these trends to remain competitive in an increasingly complex global trade environment.

Examine Final State and Display Answer

Once the pipeline completes, we retrieve the final state from the compiled graph. This includes the final Markdown-based answer with inline references and any metadata we collected along the way. The final result is displayed for clarity, concluding the entire process.

Impact of AI on Global Trade Efficiency During the Pandemic

Overview

The COVID-19 pandemic significantly disrupted global trade and supply chains, revealing vulnerabilities across industries. Artificial Intelligence (AI) emerged as a crucial tool for enhancing supply chain efficiency and resilience during this period.

Key Findings on AI's Impact During the Pandemic

  1. Enhancing Resilience: AI improved supply chain resilience by providing enhanced transparency and real-time insights, allowing companies to adapt quickly to disruptions, ensuring last-mile delivery, and minimizing the impact of supply chain interruptions. AI facilitated agile procurement strategies essential during the pandemic[1][2].

  2. Data-Driven Decision Making: Companies shifted from cost-driven approaches to data-driven strategies, utilizing AI to analyze diverse data sources for predicting disruptions and developing resilience, thereby gaining a competitive advantage post-pandemic[3]. AI's role in synchronizing manufacturing and inventory planning was crucial for mitigating pandemic disruptions[4].

  3. Transformative Automation: The integration of AI and automation transformed logistics and supply chain operations, enabling fully automated, transparent supply chains vital for future global trade[5].

  1. AI Revolutionizing Operations: By 2025, AI is expected to continue revolutionizing supply chains, focusing on enhancing efficiency and resilience. Key trends include the utilization of generative AI for improved decision-making and demand prediction[6]. Supply chain leaders must navigate geopolitical uncertainties and technological changes while embracing predictive technologies[7].

  2. Focus on Sustainability: AI is poised to drive sustainability in supply chains through optimized route planning and predictive capabilities, leading to cost reductions and improved operational efficiency[8].

  3. Dynamic Inventory Management: AI will play a crucial role in addressing challenges such as geopolitical conflicts and resource scarcity by enabling dynamic inventory management and maximizing productivity[9].

Future Projections for AI in Supply Chain Management (2035)

  1. Hyperconnected Supply Chains: By 2035, supply chains are expected to be directed by artificial superintelligence, utilizing deep learning and swarm coordination algorithms. This will lead to decentralized networks supported by blockchain protocols, enhancing tracking and coordination[10].

  2. Transforming Business Models: Companies will convert their supply chains into value networks to drive growth, leveraging AI and machine learning as key enablers for competitive advantage[11].

  3. Mitigating Risks and Enhancing Resilience: Advanced technologies, including AI, will be essential in enhancing supply chain visibility and resilience, enabling organizations to respond quickly to disruptions and minimize inefficiencies[12].

Conclusion

The pandemic underscored the importance of AI in enhancing global trade efficiency, with trends indicating a continued reliance on AI technologies to bolster supply chain resilience, operational efficiency, and sustainability. As we look toward 2025 and beyond, businesses must adapt to these trends to remain competitive in an increasingly complex global trade environment.

Last updated