Model Serialization

Author: Mark
Peer Review : Jaemin Hong, Dooil Kwak
Proofread : Two-Jay
This is a part of LangChain Open Tutorial

Overview

Serialization is the process of converting an object into a format that can be easily stored, shared, or transmitted, and later reconstructed. In the LangChain framework, classes implement standard methods for serialization, providing several advantages:

Separation of Secrets: Sensitive information, such as API keys, is separated from other parameters and can be securely reloaded into the object during deserialization.
Version Compatibility: Deserialization remains compatible across different package versions, ensuring that objects serialized with one version of LangChain can be properly deserialized with another.

All LangChain objects inheriting from Serializable are JSON-serializable, including messages, document objects (e.g., those returned from retrievers), and most Runnables such as chat models, retrievers, and chains implemented with the LangChain Expression Language.

Saving and Loading LangChain Objects

To effectively manage LangChain objects, you can serialize and deserialize them using the following functions:

dumpd: Returns a dictionary representation of an object, suitable for JSON serialization.
dumps: Returns a JSON string representation of an object.
load: Reconstructs an object from its dictionary representation.
loads: Reconstructs an object from its JSON string representation.

References

Environment Setup

Set up the environment. You may refer to Environment Setup for more details.

[Note]

langchain-opentutorial is a package that provides a set of easy-to-use environment setup, useful functions and utilities for tutorials.
You can checkout the langchain-opentutorial for more details.

%%capture --no-stderr
%pip install langchain-opentutorial

# Install required packages
from langchain_opentutorial import package

package.install(
    [
        "langsmith",
        "langchain",
        "langchain_core",
        "langchain_openai",
    ],
    verbose=False,
    upgrade=False,
)

# Set environment variables
from langchain_opentutorial import set_env

set_env(
    {
        "OPENAI_API_KEY": "Your API KEY",
        "LANGCHAIN_API_KEY": "",
        "LANGCHAIN_TRACING_V2": "true",
        "LANGCHAIN_ENDPOINT": "https://api.smith.langchain.com",
        "LANGCHAIN_PROJECT": "Caching",
    }
)

Environment variables have been set successfully.

# Alternatively, one can set environmental variables with load_dotenv
from dotenv import load_dotenv


load_dotenv(override=True)

True

from langchain_openai import ChatOpenAI
from langchain_core.prompts import PromptTemplate

# Create model
llm = ChatOpenAI(model_name="gpt-4o-mini")

# Generate prompt
prompt = PromptTemplate.from_template(
    "Sumarize about the {country} in about 200 characters"
)

# Create chain
chain = prompt | llm

`Dumps` and `Loads`

dumps : LangChain object into a JSON-formatted string
loads : JSON-formatted string into a LangChain object

from langchain_core.load.dump import dumps

# Serialize LangChain object to JSON like string

serialized_llm = dumps(llm, pretty=True)
print(serialized_llm)
print(type(serialized_llm))

serialized_prompt = dumps(prompt)
print(serialized_prompt[:100] + " ...")
print(type(serialized_prompt))

serialized_chain = dumps(chain)
print(serialized_chain[:100] + " ...")
print(type(serialized_chain))

{
      "lc": 1,
      "type": "constructor",
      "id": [
        "langchain",
        "chat_models",
        "openai",
        "ChatOpenAI"
      ],
      "kwargs": {
        "model_name": "gpt-4o-mini",
        "temperature": 0.7,
        "openai_api_key": {
          "lc": 1,
          "type": "secret",
          "id": [
            "OPENAI_API_KEY"
          ]
        },
        "max_retries": 2,
        "n": 1
      },
      "name": "ChatOpenAI"
    }
    
    {"lc": 1, "type": "constructor", "id": ["langchain", "prompts", "prompt", "PromptTemplate"], "kwargs ...
    
    {"lc": 1, "type": "constructor", "id": ["langchain", "schema", "runnable", "RunnableSequence"], "kwa ...

from langchain_core.load.load import loads

# Deserialize JSON like string to LangChain object

deserialized_llm = loads(serialized_llm)
print(type(deserialized_llm))

deserialized_prompt = loads(serialized_prompt)
print(type(deserialized_prompt))

deserialized_chain = loads(serialized_chain)
print(type(deserialized_chain))

/var/folders/q_/52ctm0y10h589cwbbptjsvrw0000gp/T/ipykernel_82468/2405148250.py:5: LangChainBetaWarning: The function `loads` is in beta. It is actively being worked on, so the API may change.
  deserialized_llm = loads(serialized_llm)

# Invoke chains

response = chain.invoke({"country": "South Korea"})
print(response.content)

deserialized_response = deserialized_chain.invoke({"country": "South Korea"})
print(deserialized_response.content)

deserialized_response_composed = (deserialized_prompt | deserialized_llm).invoke(
    {"country": "South Korea"}
)
print(deserialized_response_composed.content)

South Korea, located on the Korean Peninsula, is known for its vibrant culture, advanced technology, and rich history. Major cities include Seoul and Busan, and it has a strong economy and global influence.
    South Korea, located on the Korean Peninsula, is known for its vibrant culture, technological advancements, and dynamic economy. Seoul is its capital, and the nation is famous for K-pop, cuisine, and rich history.
    South Korea, located on the Korean Peninsula, is a vibrant democracy known for its advanced technology, rich culture, and K-pop music. It's a global leader in innovation and economic development.

`Dumpd` and `Load`

dumpd : LangChain object into a dictionary
load : dictionary into a LangChain object

from langchain_core.load.dump import dumpd

# Serialize LangChain object to dictionary

serialized_llm = dumpd(llm)
print(type(serialized_llm))

serialized_prompt = dumpd(prompt)
print(type(serialized_prompt))

serialized_chain = dumpd(chain)
print(type(serialized_chain))

from langchain_core.load.load import load

# Deserialize dictionary to LangChain object

deserialized_llm = load(serialized_llm)
print(type(deserialized_llm))

deserialized_prompt = load(serialized_prompt)
print(type(deserialized_prompt))

deserialized_chain = load(serialized_chain)
print(type(deserialized_chain))

/var/folders/q_/52ctm0y10h589cwbbptjsvrw0000gp/T/ipykernel_82468/4209275167.py:5: LangChainBetaWarning: The function `load` is in beta. It is actively being worked on, so the API may change.
  deserialized_llm = load(serialized_llm)

# Invoke chains

response = chain.invoke({"country": "South Korea"})
print(response.content)

deserialized_response = deserialized_chain.invoke({"country": "South Korea"})
print(deserialized_response.content)

deserialized_response_composed = (deserialized_prompt | deserialized_llm).invoke(
    {"country": "South Korea"}
)
print(deserialized_response_composed.content)

South Korea, located on the Korean Peninsula, is known for its vibrant culture, advanced technology, and K-pop music. Its capital, Seoul, is a bustling metropolis blending tradition and modernity.
    South Korea is a vibrant East Asian nation known for its technological advancements, rich culture, K-pop, and delicious cuisine. It has a strong economy and a unique blend of tradition and modernity.
    South Korea, located on the Korean Peninsula, is known for its vibrant culture, advanced technology, and economic strength. Major cities include Seoul and Busan. It has a rich history and a strong global presence.

Serialization with `pickle`

The pickle module in Python is used for serializing and deserializing Python object structures, also known as pickling and unpickling. Serialization involves converting a Python object hierarchy into a byte stream, while deserialization reconstructs the object hierarchy from the byte stream.

pickle - Python object serialization for more details

Key Functions

pickle.dump(obj, file): Serializes obj and writes it to the open file object file.
pickle.load(file): Reads a byte stream from the open file object file and deserializes it back into a Python object.

from langchain_core.load.dump import dumpd

# Serialize LangChain object to dictionary

serialized_llm = dumpd(llm)
print(type(serialized_llm))

serialized_prompt = dumpd(prompt)
print(type(serialized_prompt))

serialized_chain = dumpd(chain)
print(type(serialized_chain))

import pickle
import os

# Serialize dictionary to pickle file

os.makedirs("data", exist_ok=True)

with open("data/serialized_llm.pkl", "wb") as f:
    pickle.dump(serialized_llm, f)

with open("data/serialized_prompt.pkl", "wb") as f:
    pickle.dump(serialized_prompt, f)

with open("data/serialized_chain.pkl", "wb") as f:
    pickle.dump(serialized_chain, f)

# Deserialize pickle file to dictionary

with open("data/serialized_llm.pkl", "rb") as f:
    loaded_llm = pickle.load(f)
    print(type(loaded_llm))

with open("data/serialized_prompt.pkl", "rb") as f:
    loaded_prompt = pickle.load(f)
    print(type(loaded_prompt))

with open("data/serialized_chain.pkl", "rb") as f:
    loaded_chain = pickle.load(f)
    print(type(loaded_chain))

from langchain_core.load.load import load

# Deserialize dictionary to LangChain object

deserialized_llm = load(serialized_llm)
print(type(deserialized_llm))

deserialized_prompt = load(serialized_prompt)
print(type(deserialized_prompt))

deserialized_chain = load(serialized_chain)
print(type(deserialized_chain))

# Invoke chains

response = chain.invoke({"country": "South Korea"})
print(response.content)

deserialized_response = deserialized_chain.invoke({"country": "South Korea"})
print(deserialized_response.content)

deserialized_response_composed = (deserialized_prompt | deserialized_llm).invoke(
    {"country": "South Korea"}
)
print(deserialized_response_composed.content)

South Korea, located on the Korean Peninsula, is known for its technological advancements, rich culture, and history. Major cities like Seoul blend modernity with tradition, while K-pop and cuisine gain global popularity.
    South Korea, located on the Korean Peninsula, is known for its advanced technology, vibrant culture, and rich history. It features a dynamic economy, popular K-pop music, and delicious cuisine.
    South Korea is a vibrant East Asian nation known for its advanced technology, rich culture, and historical landmarks. It's famous for K-pop, delicious cuisine, and significant economic growth post-war.

Is Every Runnable Serializable?

LangChain's dumps and dumpd methods attempt to serialize objects as much as possible, but the resulting data may be incomplete.

Even if the is_lc_serializable method does not exist or returns False, a result is still generated.
Even if the is_lc_serializable method returns True and serialization is successful, deserialization may fail.

After serialization, it is essential to check if the JSON data contains "type": "not_implemented". Only then can the load or loads functions be used safely.

from langchain_core.output_parsers.string import StrOutputParser
from langchain_core.runnables import RunnableLambda
from langchain_core.load.dump import dumps
from langchain_core.load.load import loads


def custom_function(llm_response):
    return llm_response.content


# Define chains that make same results
chain_with_custom_function = chain | custom_function
print(type(chain_with_custom_function))
chain_with_str_output_parser = chain | StrOutputParser()
print(type(chain_with_str_output_parser))

response = chain_with_custom_function.invoke({"country": "South Korea"})
print(response)

response = chain_with_str_output_parser.invoke({"country": "South Korea"})
print(response)


    
    South Korea, located in East Asia, is known for its rich culture, advanced technology, and vibrant economy. It features bustling cities, traditional cuisine, and global influence through K-pop and cinema.
    South Korea, located in East Asia, is known for its advanced technology, rich culture, and vibrant economy. It's famous for K-pop, cuisine, and historical sites, blending tradition with modernity.

# Both of them are serializable
print(chain_with_custom_function.is_lc_serializable())
print(chain_with_str_output_parser.is_lc_serializable())

True
    True

try:
    print(
        "...\n"
        # You can see that the serialized string contains "type": "not_implemented"
        + ((serialized_str := dumps(chain_with_custom_function, pretty=True)))[-270:]
    )
    # First one fail to deserialize
    loads(serialized_str)
except Exception as e:
    print("Error : \n", e)

print(type(deserialized_chain := loads(dumps(chain_with_str_output_parser))))
print(deserialized_chain.invoke({"country": "South Korea"}))

...
    
        ],
        "last": {
          "lc": 1,
          "type": "not_implemented",
          "id": [
            "langchain_core",
            "runnables",
            "base",
            "RunnableLambda"
          ],
          "repr": "RunnableLambda(custom_function)"
        }
      },
      "name": "RunnableSequence"
    }
    Error : 
     Trying to load an object that doesn't implement serialization: {'lc': 1, 'type': 'not_implemented', 'id': ['langchain_core', 'runnables', 'base', 'RunnableLambda'], 'repr': 'RunnableLambda(custom_function)'}
    
    South Korea, located on the Korean Peninsula, is known for its advanced technology, rich culture, and vibrant economy. It has a democratic government and is famous for K-pop, cuisine, and historical sites.

# RunnableLambda and custom_function has no is_lc_serializable method
# But they are serializable

try:
    print(RunnableLambda(custom_function).is_lc_serializable())
except Exception as e:
    print("Error : \n", e)

print(dumps(RunnableLambda(custom_function), pretty=True))

try:
    print(custom_function.is_lc_serializable())
except Exception as e:
    print("Error : \n", e)

print(dumps(custom_function, pretty=True))

Error : 
     'RunnableLambda' object has no attribute 'is_lc_serializable'
    {
      "lc": 1,
      "type": "not_implemented",
      "id": [
        "langchain_core",
        "runnables",
        "base",
        "RunnableLambda"
      ],
      "repr": "RunnableLambda(custom_function)"
    }
    Error : 
     'function' object has no attribute 'is_lc_serializable'
    {
      "lc": 1,
      "type": "not_implemented",
      "id": [
        "__main__",
        "custom_function"
      ],
      "repr": ""
    }

from langchain_core.load.serializable import Serializable

# Serializable has is_lc_serializable method
# But it returns False
print(Serializable.is_lc_serializable())

# But also it is serializable
print(dumps(Serializable, pretty=True))
print(dumpd(Serializable))

False
    {
      "lc": 1,
      "type": "not_implemented",
      "id": [
        "langchain_core",
        "load",
        "serializable",
        "Serializable"
      ],
      "repr": ""
    }
    {'lc': 1, 'type': 'not_implemented', 'id': ['langchain_core', 'load', 'serializable', 'Serializable'], 'repr': ""}

PreviousCaching VLLM NextCheck Token Usage

Last updated 1 month ago

Overview

Saving and Loading LangChain Objects

Table of Contents