This tutorial explores the use of OpenAI Text embedding models within the LangChain framework.
It showcases how to generate embeddings for text queries and documents, reduce their dimensionality using PCA , and visualize them in 2D for better interpretability.
By analyzing relationships between the query and documents through cosine similarity, it provides insights into how embeddings can enhance workflows, including text analysis and data visualization.
[Note] If you are using a .env file, proceed as follows.
from dotenv import load_dotenv
load_dotenv(override=True)
True
Load model and set dimension
Describes the Embedding model and dimension settings supported by OpenAI.
Why Adjust Embedding Dimensions?
Optimize Resources : Shortened embeddings use less memory and compute.
Flexible Usage : Models like text-embedding-3-large allow size reduction with the dimensions API.
Key Insight : Even at 256 dimensions, performance can surpass larger models like text-embedding-ada-002.
This is a description of the models supported by OpenAI
Model
~ Pages per Dollar
Performance on MTEB Eval
Max Input
Available dimension
text-embedding-3-small
62,500
62.3%
8191
512, 1536
text-embedding-3-large
9,615
64.6%
8191
256, 1024, 3072
text-embedding-ada-002
12,500
61.0%
8191
1536
"Initialize and utilize OpenAI embedding models using langchain_openai package."
from langchain_openai import OpenAIEmbeddings
# Set desired model
openai_embedding = OpenAIEmbeddings(model="text-embedding-3-large")
[note] If dimension reduction is necessary, please set as below.
from langchain_openai import OpenAIEmbeddings
# Set desired model and dimension
openai_embedding = OpenAIEmbeddings(model="text-embedding-3-large", dimensions=1024)
This code calculates the similarity between the query and the document through Cosine Similarity .
Find the documents similar (top 3) and (bottom 3) .
from sklearn.metrics.pairwise import cosine_similarity
# Calculate Cosine Similarity
similarity = cosine_similarity([query_vector], docs_vector)
# Sorting by in descending order
sorted_idx = similarity.argsort()[0][::-1]
# Display top 3 and bottom 3 documents based on similarity
print("Top 3 most similar document:")
for i in range(0, 3):
print(
f"[{i+1}] similarity: {similarity[0][sorted_idx[i]]:.3f} | {documents[sorted_idx[i]]}"
)
print("\nBottom 3 least similar documents:")
for i in range(1, 4):
print(
f"[{i}] similarity: {similarity[0][sorted_idx[-i]]:.3f} | {documents[sorted_idx[-i]]}"
)
Top 3 most similar document:
[1] similarity: 0.514 | text-embedding-3-large
[2] similarity: 0.467 | text-embedding-ada-002
[3] similarity: 0.457 | text-embedding-3-small
Bottom 3 least similar documents:
[1] similarity: 0.050 | facebook/bart-large
[2] similarity: 0.143 | multilingual-e5-base
[3] similarity: 0.171 | all-mpnet-base-v2
Embeddings visualization(PCA)
Reduce the dimensionality of the embeddings for visualization purposes.
This code uses principal component analysis (PCA) to reduce high-dimensional embedding vectors to two dimensions.
The resulting 2D points are displayed in a scatterplot, with each point labeled for its corresponding document.
Why Dimension Reduction?
High-dimensional embedding vectors are challenging to interpret and analyze directly. By reducing them to 2D, we can:
Visually explore relationships between embeddings (e.g., clustering, grouping).
Identify patterns or anomalies in the data that may not be obvious in high dimensions.
Improve interpretability , making the data more accessible for human analysis and decision-making.
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
import numpy as np
# Combine documents and query for PCA
all_vectors = np.vstack([docs_vector, query_vector]) # Stack query vector with docs
pca = PCA(n_components=2)
reduced_vectors = pca.fit_transform(all_vectors)
# Separate reduced vectors for documents and query
doc_vectors_2d = reduced_vectors[:-1] # All but the last point (documents)
query_vector_2d = reduced_vectors[-1] # Last point (query)
# Plot the reduced vectors
plt.scatter(doc_vectors_2d[:, 0], doc_vectors_2d[:, 1], color="blue", label="Documents")
plt.scatter(
query_vector_2d[0],
query_vector_2d[1],
color="red",
label="Query",
marker="x",
s=300,
)
# Annotate document points
for i, doc in enumerate(documents):
plt.text(doc_vectors_2d[i, 0], doc_vectors_2d[i, 1], doc, fontsize=8)
# Add plot details
plt.title("2D Visualization of Embedding Vectors with Query")
plt.xlabel("PCA Dimension 1")
plt.ylabel("PCA Dimension 2")
plt.legend()
plt.show()
