llm-openai-langchain-integration

Reproducibility and Building an AI Agent by using lakeFS, LangChain and LLM/OpenAI Models

Start by ⭐️ starring lakeFS open source project.

This repository includes Jupyter Notebooks with LangChain and OpenAI libraries which you can run on your local machine.

Let's Get Started 👩🏻‍💻

Clone this repository

git clone https://github.com/treeverse/lakeFS-samples && cd lakeFS-samples/01_standalone_examples/llm-openai-langchain-integration

You now have two options:

Run a Notebook server with your existing lakeFS Server

If you have already installed lakeFS or are utilizing lakeFS cloud, all you need to run is the Jupyter notebook with LangChain and OpenAI libraries (Docker image size will be around 10GB):

docker compose up 

Don't have a lakeFS Server or Object Store?

If you want to provision a lakeFS server as well as MinIO for your object store, plus Jupyter with LangChain and OpenAI libraries then bring up the full stack:

docker compose --profile local-lakefs up

URLs and login details

• Jupyter http://localhost:8891/

If you've brought up the full stack you'll also have:

• LakeFS http://localhost:48000/ (AKIAIOSFOLKFSSAMPLES / wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY)
• MinIO http://localhost:49001/ (minioadmin/minioadmin)

Demo Instructions

Open Jupyter UI http://localhost:8891 in your web browser, open one of the provided notebooks from JupyterLab UI and follow the instructions:

1. AI Agent Demo: Build an AI Agent by using lakeFS, LangChain and OpenAI
2. LLM OpenAI LangChain Demo: Reproducibility and Data version control for LangChain and LLM/OpenAI Models