{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n", "\n", "# LLM Message History - Multiple Sessions\n", "\n", "Large Language Models are inherently stateless and have no knowledge of previous interactions with a user, or even of previous parts of the current conversation. The solution to this problem is to append the previous conversation history to each subsequent call to the LLM.\n", "This notebook will show how to use Redis to structure and store and retrieve this conversational message history and how to manage multiple conversation sessions simultaneously.\n", "\n", "## Let's Begin!\n", "\"Open\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Environment setup" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%pip install cohere \"redisvl>=0.6.0\" sentence-transformers" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Set Cohere API Key" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "import getpass\n", "\n", "\n", "if \"COHERE_API_KEY\" not in os.environ:\n", " os.environ[\"COHERE_API_KEY\"] = getpass.getpass(\"COHERE_API_KEY\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Run local redis (for colab)\n", "Use the shell script below to download, extract, and install [Redis Stack](https://redis.io/docs/getting-started/install-stack/) directly from the Redis package archive." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# NBVAL_SKIP\n", "%%sh\n", "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n", "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n", "sudo apt-get update > /dev/null 2>&1\n", "sudo apt-get install redis-stack-server > /dev/null 2>&1\n", "redis-stack-server --daemonize yes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### For Alternative Environments\n", "There are many ways to get the necessary redis-stack instance running\n", "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n", "own version of Redis Enterprise running, that works too!\n", "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n", "3. With docker: `docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define the Redis Connection URL and Client\n", "\n", "By default this notebook connects to the local instance of Redis Stack. **If you have your own Redis Enterprise instance** - replace REDIS_PASSWORD, REDIS_HOST and REDIS_PORT values with your own." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import os\n", "from redis import Redis\n", "\n", "# Replace values below with your own if using Redis Cloud instance\n", "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") # ex: \"redis-18374.c253.us-central1-1.gce.cloud.redislabs.com\"\n", "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\") # ex: 18374\n", "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\") # ex: \"1TNxTEdYRDgIDKM2gDfasupCADXXXX\"\n", "\n", "# If SSL is enabled on the endpoint, use rediss:// as the URL prefix\n", "REDIS_URL = f\"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}\"\n", "\n", "redis_client = Redis.from_url(REDIS_URL)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Initialize CohereClient as LLM layer\n", "To demonstrate how a real LLM conversation may flow we'll use a real LLM." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from typing import Dict, List\n", "import cohere\n", "import os\n", "\n", "class CohereClient():\n", " def __init__(self, api_key: str = None, model: str = 'command-r-plus-08-2024'):\n", " api_key = api_key or os.getenv(\"COHERE_API_KEY\")\n", " self.client = cohere.Client(api_key)\n", " self._model = model\n", "\n", " def converse(self, prompt: str, context: List[Dict]) -> str:\n", " context = self.remap(context)\n", " response = self.client.chat(\n", " model=self._model,\n", " chat_history = context,\n", " message=prompt,\n", " )\n", " return response.text\n", "\n", " def remap(self, context) -> List[Dict]:\n", " ''' re-index the message history to match the Cohere API requirements '''\n", " new_context = []\n", " for statement in context:\n", " if statement[\"role\"] == \"user\":\n", " new_statement = {\"role\": \"USER\", \"message\": statement[\"content\"]}\n", " elif statement[\"role\"] == \"llm\":\n", " new_statement = {\"role\": \"CHATBOT\", \"message\": statement[\"content\"]}\n", " elif statement[\"role\"] == \"system\":\n", " new_statement = {\"role\": \"SYSTEM\", \"message\": statement[\"content\"]}\n", " else:\n", " raise ValueError(f'Unknown chat role {statement[\"role\"]}')\n", " new_context.append(new_statement)\n", " return new_context\n", "\n", "client = CohereClient()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import SemanticMessageHistory\n", "\n", "redisvl provides the SemanticMessageHistory for easy management of conversational message history state.\n", "It also allows for tagging of messages to separate conversation sessions with the `session_tag` optional parameter.\n", "Let's create a few personas that can talk to our AI.\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "student = 'student'\n", "yp = 'young professional'\n", "retired = 'retired pensioner'" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from redisvl.extensions.message_history import SemanticMessageHistory\n", "\n", "history = SemanticMessageHistory(name='budgeting help')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Here we'll have multiple separate conversations simultaneously, all using the same message history object.\n", "#### Let's add some conversation history to get started.\n", "\n", "#### We'll assign each message to one of our users with their own `session_tag`." ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "# adding messages to the student session\n", "history.add_messages(\n", " [{\"role\":\"system\",\n", " \"content\":\"You are a personal assistant helping people create sound financial budgets. Be very brief and concise in your responses.\"},\n", " {\"role\":\"user\",\n", " \"content\":\"I'm a college student living in Montana and I need help creating a budget. I am a first year accounting student.\"},\n", " {\"role\":\"llm\",\n", " \"content\":\"Sure, I can help you with that. What is your monthly income and average monthly expenses?\"},\n", " {\"role\":\"user\",\n", " \"content\":\"my rent is $500, utilities are $100, and I spend $200 on groceries. I make $1000 a month as a part time tutor.\"},\n", " ],\n", " session_tag=student)\n", "\n", "#adding messages to the young professional session\n", "history.add_messages(\n", " [{\"role\":\"system\",\n", " \"content\":\"You are a personal assistant helping people create sound financial budgets. Be very brief and concise in your responses.\"},\n", " {\"role\":\"user\",\n", " \"content\":\"I'm a young professional living in New York City and I need help planning for retirement. I already have a sizable emergency fund.\"},\n", " {\"role\":\"llm\",\n", " \"content\":\"Sure I can help you with that. What is your monthly income and average monthly expenses?\"},\n", " {\"role\":\"user\",\n", " \"content\":\"I make $5000 a month as a software engineer. My rent is $2000, utilities are $200, groceries are $300, and I spend $500 on entertainment.\"},\n", " ],\n", " session_tag=yp)\n", "\n", "#adding messages to the retiree session\n", "history.add_messages(\n", " [{\"role\":\"system\",\n", " \"content\":\"You are a personal assistant helping people create sound financial budgets. Be very brief and concise in your responses.\"},\n", " {\"role\":\"user\",\n", " \"content\":\"I'm a retired pensioner living in Florida and I need help creating a budget.\"},\n", " {\"role\":\"llm\",\n", " \"content\":\"Sure I can help you with that. What is your monthly income and average monthly expenses?\"},\n", " {\"role\":\"user\",\n", " \"content\":\"I make $2000 a month from my pension. I own my home outright, utilities are $100, groceries are $200, and I spend $100 on entertainment.\"},\n", " ],\n", " session_tag=retired)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### With the same message history instance and calling the same LLM we can handle distinct conversations. There's no need to instantiate separate classes or clients.\n", "\n", "#### Just retrieve the conversation of interest using the same `session_tag` parameter when fetching context." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Student: What is the single most important thing I should focus on financially?\n", "\n", "LLM: Focus on keeping expenses low.\n" ] } ], "source": [ "prompt = \"What is the single most important thing I should focus on financially?\"\n", "context = history.get_recent(session_tag=student)\n", "response = client.converse(prompt=prompt, context=context)\n", "history.store(prompt, response, session_tag=student)\n", "print('Student: ', prompt)\n", "print('\\nLLM: ', response)" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Young Professional: What is the single most important thing I should focus on financially?\n", "\n", "LLM: Max out your 401(k) contributions to take advantage of compound interest and any employer matching.\n" ] } ], "source": [ "prompt = \"What is the single most important thing I should focus on financially?\"\n", "context = history.get_recent(session_tag=yp)\n", "response = client.converse(prompt=prompt, context=context)\n", "history.store(prompt, response, session_tag=yp)\n", "print('Young Professional: ', prompt)\n", "print('\\nLLM: ', response)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Retiree: What is the single most important thing I should focus on financially?\n", "\n", "LLM: With your current income and expenses, your focus should be on maintaining your current financial situation and ensuring your long-term financial stability. Review your budget regularly and adjust as necessary.\n" ] } ], "source": [ "prompt = \"What is the single most important thing I should focus on financially?\"\n", "context = history.get_recent(session_tag=retired)\n", "response = client.converse(prompt=prompt, context=context)\n", "history.store(prompt, response, session_tag=retired)\n", "print('Retiree: ', prompt)\n", "print('\\nLLM: ', response)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### You can see how each conversation is stored separately." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'role': 'user', 'content': \"I'm a college student living in Montana and I need help creating a budget. I am a first year accounting student.\"}\n", "{'role': 'llm', 'content': 'Sure, I can help you with that. What is your monthly income and average monthly expenses?'}\n", "{'role': 'user', 'content': 'my rent is $500, utilities are $100, and I spend $200 on groceries. I make $1000 a month as a part time tutor.'}\n", "{'role': 'user', 'content': 'What is the single most important thing I should focus on financially?'}\n", "{'role': 'llm', 'content': 'Focus on keeping expenses low.'}\n" ] } ], "source": [ "for ctx in history.get_recent(session_tag=student):\n", " print(ctx)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "history.clear()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 2 }