{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n",
    "\n",
    "# Optimize semantic cache threshold with RedisVL\n",
    "\n",
    "> **Note:** Threshold optimization with redis-retrieval-optimizer relies on `python > 3.9.`\n",
    "\n",
    "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/semantic-cache/02_semantic_cache_optimization.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Install dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install \"redisvl>=0.6.0\" \"redis-retrieval-optimizer>=0.2.0\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run a Redis instance\n",
    "\n",
    "#### For Colab\n",
    "Use the shell script below to download, extract, and install [Redis Stack](https://redis.io/docs/getting-started/install-stack/) directly from the Redis package archive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "%%sh\n",
    "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
    "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
    "sudo apt-get update  > /dev/null 2>&1\n",
    "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
    "redis-stack-server --daemonize yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### For Alternative Environments\n",
    "There are many ways to get the necessary redis-stack instance running\n",
    "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n",
    "own version of Redis Enterprise running, that works too!\n",
    "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n",
    "3. With docker: `docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# CacheThresholdOptimizer\n",
    "\n",
    "Let's say you setup the following semantic cache with a distance_threshold of `X` and store the entries:\n",
    "\n",
    "- prompt: `what is the capital of france?` response: `paris`\n",
    "- prompt: `what is the capital of morocco?` response: `rabat`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13:32:11 [RedisVL] WARNING   The default vectorizer has changed from `sentence-transformers/all-mpnet-base-v2` to `redis/langcache-embed-v1` in version 0.6.0 of RedisVL. For more information about this model, please refer to https://arxiv.org/abs/2504.02268 or visit https://huggingface.co/redis/langcache-embed-v1. To continue using the old vectorizer, please specify it explicitly in the constructor as: vectorizer=HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2')\n",
      "13:32:11 sentence_transformers.SentenceTransformer INFO   Use pytorch device_name: mps\n",
      "13:32:11 sentence_transformers.SentenceTransformer INFO   Load pretrained SentenceTransformer: redis/langcache-embed-v1\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "abd298f873404faba441d8be98e2c9de",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13:32:12 [RedisVL] WARNING   The default vectorizer has changed from `sentence-transformers/all-mpnet-base-v2` to `redis/langcache-embed-v1` in version 0.6.0 of RedisVL. For more information about this model, please refer to https://arxiv.org/abs/2504.02268 or visit https://huggingface.co/redis/langcache-embed-v1. To continue using the old vectorizer, please specify it explicitly in the constructor as: vectorizer=HFTextVectorizer(model='sentence-transformers/all-mpnet-base-v2')\n",
      "13:32:12 redisvl.index.index INFO   Index already exists, not overwriting.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0ba36412f3b84ac1be07ef0498423cc3",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "dbfe66f171584b81b9c56eb432d52a64",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from redisvl.extensions.cache.llm import SemanticCache\n",
    "\n",
    "sem_cache = SemanticCache(\n",
    "    name=\"sem_cache\",                    # underlying search index name\n",
    "    redis_url=\"redis://localhost:6379\",  # redis connection url string\n",
    "    distance_threshold=0.5               # semantic cache distance threshold\n",
    ")\n",
    "\n",
    "paris_key = sem_cache.store(prompt=\"what is the capital of france?\", response=\"paris\")\n",
    "rabat_key = sem_cache.store(prompt=\"what is the capital of morocco?\", response=\"rabat\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This works well but we want to make sure the cache only applies for the appropriate questions. If we test the cache with a question we don't want a response to we see that the current distance_threshold is too high. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e9c0bb9ee54648b7b57289013311bf34",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'entry_id': 'c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3',\n",
       "  'prompt': 'what is the capital of france?',\n",
       "  'response': 'paris',\n",
       "  'vector_distance': 0.335606694221,\n",
       "  'inserted_at': 1750699932.92,\n",
       "  'updated_at': 1750699932.92,\n",
       "  'key': 'sem_cache:c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3'}]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sem_cache.check(\"what's the capital of britain?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define test_data and optimize\n",
    "\n",
    "With the `CacheThresholdOptimizer` you can quickly tune the distance threshold by providing some test data in the form:\n",
    "\n",
    "```json\n",
    "[\n",
    "    {\n",
    "        \"query\": \"What's the capital of Britain?\",\n",
    "        \"query_match\": \"\"\n",
    "    },\n",
    "    {\n",
    "        \"query\": \"What's the capital of France??\",\n",
    "        \"query_match\": paris_key\n",
    "    },\n",
    "    {\n",
    "        \"query\": \"What's the capital city of Morocco?\",\n",
    "        \"query_match\": rabat_key\n",
    "    },\n",
    "]\n",
    "```\n",
    "\n",
    "The threshold optimizer will then efficiently execute and score different threshold against the what is currently populated in your cache and automatically update the threshold of the cache to the best setting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance threshold before: 0.5 \n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8aa7076ac42047c480c75e04a0328f35",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d3189afcfb384a4faaa0ed723bb7140b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b2a7eca240314e0d96443d85d5398699",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance threshold after: 0.10372881355932204 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "from redis_retrieval_optimizer.threshold_optimization import CacheThresholdOptimizer\n",
    "\n",
    "test_data = [\n",
    "    {\n",
    "        \"query\": \"What's the capital of Britain?\",\n",
    "        \"query_match\": \"\"\n",
    "    },\n",
    "    {\n",
    "        \"query\": \"What's the capital of France??\",\n",
    "        \"query_match\": paris_key\n",
    "    },\n",
    "    {\n",
    "        \"query\": \"What's the capital city of Morocco?\",\n",
    "        \"query_match\": rabat_key\n",
    "    },\n",
    "]\n",
    "\n",
    "print(f\"Distance threshold before: {sem_cache.distance_threshold} \\n\")\n",
    "optimizer = CacheThresholdOptimizer(sem_cache, test_data)\n",
    "optimizer.optimize()\n",
    "print(f\"Distance threshold after: {sem_cache.distance_threshold} \\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also see that we no longer match on the incorrect example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "36810080430f460781990f284c0e9091",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sem_cache.check(\"what's the capital of britain?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But still match on highly relevant prompts:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e012dfe276f44869812df0325da5f114",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "[{'entry_id': 'c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3',\n",
       "  'prompt': 'what is the capital of france?',\n",
       "  'response': 'paris',\n",
       "  'vector_distance': 0.0431383252144,\n",
       "  'inserted_at': 1750699932.92,\n",
       "  'updated_at': 1750699932.92,\n",
       "  'key': 'sem_cache:c990cc06e5e77570e5f03360426d2b7f947cbb5a67daa8af8164bfe0b3e24fe3'}]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sem_cache.check(\"what's the capital city of france?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Additional configuration\n",
    "\n",
    "By default threshold optimization is performed based on the highest `F1` score but can also be configured to rank results based on `precision` and `recall` by specifying the `eval_metric` keyword argument. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance threshold before: 0.10372881355932204 \n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "38e31141494048c79c809a6e096442ae",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6a79e76ec32c48dd807b8bf7d6791c31",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d209bcaa8ec54657824622fb6a0781ed",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance threshold after: 0.10372881355932204 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"Distance threshold before: {sem_cache.distance_threshold} \\n\")\n",
    "optimizer = CacheThresholdOptimizer(sem_cache, test_data, eval_metric=\"precision\")\n",
    "optimizer.optimize()\n",
    "print(f\"Distance threshold after: {sem_cache.distance_threshold} \\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance threshold before: 0.10372881355932204 \n",
      "\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "601b75e2c33a4d158ff69ab371e22b9d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c3ad0740353d4525b3d156f75525bfc5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "348d1826b2b543b496ba2f268ebe2280",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Batches:   0%|          | 0/1 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Distance threshold after: 0.10372881355932204 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"Distance threshold before: {sem_cache.distance_threshold} \\n\")\n",
    "optimizer = CacheThresholdOptimizer(sem_cache, test_data, eval_metric=\"recall\")\n",
    "optimizer.optimize()\n",
    "print(f\"Distance threshold after: {sem_cache.distance_threshold} \\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: the CacheThresholdOptimizer class also exposes an optional `opt_fn` which can be leveraged to define more custom logic. See implementation within [source code for reference](https://github.com/redis/redis-vl-python/blob/18ff1008c5a40353c97c176d3d30028a87ff777a/redisvl/utils/optimize/cache.py#L48-L49)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleanup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "sem_cache.delete()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "3.11.9",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}