{
 "cells": [
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "# Semantic Caching\n",
    "\n",
    "Semantic caching is an intelligent caching strategy that stores and retrieves responses based on the meaning of queries rather than exact text matches. Unlike traditional caching that requires identical strings, semantic caching can return cached responses for questions that are semantically similar, even when phrased differently.\n",
    "\n",
    "## Semantic Caching vs. Traditional Caching vs. LLM Re-generation\n",
    "\n",
    "**Traditional caching** stores responses using exact query strings as keys:\n",
    "- **Fast retrieval** for identical queries\n",
    "- **Cache misses** for any variation in phrasing, even minor differences\n",
    "- **Low cache hit rates** in conversational applications where users rarely phrase questions identically\n",
    "\n",
    "**LLM re-generation** involves calling the language model for every query:\n",
    "- **Flexible** handling of any question variation\n",
    "- **High API costs** and latency for repeated similar questions\n",
    "\n",
    "**Semantic caching** uses vector similarity to match queries with cached responses:\n",
    "- **High cache hit rates** by matching semantically similar questions\n",
    "- **Cost reduction** by avoiding redundant LLM calls for similar queries\n",
    "- **Fast retrieval** through vector similarity search\n",
    "\n",
    "In this notebook, we'll implement semantic caching using RedisVL with pre-generated FAQs about a Chevrolet Colorado vehicle brochure, demonstrating how semantic similarity can dramatically improve cache hit rates compared to exact string matching."
   ]
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Running Redis\n",
    "\n",
    "There are several options one can follow to have a running instance of Redis. For the sake of simplicity, in this notebook, we will run it in a Docker container.\n",
    "\n",
    "For production where high-availability and reliability is a concern, we recommend using [Redis Cloud](https://cloud.redis.io/).\n",
    "\n",
    "A free database can be spun up in Redis Cloud."
   ]
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Running Redis in a Docker Container using TestContainers\n",
    "\n",
    "**Docker containers** are lightweight, portable environments that package an application and all its dependencies so it runs consistently across different systems. **Testcontainers** is a library that lets us run lightweight, disposable Docker containers for integration testing, so you can test against real services like databases or message queues without complex setup.\n",
    "\n",
    "Make sure you have Docker installed: [install Docker](https://www.docker.com/get-started/)."
   ]
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "#### Installing dependencies"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:22.540393Z",
     "start_time": "2025-11-24T10:25:22.227016Z"
    }
   },
   "cell_type": "code",
   "source": "@file:DependsOn(\"org.testcontainers:testcontainers:2.0.2\")",
   "outputs": [],
   "execution_count": 1
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "#### Configuring a generic Redis Container"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:24.296117Z",
     "start_time": "2025-11-24T10:25:23.981290Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import org.testcontainers.containers.GenericContainer\n",
    "import org.testcontainers.utility.DockerImageName\n",
    "\n",
    "class RedisContainer : GenericContainer<RedisContainer>(DockerImageName.parse(\"redis:latest\")) {\n",
    "    init {\n",
    "        withExposedPorts(6379)\n",
    "    }\n",
    "}"
   ],
   "outputs": [],
   "execution_count": 2
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "#### Creating a Docker network\n",
    "\n",
    "This is necessary because later on this notebook we will spin up a Redis Insight container that needs to be in the same network."
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:26.133813Z",
     "start_time": "2025-11-24T10:25:26.081066Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import org.testcontainers.containers.Network\n",
    "\n",
    "val network = Network.newNetwork()\n",
    "val networkAlias = \"redis-network\""
   ],
   "outputs": [],
   "execution_count": 3
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "#### Start a Redis Container"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:29.231779Z",
     "start_time": "2025-11-24T10:25:27.519794Z"
    }
   },
   "cell_type": "code",
   "source": [
    "val networkAlias = \"redis\"\n",
    "val redis = RedisContainer().withNetwork(network).withNetworkAliases(networkAlias)\n",
    "redis.start()\n",
    "\n",
    "val host = redis.host\n",
    "val port = redis.getMappedPort(6379)\n",
    "println(\"Redis 8 started at $host:$port\")"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Redis 8 started at localhost:54215\n"
     ]
    }
   ],
   "execution_count": 4
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "## Implementing our Semantic Cache"
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Installing dependencies\n",
    "\n",
    "As mentioned in the beginning, we will use RedisVL's semantic routing abstraction to implement our semantic cache. Therefore, we will need to add RedisVL as a dependency."
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:32.571699Z",
     "start_time": "2025-11-24T10:25:31.160492Z"
    }
   },
   "cell_type": "code",
   "source": [
    "@file:DependsOn(\"com.redis:redisvl:0.0.1\")\n",
    "%use serialization"
   ],
   "outputs": [],
   "execution_count": 5
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Loading Pre-Generated FAQs\n",
    "\n",
    "For this semantic caching demonstration, we'll use pre-generated frequently asked questions (FAQs) about a Chevrolet Colorado vehicle brochure. These FAQs were created by processing the vehicle documentation and extracting question-answer pairs using an LLM.\n"
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:33.877137Z",
     "start_time": "2025-11-24T10:25:33.805708Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import java.io.File\n",
    "\n",
    "val jsonText = File(\"./resources/3_colorado_faqs.json\").readText(Charsets.UTF_8)\n",
    "val jsonArray = Json.parseToJsonElement(jsonText).jsonArray\n",
    "\n",
    "println(\"Loaded ${jsonArray.size} FAQs from file\")"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loaded 346 FAQs from file\n"
     ]
    }
   ],
   "execution_count": 6
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "### Setting up the Text Vectorizer\n",
    "\n",
    "The vectorizer is responsible for converting text into numerical vector representations that capture semantic meaning. RedisVL provides several vectorizer options such as OpenAI and VertexAI. We're using the HuggingFace Text Vectorizer for this example."
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:36.682315Z",
     "start_time": "2025-11-24T10:25:35.972546Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import com.redis.vl.utils.vectorize.SentenceTransformersVectorizer\n",
    "\n",
    "val vectorizer = SentenceTransformersVectorizer(\"Xenova/all-MiniLM-L6-v2\")\n",
    "\n",
    "val embedding = vectorizer.embed(\"What is the capital city of Italy?\")\n",
    "\n",
    "println(embedding.joinToString())"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-0.009056281, 0.09096523, -0.051762886, 0.08848378, -0.12719342, -0.0703391, 0.029510844, 0.013291523, -0.057980966, -0.014017097, 0.03739981, -0.13108169, 0.0018671635, 0.03550265, -0.055068597, -0.04273072, 0.0480743, 0.035149302, 0.051385034, 0.008154835, 0.02939507, -0.02790439, 0.04798433, 0.012633902, 0.050369605, 0.03730664, -0.016114296, 0.016826835, -0.05483934, -0.04307148, -0.014681098, 0.0032649112, 0.10389013, -0.085853584, 0.016533818, 0.017277544, -0.012875621, -0.008417194, 0.106101766, -3.3647308E-4, 0.03838455, -0.007070606, 0.064803414, 0.04349774, 0.027908528, -0.004982669, 0.05417708, 0.08491659, 0.01753072, -0.04387867, -0.0089426385, -0.029429087, -0.04308129, -0.0137046715, -0.049384452, 0.079110876, 0.0159977, -0.023842642, 0.010396142, -0.017871607, -0.02013254, -0.029775942, -0.057334274, 0.079562895, 0.017678022, 0.046195857, -0.025770709, -0.052720636, -0.07104178, -0.016904766, 0.005821192, -0.04959368, 0.012194841, -0.06851538, 0.024740597, -0.06589627, -0.004739907, 0.019979797, 0.012046297, 0.025945937, 0.04738658, -0.026207888, -0.024006084, 0.06688492, 0.027395409, 0.0511346, 0.033016067, 0.012410808, -7.134781E-4, -0.0053796326, 0.024619037, 0.10826674, -0.029768111, -0.008205869, -0.034382585, 0.023230458, -0.023033625, 0.06288932, -0.05734033, 0.07995538, 0.020887373, -0.048439715, 0.06801826, 0.03367503, -0.03210259, 0.030324435, 0.07029199, -0.035357255, -0.0437311, -0.026227256, -0.117252186, -0.0637143, -0.025789985, -0.076391436, 0.021659859, 0.07295666, 0.079506256, 0.030736178, -0.018580705, 0.032889977, -0.027822956, -0.056768946, -0.017266978, -0.0187513, -0.052855123, 0.0026798288, -0.09833044, -4.265337E-33, -0.04522934, -0.038603127, 0.006782485, 0.056442842, -0.047985323, 0.056943357, 0.005727432, -0.07347524, -0.06966861, -0.06284734, 6.824111E-4, -0.13293485, -0.0028802142, -0.011797879, 0.091532454, 0.039949197, 0.044991918, 0.047134932, -0.038595345, -0.050638054, 0.013533677, 0.0056156493, -0.00128937, -0.10107974, -0.0062624826, -0.006572015, -0.01721928, 0.012505045, 0.016358247, 0.027294295, 0.014827963, 0.11129013, -0.012927993, -0.026968583, 0.011228542, 0.099366866, -0.025637725, -0.017972834, -0.006258892, 0.06267472, -0.046503223, 0.023930388, -0.04911905, 0.018191079, 0.056508224, -0.034028705, -0.058674965, -0.026060073, 0.10192595, -0.07542723, 0.029558491, -0.0063548926, -0.043660775, 0.013796782, 0.05591109, 0.13179655, 0.0014945822, 0.073946476, -0.00362497, 0.09263236, -0.008254736, 0.058894735, 0.028806994, 0.041127626, 0.050720066, 0.106417455, 0.04963482, 0.07839, 0.062062256, 0.041708015, 9.6040964E-4, -0.05741613, 0.029530406, 0.10063947, -0.039977487, 0.045762528, -0.01707651, -0.048533317, -0.026703384, 0.0071982993, -0.041772787, -0.052366402, -0.036398213, 0.04329747, 0.047366947, -0.0023107647, 0.001604334, -0.02657378, -0.0017699071, -0.033968117, -0.059102736, -0.057025284, -0.058409713, 0.009664118, -0.008185311, 2.1191983E-33, 0.02503501, -0.05893375, -0.054135744, -0.013791176, -0.16847505, -0.056284852, -0.013744822, -0.0122961225, 0.0031321992, 0.053306993, -0.072457984, -0.0476266, 0.010839893, -0.03926069, 0.027747422, 0.10246861, 0.091257475, -0.015415793, -0.068407044, -0.10341207, -0.08215669, -0.03483035, -0.102931306, -0.07483412, -0.056837633, -0.027723843, -0.07740979, -0.014597713, 0.0067502027, -0.039716292, -0.04219018, -0.036527764, -0.058196455, -0.019601742, -0.032604992, 0.10461292, 0.0416616, -0.0481787, 0.091960885, 0.049276114, -0.0652168, -0.020950038, 0.07045577, 0.12440737, 0.07806728, 0.0073797563, 0.039370913, 0.002310497, -0.045826662, -0.006844643, 0.032626607, -0.039485242, -0.038203005, 0.010807971, 0.024200222, 0.017911695, -0.009244083, 0.034157734, -0.06378024, -0.09688441, 0.0928941, 0.020900233, -0.041648738, 0.062452674, 0.010918287, 0.044152, -0.08620699, 0.019485684, 0.013766632, -0.011806223, 0.059487995, 8.969072E-4, -0.08984199, 0.045840707, -0.08102404, 0.09381022, 0.049054053, 0.07864653, 0.055541668, -0.059208665, -0.0028986053, -0.0077899722, -0.04229291, 0.014191566, -0.052229255, -0.013464374, 0.08817786, -0.026562197, 0.06472866, 0.0073218145, -0.031644728, 0.065533295, 0.039485324, -0.0784605, 0.0044285893, -1.8748269E-8, 0.012146794, 0.020252327, -0.04066794, 0.0826934, 0.013210786, 0.011821769, 0.04016033, -0.027048599, 0.015920155, 0.01708129, -0.081510864, -0.0075302, -0.041846845, -0.061824482, -0.052197114, 0.04438127, 0.036889043, 0.029691536, 0.022707561, -0.007839727, 0.023343714, 0.0074147084, -0.081928894, -0.00935852, -0.024256242, -0.08159295, 0.0391832, 0.07591707, -0.04517342, 0.0011093953, 0.05846702, -0.07561812, -0.012565128, -0.08454925, -0.043759752, 0.07977247, -0.0046581454, -0.08857544, -0.012337672, -0.033817444, 0.022975812, -0.013439647, 0.05287376, 8.4293383E-4, 0.046390817, 0.015289715, 0.0053871227, -0.008255393, 0.0038129103, -0.078331016, -0.0831567, 0.020810202, 0.061217535, 0.033761967, 0.06302245, 0.0040639304, 0.062217683, 0.058209386, 0.046823863, 0.063531056, 0.04160275, -0.021602299, 0.029407658, -0.011543353\n"
     ]
    }
   ],
   "execution_count": 7
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### Creating the SemanticCache\n"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:39.497831Z",
     "start_time": "2025-11-24T10:25:39.175066Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import com.redis.vl.extensions.cache.SemanticCache\n",
    "import redis.clients.jedis.HostAndPort\n",
    "import redis.clients.jedis.UnifiedJedis\n",
    "\n",
    "val jedis = UnifiedJedis(HostAndPort(host, port))\n",
    "\n",
    "// Initialize the semantic cache with Redis connection\n",
    "val cache = SemanticCache.Builder()\n",
    "    .name(\"llmcache\")\n",
    "    .distanceThreshold(0.2F)\n",
    "    .ttl(360)\n",
    "    .redisClient(jedis)\n",
    "    .vectorizer(vectorizer)\n",
    "    .build()"
   ],
   "outputs": [],
   "execution_count": 8
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### Storing FAQs in the Semantic Cache"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:44.727233Z",
     "start_time": "2025-11-24T10:25:41.189757Z"
    }
   },
   "cell_type": "code",
   "source": [
    "jsonArray.forEachIndexed { i, el ->\n",
    "    val obj = el.jsonObject\n",
    "    val prompt = obj[\"prompt\"]?.jsonPrimitive?.content.orEmpty()\n",
    "    val response = obj[\"response\"]?.jsonPrimitive?.content.orEmpty()\n",
    "    cache.store(prompt, response)\n",
    "}"
   ],
   "outputs": [],
   "execution_count": 9
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### Testing the Semantic Cache"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:45.878043Z",
     "start_time": "2025-11-24T10:25:45.807205Z"
    }
   },
   "cell_type": "code",
   "source": [
    "val cacheHit = cache.check(\"What models of chevy colorado are available?\").get()\n",
    "println(\"Prompt: ${cacheHit.prompt}\")\n",
    "println(\"Response: ${cacheHit.response}\")\n",
    "println(\"Distance: ${cacheHit.distance}\")"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prompt: What are the available models of the Colorado?\n",
      "Response: The available models of the Colorado are WT, LT, Z71, and ZR2.\n",
      "Distance: 0.18383932\n"
     ]
    }
   ],
   "execution_count": 10
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:47.569521Z",
     "start_time": "2025-11-24T10:25:47.508860Z"
    }
   },
   "cell_type": "code",
   "source": [
    "val cacheHit = cache.check(\"What entertainment system comes with the car?\").get()\n",
    "println(\"Prompt: ${cacheHit.prompt}\")\n",
    "println(\"Response: ${cacheHit.response}\")\n",
    "println(\"Distance: ${cacheHit.distance}\")"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prompt: What entertainment system is included in the vehicle?\n",
      "Response: The vehicle includes the Chevrolet Infotainment 3 system with an 8-inch diagonal color touch-screen.\n",
      "Distance: 0.09986466\n"
     ]
    }
   ],
   "execution_count": 11
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:49.664565Z",
     "start_time": "2025-11-24T10:25:49.616993Z"
    }
   },
   "cell_type": "code",
   "source": "cache.check(\"Does the car drive on the water?\")",
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Optional.empty"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "execution_count": 12
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Redis Insight\n",
    "\n",
    "Redis Insight is a visual tool that helps you explore, monitor, and optimize your Redis data and performance through an easy-to-use interface.\n",
    "\n",
    "It can be downloaded and run locally in your machine or be run in a Docker container. To make this recipe self-contained and straightforward, we're going to run it in a Docker container using Test Containers."
   ]
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### Configuring a generic Redis Insight Container"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:53.184035Z",
     "start_time": "2025-11-24T10:25:53.108180Z"
    }
   },
   "cell_type": "code",
   "source": [
    "import org.testcontainers.containers.GenericContainer\n",
    "import org.testcontainers.containers.wait.strategy.Wait\n",
    "import org.testcontainers.utility.DockerImageName\n",
    "\n",
    "class RedisInsightContainer : GenericContainer<RedisInsightContainer>(\n",
    "    DockerImageName.parse(\"redis/redisinsight:latest\") // or latest stable version\n",
    ") {\n",
    "    init {\n",
    "        withExposedPorts(5540)\n",
    "        withEnv(\"RI_REDIS_HOST\", \"redis\")\n",
    "        withEnv(\"RI_REDIS_PORT\", \"6379\") // Since this will run in the same Docker network, we don't need to set the mapped port for the Redis Server\n",
    "        withEnv(\"RI_REDIS_ALIAS\", \"Local Redis\")\n",
    "        withEnv(\"RI_REDIS_USERNAME\", \"default\")\n",
    "        withEnv(\"RI_REDIS_PASSWORD\", \"\")\n",
    "        withEnv(\"RI_REDIS_TLS\", \"FALSE\")\n",
    "\n",
    "        waitingFor(Wait.forHttp(\"/\").forPort(5540))\n",
    "    }\n",
    "\n",
    "    fun getUiUrl(): String = \"http://${host}:${getMappedPort(5540)}\"\n",
    "}"
   ],
   "outputs": [],
   "execution_count": 13
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": "### Starting the Redis Insight container"
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:25:58.062664Z",
     "start_time": "2025-11-24T10:25:55.809086Z"
    }
   },
   "cell_type": "code",
   "source": [
    "val redisInsight = RedisInsightContainer().withNetwork(network)\n",
    "redisInsight.start()\n",
    "\n",
    "println(\"RedisInsight UI: ${redisInsight.getUiUrl()}\")"
   ],
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RedisInsight UI: http://localhost:54223\n"
     ]
    }
   ],
   "execution_count": 14
  },
  {
   "metadata": {},
   "cell_type": "markdown",
   "source": [
    "## Spinning down Docker containers\n",
    "\n",
    "Finally, once we're done, let's clean up all the resources we created for our recipe:"
   ]
  },
  {
   "metadata": {
    "ExecuteTime": {
     "end_time": "2025-11-24T10:26:00.689481Z",
     "start_time": "2025-11-24T10:26:00.129766Z"
    }
   },
   "cell_type": "code",
   "source": [
    "redis.stop()\n",
    "redisInsight.stop()\n",
    "network.close()"
   ],
   "outputs": [],
   "execution_count": 15
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Kotlin",
   "language": "kotlin",
   "name": "kotlin"
  },
  "language_info": {
   "name": "kotlin",
   "version": "2.2.20-dev-4982",
   "mimetype": "text/x-kotlin",
   "file_extension": ".kt",
   "pygments_lexer": "kotlin",
   "codemirror_mode": "text/x-kotlin",
   "nbconvert_exporter": ""
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}