{ "cells": [ { "metadata": {}, "cell_type": "markdown", "source": [ "# Semantic Tool Calling\n", "\n", "Tool calling is a pattern where AI systems determine which tools or functions to execute based on user input. The most common modern approach uses large language models (LLMs) to analyze user requests and decide which functions to call.\n", "\n", "## Semantic Tool Calling vs. LLM Tool Calling\n", "\n", "**LLM-based tool calling** involves sending user queries to models like GPT-4 or Claude with function definitions, letting the LLM decide which tools to use. This approach means:\n", "- **Cost per request**: Each decision requires an API call\n", "- **Latency**: Network round-trips add delay to every tool selection\n", "\n", "**Semantic tool calling** uses vector embeddings and similarity matching to route queries to appropriate tools:\n", "- **Speed**: Near-instantaneous tool selection through vector similarity\n", "- **Cost-effective**: No API costs after initial setup\n", "\n", "## How It Works\n", "\n", "### Creating the references and storing them in the vector database (Redis):\n", "\n", "1. Reference examples of text are generated for each tool we want to semantically trigger:\n", "\n", "\"\"\n", "\n", "2. Using an embedding model, we convert these references into embeddings (vector representation)\n", "3. These references are stored in Redis alongside the tool they refer to.\n", "\n", "\"\"\n", "\n", "### Selecting tool:\n", "\n", "1. Using the same embedding model, we convert the user prompt into an embedding (vector representation)\n", "2. We use this embedding to perform semantic search in the vector database to retrieve the most similar reference to the tools we're trying to match\n", "3. If the most similar reference is similar enough, we assume that the referring tool should be called and do it proactively.\n", "\n", "\"\"\n", "\n", "## Using RedisVL (Vector Library)\n", "\n", "RedisVL is a library that makes working with vector search easy with Redis by providing abstractions to common vector search use cases out of the box. In this notebook, we will use the *Semantic Routing* abstraction whose purpose is to classify text in the same fashion described in the previous section.\n", "\n", "## Resources\n", "- [RedisVL Java GitHub Repository](https://github.com/redis/redis-vl-java)\n", "- [RedisVL Java Documentation](https://redis.github.io/redis-vl-java/redisvl/current/index.html)\n", "- [RedisVL Python GitHub Repository](https://github.com/redis/redis-vl-python)\n", "- [RedisVL Python Documentation](https://docs.redisvl.com/en/latest/)\n", "- [Redis AI Resources Repository](https://github.com/redis-developer/redis-ai-resources)\n", "- [Redis Query Engine Documentation](https://redis.io/docs/latest/develop/ai/search-and-query/)" ] }, { "metadata": {}, "cell_type": "markdown", "source": [ "## Running Redis\n", "\n", "There are several options one can follow to have a running instance of Redis. For the sake of simplicity, in this notebook, we will run it in a Docker container.\n", "\n", "For production where high-availability and reliability is a concern, we recommend using [Redis Cloud](https://cloud.redis.io/).\n", "\n", "A free database can be spun up in Redis Cloud." ] }, { "metadata": {}, "cell_type": "markdown", "source": [ "#### Installing dependencies### Running Redis in a Docker Container using TestContainers\n", "\n", "**Docker containers** are lightweight, portable environments that package an application and all its dependencies so it runs consistently across different systems. **Testcontainers** is a library that lets us run lightweight, disposable Docker containers for integration testing, so you can test against real services like databases or message queues without complex setup.\n", "\n", "Make sure you have Docker installed: [install Docker](https://www.docker.com/get-started/)." ] }, { "metadata": {}, "cell_type": "markdown", "source": "#### Installing dependencies" }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:07.409406Z", "start_time": "2025-11-24T10:28:07.094431Z" } }, "cell_type": "code", "source": "@file:DependsOn(\"org.testcontainers:testcontainers:2.0.2\")", "outputs": [], "execution_count": 1 }, { "metadata": {}, "cell_type": "markdown", "source": "#### Configuring a generic Redis Container" }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:08.728521Z", "start_time": "2025-11-24T10:28:08.443262Z" } }, "cell_type": "code", "source": [ "import org.testcontainers.containers.GenericContainer\n", "import org.testcontainers.utility.DockerImageName\n", "\n", "class RedisContainer : GenericContainer(DockerImageName.parse(\"redis:latest\")) {\n", " init {\n", " withExposedPorts(6379)\n", " }\n", "}" ], "outputs": [], "execution_count": 2 }, { "metadata": {}, "cell_type": "markdown", "source": [ "#### Creating a Docker network\n", "\n", "This is necessary because later on this notebook we will spin up a Redis Insight container that needs to be in the same network." ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:10.704381Z", "start_time": "2025-11-24T10:28:10.643219Z" } }, "cell_type": "code", "source": [ "import org.testcontainers.containers.Network\n", "\n", "val network = Network.newNetwork()\n", "val networkAlias = \"redis-network\"" ], "outputs": [], "execution_count": 3 }, { "metadata": {}, "cell_type": "markdown", "source": "#### Start a Redis Container" }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:13.363479Z", "start_time": "2025-11-24T10:28:12.343275Z" } }, "cell_type": "code", "source": [ "val networkAlias = \"redis\"\n", "val redis = RedisContainer().withNetwork(network).withNetworkAliases(networkAlias)\n", "redis.start()\n", "\n", "val host = redis.host\n", "val port = redis.getMappedPort(6379)\n", "println(\"Redis 8 started at $host:$port\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Redis 8 started at localhost:54394\n" ] } ], "execution_count": 4 }, { "metadata": {}, "cell_type": "markdown", "source": "## Implementing our Semantic Tool Caller" }, { "metadata": {}, "cell_type": "markdown", "source": [ "### Installing dependencies\n", "\n", "As mentioned in the beginning, we will use RedisVL's semantic routing abstraction to implement our semantic tool caller. Therefore, we will need to add RedisVL as a dependency." ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:15.415175Z", "start_time": "2025-11-24T10:28:14.731399Z" } }, "cell_type": "code", "source": "@file:DependsOn(\"com.redis:redisvl:0.0.1\")", "outputs": [], "execution_count": 5 }, { "metadata": {}, "cell_type": "markdown", "source": [ "### Setting up a vectorizer\n", "\n", "In RedisVL, embedding models are called vectorizers. This is because embeddings are vector representations. The vectorizer is responsible for converting text into numerical vector representations that capture semantic meaning.\n", "\n", "This vectorizer will be passed on to our semantic routing that will convert the references and the text we're trying to classify into vectors under the hood.\n", "\n", "RedisVL provides several vectorizer options such as OpenAI and VertexAI, but for this example, we will be HuggingFace's `all-MiniLM-L6-v2` vectorizer because it's open source, lightweight, and free to use." ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:18.515846Z", "start_time": "2025-11-24T10:28:17.837886Z" } }, "cell_type": "code", "source": [ "import com.redis.vl.utils.vectorize.SentenceTransformersVectorizer\n", "\n", "val vectorizer = SentenceTransformersVectorizer(\"Xenova/all-MiniLM-L6-v2\")\n", "\n", "\n", "// Testing our vectorizer\n", "// all-MiniLM-L6-v2 is an embedding model that produces vectors of 384 dimensions, therefore we will 384 numbers printed on the screen.\n", "// Embedding models are deterministic. It doesn't matter how many times we run this cell, the same numbers will always be produced for the same string.\n", "\n", "val embedding = vectorizer.embed(\"What is the capital city of the Netherlands?\")\n", "\n", "println(embedding.joinToString())" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.10366548, 0.06542453, -0.04904806, 0.035133816, -0.030148711, -0.048898157, -0.02108736, 0.0019588028, -0.05460191, 0.027000071, 0.0186685, -0.12342901, -0.07914663, -0.0302804, -0.056598365, -0.039736673, 0.030802587, 0.005838588, 0.085851155, -0.032130066, -0.0071115145, -0.033734083, 0.100847885, -0.06491691, 0.014052424, 0.036977015, 0.04544064, -0.014863417, 0.011651148, -0.04714538, 0.019530838, -0.06317588, 0.027103335, -0.032490354, -0.06364442, 0.0034463818, -0.022536488, 0.046401046, 0.029528277, 0.023609689, 0.026152493, -0.025078116, -0.01031126, -0.0460871, -0.030701958, -0.011587745, -0.046117976, 0.0654084, -0.0105588185, -0.030012755, 0.08957275, -0.06994565, -0.07410133, -0.030177299, -0.0072215544, 0.03257758, -0.08564555, 0.06931229, 0.011757878, -0.017046366, 0.006678676, 0.005762717, -0.09732431, 0.04363133, 0.09194445, 0.0023713051, 0.032854725, 0.043560334, -0.09262396, -0.0036028812, -0.00783084, -0.051787496, 0.020866683, -0.08783279, 0.008077556, -0.061896563, -0.052876394, 0.01542515, 0.028461847, 0.055254, 0.0054902015, 0.057896607, -0.012671219, -0.016398145, 0.0065261223, 0.09946422, 0.081683286, -0.014310647, 0.016592013, -0.023128727, 0.03899589, 0.024052972, -0.036022622, 0.025064666, -0.09027798, 0.07410709, 0.033240385, 0.07689808, -0.0075047775, 0.07129043, 0.058456574, 0.0048786686, 0.042863794, -0.03333143, -0.084633105, 0.0404397, 0.0016501043, -0.043248758, 0.008720438, 0.013928717, -0.12757383, 0.0098286215, 0.005235327, -0.07792569, 0.06408246, 0.021897094, 0.05976543, -0.031112881, 0.024613243, 0.03934474, -0.028856492, -0.034532562, -0.05198242, 0.057985958, 0.012580307, 0.041744966, 0.020353135, -5.6557037E-33, -0.052744765, -0.045968663, 0.090078026, 0.05969658, -0.02187134, -0.005868256, 0.011242692, -0.08511892, -0.079772584, -0.009573086, -0.0019149515, -0.11999923, -0.03113336, -0.08176903, 0.067300566, 0.038190972, 0.06258574, 0.060455803, -0.03424304, 0.016001912, -0.005311913, 0.049972218, -0.009854142, -0.022788646, 0.0062692976, -0.047501225, -0.005984251, -0.02856334, 0.057462938, 0.018307501, -0.029511033, 0.07421711, -0.024075503, -0.0029474783, -0.07320562, 0.0706954, 0.009616033, -0.04029617, -0.01405646, -0.064900756, 0.03480194, -0.054661036, -0.020448288, 0.09943245, 0.005378907, -0.015326283, -0.033892747, -0.046929743, 0.07517493, -0.0070360987, -0.025554607, -0.00481674, -0.03463863, 0.0028586213, 0.043961085, 0.0864002, -0.015171171, 0.045793384, 0.06448305, 0.09541922, -0.03308, 0.041854348, 0.012787413, 0.046993345, 0.066325955, 0.0064544575, -0.03629538, 0.040205065, 0.06272943, 0.025750188, -0.04355145, 0.027366433, 0.03712594, 0.092217006, 0.04882238, 0.077644326, -0.0015609574, 0.011593154, -0.0020405904, 0.05031544, -0.06710352, -0.03852454, -0.13273093, 0.012622703, 0.08002853, -0.04667534, -0.07688987, -0.05683111, 0.047081694, 0.041931298, -0.015288125, -0.042343985, -0.09852148, -0.024203202, -0.018935643, 2.588187E-33, 0.010014019, -0.06895113, -0.09746031, 0.036003824, -0.06341073, 0.012032667, 0.0065762307, 0.049733743, -0.08265836, -0.06188845, -0.08704927, -0.08797185, 0.02567979, 0.04690977, 0.053944163, 0.014458485, 0.082691275, 0.005943607, 0.0030477392, -0.043116875, -0.07077661, -0.0074396897, -0.11552381, 0.005685311, -0.0045177834, -0.0017315152, -0.11645061, -0.036437806, -0.024438681, -0.029221144, 0.019877205, 0.0046609254, -0.04207322, 0.06455668, -0.076581106, 0.03378759, 0.09027628, -0.0253643, 0.015189911, 0.054381564, -0.055162173, -0.033139195, -0.009840774, 0.14639673, 0.017416827, 0.030440766, 0.0069401297, 3.176525E-4, 0.0072309277, -0.039998386, -0.0048506684, 0.040915135, -0.03348285, 0.022541454, 0.065321624, 0.031276476, -0.011164032, 0.020680353, 0.004708727, 0.009346659, 0.01592603, 0.05631898, -0.07473072, 0.076988205, 0.05419154, 0.0150421895, -0.0845003, 0.05386096, 0.021571098, -0.11888843, 0.09743547, 0.0051163672, -0.0011004083, 0.0040163654, -0.02674617, -0.024856558, 0.17286904, 0.06661335, -0.0058373064, -0.09741991, -0.01834617, 0.020193022, -0.006355057, -0.045210075, -0.08246333, 0.07894664, 0.09668022, -0.07309056, -0.042953275, -0.013248654, 0.027853789, 0.06620693, -0.05210908, -0.007688315, 0.013242694, -1.9812573E-8, -0.030713642, 0.028282253, -0.0320448, -0.009659997, -0.012524873, -3.3970224E-4, 0.10623612, -9.5831533E-4, -0.09355901, 0.054665145, 0.005950292, 0.026303494, -0.0051028966, -0.0076380824, 0.021773987, 0.0012177717, -0.005338478, 0.08560071, -0.013641202, -0.010476135, -0.024037808, 0.014412938, -0.089821, -0.014127389, -0.009554571, 0.0036026777, 0.061681934, 0.09047015, 0.030739356, -0.025442347, 0.004772291, 0.03357452, 0.010236746, 0.02115033, -0.027155366, 0.0013892107, 0.005756898, -0.008016912, -0.059457876, -0.008943728, 0.028666046, 0.022122331, 0.0048325458, -0.013133106, 0.0039171097, 0.035153743, 0.01711277, 0.02501661, 0.059882976, -0.14004363, -0.15314281, 0.030633396, 0.04397522, 0.0048514833, 0.03720228, -0.0823461, -0.0054057688, 0.0070677525, -0.02929645, -0.025880426, 0.09497929, -0.018403502, 0.043690376, 0.040304095\n" ] } ], "execution_count": 6 }, { "metadata": {}, "cell_type": "markdown", "source": [ "## Defining Tool Routes\n", "\n", "Each route represents a specific tool or function that user calls might need to trigger. We define routes by providing example phrases that represent different ways users might express the same intent.\n", "\n", "Each route contains:\n", "\n", "- **Route name**: An identifier for this classification category\n", "- **Reference examples**: Sample text that represents the category you want to classify\n", "- **Distance threshold**: How similar new text must be to the references to match the route" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:21.075438Z", "start_time": "2025-11-24T10:28:20.999660Z" } }, "cell_type": "code", "source": [ "import com.redis.vl.extensions.router.Route\n", "\n", "val getWeatherRoute = Route.builder()\n", " .name(\"get_weather_route\")\n", " .references(\n", " listOf(\n", " \"What's the weather like?\",\n", " \"What's the weather today?\",\n", " \"Will it rain today?\",\n", " \"Is it raining?\"\n", " )\n", " )\n", " .distanceThreshold(0.3)\n", " .build()\n", "\n", "val faqRoute = Route.builder()\n", " .name(\"faq_route\")\n", " .references(\n", " listOf(\n", " \"What can you do?\",\n", " \"What are you capable of?\",\n", " \"What problems can you solve?\",\n", " \"How can you help me?\"\n", " )\n", " )\n", " .distanceThreshold(0.3)\n", " .build()\n", "\n", "val notificationsRoute = Route.builder()\n", " .name(\"notifications_route\")\n", " .references(\n", " listOf(\n", " \"Do I have any notifications?\",\n", " \"Read my notifications\",\n", " \"Have I got notifications?\"\n", " )\n", " )\n", " .distanceThreshold(0.3)\n", " .build()" ], "outputs": [], "execution_count": 7 }, { "metadata": {}, "cell_type": "markdown", "source": [ "### Creating the router\n", "\n", "The SemanticRouter is the central component that orchestrates the classification process. It combines your routes, vectorizer, and Redis connection to provide fast semantic classification capabilities." ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:24.031972Z", "start_time": "2025-11-24T10:28:23.641563Z" } }, "cell_type": "code", "source": [ "import com.redis.vl.extensions.router.SemanticRouter\n", "import redis.clients.jedis.HostAndPort\n", "import redis.clients.jedis.UnifiedJedis\n", "\n", "// Configure the connection to Redis\n", "val jedis = UnifiedJedis(HostAndPort(host, port))\n", "\n", "val router = SemanticRouter.builder()\n", " .name(\"tool-router\")\n", " .jedis(jedis)\n", " .vectorizer(vectorizer)\n", " .routes(listOf(\n", " getWeatherRoute,\n", " faqRoute,\n", " notificationsRoute)\n", " ).build()" ], "outputs": [], "execution_count": 8 }, { "metadata": {}, "cell_type": "markdown", "source": "## Testing our semantic tool calling solution" }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:26.374808Z", "start_time": "2025-11-24T10:28:26.299872Z" } }, "cell_type": "code", "source": [ "val userQuery = \"Do I have new notifications?\"\n", "\n", "val routeMatch = router.route(userQuery)\n", "\n", "// This query should match the notifications route\n", "println(routeMatch)" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RouteMatch(name=notifications_route, distance=0.192504366239)\n" ] } ], "execution_count": 9 }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:27.815215Z", "start_time": "2025-11-24T10:28:27.764253Z" } }, "cell_type": "code", "source": [ "val userQuery = \"Will it be sunny today?\"\n", "\n", "val routeMatch = router.route(userQuery)\n", "\n", "// This query should match the weather route\n", "println(routeMatch)" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RouteMatch(name=get_weather_route, distance=0.244104504585)\n" ] } ], "execution_count": 10 }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:29.452642Z", "start_time": "2025-11-24T10:28:29.402769Z" } }, "cell_type": "code", "source": [ "val userQuery = \"What tasks are you capable of?\"\n", "\n", "val routeMatch = router.route(userQuery)\n", "\n", "// This query should match the faq route\n", "println(routeMatch)" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RouteMatch(name=faq_route, distance=0.269281685352)\n" ] } ], "execution_count": 11 }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:30.893974Z", "start_time": "2025-11-24T10:28:30.839753Z" } }, "cell_type": "code", "source": [ "val userQuery = \"Stamppot is the most nutritious meal\"\n", "\n", "val routeMatch = router.route(userQuery)\n", "\n", "// This query shouldn't match any route\n", "println(routeMatch)" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RouteMatch(name=null, distance=null)\n" ] } ], "execution_count": 12 }, { "metadata": {}, "cell_type": "markdown", "source": [ "## Redis Insight\n", "\n", "Redis Insight is a visual tool that helps you explore, monitor, and optimize your Redis data and performance through an easy-to-use interface.\n", "\n", "It can be downloaded and run locally in your machine or be run in a Docker container. To make this recipe self-contained and straightforward, we're going to run it in a Docker container using Test Containers." ] }, { "metadata": {}, "cell_type": "markdown", "source": "### Configuring a generic Redis Insight Container" }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:33.185527Z", "start_time": "2025-11-24T10:28:33.100178Z" } }, "cell_type": "code", "source": [ "import org.testcontainers.containers.GenericContainer\n", "import org.testcontainers.containers.wait.strategy.Wait\n", "import org.testcontainers.utility.DockerImageName\n", "\n", "class RedisInsightContainer : GenericContainer(\n", " DockerImageName.parse(\"redis/redisinsight:latest\") // or latest stable version\n", ") {\n", " init {\n", " withExposedPorts(5540)\n", " withEnv(\"RI_REDIS_HOST\", \"redis\")\n", " withEnv(\"RI_REDIS_PORT\", \"6379\") // Since this will run in the same Docker network, we don't need to set the mapped port for the Redis Server\n", " withEnv(\"RI_REDIS_ALIAS\", \"Local Redis\")\n", " withEnv(\"RI_REDIS_USERNAME\", \"default\")\n", " withEnv(\"RI_REDIS_PASSWORD\", \"\")\n", " withEnv(\"RI_REDIS_TLS\", \"FALSE\")\n", "\n", " waitingFor(Wait.forHttp(\"/\").forPort(5540))\n", " }\n", "\n", " fun getUiUrl(): String = \"http://${host}:${getMappedPort(5540)}\"\n", "}" ], "outputs": [], "execution_count": 13 }, { "metadata": {}, "cell_type": "markdown", "source": "### Starting the Redis Insight container" }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:37.632888Z", "start_time": "2025-11-24T10:28:35.386572Z" } }, "cell_type": "code", "source": [ "val redisInsight = RedisInsightContainer().withNetwork(network)\n", "redisInsight.start()\n", "\n", "println(\"RedisInsight UI: ${redisInsight.getUiUrl()}\")" ], "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "RedisInsight UI: http://localhost:54400\n" ] } ], "execution_count": 14 }, { "metadata": {}, "cell_type": "markdown", "source": [ "When accessing Redis Insight for the first time, you will have to agree with the user agreement:\n", "\n", "\"\"\n", "\n", "After agreeing, the list of configured databases will show up. In this case, there'll be only one: `Local Redis`.\n", "\n", "\"\"\n", "\n", "By clicking on `Tree View` we can organize the keys by keyspace. This will make it easier to visualize all keys in Redis Insight:\n", "\n", "\"\"\n", "\n", "The `tool-router:route_config` key holds the configuration of the router (tool caller in our case) - We can see its name, vectorizer, routes and some configuration:\n", "\n", "\"\"\n", "\n", "In the `tool-router:faq_route:`, `tool-router:get_weather_route:`, `tool-router:notifications_route:` keyspaces, we can see the details of each vectorized reference, including their respective vector representations:\n", "\n", "\"\"\n", "\n", "Make sure you change from `Unicode` to `Vector 32-bit` to see the vectors as numbers instead of a bytearray:\n", "\n", "\"\"\n", "\n", "This will be a long list of 384 floating points.\n", "\n", "On Redis Insight Workbench we can send commands directly to our Redis instance:\n", "\n", "\"\"\n", "\n", "If we send the command `FT.INFO 'tool-router'` we can see the index that was created by RedisVL to be able to perform semantic search efficiently using the [Redis Query Engine](https://redis.io/docs/latest/develop/ai/search-and-query/)\n", "\n", "\"\"" ] }, { "metadata": {}, "cell_type": "markdown", "source": [ "## Spinning down Docker containers\n", "\n", "Finally, once we're done, let's clean up all the resources we created for our recipe:" ] }, { "metadata": { "ExecuteTime": { "end_time": "2025-11-24T10:28:50.388177Z", "start_time": "2025-11-24T10:28:49.763747Z" } }, "cell_type": "code", "source": [ "redis.stop()\n", "redisInsight.stop()\n", "network.close()" ], "outputs": [], "execution_count": 15 } ], "metadata": { "kernelspec": { "display_name": "Kotlin", "language": "kotlin", "name": "kotlin" }, "language_info": { "name": "kotlin", "version": "2.2.20-dev-4982", "mimetype": "text/x-kotlin", "file_extension": ".kt", "pygments_lexer": "kotlin", "codemirror_mode": "text/x-kotlin", "nbconvert_exporter": "" } }, "nbformat": 4, "nbformat_minor": 0 }