{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n",
    "# Implementing multi vector search with Redis\n",
    "\n",
    "Multi vector search is the ability to combine the scores of multiple different vector similarity values to determine relevancy. This notebook will cover how to define a multi vector index and execute multi vector queries.\n",
    "\n",
    "## Let's Begin!\n",
    "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/05_multivector_search.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install  \"redisvl>=0.10.0\" sentence-transformers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'0.16.0'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check version\n",
    "import redisvl\n",
    "\n",
    "redisvl.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data/Index Preparation\n",
    "\n",
    "In this section:\n",
    "\n",
    "1. We prepare the data necessary for our multi vector search implementations by loading a collection of movies. Each movie object contains the following attributes:\n",
    "    - `title`\n",
    "    - `rating`\n",
    "    - `description`\n",
    "    - `genre`\n",
    "\n",
    "2. We'll generate vector embeddings from the movie descriptions. We'll use different models and generate multiple different vectors for each movie.\n",
    "\n",
    "3. After preparing the data, we populate a search index with these movie records, each with multiple vectors."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running remotely or in collab? Run this cell to download the necessary dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "!git clone https://github.com/redis-developer/redis-ai-resources.git temp_repo\n",
    "!mv temp_repo/python-recipes/vector-search/resources .\n",
    "!rm -rf temp_repo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install Redis Stack\n",
    "\n",
    "Later in this tutorial, Redis will be used to store, index, and query vector\n",
    "embeddings and full text fields. **We need to have a Redis\n",
    "instance available.**\n",
    "\n",
    "#### Local Redis\n",
    "Use the shell script below to download, extract, and install [Redis Stack](https://redis.io/docs/getting-started/install-stack/) directly from the Redis package archive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "%%sh\n",
    "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
    "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
    "sudo apt-get update  > /dev/null 2>&1\n",
    "sudo apt-get install redis-stack-server  > /dev/null 2>&1\n",
    "redis-stack-server --daemonize yes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Alternative Redis Access (Cloud, Docker, other)\n",
    "There are many ways to get the necessary redis-stack instance running\n",
    "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n",
    "own version of Redis Enterprise running, that works too!\n",
    "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n",
    "3. With docker: `docker run -d --name redis-stack-server -p 6379:6379 redis/redis-stack-server:latest`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the Redis Connection URL\n",
    "\n",
    "By default this notebook connects to the local instance of Redis Stack. **If you have your own Redis Enterprise instance** - replace REDIS_PASSWORD, REDIS_HOST and REDIS_PORT values with your own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Replace values below with your own if using Redis Cloud instance\n",
    "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") # ex: \"redis-18374.c253.us-central1-1.gce.cloud.redislabs.com\"\n",
    "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\")      # ex: 18374\n",
    "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\")  # ex: \"1TNxTEdYRDgIDKM2gDfasupCADXXXX\"\n",
    "\n",
    "# If SSL is enabled on the endpoint, use rediss:// as the URL prefix\n",
    "REDIS_URL = f\"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create redis client, load data, generate embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from redis import Redis\n",
    "\n",
    "client = Redis.from_url(REDIS_URL)\n",
    "client.ping()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "with open(\"resources/movies.json\", 'r') as file:\n",
    "    movies = json.load(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-Vector Index with Multiple Embedding Models\n",
    "\n",
    "Now let's create a multi-vector search setup by using multiple embedding models for different aspects of our movie data. This approach allows us to:\n",
    "\n",
    "1. **Use specialized embeddings** - Different models optimized for different tasks\n",
    "2. **Combine multiple perspectives** - Search across different semantic representations\n",
    "3. **Improve search quality** - Generate embeddings from different sections of your data\n",
    "\n",
    "We'll create a new index with multiple vector fields and demonstrate how to query across them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n",
      "WARNING:huggingface_hub.utils._http:Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "0ce4106ec33a435595e7ca66297324d5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: sentence-transformers/all-MiniLM-L6-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b49cde2c5f014661b05d9f29b46aa3bd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/199 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mMPNetModel LOAD REPORT\u001b[0m from: sentence-transformers/all-mpnet-base-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "48e2c968ec3046768324d7faa136f473",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Loading weights:   0%|          | 0/199 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    }
   ],
   "source": [
    "from redisvl.utils.vectorize import HFTextVectorizer\n",
    "from redisvl.extensions.cache.embeddings import EmbeddingsCache\n",
    "\n",
    "# Model 1: General purpose embeddings (what we used before)\n",
    "general_model = HFTextVectorizer(\n",
    "    model='sentence-transformers/all-MiniLM-L6-v2',\n",
    "    cache=EmbeddingsCache(\n",
    "        name=\"embedcache_general\",\n",
    "        ttl=600,\n",
    "        redis_client=client,\n",
    "    ),\n",
    "    dtype=\"float64\"\n",
    ")\n",
    "\n",
    "# Model 2: A different model that captures different aspects of the description data\n",
    "movie_model = HFTextVectorizer(\n",
    "    model='sentence-transformers/all-mpnet-base-v2',\n",
    "    cache=EmbeddingsCache(\n",
    "        name=\"embedcache_movie\",\n",
    "        ttl=600,\n",
    "        redis_client=client,\n",
    "    ),\n",
    "    dtype=\"float32\"\n",
    ")\n",
    "\n",
    "# Model 3: Genre-focused embeddings for better genre understanding\n",
    "genre_model = HFTextVectorizer(\n",
    "    model='sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2',\n",
    "    cache=EmbeddingsCache(\n",
    "        name=\"embedcache_genre\",\n",
    "        ttl=600,\n",
    "        redis_client=client,\n",
    "    ),\n",
    "    dtype=\"float32\"\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's highlight the flexibility of multi-vector search in RedisVL:\n",
    "- Different embedding models can be used for each field. Here we have 3 different HuggingFace vectorizors, but you can mix and match any vectorizer.\n",
    "- These models can have different dimensions and datatypes. Want to use a large model with high fidelity and pair it with a lightweight model? Sure thing.\n",
    "- The data source for these embeddings can be anything. In the cell below we generate embeddings from different aspects of our data. You can have a multiple text embeddings and image embeddings on the same document.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating multiple embeddings for movies...\n",
      "Generated embeddings for 20 movies\n"
     ]
    }
   ],
   "source": [
    "# Generate multiple embeddings for each movie\n",
    "print(\"Generating multiple embeddings for movies...\")\n",
    "\n",
    "multi_vector_movie_data = []\n",
    "for movie in movies:\n",
    "    movie_with_vectors = {\n",
    "        **movie,\n",
    "        \"description_vector_general\": general_model.embed(movie[\"description\"], as_buffer=True),\n",
    "        \"description_vector_movie\": movie_model.embed(movie[\"description\"], as_buffer=True),\n",
    "        \"description_vector_genre\": genre_model.embed(f\"{movie['genre']} {movie['description']}\", as_buffer=True),\n",
    "    }\n",
    "    multi_vector_movie_data.append(movie_with_vectors)\n",
    "\n",
    "print(f\"Generated embeddings for {len(multi_vector_movie_data)} movies\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Multi-vector index created and populated successfully!\n"
     ]
    }
   ],
   "source": [
    "from redisvl.schema import IndexSchema\n",
    "from redisvl.index import SearchIndex\n",
    "\n",
    "# Create a multi-vector index schema\n",
    "multi_vector_schema = IndexSchema.from_dict({\n",
    "  \"index\": {\n",
    "    \"name\": \"movies_multivector\",\n",
    "    \"prefix\": \"movie_mv\",\n",
    "    \"storage\": \"hash\"\n",
    "  },\n",
    "  \"fields\": [\n",
    "    { \"name\": \"title\", \"type\": \"text\" },\n",
    "    { \"name\": \"description\", \"type\": \"text\" },\n",
    "    { \"name\": \"genre\", \"type\": \"tag\", \"attrs\": {\"sortable\": True}},\n",
    "    { \"name\": \"rating\", \"type\": \"numeric\", \"attrs\": {\"sortable\": True}},\n",
    "    {\n",
    "        \"name\": \"description_vector_general\",\n",
    "        \"type\": \"vector\",\n",
    "        \"attrs\": {\n",
    "            \"dims\": 384,\n",
    "            \"distance_metric\": \"cosine\",\n",
    "            \"algorithm\": \"hnsw\",\n",
    "            \"datatype\": \"float64\"\n",
    "        }\n",
    "    },\n",
    "    {\n",
    "        \"name\": \"description_vector_movie\",\n",
    "        \"type\": \"vector\",\n",
    "        \"attrs\": {\n",
    "            \"dims\": 768,\n",
    "            \"distance_metric\": \"cosine\",\n",
    "            \"algorithm\": \"hnsw\",\n",
    "            \"datatype\": \"float32\"\n",
    "        }\n",
    "    },\n",
    "    {\n",
    "        \"name\": \"description_vector_genre\",\n",
    "        \"type\": \"vector\",\n",
    "        \"attrs\": {\n",
    "            \"dims\": 384,\n",
    "            \"distance_metric\": \"cosine\",\n",
    "            \"algorithm\": \"hnsw\",\n",
    "            \"datatype\": \"float32\"\n",
    "        }\n",
    "    },\n",
    "  ]\n",
    "})\n",
    "\n",
    "# Create the multi-vector index\n",
    "multi_vector_index = SearchIndex(multi_vector_schema, client, validate_on_load=True)\n",
    "multi_vector_index.create(overwrite=True, drop=True)\n",
    "\n",
    "# Load the multi-vector data\n",
    "multi_vector_index.load(multi_vector_movie_data)\n",
    "print(\"Multi-vector index created and populated successfully!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how each vector field in our index has its own definition with its own attributes.\n",
    "When Constructing an index that is intended for use with MultiVectorQuery each vector field may have different `datatype` `dims` and `algorithm`, but all must have a `cosine` `distance_metric` in order to properly compute the relative weighting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using the Official MultiVectorQuery Class\n",
    "\n",
    "Now let's demonstrate how to run a `MultiVectorQuery` in RedisVL. This class provides a clean, way to perform multi-vector search with weighted combinations. It utilizes the `Vector` class to contain the individual query vectors, which is a departure in syntax from other RedisVL queries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1. The Incredibles \n",
      "   Genre: comedy, Rating: 8\n",
      "   Description: A family of undercover superheroes, while trying to live the quiet suburban life, are forced into ac...\n",
      "\n",
      "2. The Avengers \n",
      "   Genre: action, Rating: 8\n",
      "   Description: Earth's mightiest heroes come together to stop an alien invasion that threatens the entire planet....\n",
      "\n",
      "3. Mad Max: Fury Road \n",
      "   Genre: action, Rating: 8\n",
      "   Description: In a post-apocalyptic wasteland, Max teams up with Furiosa to escape a tyrant's clutches and find fr...\n",
      "\n",
      "4. John Wick \n",
      "   Genre: action, Rating: 8\n",
      "   Description: A retired hitman seeks vengeance against those who wronged him, leaving a trail of destruction in hi...\n",
      "\n",
      "5. The Dark Knight \n",
      "   Genre: action, Rating: 9\n",
      "   Description: Batman faces off against the Joker, a criminal mastermind who threatens to plunge Gotham into chaos....\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from redisvl.query import Vector, MultiVectorQuery\n",
    "query_text = \"action movie with superheroes and explosions\"\n",
    "\n",
    "num_results = 5\n",
    "\n",
    "# Create Vector objects for each embedding model\n",
    "query_vectors = [\n",
    "    Vector(\n",
    "        vector=general_model.embed(query_text, as_buffer=True),\n",
    "        field_name=\"description_vector_general\",\n",
    "        dtype=\"float64\",\n",
    "        weight=0.3  # 30% weight for general embeddings\n",
    "    ),\n",
    "    Vector(\n",
    "        vector=movie_model.embed(query_text, as_buffer=True),\n",
    "        field_name=\"description_vector_movie\",\n",
    "        dtype=\"float32\",\n",
    "        weight=0.5  # 50% weight for movie-specific embeddings\n",
    "    ),\n",
    "    Vector(\n",
    "        vector=genre_model.embed(f\"{query_text}\", as_buffer=True),\n",
    "        field_name=\"description_vector_genre\",\n",
    "        dtype=\"float32\",\n",
    "        weight=0.2  # 20% weight for genre-focused embeddings\n",
    "    )\n",
    "]\n",
    "\n",
    "query = MultiVectorQuery(\n",
    "    vectors=query_vectors,\n",
    "    num_results=num_results,\n",
    "    return_fields=[\"title\", \"description\", \"genre\", \"rating\"],\n",
    ")\n",
    "\n",
    "results = multi_vector_index.query(query)\n",
    "\n",
    "for i, result in enumerate(results, 1):\n",
    "    print(f\"{i}. {result['title']} \")\n",
    "    print(f\"   Genre: {result['genre']}, Rating: {result['rating']}\")\n",
    "    print(f\"   Description: {result['description'][:100]}...\")\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Wrap up\n",
    "That's a wrap! Hopefully from this you were able to learn how to implement multi-vector search with multiple embedding models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# clean up!\n",
    "multi_vector_index.delete()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv (3.12.5)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}