{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![Redis](https://redis.io/wp-content/uploads/2024/04/Logotype.svg?auto=webp&quality=85,75&width=120)\n",
    "# Implementing hybrid search with Redis\n",
    "\n",
    "Hybrid search is all about combining lexical search with semantic vector search to improve result relevancy. This notebook will cover 3 different hybrid search strategies with Redis:\n",
    "\n",
    "1. Linear combination of scores from lexical search (BM25) and vector search (Cosine Distance)\n",
    "2. Reciprocal Rank Fusion (RRF)\n",
    "3. Client-Side Reranking with a cross encoder model\n",
    "\n",
    "The Redis Query Engine supports a unified interface for hybrid search with the [FT.HYBRID](https://redis.io/docs/latest/commands/ft.hybrid) command introduced in Redis Open Source 8.4.0, prior to which hybrid searches were only possible using [the aggregations API](https://redis.io/docs/latest/develop/ai/search-and-query/advanced-concepts/aggregations/). RedisVL added an interface for FT.HYBRID in 0.13.0 (via `HybridQuery`), and provided an interface for the aggregation approach for Redis prior to 8.4.0 (via `AggregateHybridQuery`). This notebook will demonstrate the usage of both approaches.\n",
    "\n",
    "## Requirements\n",
    "- Redis 8.4.0+\n",
    "- redisvl>=0.13.2\n",
    "- redispy>=7.1.0\n",
    "\n",
    "## Let's Begin!\n",
    "<a href=\"https://colab.research.google.com/github/redis-developer/redis-ai-resources/blob/main/python-recipes/vector-search/02_hybrid_search.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install \"redisvl>=0.13.2\" \"redis>7.1.0\" nltk pandas sentence-transformers "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data/Index Preparation\n",
    " \n",
    "In this section:\n",
    "\n",
    "1. We prepare the data necessary for our hybrid search implementations by loading a collection of movies. Each movie object contains the following attributes:\n",
    "    - `title`\n",
    "    - `rating`\n",
    "    - `description`\n",
    "    - `genre`\n",
    " \n",
    "2. We generate vector embeddings from the movie descriptions. This allows users to perform searches that not only rely on exact matches but also on semantic relevance, helping them find movies that align closely with their interests.\n",
    "\n",
    "3. After preparing the data, we populate a search index with these movie records, enabling efficient querying based on both lexical and vector-based search techniques."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running remotely or in collab? Run this cell to download the necessary dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "!git clone https://github.com/redis-developer/redis-ai-resources.git temp_repo\n",
    "!mv temp_repo/python-recipes/vector-search/resources .\n",
    "!rm -rf temp_repo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install Redis\n",
    "\n",
    "For this tutorial you will need a running instance of Redis if you don't already have one.\n",
    "\n",
    "#### Local Redis\n",
    "Use the shell script below to download, extract, and install [Redis](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/apt/) directly from the Redis package archive for a Linux environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# NBVAL_SKIP\n",
    "%%sh\n",
    "sudo apt-get install lsb-release curl gpg\n",
    "curl -fsSL https://packages.redis.io/gpg | sudo gpg --dearmor -o /usr/share/keyrings/redis-archive-keyring.gpg\n",
    "sudo chmod 644 /usr/share/keyrings/redis-archive-keyring.gpg\n",
    "echo \"deb [signed-by=/usr/share/keyrings/redis-archive-keyring.gpg] https://packages.redis.io/deb $(lsb_release -cs) main\" | sudo tee /etc/apt/sources.list.d/redis.list\n",
    "sudo apt-get update\n",
    "sudo apt-get install redis\n",
    "\n",
    "redis-server --version\n",
    "redis-server --daemonize yes --loadmodule /usr/lib/redis/modules/redisearch.so"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Alternative Redis Access (Cloud, Docker, other)\n",
    "There are many ways to get the necessary redis-stack instance running\n",
    "1. On cloud, deploy a [FREE instance of Redis in the cloud](https://redis.com/try-free/). Or, if you have your\n",
    "own version of Redis Enterprise running, that works too!\n",
    "2. Per OS, [see the docs](https://redis.io/docs/latest/operate/oss_and_stack/install/install-stack/)\n",
    "3. With docker: `docker run -d --name redis -p 6379:6379 redis:latest`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from packaging.version import Version\n",
    "\n",
    "from redis import __version__ as redis_version\n",
    "from redisvl import __version__ as redisvl_version\n",
    "\n",
    "\n",
    "if Version(redis_version) < Version(\"7.1.0\"):\n",
    "    raise RuntimeError(\"redis-py version must be >= 7.1.0\")\n",
    "\n",
    "if Version(redisvl_version) < Version(\"0.13.0\"):\n",
    "    raise RuntimeError(\"redisvl version must be >= 0.13.0\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the Redis Connection URL\n",
    "\n",
    "By default this notebook connects to the local instance of Redis Stack. **If you have your own Redis Enterprise instance** - replace REDIS_PASSWORD, REDIS_HOST and REDIS_PORT values with your own."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import warnings\n",
    "\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "# Replace values below with your own if using Redis Cloud instance\n",
    "REDIS_HOST = os.getenv(\"REDIS_HOST\", \"localhost\") # ex: \"redis-18374.c253.us-central1-1.gce.cloud.redislabs.com\"\n",
    "REDIS_PORT = os.getenv(\"REDIS_PORT\", \"6379\")      # ex: 18374\n",
    "REDIS_PASSWORD = os.getenv(\"REDIS_PASSWORD\", \"\")  # ex: \"1TNxTEdYRDgIDKM2gDfasupCADXXXX\"\n",
    "\n",
    "# If SSL is enabled on the endpoint, use rediss:// as the URL prefix\n",
    "REDIS_URL = f\"redis://:{REDIS_PASSWORD}@{REDIS_HOST}:{REDIS_PORT}\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create redis client, load data, generate embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "from redis import Redis\n",
    "from redisvl.redis.connection import RedisConnectionFactory\n",
    "\n",
    "client = Redis.from_url(REDIS_URL)\n",
    "client.ping()\n",
    "\n",
    "if Version(client.info()[\"redis_version\"]) < Version(\"8.4.0\"):\n",
    "    raise RuntimeError(\"Redis version must be >= 8.4.0\")\n",
    "\n",
    "installed_modules = RedisConnectionFactory.get_modules(client)\n",
    "if \"search\" not in installed_modules:\n",
    "    raise RuntimeError(\"Redisearch module is not installed\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "\n",
    "with open(\"resources/movies.json\", 'r') as file:\n",
    "    movies = json.load(file)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n",
      "WARNING:huggingface_hub.utils._http:Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n",
      "Loading weights: 100%|██████████| 103/103 [00:00<00:00, 7941.42it/s]\n",
      "\u001b[1mBertModel LOAD REPORT\u001b[0m from: sentence-transformers/all-MiniLM-L6-v2\n",
      "Key                     | Status     |  | \n",
      "------------------------+------------+--+-\n",
      "embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    }
   ],
   "source": [
    "from redisvl.utils.vectorize import HFTextVectorizer\n",
    "from redisvl.extensions.cache.embeddings import EmbeddingsCache\n",
    "\n",
    "\n",
    "# load model for embedding our movie descriptions\n",
    "model = HFTextVectorizer(\n",
    "    model='sentence-transformers/all-MiniLM-L6-v2',\n",
    "    cache=EmbeddingsCache(\n",
    "        name=\"embedcache\",\n",
    "        ttl=600,\n",
    "        redis_client=client,\n",
    "    )\n",
    ")\n",
    "\n",
    "# embed movie descriptions\n",
    "movie_data = [\n",
    "    {\n",
    "        **movie,\n",
    "        \"description_vector\": model.embed(movie[\"description\"], as_buffer=True)\n",
    "    } for movie in movies\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'id': 1,\n",
       "  'title': 'Explosive Pursuit',\n",
       "  'genre': 'action',\n",
       "  'rating': 7,\n",
       "  'description': 'A daring cop chases a notorious criminal across the city in a high-stakes game of cat and mouse.',\n",
       "  'description_vector': b'\\xa0f|=\\xee`\\n;\\xe2\\x91\\xb7;.\\xcb~\\xbd9e\\xce\\xbb\\xd6\\x16J=K\\xa7?=\\xdev\\x95<j\\xfa\\x06\\xbe\\x12Y\\xcf=/\\x07p=C\\xdb\\r\\xbd\\x8a\\xf2H\\xbd\\x81e\\xc6<E\\xdfa=~8\\x16\\xbc\\xd9\\xd3\\x13<>\\xaa\\x1c=\\xff\\xee\\x89<\\xcc\\xb0-<\\xab\\xb2\\x9f\\xbc]\\x0b\\xc3\\xbd\\xa2NR=fl\\xf7\\xbcO>\\x17\\xbe1\\x18\\x05\\xb9Cu\\xbf<\\x1d\\xe2b\\xba\\xd3\\xa6\\xa8\\xbdy\\xdc\\xec\\xbcTc%=\\xb2\\xe7r\\xbb#OG=;(\\x85=o@\\xa2\\xbc/Z\\xd0\\xbdH%K\\xbd\\xcd\\xed\\x94\\xbc`\\xddH=\\x99&F<\\xc1*\\xec<\\x8e\\xd8\\x8d\\xbd\\xdbZ\\x98<\\r\\xa3\\xa3==g3\\xbd$\\xcd\\xbd\\xbd\\xac$\\xf7;\\xf4\\xf4z=\\xff\\xb4\\x8c=\\x8d\\x0e\\xc6\\xbdmI\\x90\\xbdD\\x16\\xbd;\\x83\\xe7\\x0c\\xbd\\x193\\xc9\\xbc\\x9c\\xf8\\xbb\\xbcf&u\\xbb1\\x8f\\xca<\\xf6\\x7fJ=\\x10\\xaf*=\\x86OU\\xbd\\xd2\\xf0\\x95\\xbc#\\x02\\x19=1\\xf4K<\\xcd\\xc2\\t=H\\x83\\xac=\\xa0\\xd7\\xb8\\xbd\\xf4\\xb5\\x9c\\xbd?\\x85\\x18=\\x9ad&=03\\xf8<\\xd5\\xf7\\x88<[v\\xf2\\xbb==[\\xbd\\x06\\xad\\xee\\xbb;:A\\xbd\\xdbd\\x19\\xbd\\x13d\\xf2\\xbb\\xde\\xb9x;\\xc4;O<\\xcf1,\\xbc\\xeb\\xae\\xae=\\x8c\\x00-\\xbc\\x14\\x06\\xae\\xbdo\\xd6\\x1a=\\xcc\\xbf\\xcd=\\'\\x150=\\xe4\\xf1\\x9d\\xbc\\xaaGK=\\xae\\xb8 =\\xa8\\xf1I\\xbd+e\\x9e\\xbbp\\x8b\\xf7:\\x95\\xf8\\x1c=\\xa3\\xba\\xde<=o\\x16\\xbb\\xc2]p\\xbb\\x9d\\xd5<<\\x8b\\x91\\xa3\\xb8\\xda9s</4&<\\x7f\\x1c\\x18<\\x10\\xd9-\\xbd\\xd3\\xe6\\x98< \\xa1N=\\xa1/\\xa5=*\\xf3\\xdd<>L\\x13<\\xa4\\x10\\xce\\xbau\\x9e\\xdc\\xbc\\xa28\\x05=-\\xa1\\xf5<z\\xd8R\\xbc\\xf0\\x93-\\xbc\\x92\\xeb\\\\9\\xa01\\xec=Z\\xf5V=(G\\xd6\\xbc\\x93\"S=\\xce\\xd9^\\xbd\\xb1\\xa3\\x91<\\xf7\\xd9\\x13<\\xc0\\xb2y\\xbbv\\x8d/\\xbd\\xf5\\x06p<\\xe0\\x94\\x88\\xbb\\xbe\\xd0\\x9d<H\\x02\\xb6=\\x1b\\xf1\\xad\\x89\\x1c\\xf2\\xa6\\xbb\\xba\\xfbe\\xbd\\xa6QK\\xbb\\xeb\\x0f\\x90\\xbb\\tt\\xb9\\xbb&\\xc9\\xb9\\xbb\\xf7hk=\\x9e\\r\\xad<8f\\xa8\\xbd\\xc3]\\xcc=\\x18\\xe0 \\xbe\\xc24/\\xbdkf\\xf7\\xbcZ\\x9av=\\xed\\x0bq<0\\xda\\x1c\\xbd\\x00\\t\\x8b<\\xe3n\\xa6\\xbc\\xc2t\\x86<\\x81\\x87\\x19=v\\xae\\xe4\\xbc\\xeal^\\xbc\\x10V\\x0e\\xbdW\\xb0\\xe3\\xbc\\xd4FU;\\xaaG|\\xbd^\\xfb\\x8b\\xbd\\x83\\x81*\\xbdz\\x83\\xf4=Q\\xb7\\x10;\\x11!\\x0e\\xbd\\xf6\\xd3\\xb4=)&\\x15\\xbdR\\x86\\x83=7$:<\\x17\\xea\\xa1<\\x1d\\x08\\x8d\\xbd\\x9e\\x00i\\xbd)\\x01\\xdd<\\x05\\x87\\xd6=6\\x98\\x9c\\xbd]|!\\xbd\\xb1\\x1c\\xc5\\xb9\\xc1\\xb3\\x93\\xbc\\xec\\xcb>\\xbdy\\x1bF\\xbd\\x9f?\\x14\\xbe\\xc1\\x8f(\\xbd\\xdeO\\x89\\xbd\\xfd\\xad\\xd4<\\xa5\\x12\\xc3=\\xb9\\x05O\\xbdu\\x8ep\\xbc.\\xb5\\xac\\xbc\\xc9\\x9ee\\xbdf\\x8es;ga\\xc1;\\xd1\\xfaB\\xbdv$\\xfe:\\x95\\xe6\\xf4=\\xcb\\x15*<\\x81\\xf8\\x1b=\\xfb\\xfbV\\xbd\\xd7\\xd1\\r=0\\xee\\x06=\\x17u\\xba\\xbd\\xfd\\xa3\\xd6<\\xb6\\xeb\\xd9;\\xbc9/=\\xa8\\xc2\\x85=|\\x0b\"=\\xf8i\\xef<@\\xe8c=\\xfd2\\x08\\xbe\\xe1\\x12;=\\x0cVW;Z\\xa4b<\\xd9\\x9d\\xb7<\\x8br;\\xbdhz\\x91\\xbcM\\x00<\\xbd\\x11\\x1a\\xa3<\\xfeJ%\\xbc\\x1d\\xe7\\xbf\\xbbs\\x87\\x12=\\x9b\\x1d\\x95=\\x80|\\xfd\\xbc\\xf0\\xf1\\xd1\\xbdaz\\x84;\\xc5\\tu=7\\x8ai<9\\x91R\\xbd\\xec\\xf3m\\xbd\\x85\\xb83=]\\xedF=#\\xf3\\xd1\\x08`A\\xba<\\x13\\xacO\\xbdX\\x0f\\xc7;\\x82\\xf4\\x04\\xbdN\\x82\\x92\\xbd\\xa4\\xddD={\\xd8;\\xbc\\xb7;\\xf4\\xbc\\xb2\\x8f\\x97\\xbd7\\\\\\r\\xbd\\xe1\\x8c\\xf5\\xbd\\x9d\\x13(=\\xa3\\xc8\\xc6=\\xab\\xed\\x1a=\\x95\\xa8\\xf8=\\x9b\\xc1\\xee\\xbc\\xd6.\\x18\\xbb\\xb7~;<\\xd7F\\t\\xbd\\x19\\x08\\x17=\\xa6\\xa5\\x1e=\\x14K\\xcb\\xbd0\\xf7\\x8c\\xbdQb\\xed\\xbb\\x9f[\\x19\\xbc\\x19\\x0c\\x13\\xbccq\\x83=\\xe7wd\\xbd\\x86\\xc7\\xd1\\xbb^lY\\xbc\\xa6|a=Z\\xcf\\xfd\\xbc\\x08\\xa5\\x83\\xbb\\xa5O\\x19\\xbd-\\x02]\\xbd\\xc9\\xeaz=\\xf95\\x9c=2^\\xa9\\xbdy^9\\xbcG\\xe4N\\xbc}\\x07x\\xbd\\x1b{\\xa0=^\\x9f\\x96<\\xc0r8\\xba\\x9a\\xbb=\\xbd\\x03}(<\\x90\\xdf\\xb4\\xbbs\\xc9\\x0b\\xbd\\xc3\\x01\\x95\\xbd\\xf3\\xc6T=\\xf1o\\xd1<m3\\'\\xbd\\xc2\\x00H<y\\x0f\\xb8<\\x1fU}\\xbdw}\\x01=\\xfb\\xc9[\\xbd2\\xed\\x03<\\x04c\\x0f\\xbd\\xd3\\x8e\\xe7\\xbc\\x01\\x17\\xc8<\\xb9\\x08\\x9c=\\xfb\\xd4\\xa6\\xbdIQ\\xe5=s-O<\\xde\\xf22=\\xd626\\xbd\\xfc\\xe2\\xf2\\xbc\\xf9Z\\xa7<u\\xb2\\x02\\xbe\\xefn2\\xbd\\xad\\x80\\x8a=\\xfe\\xc3\\t\\xbd!\\xb8\\xcd\\xbc1\\xdb\\xad\\xbb\\x9c\\xfd\\xde\\xbc\\x02\\x17E=~\\x06\\xbd= &m=\\x16X\\x8e=\\xe7\\xae\\xa4=$\\xa3\\xca\\xbb\\xa6?\\r\\xbc\\xc59\\x9c<Jl\\x8e\\xbd+\\x10k=\\xb3\\xff\\xbe;w\\x16\\xff<g8\\x92<Q2\\x8c;M\\xac\\xa3=\\xcb+v<Z\\x89\\xf0\\xbb\\xa3\\xdb\\x07\\xbc\\x8a\\xae\\xbd\\xbc\\xb3\\x8f\\xb4\\xb2\\xae\\x88\\xd3\\xbd}\\xe9j=\\xea\\xdb\\xa7\\xbd\\xec\\x16\\xe5<8\\x19p=\\xf1\\xe6\\x9f=\\xe2Gh\\xbd\\xee\\x9a\\xf4\\xbdX\\xd6\\x07\\xbcS\\xddj<&tw=\\x87\\xd0\\xe1<\\xc5\\xae\\x1e=\\xdfF\\xc4<\\x9c0\\n<\\x11jd\\xbc\\xb3\\xfa\\x8c=\\xf0\\x8a\\xa6\\xbc&.\\xcd\\xba\\xaap\\xcf=,T-\\xbd\\xc7Qn=7\\x16\\x9f\\xbd\\x8e\\x8a_=F\\x13#<\\xf8\\xabm;&\\x86\\x00\\xbe\\xc6y\\x8b\\xbcuUG=\\x9d\\xa3\\xca=\\xd4\\xbdq\\xbc\\x9a\\xe3:\\xbd\\x9a\\x93]</]\\xd1;\\xc2d\\xae=\\xa7\\x1f\\x03\\xbd\\x05L\\x89=r:s\\xbd\\xa0\\x1eb=\\xef\\xc9\\t\\xbb\\x95\\xfb\\xe7<\\x91\\xce\\xe4\\xbcLV#=t\\x8bk\\xbd\\xc6h\\x01<\\xf5\\xe5N\\xbdN\\x0c\\xb1\\xbcg\\x01J\\xbde\\xf4~==\\xe3Z\\xbd)/\\xf1\\xbbG98=\\x912T\\xbc(\\'\\x81<\\xbd\\xa0M=\\xa7\\xde\\x05<\\x18I|\\xbd\\xbe\\x98w<\\xdf\\xd3\\xa7\\xbd\\xdcS \\xbd\\x83\\x13\\x07=\\xdc%\\x14\\xbc\\xafv\\xe9<\\xf1,\\x97='}]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "movie_data[:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define Redis index schema\n",
    "\n",
    "Below, we build a schema that represents our movie objects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from redisvl.schema import IndexSchema\n",
    "from redisvl.index import SearchIndex\n",
    "\n",
    "\n",
    "schema = IndexSchema.from_dict({\n",
    "  \"index\": {\n",
    "    \"name\": \"movies\",\n",
    "    \"prefix\": \"movie\",\n",
    "    \"storage\": \"hash\"\n",
    "  },\n",
    "  \"fields\": [\n",
    "    { \"name\": \"title\", \"type\": \"text\" },\n",
    "    { \"name\": \"description\", \"type\": \"text\" },\n",
    "    { \"name\": \"genre\", \"type\": \"tag\", \"attrs\": {\"sortable\": True}},\n",
    "    { \"name\": \"rating\", \"type\": \"numeric\", \"attrs\": {\"sortable\": True}},\n",
    "    {\n",
    "        \"name\": \"description_vector\",\n",
    "        \"type\": \"vector\",\n",
    "        \"attrs\": {\n",
    "            \"dims\": 384,\n",
    "            \"distance_metric\": \"cosine\",\n",
    "            \"algorithm\": \"hnsw\",\n",
    "            \"datatype\": \"float32\"\n",
    "        }\n",
    "    }\n",
    "  ]\n",
    "})\n",
    "\n",
    "\n",
    "index = SearchIndex(schema, client, validate_on_load=True)\n",
    "index.create(overwrite=True, drop=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Populate index\n",
    "\n",
    "Load movie objects into Redis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['movie:01KNMR6J0SSDJPRSNPBJK0CJCP',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76C',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76D',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76E',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76F',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76G',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76H',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76J',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76K',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76M',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76N',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76P',\n",
       " 'movie:01KNMR6J0V9MCZ689A93H5S76Q',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG1W',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG1X',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG1Y',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG1Z',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG20',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG21',\n",
       " 'movie:01KNMR6J0WNHH9FFQSW5R5RG22']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "index.load(movie_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hybrid Search Approaches\n",
    "\n",
    "Now that our search index is populated and ready, we will build out a few different hybrid search techniques in Redis.\n",
    "\n",
    "To start, we will use the `HybridQuery` class that accepts a text string and vector to automatically combine text similarity and vector similarity scores.\n",
    "\n",
    "At a minimum, the `HybridQuery` class requires the following arguments:\n",
    "\n",
    "```python\n",
    "query = HybridQuery(\n",
    "    text=\"your query string here\",\n",
    "    text_field_name=\"<name of the text field in the index to do text search in>\",\n",
    "    vector=<bytes or numeric array, ex: [0.1, 0.2, 0.3]>,\n",
    "    vector_field_name=\"<name of the vector field in the index to compute vector similarity>\",\n",
    ")\n",
    "```\n",
    "\n",
    "This defaults to using the reciprocal rank fusion (RRF) method to combine scores, and only outputs the final keys and combined scores. A more common minimal usage might be:\n",
    "\n",
    "```python\n",
    "query = HybridQuery(\n",
    "    text=\"your query string here\",\n",
    "    text_field_name=\"<name of the text field in the index to do text search in>\",\n",
    "    vector=<bytes or numeric array, ex: [0.1, 0.2, 0.3]>,\n",
    "    vector_field_name=\"<name of the vector field in the index to compute vector similarity>\",\n",
    "    combination_method=\"RRF\",\n",
    "    rrf_window=20,\n",
    "    yield_text_score_as=\"text_score\",\n",
    "    yield_vsim_score_as=\"vector_similarity\",\n",
    "    yield_combined_score_as=\"hybrid_score\",\n",
    "    return_fields=[\"<list of fields to return>\"],\n",
    ")\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Linear Combination\n",
    "\n",
    "The goal of this technique is to calculate a weighted sum of the text similarity score for our provided text search and the vector similarity score for our provided vector.\n",
    "\n",
    "The FT.HYBRID API introduced in Redis 8.4.0 supports a linear combination of text and vector scores (accessible as of RedisVL 0.13.0 in `HybridQuery`), and it is also possible with the aggregations API, as of `Redis 7.4.x` (search version `2.10.5` - accessible as of RedisVl 0.5.0 in `AggregateHybridQuery`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Sample user query (can be changed for comparisons)\n",
    "user_query = \"action adventure movie with great fighting scenes against a dangerous criminal, crime busting, superheroes, and magic\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text_score</th>\n",
       "      <th>title</th>\n",
       "      <th>vector_similarity</th>\n",
       "      <th>hybrid_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9.1624524482</td>\n",
       "      <td>The Incredibles</td>\n",
       "      <td>0.677012205124</td>\n",
       "      <td>3.22264427805</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5.02411250758</td>\n",
       "      <td>Skyfall</td>\n",
       "      <td>0.601227432489</td>\n",
       "      <td>1.92809295502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.13361061261</td>\n",
       "      <td>Explosive Pursuit</td>\n",
       "      <td>0.695675313473</td>\n",
       "      <td>1.72705590321</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      text_score              title vector_similarity   hybrid_score\n",
       "0   9.1624524482    The Incredibles    0.677012205124  3.22264427805\n",
       "1  5.02411250758            Skyfall    0.601227432489  1.92809295502\n",
       "2  4.13361061261  Explosive Pursuit    0.695675313473  1.72705590321"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "from redisvl.query.hybrid import HybridQuery\n",
    "\n",
    "vector = model.embed(user_query, as_buffer=True)\n",
    "\n",
    "query = HybridQuery(\n",
    "\ttext=user_query,\n",
    "\ttext_field_name=\"description\",\n",
    "\tvector=vector,\n",
    "\tvector_field_name=\"description_vector\",\n",
    "\tcombination_method=\"LINEAR\",\n",
    "\tyield_text_score_as=\"text_score\",\n",
    "\tyield_vsim_score_as=\"vector_similarity\",\n",
    "\tyield_combined_score_as=\"hybrid_score\",\n",
    "\treturn_fields=[\"title\"],\n",
    ")\n",
    "\n",
    "results = index.query(query)\n",
    "pd.DataFrame(results[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, for Redis versions prior to 8.4.0, we can use the aggregations API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Query being executed:\n",
      "(~@description:(action | adventure | movie | great | fighting | scenes | dangerous | criminal | crime | busting | superheroes | magic))=>[KNN 10 @description_vector $vector AS vector_distance]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>vector_distance</th>\n",
       "      <th>title</th>\n",
       "      <th>vector_similarity</th>\n",
       "      <th>text_score</th>\n",
       "      <th>hybrid_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.645975589752</td>\n",
       "      <td>The Incredibles</td>\n",
       "      <td>0.677012205124</td>\n",
       "      <td>9.1624524482</td>\n",
       "      <td>3.22264427805</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.797545135021</td>\n",
       "      <td>Skyfall</td>\n",
       "      <td>0.601227432489</td>\n",
       "      <td>5.02411250758</td>\n",
       "      <td>1.92809295502</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.608649373055</td>\n",
       "      <td>Explosive Pursuit</td>\n",
       "      <td>0.695675313473</td>\n",
       "      <td>4.13361061261</td>\n",
       "      <td>1.72705590321</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  vector_distance              title vector_similarity     text_score  \\\n",
       "0  0.645975589752    The Incredibles    0.677012205124   9.1624524482   \n",
       "1  0.797545135021            Skyfall    0.601227432489  5.02411250758   \n",
       "2  0.608649373055  Explosive Pursuit    0.695675313473  4.13361061261   \n",
       "\n",
       "    hybrid_score  \n",
       "0  3.22264427805  \n",
       "1  1.92809295502  \n",
       "2  1.72705590321  "
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from redisvl.query.aggregate import AggregateHybridQuery\n",
    "\n",
    "agg_query = AggregateHybridQuery(\n",
    "    text=user_query,\n",
    "    text_field_name=\"description\",\n",
    "    vector=vector,\n",
    "    vector_field_name=\"description_vector\",\n",
    "    return_fields=[\"title\"],\n",
    ")\n",
    "\n",
    "print(f\"Query being executed:\\n{agg_query._build_query_string()}\")\n",
    "\n",
    "results = index.query(agg_query)\n",
    "pd.DataFrame(results[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Choosing your stopwords for better queries\n",
    "You can see that the user query string has been tokenized and certain stopwords like 'and', 'for', 'with', 'but', have been removed, otherwise you would get matches on irrelevant words.\n",
    "RedisVL uses [NLTK](https://www.nltk.org/index.html) english stopwords as the the default. You can change which default language stopwords to use with the `stopwords` argument.\n",
    "You specify a language, like 'german', 'arabic', 'greek' and many others, provide your own list of stopwords, or set it to `None` to not remove any.\n",
    "\n",
    "Note that both `HybridQuery` and `AggregateHybridQuery` process stopwords identically."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(~@description:(film | d\\'action | d\\'aventure | superbes | scènes | combat | enquêtes | criminelles | super\\-héros | magie))\n",
      "(~@description:(action | adventure | movie | great | fighting | scenes | against | dangerous | criminal | crime | busting | superheroes | magic))\n",
      "(~@description:(action | adventure | movie | with | great | fighting | scenes | against | a | dangerous | criminal | crime | busting | superheroes | and | magic))\n"
     ]
    }
   ],
   "source": [
    "# translate our user query to French and use nltk french stopwords\n",
    "french_query_text = \"Film d'action et d'aventure avec de superbes scènes de combat, des enquêtes criminelles, des super-héros et de la magie\"\n",
    "\n",
    "french_film_query = HybridQuery(\n",
    "    text=french_query_text,\n",
    "    text_field_name=\"description\",\n",
    "    vector=model.embed(french_query_text, as_buffer=True),\n",
    "    vector_field_name=\"description_vector\",\n",
    "    stopwords=\"french\",\n",
    ")\n",
    "\n",
    "print(french_film_query.query._search_query.query_string())\n",
    "\n",
    "# specify your own stopwords\n",
    "custom_stopwords = set([\n",
    "    \"a\", \"is\", \"the\", \"an\", \"and\", \"are\", \"as\", \"at\", \"be\", \"but\", \"by\", \"for\",\n",
    "    \"if\", \"in\", \"into\", \"it\", \"no\", \"not\", \"of\", \"on\", \"or\", \"such\", \"that\", \"their\",\n",
    "    \"then\", \"there\", \"these\", \"they\", \"this\", \"to\", \"was\", \"will\", \"with\"\n",
    "])\n",
    "\n",
    "stopwords_query = HybridQuery(\n",
    "    text=user_query,\n",
    "    text_field_name=\"description\",\n",
    "    vector=vector,\n",
    "    vector_field_name=\"description_vector\",\n",
    "    stopwords=custom_stopwords,\n",
    ")\n",
    "\n",
    "print(stopwords_query.query._search_query.query_string())\n",
    "\n",
    "# don't use any stopwords\n",
    "no_stopwords_query = HybridQuery(\n",
    "    text=user_query,\n",
    "    text_field_name=\"description\",\n",
    "    vector=vector,\n",
    "    vector_field_name=\"description_vector\",\n",
    "    stopwords=None,\n",
    ")\n",
    "\n",
    "print(no_stopwords_query.query._search_query.query_string())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Choosing your text scoring function and weights\n",
    "There are different ways to calculate the similarity between sets of text. Options for text scoring functions are TFIDF, TFIDF.DOCNORM, BM25STD, BM25STD.NORM, BM25STD.TANH, DISMAX, DOCSCORE, and HAMMING; the default is BM25STD and is easy to configure with the `text_scorer` parameter. Just like changing you embedding model can change your vector similarity scores, changing your text similarity measure can change your text scores.\n",
    "\n",
    ">  For more information about supported scoring algorithms, see [the Redis documentation on scoring](https://redis.io/docs/latest/develop/ai/search-and-query/advanced-concepts/scoring/).\n",
    "\n",
    "When combining text and vector scores using a linear combination (`combination_method=\"LINEAR\"` in `HybridQuery` and the only option for `AggregateHybridQuery`), you can control the relative balance of these scores with tunable parameters.\n",
    "\n",
    "The FT.HYBRID API calculates the combined score as:\n",
    "\n",
    "```python\n",
    "hybrid_score = {alpha} * text_score + {beta} * vector_similarity\n",
    "```\n",
    "\n",
    "Where `alpha` can be provided to `HybridQuery` via the `linear_alpha` and `beta` is calculated as `1 - alpha`. FT.HYBRID defaults to `alpha=0.3`.\n",
    "\n",
    "`AggregateHybridQuery` defines the combined score in reverse as:\n",
    "\n",
    "```python\n",
    "hybrid_score = {1-alpha} * text_score + {alpha} * vector_similarity\n",
    "```\n",
    "\n",
    "Where the `alpha` parameter is configurable on the `AggregateHybridQuery` class. If not specified, it defaults to `0.7`.\n",
    "\n",
    "Try changing the `text_scorer` and `linear_alpha` parameters in the query below to see how results may change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text_score</th>\n",
       "      <th>title</th>\n",
       "      <th>description</th>\n",
       "      <th>vector_similarity</th>\n",
       "      <th>hybrid_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>6</td>\n",
       "      <td>Explosive Pursuit</td>\n",
       "      <td>A daring cop chases a notorious criminal acros...</td>\n",
       "      <td>0.695675313473</td>\n",
       "      <td>4.67391882837</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>6</td>\n",
       "      <td>Despicable Me</td>\n",
       "      <td>When a criminal mastermind uses a trio of orph...</td>\n",
       "      <td>0.651065170765</td>\n",
       "      <td>4.66276629269</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6</td>\n",
       "      <td>Skyfall</td>\n",
       "      <td>James Bond returns to track down a dangerous n...</td>\n",
       "      <td>0.601227432489</td>\n",
       "      <td>4.65030685812</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  text_score              title  \\\n",
       "0          6  Explosive Pursuit   \n",
       "1          6      Despicable Me   \n",
       "2          6            Skyfall   \n",
       "\n",
       "                                         description vector_similarity  \\\n",
       "0  A daring cop chases a notorious criminal acros...    0.695675313473   \n",
       "1  When a criminal mastermind uses a trio of orph...    0.651065170765   \n",
       "2  James Bond returns to track down a dangerous n...    0.601227432489   \n",
       "\n",
       "    hybrid_score  \n",
       "0  4.67391882837  \n",
       "1  4.66276629269  \n",
       "2  4.65030685812  "
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tfidf_query = HybridQuery(\n",
    "    text=user_query,\n",
    "    text_field_name=\"description\",\n",
    "    vector=vector,\n",
    "    vector_field_name=\"description_vector\",\n",
    "    text_scorer=\"TFIDF\", # can be one of [TFIDF, TFIDF.DOCNORM, BM25, DISMAX, DOCSCORE, BM25STD]\n",
    "    stopwords=None,\n",
    "\tcombination_method=\"LINEAR\",\n",
    "    linear_alpha=0.75, # weight the text score higher\n",
    "    return_fields=[\"title\", \"description\"],\n",
    "\tyield_text_score_as=\"text_score\",\n",
    "    yield_vsim_score_as=\"vector_similarity\",\n",
    "    yield_combined_score_as=\"hybrid_score\",\n",
    ")\n",
    "\n",
    "results = index.query(tfidf_query)\n",
    "pd.DataFrame(results[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Reciprocal Rank Fusion (RRF)\n",
    "\n",
    "Instead of relying on document scores like cosine similarity and BM25/TFIDF, we can fetch items and focus on their rank. This rank can be utilized to create a new ranking metric known as [Reciprocal Rank Fusion (RRF)](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf). RRF is powerful because it can handle ranked lists of different length, scores of different scales, and other complexities.\n",
    "\n",
    "The FT.HYBRID API introduced in Redis 8.4.0 supports using RRF to combine results from text and vector queries (accessible as of RedisVL 0.13.0 in `HybridQuery`). Unless otherwise specified, RRF is the default combination method.\n",
    "\n",
    "The parameters available to customize the behaviour of RRF are `rrf_window` and `rrf_constant`. The `rrf_window` parameter controls the size of the window over which the RRF score is calculated, and the `rrf_constant` parameter controls the constant used in the RRF formula. Try changing these parameters to see how results may change."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text_score</th>\n",
       "      <th>title</th>\n",
       "      <th>description</th>\n",
       "      <th>vector_similarity</th>\n",
       "      <th>hybrid_score</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>9.1624524482</td>\n",
       "      <td>The Incredibles</td>\n",
       "      <td>A family of undercover superheroes, while tryi...</td>\n",
       "      <td>0.677012205124</td>\n",
       "      <td>0.032522474881</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>4.13361061261</td>\n",
       "      <td>Explosive Pursuit</td>\n",
       "      <td>A daring cop chases a notorious criminal acros...</td>\n",
       "      <td>0.695675313473</td>\n",
       "      <td>0.032266458496</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4.13361061261</td>\n",
       "      <td>The Dark Knight</td>\n",
       "      <td>Batman faces off against the Joker, a criminal...</td>\n",
       "      <td>0.673311829567</td>\n",
       "      <td>0.031498015873</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      text_score              title  \\\n",
       "0   9.1624524482    The Incredibles   \n",
       "1  4.13361061261  Explosive Pursuit   \n",
       "2  4.13361061261    The Dark Knight   \n",
       "\n",
       "                                         description vector_similarity  \\\n",
       "0  A family of undercover superheroes, while tryi...    0.677012205124   \n",
       "1  A daring cop chases a notorious criminal acros...    0.695675313473   \n",
       "2  Batman faces off against the Joker, a criminal...    0.673311829567   \n",
       "\n",
       "     hybrid_score  \n",
       "0  0.032522474881  \n",
       "1  0.032266458496  \n",
       "2  0.031498015873  "
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query = HybridQuery(\n",
    "\ttext=user_query,\n",
    "\ttext_field_name=\"description\",\n",
    "\tvector=vector,\n",
    "\tvector_field_name=\"description_vector\",\n",
    "\tcombination_method=\"RRF\",\n",
    "\trrf_window=20,\n",
    "\trrf_constant=60,\n",
    "\tyield_text_score_as=\"text_score\",\n",
    "\tyield_vsim_score_as=\"vector_similarity\",\n",
    "\tyield_combined_score_as=\"hybrid_score\",\n",
    "\treturn_fields=[\"title\", \"description\"],\n",
    ")\n",
    "\n",
    "results = index.query(query)\n",
    "pd.DataFrame(results[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Client-side RRF for older Redis versions\n",
    "\n",
    "When using Redis versions prior to 8.4.0, you can still perform RRF by fetching the top-k results from both the text and vector queries, and then fusing them together on the client-side."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fuse_rankings_rrf(*ranked_lists, weights=None, k=60):\n",
    "    \"\"\"\n",
    "    Perform Weighted Reciprocal Rank Fusion on N number of ordered lists.\n",
    "    \"\"\"\n",
    "    item_scores = {}\n",
    "    \n",
    "    if weights is None:\n",
    "        weights = [1.0] * len(ranked_lists)\n",
    "    else:\n",
    "        assert len(weights) == len(ranked_lists), \"Number of weights must match number of ranked lists\"\n",
    "        assert all(0 <= w <= 1 for w in weights), \"Weights must be between 0 and 1\"\n",
    "    \n",
    "    for ranked_list, weight in zip(ranked_lists, weights):\n",
    "        for rank, item in enumerate(ranked_list, start=1):\n",
    "            if item not in item_scores:\n",
    "                item_scores[item] = 0\n",
    "            item_scores[item] += weight * (1 / (rank + k))\n",
    "    \n",
    "    # Sort items by their weighted RRF scores in descending order\n",
    "    return sorted(item_scores.items(), key=lambda x: x[1], reverse=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(2, 0.04814747488101534),\n",
       " (1, 0.032266458495966696),\n",
       " (6, 0.03200204813108039),\n",
       " (5, 0.01639344262295082),\n",
       " (4, 0.016129032258064516),\n",
       " (3, 0.015873015873015872),\n",
       " (7, 0.015625),\n",
       " (8, 0.015384615384615385)]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Below is a simple example of RRF over a few lists of numbers\n",
    "fuse_rankings_rrf([1, 2, 3], [2, 4, 6, 7, 8], [5, 6, 1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll want some helper functions to  construct our individual text and vector queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to create a vector query using RedisVL helpers for ease of use\n",
    "from redisvl.query import VectorQuery, TextQuery\n",
    "\n",
    "\n",
    "def make_vector_query(user_query: str, num_results: int, filters = None) -> VectorQuery:\n",
    "    \"\"\"Generate a Redis vector query given user query string.\"\"\"\n",
    "    vector = model.embed(user_query, as_buffer=True)\n",
    "    query = VectorQuery(\n",
    "        vector=vector,\n",
    "        vector_field_name=\"description_vector\",\n",
    "        num_results=num_results,\n",
    "        return_fields=[\"title\", \"description\"]\n",
    "    )\n",
    "    if filters:\n",
    "        query.set_filter(filters)\n",
    "    return query\n",
    "\n",
    "\n",
    "def make_ft_query(text_field: str, user_query: str, num_results: int) -> TextQuery:\n",
    "    \"\"\"Generate a Redis full-text query given a user query string.\"\"\"\n",
    "    return TextQuery(\n",
    "        text=user_query,\n",
    "        text_field_name=text_field,\n",
    "        text_scorer=\"BM25\",\n",
    "        num_results=num_results,\n",
    "        return_fields=[\"title\", \"description\"],\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import List, Dict, Any\n",
    "\n",
    "\n",
    "def weighted_rrf(\n",
    "    user_query: str,\n",
    "    alpha: float = 0.5,\n",
    "    num_results: int = 4,\n",
    "    k: int = 60,\n",
    ") -> List[Dict[str, Any]]:\n",
    "    \"\"\"Implemented client-side RRF after querying from Redis.\"\"\"\n",
    "    # Create the vector query\n",
    "    vector_query = make_vector_query(user_query, num_results=len(movie_data))\n",
    "\n",
    "    # Create the full-text query\n",
    "    full_text_query = make_ft_query(\"description\", user_query, num_results=len(movie_data))\n",
    "\n",
    "    # Run queries individually\n",
    "    vector_query_results = index.query(vector_query)\n",
    "    full_text_query_results = index.query(full_text_query)\n",
    "\n",
    "    # Extract titles from results\n",
    "    vector_titles = [movie[\"title\"] for movie in vector_query_results]\n",
    "    full_text_titles = [movie[\"title\"] for movie in full_text_query_results]\n",
    "\n",
    "    # Perform weighted RRF\n",
    "    return fuse_rankings_rrf(vector_titles, full_text_titles, weights=[alpha, 1-alpha], k=k)[:num_results]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Explosive Pursuit', 0.01639344262295082),\n",
       " ('The Dark Knight', 0.015873015873015872),\n",
       " ('Despicable Me', 0.015625),\n",
       " ('The Incredibles', 0.015417457305502846),\n",
       " ('Skyfall', 0.0152073732718894),\n",
       " ('Finding Nemo', 0.014242424242424244)]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test it out!\n",
    "weighted_rrf(user_query, num_results=6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But say we want to give more weight to the vector search rankings in this case to boost semantic similarities contribution to the final rank:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('Explosive Pursuit', 0.01639344262295082),\n",
       " ('The Dark Knight', 0.015873015873015872),\n",
       " ('The Incredibles', 0.015702087286527514),\n",
       " ('Despicable Me', 0.015625),\n",
       " ('Skyfall', 0.014838709677419354),\n",
       " ('Finding Nemo', 0.01387878787878788)]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "weighted_rrf(user_query, alpha=0.7, num_results=6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Client-side reranking\n",
    "\n",
    "An alternative approach to RRF is to simply use an external reranker to order the final recommendations. RedisVL has built-in integrations to a few popular reranking modules."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Loading weights: 100%|██████████| 105/105 [00:00<00:00, 17207.23it/s]\n",
      "\u001b[1mBertForSequenceClassification LOAD REPORT\u001b[0m from: cross-encoder/ms-marco-MiniLM-L-6-v2\n",
      "Key                          | Status     |  | \n",
      "-----------------------------+------------+--+-\n",
      "bert.embeddings.position_ids | UNEXPECTED |  | \n",
      "\n",
      "Notes:\n",
      "- UNEXPECTED:\tcan be ignored when loading from different task/architecture; not ok if you expect identical arch.\n"
     ]
    }
   ],
   "source": [
    "from redisvl.utils.rerank import HFCrossEncoderReranker\n",
    "\n",
    "# Load the ms marco MiniLM cross encoder model from huggingface\n",
    "reranker = HFCrossEncoderReranker(\"cross-encoder/ms-marco-MiniLM-L-6-v2\")\n",
    "\n",
    "\n",
    "def rerank(\n",
    "    user_query: str,\n",
    "    num_results: int = 4,\n",
    ") -> List[Dict[str, Any]]:\n",
    "    \"\"\"Rerank the candidates based on the user query with an external model/module.\"\"\"\n",
    "    # Create the vector query\n",
    "    vector_query = make_vector_query(user_query, num_results=num_results)\n",
    "\n",
    "    # Create the full-text query\n",
    "    full_text_query = make_ft_query(\"description\", user_query, num_results=num_results)\n",
    "\n",
    "    # Run queries individually\n",
    "    vector_query_results = index.query(vector_query)\n",
    "    full_text_query_results = index.query(full_text_query)\n",
    "\n",
    "    # Assemble list of potential movie candidates with their IDs\n",
    "    movie_map = {}\n",
    "    for movie in vector_query_results + full_text_query_results:\n",
    "        candidate = f\"Title: {movie['title']}. Description: {movie['description']}\"\n",
    "        if candidate not in movie_map:\n",
    "            movie_map[candidate] = movie\n",
    "\n",
    "    # Rerank candidates\n",
    "    reranked_movies, scores = reranker.rank(\n",
    "        query=user_query,\n",
    "        docs=list(movie_map.keys()),\n",
    "        limit=num_results,\n",
    "        return_score=True\n",
    "    )\n",
    "\n",
    "    # Fetch full movie objects for the reranked results\n",
    "    return [\n",
    "        (movie_map[movie['content']][\"title\"], score)\n",
    "        for movie, score in zip(reranked_movies, scores)\n",
    "    ]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('The Incredibles', -4.1636810302734375),\n",
       " ('Explosive Pursuit', 0.8551025390625),\n",
       " ('The Dark Knight', -4.403158664703369),\n",
       " ('Skyfall', -7.830077171325684),\n",
       " ('Mad Max: Fury Road', -7.7119951248168945),\n",
       " ('Despicable Me', -8.742402076721191)]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Test it out!\n",
    "rerank(user_query, num_results=6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This technique is certainly much slower than simple RRF as it's running an additional cross-encoder model to rerank the results. This can be fairly computationally expensive, but tunable with enough clarity on the use case and focus (how many items to retrieve? how many items to rerank? model accleration via GPU?)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Post-processing configuration with FT.HYBRID\n",
    "\n",
    "The FT.HYBRID API also allows for post-processing of the results (e.g. aggregations and aliasing)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>genre</th>\n",
       "      <th>max_hybrid_score</th>\n",
       "      <th>avg_hybrid_score</th>\n",
       "      <th>count</th>\n",
       "      <th>max_rating</th>\n",
       "      <th>min_rating</th>\n",
       "      <th>rating_range</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>comedy</td>\n",
       "      <td>0.032522474881</td>\n",
       "      <td>0.0172438509297</td>\n",
       "      <td>10</td>\n",
       "      <td>8</td>\n",
       "      <td>6</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>action</td>\n",
       "      <td>0.032266458496</td>\n",
       "      <td>0.0266136701383</td>\n",
       "      <td>10</td>\n",
       "      <td>9</td>\n",
       "      <td>6</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    genre max_hybrid_score avg_hybrid_score count max_rating min_rating  \\\n",
       "0  comedy   0.032522474881  0.0172438509297    10          8          6   \n",
       "1  action   0.032266458496  0.0266136701383    10          9          6   \n",
       "\n",
       "  rating_range  \n",
       "0            2  \n",
       "1            3  "
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from redis.commands.search import reducers\n",
    "\n",
    "query = HybridQuery(\n",
    "\ttext=user_query,\n",
    "\ttext_field_name=\"description\",\n",
    "\tvector=vector,\n",
    "\tvector_field_name=\"description_vector\",\n",
    "\tcombination_method=\"RRF\",\n",
    "\trrf_window=20,\n",
    "\tyield_text_score_as=\"text_score\",\n",
    "\tyield_vsim_score_as=\"vector_similarity\",\n",
    "\tyield_combined_score_as=\"hybrid_score\",\n",
    "\treturn_fields=[\"title\", \"genre\", \"description\", \"rating\"],\n",
    "\tnum_results=20,\n",
    ")\n",
    "\n",
    "query.postprocessing_config.group_by(\n",
    "\t\"@genre\",\n",
    "\treducers.max(\"@hybrid_score\").alias(\"max_hybrid_score\"),\n",
    "\treducers.avg(\"@hybrid_score\").alias(\"avg_hybrid_score\"),\n",
    "\treducers.count().alias(\"count\"),\n",
    "\treducers.max(\"@rating\").alias(\"max_rating\"),\n",
    "\treducers.min(\"@rating\").alias(\"min_rating\"),\n",
    ").apply(\n",
    "\trating_range=\"@max_rating - @min_rating\",\n",
    ")\n",
    "\n",
    "results = index.query(query)\n",
    "pd.DataFrame(results)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Comparing Approaches\n",
    "\n",
    "While each approach has strengths and weaknesses, it's important to understand that each might work better in some use cases than others. Below we will run through a sample of user queries and generate matches for each using different hybrid search techniques."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_user_queries = [\n",
    "    \"I'm in the mood for a high-rated action movie with a complex plot\",\n",
    "    \"What's a funny animated film about unlikely friendships?\",\n",
    "    \"Any movies featuring superheroes or extraordinary abilities\", \n",
    "    \"I want to watch a thrilling movie with spies or secret agents\",\n",
    "    \"Are there any comedies set in unusual locations or environments?\",\n",
    "    \"Find me an action-packed movie with car chases or explosions\",\n",
    "    \"What's a good family-friendly movie with talking animals?\",\n",
    "    \"I'm looking for a film that combines action and mind-bending concepts\",\n",
    "    \"Suggest a movie with a strong female lead character\",\n",
    "    \"What are some movies that involve heists or elaborate plans?\",\n",
    "    \"I need a feel-good movie about personal growth or transformation\",\n",
    "    \"Are there any films that blend comedy with action elements?\", \n",
    "    \"Show me movies set in dystopian or post-apocalyptic worlds\",\n",
    "    \"I'm interested in a movie with themes of revenge or justice\",\n",
    "    \"What are some visually stunning movies with impressive special effects?\"\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Tuple\n",
    "\n",
    "\n",
    "def hybrid_query(text, num_results: int, **kwargs) -> List[Tuple[str, float]]:\n",
    "\n",
    "    query = HybridQuery(\n",
    "\t\ttext,\n",
    "\t\ttext_field_name=\"description\",\n",
    "\t\tvector=model.embed(text, as_buffer=True),\n",
    "\t\tvector_field_name=\"description_vector\",\n",
    "\t\tstopwords=\"english\",\n",
    "\t\tnum_results=num_results,\n",
    "\t\treturn_fields=[\"title\"],\n",
    "\t\tyield_combined_score_as=\"hybrid_score\",\n",
    "\t\t**kwargs,\n",
    "    )\n",
    "\n",
    "    results = index.query(query)\n",
    "\n",
    "    return [\n",
    "        (\n",
    "            movie[\"title\"],\n",
    "            movie[\"hybrid_score\"]\n",
    "        )\n",
    "        for movie in results\n",
    "    ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "\n",
    "rankings = pd.DataFrame()\n",
    "rankings[\"query\"] = movie_user_queries\n",
    "\n",
    "# First, add new columns to the DataFrame\n",
    "rankings[\"hf-cross-encoder\"] = \"\"\n",
    "rankings[\"rrf\"] = \"\"\n",
    "rankings[\"linear\"] = \"\"\n",
    "\n",
    "rankings = rankings.astype({\n",
    "    \"query\": \"string\",\n",
    "    \"hf-cross-encoder\": \"object\",\n",
    "    \"rrf\": \"object\",\n",
    "    \"linear\": \"object\"\n",
    "})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now iterate through the queries and add results\n",
    "for i, user_query in enumerate(movie_user_queries):\n",
    "    rankings.at[i, \"hf-cross-encoder\"] = rerank(user_query, num_results=4)\n",
    "    rankings.at[i, \"rrf\"] = hybrid_query(user_query, num_results=4, combination_method=\"RRF\", rrf_window=20)\n",
    "    rankings.at[i, \"linear\"] = hybrid_query(user_query, num_results=4, combination_method=\"LINEAR\", linear_alpha=0.3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>query</th>\n",
       "      <th>hf-cross-encoder</th>\n",
       "      <th>rrf</th>\n",
       "      <th>linear</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>I'm in the mood for a high-rated action movie ...</td>\n",
       "      <td>[(Mad Max: Fury Road, -11.244140625), (Toy Sto...</td>\n",
       "      <td>[(The Incredibles, 0.032266458496), (Toy Story...</td>\n",
       "      <td>[(The Incredibles, 1.02685218482), (Toy Story,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>What's a funny animated film about unlikely fr...</td>\n",
       "      <td>[(Despicable Me, -10.44190788269043), (The Inc...</td>\n",
       "      <td>[(Monsters, Inc., 0.0312805474096), (Madagasca...</td>\n",
       "      <td>[(Madagascar, 1.23686656547), (Monsters, Inc.,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Any movies featuring superheroes or extraordin...</td>\n",
       "      <td>[(The Incredibles, -3.6648080348968506), (The ...</td>\n",
       "      <td>[(The Incredibles, 0.0327868852459), (Mad Max:...</td>\n",
       "      <td>[(The Incredibles, 1.45202633159), (The Avenge...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>I want to watch a thrilling movie with spies o...</td>\n",
       "      <td>[(Inception, -10.843633651733398), (The Incred...</td>\n",
       "      <td>[(Skyfall, 0.032266458496), (Explosive Pursuit...</td>\n",
       "      <td>[(Inception, 1.31241820902), (Skyfall, 0.44384...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Are there any comedies set in unusual location...</td>\n",
       "      <td>[(The Incredibles, -11.45376968383789), (Findi...</td>\n",
       "      <td>[(Finding Nemo, 0.0315449577745), (Explosive P...</td>\n",
       "      <td>[(Finding Nemo, 1.23817388011), (Madagascar, 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                               query  \\\n",
       "0  I'm in the mood for a high-rated action movie ...   \n",
       "1  What's a funny animated film about unlikely fr...   \n",
       "2  Any movies featuring superheroes or extraordin...   \n",
       "3  I want to watch a thrilling movie with spies o...   \n",
       "4  Are there any comedies set in unusual location...   \n",
       "\n",
       "                                    hf-cross-encoder  \\\n",
       "0  [(Mad Max: Fury Road, -11.244140625), (Toy Sto...   \n",
       "1  [(Despicable Me, -10.44190788269043), (The Inc...   \n",
       "2  [(The Incredibles, -3.6648080348968506), (The ...   \n",
       "3  [(Inception, -10.843633651733398), (The Incred...   \n",
       "4  [(The Incredibles, -11.45376968383789), (Findi...   \n",
       "\n",
       "                                                 rrf  \\\n",
       "0  [(The Incredibles, 0.032266458496), (Toy Story...   \n",
       "1  [(Monsters, Inc., 0.0312805474096), (Madagasca...   \n",
       "2  [(The Incredibles, 0.0327868852459), (Mad Max:...   \n",
       "3  [(Skyfall, 0.032266458496), (Explosive Pursuit...   \n",
       "4  [(Finding Nemo, 0.0315449577745), (Explosive P...   \n",
       "\n",
       "                                              linear  \n",
       "0  [(The Incredibles, 1.02685218482), (Toy Story,...  \n",
       "1  [(Madagascar, 1.23686656547), (Monsters, Inc.,...  \n",
       "2  [(The Incredibles, 1.45202633159), (The Avenge...  \n",
       "3  [(Inception, 1.31241820902), (Skyfall, 0.44384...  \n",
       "4  [(Finding Nemo, 1.23817388011), (Madagascar, 0...  "
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rankings.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "query               Show me movies set in dystopian or post-apocal...\n",
       "hf-cross-encoder    [(Mad Max: Fury Road, -3.4906256198883057), (D...\n",
       "rrf                 [(The Incredibles, 0.032522474881), (Mad Max: ...\n",
       "linear              [(The Incredibles, 1.35715968095), (Finding Ne...\n",
       "Name: 12, dtype: object"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rankings.loc[12].T"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Wrap up\n",
    "That's a wrap! Hopefully from this you were able to learn:\n",
    "- How to implement simple vector search queries in Redis\n",
    "- How to implement vector search queries with full-text filters\n",
    "- How to implement hybrid search queries using the Redis hybrid and aggregation APIs\n",
    "- How to perform client-side fusion and reranking techniques"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv (3.12.5)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}