{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8dbf2103",
   "metadata": {},
   "source": [
    "# 02 — Hybrid Search Patterns\n",
    "\n",
    "This notebook digs into the three knobs that most change retrieval quality:\n",
    "\n",
    "1. `target_vector` — `content_vec` vs `title_vec` vs `summary_vec`.\n",
    "2. `alpha` — slider between pure-keyword (0.0) and pure-vector (1.0).\n",
    "3. `filters` — restrict by `source_type`, `year`, or other properties.\n",
    "\n",
    "We'll run the same query under different settings so you can build intuition\n",
    "for when to reach for which."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff4c66c8",
   "metadata": {},
   "outputs": [],
   "source": [
    "from fractal_rag.client import client_session\n",
    "from fractal_rag.config import get_settings\n",
    "from weaviate.classes.query import Filter, MetadataQuery\n",
    "\n",
    "settings = get_settings(refresh=True)\n",
    "QUERY = 'Barnsley fern iterated function system'\n",
    "\n",
    "def hybrid(target_vector, alpha=0.5, filters=None, limit=5):\n",
    "    with client_session() as client:\n",
    "        art = client.collections.get(settings.artifact_collection)\n",
    "        return art.query.hybrid(\n",
    "            query=QUERY, limit=limit, target_vector=target_vector,\n",
    "            alpha=alpha, filters=filters,\n",
    "            return_metadata=MetadataQuery(distance=True),\n",
    "        ).objects\n",
    "\n",
    "def show(objs, label):\n",
    "    print(f'--- {label} ---')\n",
    "    for o in objs:\n",
    "        p = o.properties\n",
    "        print(f\"  [{p.get('source_type'):>10}] {p.get('title')[:70]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a6d8035",
   "metadata": {},
   "source": [
    "## target_vector matters\n",
    "\n",
    "The same query against different named vectors returns different rankings. Use `title_vec` for 'find me the notebook/page called X' and `content_vec` for 'find passages about X'."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35f2e0d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "show(hybrid('content_vec'), 'content_vec, alpha=0.5')\n",
    "show(hybrid('title_vec'),   'title_vec,   alpha=0.5')\n",
    "show(hybrid('summary_vec'), 'summary_vec, alpha=0.5')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef73235c",
   "metadata": {},
   "source": [
    "## alpha shifts the keyword/vector balance\n",
    "\n",
    "`alpha=0.0` is pure BM25 (lexical), `alpha=1.0` is pure cosine. For technical vocabulary that the embedding model may not know, lower alpha. For paraphrases and conceptual matches, raise alpha."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3652e191",
   "metadata": {},
   "outputs": [],
   "source": [
    "show(hybrid('content_vec', alpha=0.0), 'alpha=0.0 (keyword)')\n",
    "show(hybrid('content_vec', alpha=0.5), 'alpha=0.5 (balanced)')\n",
    "show(hybrid('content_vec', alpha=1.0), 'alpha=1.0 (vector only)')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dd32de9",
   "metadata": {},
   "source": [
    "## Filtering by source_type\n",
    "\n",
    "Often you want notebooks **only**, or articles **only**. Pass a `Filter` from `weaviate.classes.query`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "662e4ecb",
   "metadata": {},
   "outputs": [],
   "source": [
    "notebook_only = Filter.by_property('source_type').contains_any(['notebook'])\n",
    "articles_only = Filter.by_property('source_type').contains_any(['json_corpus'])\n",
    "\n",
    "show(hybrid('content_vec', filters=notebook_only), 'notebooks only')\n",
    "show(hybrid('content_vec', filters=articles_only), 'json_corpus only')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "163835b2",
   "metadata": {},
   "source": [
    "## Filtering by year\n",
    "\n",
    "Useful for historical asks. Combine with `Filter.all_of(...)` if you need multiple conditions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d1efc4c4",
   "metadata": {},
   "outputs": [],
   "source": [
    "before_1990 = Filter.all_of([\n",
    "    Filter.by_property('source_type').contains_any(['json_corpus']),\n",
    "    Filter.by_property('year').less_or_equal(1990),\n",
    "])\n",
    "show(hybrid('content_vec', filters=before_1990, limit=8), 'pre-1990 articles')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c504facb",
   "metadata": {},
   "source": [
    "## near_object: 'more like this'\n",
    "\n",
    "Once you find a single strong hit, `near_object` returns its neighbors in the embedding space. This is the workhorse for synthesis: find one anchor, then expand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "81969042",
   "metadata": {},
   "outputs": [],
   "source": [
    "anchor = hybrid('content_vec', limit=1)[0]\n",
    "with client_session() as client:\n",
    "    art = client.collections.get(settings.artifact_collection)\n",
    "    neighbors = art.query.near_object(\n",
    "        near_object=str(anchor.uuid),\n",
    "        limit=5,\n",
    "        target_vector='content_vec',\n",
    "        return_metadata=MetadataQuery(distance=True),\n",
    "    ).objects\n",
    "\n",
    "print(f\"Anchor: {anchor.properties.get('title')[:70]}\")\n",
    "print('Neighbors:')\n",
    "for n in neighbors:\n",
    "    d = getattr(n.metadata, 'distance', None)\n",
    "    print(f\"  [{d:.3f}] {n.properties.get('title')[:70]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b3e5cf4b",
   "metadata": {},
   "source": [
    "## Next steps\n",
    "\n",
    "- [03 — Concept ontology](03_concept_ontology.ipynb) — follow cross-refs to find every artifact that mentions a given concept."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}