{"cells": [{"cell_type": "markdown", "metadata": {}, "source": ["# 在Intel CPU上使用IPEX-LLM进行本地嵌入\n", "\n", "> [IPEX-LLM](https://github.com/intel-analytics/ipex-llm/) 是一个用于在Intel CPU和GPU上运行LLM的PyTorch库(例如,带有iGPU的本地PC,离散GPU,如Arc、Flex和Max),具有非常低的延迟。\n", "\n", "本示例介绍了如何使用LlamaIndex在Intel CPU上进行嵌入任务,同时利用`ipex-llm`进行优化。这对于RAG、文档QA等应用非常有帮助。\n", "\n", "> **注意**\n", ">\n", "> 您可以参考[这里](https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/embeddings/llama-index-embeddings-ipex-llm/examples)获取`IpexLLMEmbedding`的完整示例。请注意,在Intel CPU上运行示例时,请在命令参数中指定`-d 'cpu'`。\n", "\n", "## 安装`llama-index-embeddings-ipex-llm`\n", "\n", "这也将安装`ipex-llm`及其依赖项。\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["%pip install llama-index-embeddings-ipex-llm"]}, {"cell_type": "markdown", "metadata": {}, "source": ["## `IpexLLMEmbedding`\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": ["from llama_index.embeddings.ipex_llm import IpexLLMEmbedding\n", "\n", "embedding_model = IpexLLMEmbedding(model_name=\"BAAI/bge-large-en-v1.5\")"]}, {"cell_type": "markdown", "metadata": {}, "source": ["请注意,`IpexLLMEmbedding` 目前仅为 Hugging Face Bge 模型提供优化。\n"]}, {"cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [{"data": {"application/vnd.jupyter.widget-view+json": {"model_id": "6810cbccd883423b9b7ce2ad464d1be8", "version_major": 2, "version_minor": 0}, "text/plain": ["Batches: 0%| | 0/1 [00:00