{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Advanced Paraphrase Mining with Sentence Transformers and MLflow\n", "\n", "Embark on an enriching journey through advanced paraphrase mining using Sentence Transformers, enhanced by MLflow." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Learning Objectives\n", "\n", "- Apply `sentence-transformers` for advanced paraphrase mining.\n", "- Develop a custom `PythonModel` in MLflow tailored for this task.\n", "- Effectively manage and track models within the MLflow ecosystem.\n", "- Deploy paraphrase mining models using MLflow's deployment capabilities.\n", "\n", "#### Exploring Paraphrase Mining\n", "Discover the process of identifying semantically similar but textually distinct sentences, a key aspect in various NLP applications such as document summarization and chatbot development.\n", "\n", "#### The Role of Sentence Transformers in Paraphrase Mining\n", "Learn how Sentence Transformers, specialized for generating rich sentence embeddings, are used to capture deep semantic meanings and compare textual content.\n", "\n", "#### MLflow: Simplifying Model Management and Deployment\n", "Delve into how MLflow streamlines the process of managing and deploying NLP models, with a focus on efficient tracking and customizable model implementations.\n", "\n", "Join us to develop a nuanced understanding of paraphrase mining and master the art of managing and deploying NLP models with MLflow." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import warnings\n", "\n", "# Disable a few less-than-useful UserWarnings from setuptools and pydantic\n", "warnings.filterwarnings(\"ignore\", category=UserWarning)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Introduction to the Paraphrase Mining Model\n", "Initiate the Paraphrase Mining Model, integrating Sentence Transformers and MLflow for advanced NLP tasks.\n", "\n", "#### Overview of the Model Structure\n", "\n", "- **Loading Model and Corpus `load_context` Method**: Essential for loading the Sentence Transformer model and the text corpus for paraphrase identification.\n", "- **Paraphrase Mining Logic `predict` Method**: Integrates custom logic for input validation and paraphrase mining, offering customizable parameters.\n", "- **Sorting and Filtering Matches `_sort_and_filter_matches` Helper Method**: Ensures relevant and unique paraphrase identification by sorting and filtering based on similarity scores.\n", "\n", "#### Key Features\n", "\n", "- **Advanced NLP Techniques**: Utilizes Sentence Transformers for semantic text understanding.\n", "- **Custom Logic Integration**: Demonstrates flexibility in model behavior customization.\n", "- **User Customization Options**: Allows end users to adjust match criteria for various use cases.\n", "- **Efficiency in Processing**: Pre-encodes the corpus for efficient paraphrase mining operations.\n", "- **Robust Error Handling**: Incorporates validations for reliable model performance.\n", "\n", "#### Practical Implications\n", "This model provides a powerful tool for paraphrase detection in diverse applications, exemplifying the effective use of custom models within the MLflow framework." ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import warnings\n", "\n", "import pandas as pd\n", "from sentence_transformers import SentenceTransformer, util\n", "\n", "import mlflow\n", "from mlflow.models.signature import infer_signature\n", "from mlflow.pyfunc import PythonModel\n", "\n", "\n", "class ParaphraseMiningModel(PythonModel):\n", " def load_context(self, context):\n", " \"\"\"Load the model context for inference, including the customer feedback corpus.\"\"\"\n", " try:\n", " # Load the pre-trained sentence transformer model\n", " self.model = SentenceTransformer.load(context.artifacts[\"model_path\"])\n", "\n", " # Load the customer feedback corpus from the specified file\n", " corpus_file = context.artifacts[\"corpus_file\"]\n", " with open(corpus_file) as file:\n", " self.corpus = file.read().splitlines()\n", "\n", " except Exception as e:\n", " raise ValueError(f\"Error loading model and corpus: {e}\")\n", "\n", " def _sort_and_filter_matches(\n", " self,\n", " query: str,\n", " paraphrase_pairs: list[tuple[float, int, int]],\n", " similarity_threshold: float,\n", " ):\n", " \"\"\"Sort and filter the matches by similarity score.\"\"\"\n", "\n", " # Convert to list of tuples and sort by score\n", " sorted_matches = sorted(paraphrase_pairs, key=lambda x: x[1], reverse=True)\n", "\n", " # Filter and collect paraphrases for the query, avoiding duplicates\n", " query_paraphrases = {}\n", " for score, i, j in sorted_matches:\n", " if score < similarity_threshold:\n", " continue\n", "\n", " paraphrase = self.corpus[j] if self.corpus[i] == query else self.corpus[i]\n", " if paraphrase == query:\n", " continue\n", "\n", " if paraphrase not in query_paraphrases or score > query_paraphrases[paraphrase]:\n", " query_paraphrases[paraphrase] = score\n", "\n", " return sorted(query_paraphrases.items(), key=lambda x: x[1], reverse=True)\n", "\n", " def predict(self, context, model_input, params=None):\n", " \"\"\"Predict method to perform paraphrase mining over the corpus.\"\"\"\n", "\n", " # Validate and extract the query input\n", " if isinstance(model_input, pd.DataFrame):\n", " if model_input.shape[1] != 1:\n", " raise ValueError(\"DataFrame input must have exactly one column.\")\n", " query = model_input.iloc[0, 0]\n", " elif isinstance(model_input, dict):\n", " query = model_input.get(\"query\")\n", " if query is None:\n", " raise ValueError(\"The input dictionary must have a key named 'query'.\")\n", " else:\n", " raise TypeError(\n", " f\"Unexpected type for model_input: {type(model_input)}. Must be either a Dict or a DataFrame.\"\n", " )\n", "\n", " # Determine the minimum similarity threshold\n", " similarity_threshold = params.get(\"similarity_threshold\", 0.5) if params else 0.5\n", "\n", " # Add the query to the corpus for paraphrase mining\n", " extended_corpus = self.corpus + [query]\n", "\n", " # Perform paraphrase mining\n", " paraphrase_pairs = util.paraphrase_mining(\n", " self.model, extended_corpus, show_progress_bar=False\n", " )\n", "\n", " # Convert to list of tuples and sort by score\n", " sorted_paraphrases = self._sort_and_filter_matches(\n", " query, paraphrase_pairs, similarity_threshold\n", " )\n", "\n", " # Warning if no paraphrases found\n", " if not sorted_paraphrases:\n", " warnings.warn(\"No paraphrases found above the similarity threshold.\", UserWarning)\n", "\n", " return {sentence[0]: str(sentence[1]) for sentence in sorted_paraphrases}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Preparing the Corpus for Paraphrase Mining\n", "Set up the foundation for paraphrase mining by creating and preparing a diverse corpus.\n", "\n", "#### Corpus Creation\n", "\n", "- Define a `corpus` comprising a range of sentences from various topics, including space exploration, AI, gardening, and more. This diversity enables the model to identify paraphrases across a broad spectrum of subjects.\n", "\n", "#### Writing the Corpus to a File\n", "\n", "- The corpus is saved to a file named `feedback.txt`, mirroring a common practice in large-scale data handling.\n", "- This step also prepares the corpus for efficient processing within the Paraphrase Mining Model.\n", "\n", "#### Significance of the Corpus\n", "The corpus serves as the key dataset for the model to find semantically similar sentences. Its variety ensures the model's adaptability and effectiveness across diverse use cases." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "corpus = [\n", " \"Exploring ancient cities in Europe offers a glimpse into history.\",\n", " \"Modern AI technologies are revolutionizing industries.\",\n", " \"Healthy eating contributes significantly to overall well-being.\",\n", " \"Advancements in renewable energy are combating climate change.\",\n", " \"Learning a new language opens doors to different cultures.\",\n", " \"Gardening is a relaxing hobby that connects you with nature.\",\n", " \"Blockchain technology could redefine digital transactions.\",\n", " \"Homemade Italian pasta is a delight to cook and eat.\",\n", " \"Practicing yoga daily improves both physical and mental health.\",\n", " \"The art of photography captures moments in time.\",\n", " \"Baking bread at home has become a popular quarantine activity.\",\n", " \"Virtual reality is creating new experiences in gaming.\",\n", " \"Sustainable travel is becoming a priority for eco-conscious tourists.\",\n", " \"Reading books is a great way to unwind and learn.\",\n", " \"Jazz music provides a rich tapestry of sound and rhythm.\",\n", " \"Marathon training requires discipline and perseverance.\",\n", " \"Studying the stars helps us understand our universe.\",\n", " \"The rise of electric cars is an important environmental development.\",\n", " \"Documentary films offer deep insights into real-world issues.\",\n", " \"Crafting DIY projects can be both fun and rewarding.\",\n", " \"The history of ancient civilizations is fascinating to explore.\",\n", " \"Exploring the depths of the ocean reveals a world of marine wonders.\",\n", " \"Learning to play a musical instrument can be a rewarding challenge.\",\n", " \"Artificial intelligence is shaping the future of personalized medicine.\",\n", " \"Cycling is not only a great workout but also eco-friendly transportation.\",\n", " \"Home automation with IoT devices is enhancing living experiences.\",\n", " \"Understanding quantum computing requires a grasp of complex physics.\",\n", " \"A well-brewed cup of coffee is the perfect start to the day.\",\n", " \"Urban farming is gaining popularity as a sustainable food source.\",\n", " \"Meditation and mindfulness can lead to a more balanced life.\",\n", " \"The popularity of podcasts has revolutionized audio storytelling.\",\n", " \"Space exploration continues to push the boundaries of human knowledge.\",\n", " \"Wildlife conservation is essential for maintaining biodiversity.\",\n", " \"The fusion of technology and fashion is creating new trends.\",\n", " \"E-learning platforms have transformed the educational landscape.\",\n", " \"Dark chocolate has surprising health benefits when enjoyed in moderation.\",\n", " \"Robotics in manufacturing is leading to more efficient production.\",\n", " \"Creating a personal budget is key to financial well-being.\",\n", " \"Hiking in nature is a great way to connect with the outdoors.\",\n", " \"3D printing is innovating the way we create and manufacture objects.\",\n", " \"Sommeliers can identify a wine's characteristics with just a taste.\",\n", " \"Mind-bending puzzles and riddles are great for cognitive exercise.\",\n", " \"Social media has a profound impact on communication and culture.\",\n", " \"Urban sketching captures the essence of city life on paper.\",\n", " \"The ethics of AI is a growing field in tech philosophy.\",\n", " \"Homemade skincare remedies are becoming more popular.\",\n", " \"Virtual travel experiences can provide a sense of adventure at home.\",\n", " \"Ancient mythology still influences modern storytelling and literature.\",\n", " \"Building model kits is a hobby that requires patience and precision.\",\n", " \"The study of languages opens windows into different worldviews.\",\n", " \"Professional esports has become a major global phenomenon.\",\n", " \"The mysteries of the universe are unveiled through space missions.\",\n", " \"Astronauts' experiences in space stations offer unique insights into life beyond Earth.\",\n", " \"Telescopic observations bring distant galaxies within our view.\",\n", " \"The study of celestial bodies helps us understand the cosmos.\",\n", " \"Space travel advancements could lead to interplanetary exploration.\",\n", " \"Observing celestial events provides valuable data for astronomers.\",\n", " \"The development of powerful rockets is key to deep space exploration.\",\n", " \"Mars rover missions are crucial in searching for extraterrestrial life.\",\n", " \"Satellites play a vital role in our understanding of Earth's atmosphere.\",\n", " \"Astrophysics is central to unraveling the secrets of space.\",\n", " \"Zero gravity environments in space pose unique challenges and opportunities.\",\n", " \"Space tourism might soon become a reality for many.\",\n", " \"Lunar missions have contributed significantly to our knowledge of the moon.\",\n", " \"The International Space Station is a hub for groundbreaking space research.\",\n", " \"Studying comets and asteroids reveals information about the early solar system.\",\n", " \"Advancements in space technology have implications for many scientific fields.\",\n", " \"The possibility of life on other planets continues to intrigue scientists.\",\n", " \"Black holes are among the most mysterious phenomena in space.\",\n", " \"The history of space exploration is filled with remarkable achievements.\",\n", " \"Future space missions could unlock the mysteries of dark matter.\",\n", "]\n", "\n", "# Write out the corpus to a file\n", "corpus_file = \"/tmp/feedback.txt\"\n", "with open(corpus_file, \"w\") as file:\n", " for sentence in corpus:\n", " file.write(sentence + \"\\n\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Setting Up the Paraphrase Mining Model\n", "\n", "Prepare the Sentence Transformer model for integration with MLflow to harness its paraphrase mining capabilities.\n", "\n", "#### Loading the Sentence Transformer Model\n", "\n", "- Initialize the `all-MiniLM-L6-v2` Sentence Transformer model, ideal for generating sentence embeddings suitable for paraphrase mining.\n", "\n", "#### Preparing the Input Example\n", "\n", "- Create a DataFrame as an input example to illustrate the type of query the model will handle, aiding in defining the model's input structure.\n", "\n", "#### Saving the Model\n", "\n", "- Save the model to `/tmp/paraphrase_search_model` for portability and ease of loading during deployment with MLflow.\n", "\n", "#### Defining Artifacts and Corpus Path\n", "\n", "- Specify paths to the saved model and corpus as artifacts in MLflow, crucial for model logging and reproduction.\n", "\n", "#### Generating Test Output for Signature\n", "\n", "- Generate a sample output, illustrating the model's expected output format for paraphrase mining.\n", "\n", "#### Creating the Model Signature\n", "\n", "- Use MLflow's `infer_signature` to define the model's input and output schema, adding the `similarity_threshold` parameter for inference flexibility." ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "inputs: \n", " ['query': string]\n", "outputs: \n", " ['This product is satisfactory and functions as expected.': string]\n", "params: \n", " ['similarity_threshold': double (default: 0.5)]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load a pre-trained sentence transformer model\n", "model = SentenceTransformer(\"all-MiniLM-L6-v2\")\n", "\n", "# Create an input example DataFrame\n", "input_example = pd.DataFrame({\"query\": [\"This product works well. I'm satisfied.\"]})\n", "\n", "# Save the model in the /tmp directory\n", "model_directory = \"/tmp/paraphrase_search_model\"\n", "model.save(model_directory)\n", "\n", "# Define the path for the corpus file\n", "corpus_file = \"/tmp/feedback.txt\"\n", "\n", "# Define the artifacts (paths to the model and corpus file)\n", "artifacts = {\"model_path\": model_directory, \"corpus_file\": corpus_file}\n", "\n", "# Generate test output for signature\n", "# Sample output for paraphrase mining could be a list of tuples (paraphrase, score)\n", "test_output = [{\"This product is satisfactory and functions as expected.\": \"0.8\"}]\n", "\n", "# Define the signature associated with the model\n", "# The signature includes the structure of the input and the expected output, as well as any parameters that\n", "# we would like to expose for overriding at inference time (including their default values if they are not overridden).\n", "signature = infer_signature(\n", " model_input=input_example, model_output=test_output, params={\"similarity_threshold\": 0.5}\n", ")\n", "\n", "# Visualize the signature, showing our overridden inference parameter and its default.\n", "signature" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating an experiment\n", "\n", "We create a new MLflow Experiment so that the run we're going to log our model to does not log to the default experiment and instead has its own contextually relevant entry." ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# If you are running this tutorial in local mode, leave the next line commented out.\n", "# Otherwise, uncomment the following line and set your tracking uri to your local or remote tracking server.\n", "\n", "# mlflow.set_tracking_uri(\"http://127.0.0.1:8080\")\n", "\n", "mlflow.set_experiment(\"Paraphrase Mining\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Logging the Paraphrase Mining Model with MLflow\n", "Log the custom Paraphrase Mining Model with MLflow, a key step for model management and deployment.\n", " \n", "#### Initiating an MLflow Run\n", "\n", "- Start an MLflow run to create a comprehensive record of model logging and tracking within the MLflow framework.\n", "\n", "#### Logging the Model in MLflow\n", "\n", "- Use MLflow's Python model logging function to integrate the custom model into the MLflow ecosystem.\n", "- Provide a unique name for the model for easy identification in MLflow.\n", "- Log the instantiated Paraphrase Mining Model, along with an input example, model signature, artifacts, and Python dependencies.\n", "\n", "#### Outcomes and Benefits of Model Logging\n", "\n", "- Register the model within MLflow for streamlined management and deployment, enhancing its accessibility and trackability.\n", "- Ensure model reproducibility and version control across deployment environments." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "1fa5a17ce9f84f628c9cee02b633222a", "version_major": 2, "version_minor": 0 }, "text/plain": [ "Downloading artifacts: 0%| | 0/11 [00:00