{ "cells": [ { "cell_type": "markdown", "id": "65e52c90c0c5ff06", "metadata": { "collapsed": false, "id": "65e52c90c0c5ff06" }, "source": [ "# UD03 · Notebook 3 — Modelos de lenguaje, DistilBERT\n", "\n", "En esta sección se realiza un análisis de sentimiento de tweets, como un ejemplo del uso de modelos de lenguaje previamente entrenado. En este caso, se utiliza el modelo [Distilbert](https://arxiv.org/abs/1910.01108) para el análisis de sentimientos. Este modelo es una versión más ligera del modelo [BERT](https://arxiv.org/abs/1810.04805), que es un modelo de lenguaje previo a la entrada que se ha utilizado con muy buenos resultados en diferentes tareas de procesamiento del lenguaje natural, como [Análisis de sentimientos](https://arxiv.org/abs/1905.05583), [Clasificación de texto](https://arxiv.org/abs/1904.09077) o [Extracción de información](https://arxiv.org/abs/1906.05237).\n", "\n", "En este caso, se utiliza el modelo previamente entrenado para el análisis de sentimientos en inglés." ] }, { "cell_type": "markdown", "id": "9012845d4dfa4e1f", "metadata": { "collapsed": false, "id": "9012845d4dfa4e1f" }, "source": [ "## Carga del conjunto de datos`\n", "\n", "Usaremos la librería [datasets](https://huggingface.co/docs/datasets/) para cargar el `dataset` de los tweets. Esta La librería te permite cargar `datasets` de diferentes fuentes como [Hugging Face Hub](https://huggingface.co/datasets), [Amazon AWS](https://docs.aws.amazon.com/es_es/marketplace/latest/userguide/datasets.html) o [Google Cloud](https://cloud.google.com/ai-platform/training/docs/datasets). En este caso, cargaremos el `dataset` de tuits desde [Hugging Face Hub](https://huggingface.co/datasets/dair-ai/emotion).\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "d2a676e1e94a413c", "metadata": { "ExecuteTime": { "end_time": "2024-01-20T18:36:46.715988Z", "start_time": "2024-01-20T18:36:28.346946Z" }, "colab": { "base_uri": "https://localhost:8080/" }, "id": "d2a676e1e94a413c", "outputId": "ddc6bdcd-9620-4f4e-856b-e4a2011b82bb" }, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Requirement already satisfied: transformers in /usr/local/lib/python3.12/dist-packages (4.57.3)\n", "Requirement already satisfied: datasets in /usr/local/lib/python3.12/dist-packages (4.0.0)\n", "Collecting datasets\n", " Downloading datasets-4.4.1-py3-none-any.whl.metadata (19 kB)\n", "Collecting evaluate\n", " Downloading evaluate-0.4.6-py3-none-any.whl.metadata (9.5 kB)\n", "Requirement already satisfied: accelerate in /usr/local/lib/python3.12/dist-packages (1.12.0)\n", "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.12/dist-packages (1.6.1)\n", "Collecting scikit-learn\n", " Downloading scikit_learn-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl.metadata (11 kB)\n", "Collecting accuracy\n", " Downloading accuracy-0.1.1-py2.py3-none-any.whl.metadata (4.8 kB)\n", "Requirement already satisfied: filelock in /usr/local/lib/python3.12/dist-packages (from transformers) (3.20.0)\n", "Requirement already satisfied: huggingface-hub<1.0,>=0.34.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.36.0)\n", "Requirement already satisfied: numpy>=1.17 in /usr/local/lib/python3.12/dist-packages (from transformers) (2.0.2)\n", "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (25.0)\n", "Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.12/dist-packages (from transformers) (6.0.3)\n", "Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.12/dist-packages (from transformers) (2025.11.3)\n", "Requirement already satisfied: requests in /usr/local/lib/python3.12/dist-packages (from transformers) (2.32.4)\n", "Requirement already satisfied: tokenizers<=0.23.0,>=0.22.0 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.22.1)\n", "Requirement already satisfied: safetensors>=0.4.3 in /usr/local/lib/python3.12/dist-packages (from transformers) (0.7.0)\n", "Requirement already satisfied: tqdm>=4.27 in /usr/local/lib/python3.12/dist-packages (from transformers) (4.67.1)\n", "Collecting pyarrow>=21.0.0 (from datasets)\n", " Downloading pyarrow-22.0.0-cp312-cp312-manylinux_2_28_x86_64.whl.metadata (3.2 kB)\n", "Requirement already satisfied: dill<0.4.1,>=0.3.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.3.8)\n", "Requirement already satisfied: pandas in /usr/local/lib/python3.12/dist-packages (from datasets) (2.2.2)\n", "Requirement already satisfied: httpx<1.0.0 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.28.1)\n", "Requirement already satisfied: xxhash in /usr/local/lib/python3.12/dist-packages (from datasets) (3.6.0)\n", "Requirement already satisfied: multiprocess<0.70.19 in /usr/local/lib/python3.12/dist-packages (from datasets) (0.70.16)\n", "Requirement already satisfied: fsspec<=2025.10.0,>=2023.1.0 in /usr/local/lib/python3.12/dist-packages (from fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (2025.3.0)\n", "Requirement already satisfied: psutil in /usr/local/lib/python3.12/dist-packages (from accelerate) (5.9.5)\n", "Requirement already satisfied: torch>=2.0.0 in /usr/local/lib/python3.12/dist-packages (from accelerate) (2.9.0+cu126)\n", "Requirement already satisfied: scipy>=1.10.0 in /usr/local/lib/python3.12/dist-packages (from scikit-learn) (1.16.3)\n", "Requirement already satisfied: joblib>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from scikit-learn) (1.5.2)\n", "Requirement already satisfied: threadpoolctl>=3.2.0 in /usr/local/lib/python3.12/dist-packages (from scikit-learn) (3.6.0)\n", "Requirement already satisfied: Jinja2>=3.1.1 in /usr/local/lib/python3.12/dist-packages (from accuracy) (3.1.6)\n", "Requirement already satisfied: altair>=4.2.0 in /usr/local/lib/python3.12/dist-packages (from accuracy) (5.5.0)\n", "Collecting clumper>=0.2.15 (from accuracy)\n", " Downloading clumper-0.2.15-py2.py3-none-any.whl.metadata (1.2 kB)\n", "Requirement already satisfied: rich>=10.3.0 in /usr/local/lib/python3.12/dist-packages (from accuracy) (13.9.4)\n", "Requirement already satisfied: spacy>=3.0.0 in /usr/local/lib/python3.12/dist-packages (from accuracy) (3.8.11)\n", "Requirement already satisfied: typer>=0.3.0 in /usr/local/lib/python3.12/dist-packages (from accuracy) (0.20.0)\n", "Requirement already satisfied: jsonschema>=3.0 in /usr/local/lib/python3.12/dist-packages (from altair>=4.2.0->accuracy) (4.25.1)\n", "Requirement already satisfied: narwhals>=1.14.2 in /usr/local/lib/python3.12/dist-packages (from altair>=4.2.0->accuracy) (2.13.0)\n", "Requirement already satisfied: typing-extensions>=4.10.0 in /usr/local/lib/python3.12/dist-packages (from altair>=4.2.0->accuracy) (4.15.0)\n", "Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /usr/local/lib/python3.12/dist-packages (from fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (3.13.2)\n", "Requirement already satisfied: anyio in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (4.12.0)\n", "Requirement already satisfied: certifi in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (2025.11.12)\n", "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (1.0.9)\n", "Requirement already satisfied: idna in /usr/local/lib/python3.12/dist-packages (from httpx<1.0.0->datasets) (3.11)\n", "Requirement already satisfied: h11>=0.16 in /usr/local/lib/python3.12/dist-packages (from httpcore==1.*->httpx<1.0.0->datasets) (0.16.0)\n", "Requirement already satisfied: hf-xet<2.0.0,>=1.1.3 in /usr/local/lib/python3.12/dist-packages (from huggingface-hub<1.0,>=0.34.0->transformers) (1.2.0)\n", "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.12/dist-packages (from Jinja2>=3.1.1->accuracy) (3.0.3)\n", "Requirement already satisfied: charset_normalizer<4,>=2 in /usr/local/lib/python3.12/dist-packages (from requests->transformers) (3.4.4)\n", "Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.12/dist-packages (from requests->transformers) (2.5.0)\n", "Requirement already satisfied: markdown-it-py>=2.2.0 in /usr/local/lib/python3.12/dist-packages (from rich>=10.3.0->accuracy) (4.0.0)\n", "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /usr/local/lib/python3.12/dist-packages (from rich>=10.3.0->accuracy) (2.19.2)\n", "Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.11 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (3.0.12)\n", "Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (1.0.5)\n", "Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (1.0.15)\n", "Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (2.0.13)\n", "Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (3.0.12)\n", "Requirement already satisfied: thinc<8.4.0,>=8.3.4 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (8.3.10)\n", "Requirement already satisfied: wasabi<1.2.0,>=0.9.1 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (1.1.3)\n", "Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (2.5.2)\n", "Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (2.0.10)\n", "Requirement already satisfied: weasel<0.5.0,>=0.4.2 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (0.4.3)\n", "Requirement already satisfied: typer-slim<1.0.0,>=0.3.0 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (0.20.0)\n", "Requirement already satisfied: pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4 in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (2.12.3)\n", "Requirement already satisfied: setuptools in /usr/local/lib/python3.12/dist-packages (from spacy>=3.0.0->accuracy) (75.2.0)\n", "Requirement already satisfied: sympy>=1.13.3 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (1.14.0)\n", "Requirement already satisfied: networkx>=2.5.1 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.6.1)\n", "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.6.77 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.6.77)\n", "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.6.77 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.6.77)\n", "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.6.80 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.6.80)\n", "Requirement already satisfied: nvidia-cudnn-cu12==9.10.2.21 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (9.10.2.21)\n", "Requirement already satisfied: nvidia-cublas-cu12==12.6.4.1 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.6.4.1)\n", "Requirement already satisfied: nvidia-cufft-cu12==11.3.0.4 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (11.3.0.4)\n", "Requirement already satisfied: nvidia-curand-cu12==10.3.7.77 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (10.3.7.77)\n", "Requirement already satisfied: nvidia-cusolver-cu12==11.7.1.2 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (11.7.1.2)\n", "Requirement already satisfied: nvidia-cusparse-cu12==12.5.4.2 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.5.4.2)\n", "Requirement already satisfied: nvidia-cusparselt-cu12==0.7.1 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (0.7.1)\n", "Requirement already satisfied: nvidia-nccl-cu12==2.27.5 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (2.27.5)\n", "Requirement already satisfied: nvidia-nvshmem-cu12==3.3.20 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.3.20)\n", "Requirement already satisfied: nvidia-nvtx-cu12==12.6.77 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.6.77)\n", "Requirement already satisfied: nvidia-nvjitlink-cu12==12.6.85 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (12.6.85)\n", "Requirement already satisfied: nvidia-cufile-cu12==1.11.1.6 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (1.11.1.6)\n", "Requirement already satisfied: triton==3.5.0 in /usr/local/lib/python3.12/dist-packages (from torch>=2.0.0->accelerate) (3.5.0)\n", "Requirement already satisfied: click>=8.0.0 in /usr/local/lib/python3.12/dist-packages (from typer>=0.3.0->accuracy) (8.3.1)\n", "Requirement already satisfied: shellingham>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from typer>=0.3.0->accuracy) (1.5.4)\n", "Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2.9.0.post0)\n", "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2025.2)\n", "Requirement already satisfied: tzdata>=2022.7 in /usr/local/lib/python3.12/dist-packages (from pandas->datasets) (2025.2)\n", "Requirement already satisfied: aiohappyeyeballs>=2.5.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (2.6.1)\n", "Requirement already satisfied: aiosignal>=1.4.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (1.4.0)\n", "Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (25.4.0)\n", "Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (1.8.0)\n", "Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (6.7.0)\n", "Requirement already satisfied: propcache>=0.2.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (0.4.1)\n", "Requirement already satisfied: yarl<2.0,>=1.17.0 in /usr/local/lib/python3.12/dist-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->fsspec[http]<=2025.10.0,>=2023.1.0->datasets) (1.22.0)\n", "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /usr/local/lib/python3.12/dist-packages (from jsonschema>=3.0->altair>=4.2.0->accuracy) (2025.9.1)\n", "Requirement already satisfied: referencing>=0.28.4 in /usr/local/lib/python3.12/dist-packages (from jsonschema>=3.0->altair>=4.2.0->accuracy) (0.37.0)\n", "Requirement already satisfied: rpds-py>=0.7.1 in /usr/local/lib/python3.12/dist-packages (from jsonschema>=3.0->altair>=4.2.0->accuracy) (0.30.0)\n", "Requirement already satisfied: mdurl~=0.1 in /usr/local/lib/python3.12/dist-packages (from markdown-it-py>=2.2.0->rich>=10.3.0->accuracy) (0.1.2)\n", "Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.12/dist-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->accuracy) (0.7.0)\n", "Requirement already satisfied: pydantic-core==2.41.4 in /usr/local/lib/python3.12/dist-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->accuracy) (2.41.4)\n", "Requirement already satisfied: typing-inspection>=0.4.2 in /usr/local/lib/python3.12/dist-packages (from pydantic!=1.8,!=1.8.1,<3.0.0,>=1.7.4->spacy>=3.0.0->accuracy) (0.4.2)\n", "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.12/dist-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.17.0)\n", "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /usr/local/lib/python3.12/dist-packages (from sympy>=1.13.3->torch>=2.0.0->accelerate) (1.3.0)\n", "Requirement already satisfied: blis<1.4.0,>=1.3.0 in /usr/local/lib/python3.12/dist-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.0.0->accuracy) (1.3.3)\n", "Requirement already satisfied: confection<1.0.0,>=0.0.1 in /usr/local/lib/python3.12/dist-packages (from thinc<8.4.0,>=8.3.4->spacy>=3.0.0->accuracy) (0.1.5)\n", "Requirement already satisfied: cloudpathlib<1.0.0,>=0.7.0 in /usr/local/lib/python3.12/dist-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.0.0->accuracy) (0.23.0)\n", "Requirement already satisfied: smart-open<8.0.0,>=5.2.1 in /usr/local/lib/python3.12/dist-packages (from weasel<0.5.0,>=0.4.2->spacy>=3.0.0->accuracy) (7.5.0)\n", "Requirement already satisfied: wrapt in /usr/local/lib/python3.12/dist-packages (from smart-open<8.0.0,>=5.2.1->weasel<0.5.0,>=0.4.2->spacy>=3.0.0->accuracy) (2.0.1)\n", "Downloading datasets-4.4.1-py3-none-any.whl (511 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m511.6/511.6 kB\u001b[0m \u001b[31m21.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading evaluate-0.4.6-py3-none-any.whl (84 kB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m6.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading scikit_learn-1.8.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (8.9 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.9/8.9 MB\u001b[0m \u001b[31m62.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hDownloading accuracy-0.1.1-py2.py3-none-any.whl (7.8 kB)\n", "Downloading clumper-0.2.15-py2.py3-none-any.whl (18 kB)\n", "Downloading pyarrow-22.0.0-cp312-cp312-manylinux_2_28_x86_64.whl (47.7 MB)\n", "\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m47.7/47.7 MB\u001b[0m \u001b[31m12.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n", "\u001b[?25hInstalling collected packages: clumper, pyarrow, scikit-learn, datasets, evaluate, accuracy\n", " Attempting uninstall: pyarrow\n", " Found existing installation: pyarrow 18.1.0\n", " Uninstalling pyarrow-18.1.0:\n", " Successfully uninstalled pyarrow-18.1.0\n", " Attempting uninstall: scikit-learn\n", " Found existing installation: scikit-learn 1.6.1\n", " Uninstalling scikit-learn-1.6.1:\n", " Successfully uninstalled scikit-learn-1.6.1\n", " Attempting uninstall: datasets\n", " Found existing installation: datasets 4.0.0\n", " Uninstalling datasets-4.0.0:\n", " Successfully uninstalled datasets-4.0.0\n", "Successfully installed accuracy-0.1.1 clumper-0.2.15 datasets-4.4.1 evaluate-0.4.6 pyarrow-22.0.0 scikit-learn-1.8.0\n" ] } ], "source": [ "# Instalamos las librerias que vamos a usar\n", "\n", "import os\n", "os.environ[\"WANDB_DISABLED\"] = \"true\"\n", "\n", "%pip install -U transformers datasets evaluate accelerate scikit-learn accuracy" ] }, { "cell_type": "code", "execution_count": 2, "id": "47ff17e0f41702d2", "metadata": { "ExecuteTime": { "end_time": "2024-01-20T18:36:48.675124Z", "start_time": "2024-01-20T18:36:46.721737Z" }, "colab": { "base_uri": "https://localhost:8080/", "height": 370, "referenced_widgets": [ "32ffd5638dcf40538db1584dd69caea2", "bff4b847e0134af3a6d503948cf738e9", "b27397c16a4c4496a746b1bf1ffcd644", "e1dfd4292ddb411b9cdcb160af316ff1", "5c32c5639dd8481480021f1e75115a7a", "450c0b7809354ebc93d83739efe6301a", "10750272ace6487fba0f79c8a9965be8", "6048de36fa954b9894e3a27bee51bdc3", "04876e83a19244d6b9e242ab37c2c334", "c79aded1391c4540a359096af00308dc", "47691a73b95a4f34a30ae87e9a6d08d3", "78859832137943bb8b5393be89159ab1", "191553d5edc143c5a5881c0ef2a3e6f0", "58c3d1ffc20c4fe399aa4d56eaf8a673", "6c988f2547bc4ed5b4740b1703239f51", "abaececafaaa45b49f60c8573e897255", "2585e4fdaa8e4efe93a66234c2649979", "cf967ee71ccf4f4ba572ca8f66757b59", "8907ac3a04cb44a38847238367ed4946", "86b25111c5d84a999a348d52f09a209a", "91f9c001f36e4a01be534adb30b97dbd", "f2b41dc8474d4584bd55bac610446ab0", "d8df940e66c04170965702f364046962", "f4732e7ded224082a6a8cb15cedff9d4", "1db7e6c26c61491e8e42475a005911fd", "2ac2e04e95db45e19f2e1c3450251cd1", "1c277b26dccf4026a8fd32c551fa2f55", "619ffe5d708d41c2b353f8979f3bd210", "8cf83e45960b4da5bd65e24999deb690", "3f07e772cd2f4d99bc3f1e84328115d9", "01d367825f024966a4bcb19c91daac49", "dc1428f7625940328fc581e4655e8bd9", "b3fa6ba0ad0f4473b810eff8d39ddc02", "9635dee4ead64f7bada62ea7ddca043d", "0a9b960d33e349a8a166e266ed30c091", "59060f72bc20432a9404c556ae216d44", "655abc0f05a7422888ecc1cf92117a2d", "0f30422325c546b4a1dce2436909f299", "020bae401b644380a73996ca354ec9cb", "a2e7c0e743a74b298de97925a883148b", "16a098e44bb84e6d9bccfdc147bde914", "7558fdd152284b79a49fc7f3fef4cf49", "fffd422ff5a8440595e66dfe2dd40d5f", "ad3178efdce44dcb89834fb4c16fa55b", "3fd752f843234f72912663bff9f02c30", "f634d234fcf844c6b0a0e61a216fa815", "b6f085183d074db49c6848e1e48a614b", "8a0f4f1465e24cd3a1089224f0681377", "290b495521c64d34a5140819b1a47b44", "649805290155424ab3a79fec0b14dc46", "d3a9b112686f444bb3e5ea0cbafcd3e7", "1c41eada8e83424492e59756c4ab2542", "834e6a3fc1c34a96adc83f4c49d1a6fe", "ed67c7e0a5c64c549ee270d8ba96a2bf", "d215d3e17bcf4a8fba4e4938be2a05ac", "873500d5d1444dc386d2743249c1b4a5", "cb9086e3419c4cd48c4da879accb0338", "9a4ed7ed072942ffb9fcd7988324eb94", "9fe3931d6ce14b7bafb74c501794effb", "076ed893328649faa728e9af98c0d0fe", "953e703e34214c838f492d63aace6bd4", "eb672a17ab1947d68bc5769a74196c5d", "c951fd79d7224552a01c43d61d693487", "31680e277c254322869342ae560c25f3", "5c31eee39b9b4849b9df57560e1f6a25", "873091e5fe13469789e9dc426bce88ca", "b49e72b2d8cf4b8db5f043e8eb5332a4", "b01cb0d8ca7f4ad6999800c889123d4e", "7f25350a364744eba11d211312064f18", "14677007ffde4a6b9e0275d26048a5f5", "d0fe326b601a437e99b72aa2ef457b8d", "a3e92c5d31a9466a94a4e3f6b1f579db", "b4d2d879ce88430e9784206e32f39149", "10d5e68b4ff14e66b3fe706b9e9b9a07", "1b03d1e0fbd34ac59a0dfb3b6c2a3887", "908ab742de6c42ac9152fc0f74073525", "af76a65cb1834e399e13732d395a3904" ] }, "id": "47ff17e0f41702d2", "outputId": "75251786-8b8c-4134-9633-a9d9e1fcce46" }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "/usr/local/lib/python3.12/dist-packages/huggingface_hub/utils/_auth.py:94: UserWarning: \n", "The secret `HF_TOKEN` does not exist in your Colab secrets.\n", "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n", "You will be able to reuse this secret in all of your notebooks.\n", "Please note that authentication is recommended but still optional to access public models or datasets.\n", " warnings.warn(\n" ] }, { "output_type": "display_data", "data": { "text/plain": [ "README.md: 0.00B [00:00, ?B/s]" ], "application/vnd.jupyter.widget-view+json": { "version_major": 2, "version_minor": 0, "model_id": "32ffd5638dcf40538db1584dd69caea2" } }, "metadata": {} }, { "output_type": "display_data", "data": { "text/plain": [ "split/train-00000-of-00001.parquet: 0%| | 0.00/1.03M [00:00\n", " \n", " \n", " [ 785/1000 09:00 < 02:28, 1.45 it/s, Epoch 0.78/1]\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
EpochTraining LossValidation Loss

" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "output_type": "display_data", "data": { "text/plain": [ "" ], "text/html": [ "\n", "

\n", " \n", " \n", " [1000/1000 12:05, Epoch 1/1]\n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
EpochTraining LossValidation LossAccuracy
10.2054000.1794990.927000

" ] }, "metadata": {} }, { "output_type": "execute_result", "data": { "text/plain": [ "TrainOutput(global_step=1000, training_loss=0.379160888671875, metrics={'train_runtime': 728.165, 'train_samples_per_second': 21.973, 'train_steps_per_second': 1.373, 'total_flos': 2119629570048000.0, 'train_loss': 0.379160888671875, 'epoch': 1.0})" ] }, "metadata": {}, "execution_count": 12 } ], "source": [ "trainer = Trainer(\n", " model=model,\n", " args=training_args,\n", " train_dataset=datos_tokenizados['train'],\n", " eval_dataset=datos_tokenizados['validation'],\n", " compute_metrics=compute_metrics,\n", ")\n", "\n", "trainer.train()" ] }, { "cell_type": "markdown", "id": "ONNQFTY5qCTj", "metadata": { "id": "ONNQFTY5qCTj" }, "source": [ "## Inferencia\n", "\n", "Para hacer inferencia usando el modelo, crearemos una pipeline Huggingface. Esta pipeline utilizará el modelo y el tokenizador que importamos anteriormente.\n", "\n", "Luego usaremos la pipeline para hacer una inferencia con un texto de ejemplo." ] }, { "cell_type": "code", "execution_count": 13, "id": "cRR9pIOThniA", "metadata": { "id": "cRR9pIOThniA", "outputId": "70d1c7bc-08c6-49a4-bb5d-7296be60b833", "colab": { "base_uri": "https://localhost:8080/" } }, "outputs": [ { "output_type": "stream", "name": "stderr", "text": [ "Device set to use cuda:0\n" ] }, { "output_type": "stream", "name": "stdout", "text": [ "[{'label': 'ANGER', 'score': 0.34769198298454285}]\n", "[{'label': 'JOY', 'score': 0.9859300255775452}]\n", "[{'label': 'ANGER', 'score': 0.48408079147338867}]\n", "[{'label': 'ANGER', 'score': 0.9862304329872131}]\n" ] } ], "source": [ "from transformers import pipeline\n", "\n", "classifier = pipeline(\"sentiment-analysis\", model=model, tokenizer=tokenizer)\n", "print(classifier(\"Suddenly, I'm not half the man I used to be, There's a shadow hanging over me, Oh, yesterday came suddenly\"))\n", "print(classifier(\"Don't stop me now. I'm havin' such a good time, I'm havin' a ball. If you wanna have a good time, just give me a call\"))\n", "print(classifier(\"Remember those who win the game. Lose the love they sought to gain. In debentures of quality. And dubious integrity. Their small town eyes will gape at you. In dull surprise when payment due. Exceeds accounts received. At seventeen\"))\n", "print(classifier(\"You got your bitches with the silicone injections. Crystal meth and yeast infections. Bleached blond hair, collagen lip injections. Who are you to criticize my intentions?. Got your subtle, manipulative devices. Just like you, I got my vices. I got a thought that would be nice. I'd like to crush your head, tight in my vice. Pain\"))" ] } ], "metadata": { "accelerator": "GPU", "colab": { "gpuType": "T4", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.4" } }, "nbformat": 4, "nbformat_minor": 5 }