{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/duoan/TorchCode/blob/master/templates/26_lora.ipynb)\n",
    "\n",
    "# 🟠 Medium: LoRA (Low-Rank Adaptation)\n",
    "\n",
    "Implement **LoRA** — parameter-efficient fine-tuning for large models.\n",
    "\n",
    "$$h = W_0 x + \\frac{\\alpha}{r} B A x$$\n",
    "\n",
    "### Signature\n",
    "```python\n",
    "class LoRALinear(nn.Module):\n",
    "    def __init__(self, in_features, out_features, rank, alpha=1.0): ...\n",
    "    def forward(self, x: Tensor) -> Tensor: ...\n",
    "```\n",
    "\n",
    "### Requirements\n",
    "- `self.linear`: frozen `nn.Linear` (weight & bias `requires_grad=False`)\n",
    "- `self.lora_A`: `nn.Parameter(rank, in_features)` — random init\n",
    "- `self.lora_B`: `nn.Parameter(out_features, rank)` — **zero** init\n",
    "- Scaling: `alpha / rank`"
   ],
   "outputs": []
  },
  {
   "cell_type": "code",
   "metadata": {},
   "source": [
    "# Install torch-judge in Colab (no-op in JupyterLab/Docker)\n",
    "try:\n",
    "    import google.colab\n",
    "    get_ipython().run_line_magic('pip', 'install -q torch-judge')\n",
    "except ImportError:\n",
    "    pass\n"
   ],
   "outputs": [],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn as nn"
   ],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ✏️ YOUR IMPLEMENTATION HERE\n",
    "\n",
    "class LoRALinear(nn.Module):\n",
    "    def __init__(self, in_features, out_features, rank, alpha=1.0):\n",
    "        super().__init__()\n",
    "        pass  # frozen linear + lora_A + lora_B\n",
    "\n",
    "    def forward(self, x):\n",
    "        pass  # base + lora"
   ],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 🧪 Debug\n",
    "layer = LoRALinear(16, 8, rank=4)\n",
    "x = torch.randn(2, 16)\n",
    "print('Output:', layer(x).shape)\n",
    "print('Trainable:', sum(p.numel() for p in layer.parameters() if p.requires_grad))\n",
    "print('Total:    ', sum(p.numel() for p in layer.parameters()))"
   ],
   "execution_count": null
  },
  {
   "cell_type": "code",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ✅ SUBMIT\n",
    "from torch_judge import check\n",
    "check('lora')"
   ],
   "execution_count": null
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}