{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "source": [
    "<!---\n",
    "Latex Macros\n",
    "-->\n",
    "$$\n",
    "\\newcommand{\\Xs}{\\mathcal{X}}\n",
    "\\newcommand{\\Ys}{\\mathcal{Y}}\n",
    "\\newcommand{\\y}{\\mathbf{y}}\n",
    "\\newcommand{\\balpha}{\\boldsymbol{\\alpha}}\n",
    "\\newcommand{\\bbeta}{\\boldsymbol{\\beta}}\n",
    "\\newcommand{\\aligns}{\\mathbf{a}}\n",
    "\\newcommand{\\align}{a}\n",
    "\\newcommand{\\source}{\\mathbf{s}}\n",
    "\\newcommand{\\target}{\\mathbf{t}}\n",
    "\\newcommand{\\ssource}{s}\n",
    "\\newcommand{\\starget}{t}\n",
    "\\newcommand{\\repr}{\\mathbf{f}}\n",
    "\\newcommand{\\repry}{\\mathbf{g}}\n",
    "\\newcommand{\\x}{\\mathbf{x}}\n",
    "\\newcommand{\\prob}{p}\n",
    "\\newcommand{\\vocab}{V}\n",
    "\\newcommand{\\params}{\\boldsymbol{\\theta}}\n",
    "\\newcommand{\\param}{\\theta}\n",
    "\\DeclareMathOperator{\\perplexity}{PP}\n",
    "\\DeclareMathOperator{\\argmax}{argmax}\n",
    "\\DeclareMathOperator{\\argmin}{argmin}\n",
    "\\newcommand{\\train}{\\mathcal{D}}\n",
    "\\newcommand{\\counts}[2]{\\#_{#1}(#2) }\n",
    "\\newcommand{\\length}[1]{\\text{length}(#1) }\n",
    "\\newcommand{\\indi}{\\mathbb{I}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 2\n",
    "\n",
    "* Fixed [shirtless slide](/notebooks/chapters/introduction.ipynb#/slide-9-0)!\n",
    "* Workload for assignments: 5h-20h per assignment, depending on your background, skills etc\n",
    "* Module Selection Priorities:\n",
    "    * Optional\n",
    "    * Elective\n",
    "    * Non-CS (update: will not be impossible)\n",
    "* First Come First Serve, but Elective students will only know if they have a spot after next Friday"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 3\n",
    "\n",
    "* Please use the moodle for questions regarding the assignment\n",
    "* No office hour today\n",
    "* Next: [Language Models: MLE](chapters/language_models_slides.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "# Open Items Lecture 4\n",
    "\n",
    "* Assignment questions?\n",
    "    * Please use the moodle for questions regarding the assignment\n",
    "* I will not be available for office hours today\n",
    "* Next: [Language Models: MLE](chapters/language_models_slides.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "# Open Items Lecture 5\n",
    "\n",
    "* Assignment Questions\n",
    "    * Are we allowed to use the `replace_OOVs` function on `_snlp_test_song_words`?\n",
    "    * Will the hidden test set we are assessed on already have the replace OOV's function applied to it? \n",
    "    * Can we use helper functions from the exercices in the coursework?\n",
    "    * Question on the `OOVAwareLM`\n",
    "* Next: Word-Based Machine Translation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 6\n",
    "\n",
    "* Do read the course notes\n",
    "* Assignment:\n",
    "    * You are allowed to use both development and training set as \"larger training set\" but understand the potential drawbacks.\n",
    "    * If you try something interesting that doesn't work, do report it!\n",
    "    * There is a potential to cheat (don't)\n",
    "* Re question on simplification: http://nlp.cs.swarthmore.edu/semeval/tasks/task10/summary.shtml\n",
    "* Next: MT Noisy Channel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 7 \n",
    "\n",
    "* LectureCast: online! \n",
    "* Next: [Word-based MT](/notebooks/chapters/word_mt_slides.ipynb), EM Algorithm\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 8\n",
    "\n",
    "* Assignments: \n",
    "    * It's possible to train most (unless neural) LMs by scanning through the data once and gathering counts\n",
    "    * You often do not need to calculate normalizers by explicit summing (e.g. in Absolute Discounting)\n",
    "* Next: CFGs in [Parsing](/notebooks/chapters/parsing_slides.ipynb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 9\n",
    "\n",
    "* Assignment 2 coming out today!\n",
    "    * will be about feature engineering\n",
    "    * see lecture notes on text classification and sequence labelling\n",
    "    * you have 4 weeks! \n",
    "* After reading week we will discuss possible MSc projects\n",
    "* Next: [Parsing](/notebooks/chapters/parsing_slides.ipynb), PCFG\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 10\n",
    "\n",
    "* [South England NLP Meetups](https://www.meetup.com/South-England-Natural-Language-Processing-Meetup/?_cookie-check=BhaN3MFi2WYR6z7T)\n",
    "* Next: text classification\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 11\n",
    "\n",
    "* Office hours: now at 90 High Holborn, still Friday 5 PM - 6 PM\n",
    "* Dec 6 & 8: Pontus will present\n",
    "* Last Week: Industry Speakers (currently: DeepMind, Benevolent AI)\n",
    "* Next: Conditional Log-Linear Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 12\n",
    "\n",
    "* Please fill out the attendance form\n",
    "* Assignment 1 Marking: \n",
    "  * Students did change the \"do not change\" fields\n",
    "  * Students did submit code that didn't work (I will get in touch with you separately) \n",
    "* Assignment 2 Typo: You get 20pts for Task 1!  \n",
    "* Office hours: now at 90 High Holborn, still Friday 5 PM - 6 PM\n",
    "    * But I will head over to the cafeteria in the Roberts building to chat for a while \n",
    "* Next: Sequence Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 13\n",
    "\n",
    "* Assignment 2: Annotation Noise?\n",
    "* Next: Maximum Entropy Markov Models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 14\n",
    "\n",
    "* Assignment 2: You can't save/submit your training parameters, but you can fix your hyperparameters\n",
    "* Thanks for amazing support on Moodle! \n",
    "* Next: Unsupervised Relation Extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 15\n",
    "\n",
    "* Assignment 1: Marks out today.\n",
    "  * Scores for description come with short justifications based on a global point system \n",
    "  * Creativity vs Substance: KN standard on creativity, high on substance\n",
    "* Assignment 2: Due today! \n",
    "* Assignment 3: coming out today \n",
    "* Next: Dependency Parsing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 16\n",
    " \n",
    "* Assignment 1 Marks\n",
    "  * Please be polite when requesting re-assessment or raise issues  \n",
    "  * We will not change the valuation of contributions, but if we find that other students got more points for \"the same thing\", we will adapt. \n",
    "  * Please do make sure your code runs, and your text renders.\n",
    "* MSc Projects: \n",
    "  * [Initial Ideas](https://docs.google.com/presentation/d/1fdgNcF49uAHUcyJyVp16mzMJ7pojPQ2zK7g0eRvjXwg/edit#slide=id.g2b7a5d13b8_4_2)\n",
    "  * [Previous projects](http://mr.cs.ucl.ac.uk/prev_msc_dissertations)\n",
    "* Next Week: Pontus Stenetorp\n",
    "* Next: More Deep Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Open Items Lecture 19\n",
    " \n",
    "* Assignment 1 example solutions:\n",
    "  * modelling lower-case/upper case \n",
    "  * caching: looking at the most recent histories and build a local LM + interpolate\n",
    "      * motivate by repetition in songs\n",
    "  * special treatment of `BAR`\n",
    "  * OOV parameters\n",
    "  * cross-validation of parameters to see robustness\n",
    "  * motivate higher order models by showing hit-rate\n",
    "* Next: injecting prior knowledge into NNs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}