{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "abstract: |\n", " Machine learning solutions, in particular those based on deep learning\n", " methods, form an underpinning of the current revolution in “artificial\n", " intelligence” that has dominated popular press headlines and is having a\n", " significant influence on the wider tech agenda. In this talk I will give\n", " an overview of where we are now with machine learning solutions, and\n", " what challenges we face both in the near and far future. These include\n", " practical application of existing algorithms in the face of the need to\n", " explain decision making, mechanisms for improving the quality and\n", " availability of data, dealing with large unstructured datasets.\n", "affiliation: University of Sheffield\n", "author: 'Neil D. Lawrence'\n", "bibliography:\n", "- '../other.bib'\n", "- '../lawrence.bib'\n", "- '../zbooks.bib'\n", "csl: '../elsevier-harvard.csl'\n", "title: Outlook for AI and Machine Learning\n", "transition: None\n", "venue: HM Treasury\n", "---\n", "\n", "#### Outlook for UK AI and Machine Learning\n", "\n", "#### 2018-05-11\n", "\n", "#### Neil D. Lawrence\n", "\n", "#### Amazon Cambridge and **University of Sheffield**\n", "\n", "`@lawrennd` [inverseprobability.com](http://inverseprobability.com)\n", "\n", "The aim of this presentation is give a sense of the current situation in\n", "machine learning and artificial intelligence as well as some perspective\n", "on the immediate outlook for the field.\n", "\n", "This presentation represents my personal opinion as an academic with 20\n", "years experience in machine learning, computational biology and data\n", "science. This is not in any sense *Amazon* policy, but since September\n", "2016 I have been on leave of absence at Amazon.\n", "\n", "The Gartner Hype Cycle\n", "----------------------\n", "\n", "\n", "\n", "The [Gartner Hype Cycle](https://en.wikipedia.org/wiki/Hype_cycle) tries\n", "to assess where an idea is in terms of maturity and adoption. It splits\n", "the evolution of technology into a technological trigger, a peak of\n", "expectations followed by a trough of disillusionment and a final\n", "ascension into a useful technology. It looks rather like a classical\n", "control response to a final set point." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import pods\n", "import mlai\n", "import teaching_plots as plot" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# calling without arguments uses the default query terms\n", "data = pods.datasets.google_trends(['big data', 'data science', 'internet of things', 'machine learning']) \n", "data['data frame'].set_index('Date', inplace=True)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "fig, ax = plt.subplots(figsize=plot.big_wide_figsize)\n", "data['data frame'].plot(ax=ax)\n", "_ = ax.set_xticklabels(ax.xaxis.get_majorticklabels(), rotation=45)\n", "plt.gcf().subplots_adjust(bottom=0.3)\n", "handles = ax.get_lines()\n", "for handle in handles:\n", " handle.set_visible(False)\n", "for i, handle in enumerate(handles):\n", " handle.set_visible(True)\n", " handle.set_linewidth(3)\n", " mlai.write_figure('../slides/diagrams/data-science/bd-ds-iot-ml-google-trends{sample:0>3}.svg'.format(sample=i))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "pods.notebook.display_plots('bd-ds-iot-ml-google-trends{sample:0>3}.svg', \n", " '../slides/diagrams/data-science', sample=(0,3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Google trends gives us insight into how far along various technological\n", "terms are on the hype cycle.\n", "\n", "What is Machine Learning?\n", "-------------------------\n", "\n", "What is machine learning? At its most basic level machine learning is a\n", "combination of\n", "\n", "$$ \\text{data} + \\text{model} \\xrightarrow{\\text{compute}} \\text{prediction}$$\n", "\n", "where *data* is our observations. They can be actively or passively\n", "acquired (meta-data). The *model* contains our assumptions, based on\n", "previous experience. THat experience can be other data, it can come from\n", "transfer learning, or it can merely be our beliefs about the\n", "regularities of the universe. In humans our models include our inductive\n", "biases. The *prediction* is an action to be taken or a categorization or\n", "a quality score. The reason that machine learning has become a mainstay\n", "of artificial intelligence is the importance of predictions in\n", "artificial intelligence. The data and the model are combined through\n", "computation.\n", "\n", "In practice we normally perform machine learning using two functions. To\n", "combine data with a model we typically make use of:\n", "\n", "**a prediction function** a function which is used to make the\n", "predictions. It includes our beliefs about the regularities of the\n", "universe, our assumptions about how the world works, e.g. smoothness,\n", "spatial similarities, temporal similarities.\n", "\n", "**an objective function** a function which defines the cost of\n", "misprediction. Typically it includes knowledge about the world’s\n", "generating processes (probabilistic objectives) or the costs we pay for\n", "mispredictions (empiricial risk minimization).\n", "\n", "The combination of data and model through the prediction function and\n", "the objectie function leads to a *learning algorithm*. The class of\n", "prediction functions and objective functions we can make use of is\n", "restricted by the algorithms they lead to. If the prediction function or\n", "the objective function are too complex, then it can be difficult to find\n", "an appropriate learning algorithm. Much of the acdemic field of machine\n", "learning is the quest for new learning algorithms that allow us to bring\n", "different types of models and data together.\n", "\n", "A useful reference for state of the art in machine learning is the UK\n", "Royal Society Report, [Machine Learning: Power and Promise of Computers\n", "that Learn by\n", "Example](https://royalsociety.org/~/media/policy/projects/machine-learning/publications/machine-learning-report.pdf).\n", "\n", "You can also check my blog post on [“What is Machine\n", "Learning?”](http://inverseprobability.com/2017/07/17/what-is-machine-learning)\n", "\n", "Natural and Artificial Intelligence: Embodiment Factors\n", "-------------------------------------------------------\n", "\n", "
\n", " | \n", "\n", "\n", " | \n", "\n", "\n", " | \n", "
\n", "compute\n", " | \n", "\n", "$$\\approx 100 \\text{ gigaflops}$$\n", " | \n", "\n", "$$\\approx 16 \\text{ petaflops}$$\n", " | \n", "
\n", "communicate\n", " | \n", "\n", "$$1 \\text{ gigbit/s}$$\n", " | \n", "\n", "$$100 \\text{ bit/s}$$\n", " | \n", "
\n", "(compute/communicate)\n", " | \n", "\n", "$$10^{4}$$\n", " | \n", "\n", "$$10^{14}$$\n", " | \n", "
\n", "Source: DeepFace\n", "
\n", "### \n", "\n", "\n", "\n", "We can think of what these models are doing as being similar to early\n", "pin ball machines. In a neural network, we input a number (or numbers),\n", "whereas in pinball, we input a ball. The location of the ball on the\n", "left-right axis can be thought of as the number. As the ball falls\n", "through the machine, each layer of pins can be thought of as a different\n", "layer of neurons. Each layer acts to move the ball from left to right.\n", "\n", "In a pinball machine, when the ball gets to the bottom it might fall\n", "into a hole defining a score, in a neural network, that is equivalent to\n", "the decision: a classification of the input object.\n", "\n", "An image has more than one number associated with it, so it’s like\n", "playing pinball in a *hyper-space*." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "import pods\n", "pods.notebook.display_plots('pinball{sample:0>3}.svg', \n", " '../slides/diagrams', sample=(1,2))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "At initialization, the pins aren’t in the right place to bring the ball\n", "to the correct decision.\n", "\n", "Learning involves moving all the pins to be in the right position, so\n", "that the ball falls in the right place. But moving all these pins in\n", "hyperspace can be difficult. In a hyper space you have to put a lot of\n", "data through the machine for to explore the positions of all the pins.\n", "Adversarial learning reflects the fact that a ball can be moved a small\n", "distance and lead to a very different result.\n", "\n", "Probabilistic methods explore more of the space by considering a range\n", "of possible paths for the ball through the machine.\n", "\n", "### Uncertainty and Learning\n", "\n", "- In this “vanilla” form these machines “don’t know when they don’t\n", " know”.\n", "\n", "- Doubt is vital in real world decision making.\n", "\n", "- Incorporating this in systems is a long time focus of my technical\n", " research.\n", "\n", "### Comparison with Human Learning & Embodiment\n", "\n", "- The emulation of intelligence does not exhibit all the\n", " meta-modelling humans perform.\n", "\n", "### Data Science\n", "\n", "- Industrial Revolution 4.0?\n", "- *Industrial Revolution* (1760-1840) term coined by Arnold Toynbee,\n", " late 19th century.\n", "- Maybe: But this one is dominated by *data* not *capital*\n", "- That presents *challenges* and *opportunities*\n", "\n", "compare [digital\n", "oligarchy](https://www.theguardian.com/media-network/2015/mar/05/digital-oligarchy-algorithms-personal-data)\n", "vs [how Africa can benefit from the data\n", "revolution](https://www.theguardian.com/media-network/2015/aug/25/africa-benefit-data-science-information)\n", "\n", "- Apple vs Nokia: How you handle disruption.\n", "\n", "Disruptive technologies take time to assimilate, and best practices, as\n", "well as the pitfalls of new technologies take time to share.\n", "Historically, new technologies led to new professions. [Isambard Kingdom\n", "Brunel](https://en.wikipedia.org/wiki/Isambard_Kingdom_Brunel) (born\n", "1806) was a leading innovator in civil, mechanical and naval\n", "engineering. Each of these has its own professional institutions founded\n", "in 1818, 1847, and 1860 respectively.\n", "\n", "[Nikola Tesla](https://en.wikipedia.org/wiki/Nikola_Tesla) developed the\n", "modern approach to electrical distribution, he was born in 1856 and the\n", "American Instiute for Electrical Engineers was founded in 1884, the UK\n", "equivalent was founded in 1871.\n", "\n", "[William Schockley Jr](https://en.wikipedia.org/wiki/William_Shockley),\n", "born 1910, led the group that developed the transistor, referred to as\n", "“the man who brought silicon to Silicon Valley”, in 1963 the American\n", "Institute for Electical Engineers merged with the Institute of Radio\n", "Engineers to form the Institute of Electrical and Electronic Engineers.\n", "\n", "[Watts S. Humphrey](https://en.wikipedia.org/wiki/Watts_Humphrey), born\n", "1927, was known as the “father of software quality”, in the 1980s he\n", "founded a program aimed at understanding and managing the software\n", "process. The British Computer Society was founded in 1956.\n", "\n", "Why the need for these professions? Much of it is about codification of\n", "best practice and developing trust between the public and practitioners.\n", "These fundamental characteristics of the professions are shared with the\n", "oldest professions (Medicine, Law) as well as the newest (Information\n", "Technology).\n", "\n", "So where are we today? My best guess is we are somewhere equivalent to\n", "the 1980s for Software Engineering. In terms of professional deployment\n", "we have a basic understanding of the equivalent of “programming” but\n", "much less understanding of *machine learning systems design* and *data\n", "infrastructure*. How the components we ahve developed interoperate\n", "together in a reliable and accountable manner. Best practice is still\n", "evolving, but perhaps isn’t being shared widely enough.\n", "\n", "One problem is that the art of data science is superficially similar to\n", "regular software engineering. Although in practice it is rather\n", "different. Modern software engineering practice operates to generate\n", "code which is well tested as it is written, agile programming techniques\n", "provide the appropriate degree of flexibility for the individual\n", "programmers alongside sufficient formalization and testing. These\n", "techniques have evolved from an overly restrictive formalization that\n", "was proposed in the early days of software engineering.\n", "\n", "While data science involves programming, it is different in the\n", "following way. Most of the work in data science involves understanding\n", "the data and the appropriate manipulations to apply to extract knowledge\n", "from the data. The eventual number of lines of code that are required to\n", "extract that knowledge are often very few, but the amount of thought and\n", "attention that needs to be applied to each line is much more than a\n", "traditional line of software code. Testing of those lines is also of a\n", "different nature, provisions have to be made for evolving data\n", "environments. Any development work is often done on a static snapshot of\n", "data, but deployment is made in a live environment where the nature of\n", "data changes. Quality control involves checking for degradation in\n", "performance arising form unanticipated changes in data quality. It may\n", "also need to check for regulatory conformity. For example, in the UK the\n", "General Data Protection Regulation stipulates standards of\n", "explainability and fairness that may need to be monitored. These\n", "concerns do not affect traditional software deployments.\n", "\n", "Others are also pointing out these challenges, [this\n", "post](https://medium.com/@karpathy/software-2-0-a64152b37c35) from\n", "Andrej Karpathy (now head of AI at Tesla) covers the notion of “Software\n", "2.0”. Google researchers have highlighted the challenges of “Technical\n", "Debt” in machine learning [@Sculley:debt15]. Researchers at Berkeley\n", "have characterized the systems challenges associated with machine\n", "learning [@Stoica:systemsml17].\n", "\n", "Data science is not only about technical expertise and analysis of data,\n", "we need to also generate a culture of decision making that acknowledges\n", "the true challenges in data-driven automated decision making. In\n", "particular, a focus on algorithms has neglected the importance of data\n", "in driving decisions. The quality of data is paramount in that poor\n", "quality data will inevitably lead to poor quality decisions. Anecdotally\n", "most data scientists will suggest that 80% of their time is spent on\n", "data clean up, and only 20% on actually modelling.\n", "\n", "### The Software Crisis\n", "\n", "> The major cause of the software crisis is that the machines have\n", "> become several orders of magnitude more powerful! To put it quite\n", "> bluntly: as long as there were no machines, programming was no problem\n", "> at all; when we had a few weak computers, programming became a mild\n", "> problem, and now we have gigantic computers, programming has become an\n", "> equally gigantic problem.\n", ">\n", "> Edsger Dijkstra (1930-2002), The Humble Programmer\n", "\n", "In the late sixties early software programmers made note of the\n", "increasing costs of software development and termed the challenges\n", "associated with it as the “[Software\n", "Crisis](https://en.wikipedia.org/wiki/Software_crisis)”. Edsger Dijkstra\n", "referred to the crisis in his 1972 Turing Award winner’s address.\n", "\n", "### The Data Crisis\n", "\n", "> The major cause of the data crisis is that machines have become more\n", "> interconnected than ever before. Data access is therefore cheap, but\n", "> data quality is often poor. What we need is cheap high quality data.\n", "> That implies that we develop processes for improving and verifying\n", "> data quality that are efficient.\n", ">\n", "> There would seem to be two ways for improving efficiency. Firstly, we\n", "> should not duplicate work. Secondly, where possible we should automate\n", "> work.\n", "\n", "What I term “The Data Crisis” is the modern equivalent of this problem.\n", "The quantity of modern data, and the lack of attention paid to data as\n", "it is initially “laid down” and the costs of data cleaning are bringing\n", "about a crisis in data-driven decision making. Just as with software,\n", "the crisis is most correctly addressed by ‘scaling’ the manner in which\n", "we process our data. Duplication of work occurs because the value of\n", "data cleaning is not correctly recognised in management decision making\n", "processes. Automation of work is increasingly possible through\n", "techniques in “artificial intelligence”, but this will also require\n", "better management of the data science pipeline so that data about data\n", "science (meta-data science) can be correctly assimilated and processed.\n", "The Alan Turing institute has a program focussed on this area, [AI for\n", "Data\n", "Analytics](https://www.turing.ac.uk/research_projects/artificial-intelligence-data-analytics/).\n", "\n", "\n", "\n", "Our current information infrastructure bears a close relation with\n", "*feudal systems* of government. In the feudal system a lord had a duty\n", "of care over his serfs and vassals, a duty to protect subjects. But in\n", "practice there was a power-asymetry. In feudal days protection was\n", "against Viking raiders, today, it is against information raiders.\n", "However, when there is an information leak, when there is a failure it\n", "is too late. Alternatively, our data is publicly shared, in an\n", "information commons. Akin to common land of the medieval village. But\n", "just as commons were subject to overgrazing and poor management, so it\n", "is that much of our data cannot be managed in this way. In particularly\n", "personal, sensitive data.\n", "\n", "I explored this idea further in [this Guardian Op-Ed from\n", "2015](https://www.theguardian.com/media-network/2015/nov/16/information-barons-threaten-autonomy-privacy-online).\n", "\n", "### Rest of the Talk\n", "\n", "- Importance of data infrastructure\n", "\n", "\n", "\n", "\n", "Public Use of Data for Public Good\n", "----------------------------------\n", "\n", "Since machine learning methods are so dependent on data, Understanding\n", "public attitudes to the use of their data is key to developing machine\n", "learning methods that maintain the trust of the public. Nowhere are the\n", "benefits of machine learning more profound, and the potential pitfalls\n", "more catastrophic than in the use of machine learning in health data.\n", "\n", "The promise is for methods that take a personalized perspective on our\n", "individual health, but health data is some of the most sensitive data\n", "available to us. This is recognised both by the public and by\n", "regulation.\n", "\n", "With this in mind The Wellcome Trust launched a report on\n", "[“Understanding Patient\n", "Data”](https://wellcome.ac.uk/news/understanding-patient-data-launches-today)\n", "authored by Nicola Perrin, driven by the National Data Guardian’s\n", "recommendations.\n", "\n", "From this report we know that patients trust Universities and hospitals\n", "more than the trust commercial entities and insurers. However, there are\n", "a number of different ways in which data can be mishandled, it is not\n", "only the intent of the data-controllers that effects our data security.\n", "\n", "For example, the recent WannaCry virus attack which demonstrated the\n", "unpreparedness of much of the NHS IT infrastructure for a virus\n", "exhibiting an exploit that was well known to the security community. The\n", "key point is that the public trust the *intent* of academics and medical\n", "professionals, but actual *capability* could be at variance with the\n", "intent.\n", "\n", "\n", "\n", "