{ "cells": [ { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from fastai2.text.all import *" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "chunked??" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's look at how long it takes to tokenize a sample of 1000 IMDB review." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
labeltextis_valid
0negativeUn-bleeping-believable! Meg Ryan doesn't even look her usual pert lovable self in this, which normally makes me forgive her shallow ticky acting schtick. Hard to believe she was the producer on this dog. Plus Kevin Kline: what kind of suicide trip has his career been on? Whoosh... Banzai!!! Finally this was directed by the guy who did Big Chill? Must be a replay of Jonestown - hollywood style. Wooofff!False
1positiveThis is a extremely well-made film. The acting, script and camera-work are all first-rate. The music is good, too, though it is mostly early in the film, when things are still relatively cheery. There are no really superstars in the cast, though several faces will be familiar. The entire cast does an excellent job with the script.<br /><br />But it is hard to watch, because there is no good end to a situation like the one presented. It is now fashionable to blame the British for setting Hindus and Muslims against each other, and then cruelly separating them into two countries. There is som...False
\n", "
" ], "text/plain": [ " label \\\n", "0 negative \n", "1 positive \n", "\n", " text \\\n", "0 Un-bleeping-believable! Meg Ryan doesn't even look her usual pert lovable self in this, which normally makes me forgive her shallow ticky acting schtick. Hard to believe she was the producer on this dog. Plus Kevin Kline: what kind of suicide trip has his career been on? Whoosh... Banzai!!! Finally this was directed by the guy who did Big Chill? Must be a replay of Jonestown - hollywood style. Wooofff! \n", "1 This is a extremely well-made film. The acting, script and camera-work are all first-rate. The music is good, too, though it is mostly early in the film, when things are still relatively cheery. There are no really superstars in the cast, though several faces will be familiar. The entire cast does an excellent job with the script.

But it is hard to watch, because there is no good end to a situation like the one presented. It is now fashionable to blame the British for setting Hindus and Muslims against each other, and then cruelly separating them into two countries. There is som... \n", "\n", " is_valid \n", "0 False \n", "1 False " ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "path = untar_data(URLs.IMDB_SAMPLE)\n", "df = pd.read_csv(path/'texts.csv')\n", "df.head(2)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "\"Un-bleeping-believable! Meg Ryan doesn't even look her usual pert lovable self in this, which normally makes me forgive her shallow ticky acting schtick. Hard to believe she was the producer on this dog. Plus Kevin Kline: what kind of suicide trip has his career been on? Whoosh... Banzai!!! Finally this was directed by the guy who did Big Chill? Must be a replay of Jonestown - hollywood style. Wooofff!\"" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ss = L(list(df.text))\n", "ss[0]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We'll start with the simplest approach:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(#69) ['Un-bleeping-believable!','Meg','Ryan',\"doesn't\",'even','look','her','usual','pert','lovable'...]" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def delim_tok(s, delim=' '): return L(s.split(delim))\n", "s = ss[0]\n", "delim_tok(s)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "...and a general way to tokenize a bunch of strings:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def apply(func, items): return list(map(func, items))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's time it:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "29.9 ms ± 722 µs per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "global t\n", "t = apply(delim_tok, ss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "...and the same thing with 2 workers:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "325 ms ± 3.52 ms per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "parallel(delim_tok, ss, n_workers=2, progress=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "How about if we put half the work in each worker?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "batches32 = [L(list(o)).map(str) for o in np.array_split(ss, 32)]\n", "batches8 = [L(list(o)).map(str) for o in np.array_split(ss, 8 )]\n", "batches = [L(list(o)).map(str) for o in np.array_split(ss, 2 )]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "146 ms ± 7.26 ms per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "parallel(partial(apply, delim_tok), batches, progress=False, n_workers=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So there's a lot of overhead in using parallel processing in Python. :(\n", "\n", "Let's see why. What if we do nothing interesting in our function?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "52 ms ± 5.06 ms per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "global t\n", "t = parallel(noop, batches, progress=False, n_workers=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's quite fast! (Although still slower than single process.)\n", "\n", "What if we don't return much data?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def f(x): return 1" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "44.7 ms ± 2.82 ms per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "global t\n", "t = parallel(f, batches, progress=False, n_workers=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's a bit faster still.\n", "\n", "What if we don't actually return the lists of tokens, but create them still?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def f(items):\n", " o = [s.split(' ') for s in items]\n", " return [s for s in items]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So creating the tokens, isn't taking the time, but returning them over the process boundary is." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "65.4 ms ± 445 µs per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "global t\n", "t = parallel(f, batches, progress=False, n_workers=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Is numpy any faster?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sarr = np.array(ss)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "53.2 ms ± 55.4 µs per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "global t\n", "t = np.char.split(sarr)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Spacy" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from spacy.lang.en import English\n", "\n", "def conv_sp(doc): return L(doc).map(str)\n", "\n", "class SpTok:\n", " def __init__(self):\n", " nlp = English()\n", " self.tok = nlp.Defaults.create_tokenizer(nlp)\n", " \n", " def __call__(self, x): return L(self.tok(str(x))).map(conv_sp)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's see how long it takes to create a tokenizer in Spacy:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "478 ms ± 21 ms per loop (mean ± std. dev. of 3 runs, 2 loops each)\n" ] } ], "source": [ "%%timeit -n 2 -r 3\n", "SpTok()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "nlp = English()\n", "sp_tokenizer = nlp.Defaults.create_tokenizer(nlp)\n", "def spacy_tok(s): return L(sp_tokenizer(str(s))).map(str)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Time tokenize in Spacy using a loop:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2.63 s ± 37.5 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = apply(spacy_tok, ss)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "...and the same thing in parallel:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "1.65 s ± 28.1 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = parallel(partial(apply, spacy_tok), batches, progress=False, n_workers=2)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "...and with more workers:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "527 ms ± 5.1 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = parallel(partial(apply, spacy_tok), batches8, progress=False, n_workers=8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "...and with creating the tokenizer in the child process:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def f(its):\n", " tok = SpTok()\n", " return [[str(o) for o in tok(p)] for p in its]" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2.08 s ± 10.2 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = parallel(f, batches8, progress=False, n_workers=8)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's try `pipe`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "2.51 s ± 41.3 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = L(nlp.tokenizer.pipe(ss)).map(conv_sp)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def f(its): return L(nlp.tokenizer.pipe(its)).map(conv_sp)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "539 ms ± 5.86 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = parallel(f, batches8, progress=False, n_workers=8)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "test_eq(chunked(range(12),n_chunks=4), [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10, 11]])\n", "test_eq(chunked(range(11),n_chunks=4), [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9, 10]])\n", "test_eq(chunked(range(10),n_chunks=4), [[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]])\n", "test_eq(chunked(range( 9),n_chunks=3), [[0, 1, 2], [3, 4, 5], [6, 7, 8]])" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "607 ms ± 7.87 ms per loop (mean ± std. dev. of 3 runs, 1 loop each)\n" ] } ], "source": [ "%%timeit -r 3\n", "global t\n", "t = parallel_chunks(f, ss, n_workers=8, progress=False)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def array_split(arr, n): return chunked(arr, math.floor(len(arr)/n))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 4 }