{ "metadata": { "name": "" }, "nbformat": 3, "nbformat_minor": 0, "worksheets": [ { "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "[Python para Desenvolvedores](http://ricardoduarte.github.io/python-para-desenvolvedores/#conteudo)\n", "===================================\n", "2ª edi\u00e7\u00e3o, revisada e ampliada\n", "-----------------------------------\n", "\n", "Cap\u00edtulo 39: Performance\n", "=============================\n", "_____________________________\n", "O Python prov\u00ea algumas ferramentas para avaliar performance e localizar gargalos na aplica\u00e7\u00e3o. Entre estas ferramentas est\u00e3o os m\u00f3dulos *cProfile* e *timeit*.\n", "\n", "O m\u00f3dulo cProfile faz uma an\u00e1lise detalhada de performance, incluindo as chamadas de fun\u00e7\u00e3o, retornos de fun\u00e7\u00e3o e exce\u00e7\u00f5es.\n", "\n", "Exemplo:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import cProfile\n", "\n", "def rgb1():\n", " \"\"\"\n", " Fun\u00e7\u00e3o usando range()\n", " \"\"\"\n", " rgbs = []\n", " for r in range(256):\n", " for g in range(256):\n", " for b in range(256):\n", " rgbs.append('#%02x%02x%02x' % (r, g, b))\n", " return rgbs\n", "\n", "def rgb2():\n", " \"\"\"\n", " Fun\u00e7\u00e3o usando xrange()\n", " \"\"\"\n", " rgbs = []\n", " for r in xrange(256):\n", " for g in xrange(256):\n", " for b in xrange(256):\n", " rgbs.append('#%02x%02x%02x' % (r, g, b))\n", " return rgbs\n", "\n", "def rgb3():\n", " \"\"\"\n", " Gerador usando xrange()\n", " \"\"\"\n", " for r in xrange(256):\n", " for g in xrange(256):\n", " for b in xrange(256):\n", " yield '#%02x%02x%02x' % (r, g, b)\n", "\n", "def rgb4():\n", " \"\"\"\n", " Fun\u00e7\u00e3o usando uma lista v\u00e1rias vezes\n", " \"\"\"\n", " rgbs = []\n", " ints = range(256)\n", " for r in ints:\n", " for g in ints:\n", " for b in ints:\n", " rgbs.append('#%02x%02x%02x' % (r, g, b))\n", " return rgbs\n", "\n", "def rgb5():\n", " \"\"\"\n", " Gerador usando apenas uma lista\n", " \"\"\"\n", " for i in range(256 ** 3):\n", " yield '#%06x' % i\n", "\n", "def rgb6():\n", " \"\"\"\n", " Gerador usando xrange() uma vez\n", " \"\"\"\n", " for i in xrange(256 ** 3):\n", " yield '#%06x' % i\n", "\n", "# Benchmarks\n", "print 'rgb1:'\n", "cProfile.run('rgb1()')\n", "\n", "print 'rgb2:'\n", "cProfile.run('rgb2()')\n", "\n", "print 'rgb3:'\n", "cProfile.run('list(rgb3())')\n", "\n", "print 'rgb4:'\n", "cProfile.run('rgb4()')\n", "\n", "print 'rgb5:'\n", "cProfile.run('list(rgb5())')\n", "\n", "print 'rgb6:'\n", "cProfile.run('list(rgb6())')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "rgb1:\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 16843012 function calls in 24.060 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 1 23.070 23.070 23.875 23.875 :3(rgb1)\n", " 1 0.185 0.185 24.060 24.060 :1()\n", " 16777216 0.739 0.000 0.739 0.000 {method 'append' of 'list' objects}\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", " 65793 0.067 0.000 0.067 0.000 {range}\n", "\n", "\n", "rgb2:\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 16777219 function calls in 23.214 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 1 22.293 22.293 23.027 23.027 :14(rgb2)\n", " 1 0.187 0.187 23.214 23.214 :1()\n", " 16777216 0.734 0.000 0.734 0.000 {method 'append' of 'list' objects}\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", "\n", "\n", "rgb3:\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 16777219 function calls in 23.711 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 16777217 22.172 0.000 22.172 0.000 :25(rgb3)\n", " 1 1.540 1.540 23.711 23.711 :1()\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", "\n", "\n", "rgb4:\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 16777220 function calls in 23.812 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 1 22.840 22.840 23.609 23.609 :34(rgb4)\n", " 1 0.203 0.203 23.812 23.812 :1()\n", " 16777216 0.770 0.000 0.770 0.000 {method 'append' of 'list' objects}\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", " 1 0.000 0.000 0.000 0.000 {range}\n", "\n", "\n", "rgb5:\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 16777220 function calls in 10.513 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 16777217 8.896 0.000 9.111 0.000 :46(rgb5)\n", " 1 1.402 1.402 10.513 10.513 :1()\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", " 1 0.215 0.215 0.215 0.215 {range}\n", "\n", "\n", "rgb6:\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 16777219 function calls in 10.369 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 16777217 8.888 0.000 8.888 0.000 :53(rgb6)\n", " 1 1.481 1.481 10.369 10.369 :1()\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", "\n", "\n" ] } ], "prompt_number": 1 }, { "cell_type": "markdown", "metadata": {}, "source": [ "O relat\u00f3rio do *cProfile* mostra no inicio as duas informa\u00e7\u00f5es mais importantes: o tempo de CPU consumido em segundos e a quantidade de chamadas de fun\u00e7\u00e3o. As outras linhas mostram os detalhes por fun\u00e7\u00e3o, incluindo o tempo total e por chamada.\n", "\n", "As cinco rotinas do exemplo t\u00eam a mesma funcionalidade: geram uma escala de cores RGB. Por\u00e9m, o tempo de execu\u00e7\u00e3o \u00e9 diferente.\n", "\n", "Comparando os resultados:\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
RotinaTipoTempoLa\u00e7osx/range()
rgb1()Fun\u00e7\u00e3o24.0603range()
rgb2()Fun\u00e7\u00e3o23.2143xrange()
rgb3()Gerador23.7113xrange()
rgb4()Fun\u00e7\u00e3o23.8123range()
rgb5()Gerador10.5131range()
rgb6()Gerador10.3691xrange()
\n", "\n", "Fatores observados que pesaram no desempenho:\n", "\n", "+ A complexidade do algoritmo.\n", "+ Geradores apresentaram melhores resultados do que as fun\u00e7\u00f5es tradicionais.\n", "+ O gerador `xrange()` apresentou uma performance ligeiramente melhor do que a fun\u00e7\u00e3o `range()`.\n", "\n", "O gerador `rgb6()`, que usa apenas um la\u00e7o e `xrange()`, \u00e9 bem mais eficiente que as outras rotinas.\n", "\n", "Outro exemplo:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import cProfile\n", "\n", "def fib1(n):\n", " \"\"\"\n", " Fibonacci calculado de forma recursiva.\n", " \"\"\"\n", " if n > 1:\n", " return fib1(n - 1) + fib1(n - 2)\n", " else:\n", " return 1\n", "\n", "def fib2(n):\n", " \"\"\"\n", " Fibonacci calculado por um loop.\n", " \"\"\"\n", " if n > 1:\n", "\n", " # O dicion\u00e1rio guarda os resultados\n", " fibs = {0:1, 1:1}\n", " for i in xrange(2, n + 1):\n", " fibs[i] = fibs[i - 1] + fibs[i - 2]\n", " return fibs[n]\n", " else:\n", " return 1\n", "\n", "print 'fib1'\n", "cProfile.run('[fib1(x) for x in xrange(1, 31)]')\n", "print 'fib2'\n", "cProfile.run('[fib2(x) for x in xrange(1, 31)]')" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "fib1\n", " " ] }, { "output_type": "stream", "stream": "stdout", "text": [ " 7049124 function calls (32 primitive calls) in 1.449 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", "7049122/30 1.448 0.000 1.448 0.048 :3(fib1)\n", " 1 0.000 0.000 1.449 1.449 :1()\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", "\n", "\n", "fib2\n", " 32 function calls in 0.000 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 30 0.000 0.000 0.000 0.000 :12(fib2)\n", " 1 0.000 0.000 0.000 0.000 :1()\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", "\n", "\n" ] } ], "prompt_number": 2 }, { "cell_type": "markdown", "metadata": {}, "source": [ "A performance do c\u00e1lculo da s\u00e9rie de Fibonacci usando um la\u00e7o que preenche um dicion\u00e1rio \u00e9 muito mais eficiente do que a vers\u00e3o usando recurs\u00e3o, que faz muitas chamadas de fun\u00e7\u00e3o.\n", "\n", "O m\u00f3dulo *timeit* serve para fazer *benchmark* de pequenos trechos de c\u00f3digo. O m\u00f3dulo foi projetado para evitar as falhas mais comuns que afetam programas usados para fazer *benchmarks*.\n", "\n", "Exemplo:" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import timeit\n", "\n", "# Lista dos quadrados de 1 a 1000\n", "cod = '''s = []\n", "for i in xrange(1, 1001):\n", " s.append(i ** 2)\n", "'''\n", "print timeit.Timer(cod).timeit()\n", "\n", "# Com Generator Expression\n", "cod = 'list(x ** 2 for x in xrange(1, 1001))'\n", "print timeit.Timer(cod).timeit()\n", "\n", "# Com List Comprehesion\n", "cod = '[x ** 2 for x in xrange(1, 1001)]'\n", "print timeit.Timer(cod).timeit()" ], "language": "python", "metadata": {}, "outputs": [ { "output_type": "stream", "stream": "stdout", "text": [ "111.733070135\n", "80.269646883" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n", "71.9685299397" ] }, { "output_type": "stream", "stream": "stdout", "text": [ "\n" ] } ], "prompt_number": 3 }, { "cell_type": "markdown", "metadata": {}, "source": [ "O *List Comprehension* \u00e9 mais eficiente do que o la\u00e7o tradicional.\n", "\n", "Outra forma de melhorar a performance de uma aplica\u00e7\u00e3o \u00e9 usando o Psyco, que \u00e9 uma esp\u00e9cie de *Just In Time Compiler* (JIT). Durante a execu\u00e7\u00e3o, ele tenta otimizar o c\u00f3digo da aplica\u00e7\u00e3o e, por isso, o m\u00f3dulo deve ser importado antes do c\u00f3digo a ser otimizado (o inicio do m\u00f3dulo principal da aplica\u00e7\u00e3o \u00e9 um lugar adequado).\n", "\n", "Exemplo (com o \u00faltimo trecho de c\u00f3digo avaliado no exemplo anterior):" ] }, { "cell_type": "code", "collapsed": false, "input": [ "import psyco\n", "\n", "# Tente otimizar tudo\n", "psyco.full()\n", "\n", "import timeit\n", "\n", "# Lista dos quadrados de 1 a 1000\n", "cod = '[x ** 2 for x in xrange(1, 1001)]'\n", "print timeit.Timer(cod).timeit()" ], "language": "python", "metadata": {}, "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "26.678481102\n", "\n", "O c\u00f3digo foi executado mais de duas vezes mais r\u00e1pido do que antes. Para isso, foi necess\u00e1rio apenas acrescentar duas linhas de c\u00f3digo.\n", "\n", "Por\u00e9m, o Psyco deve ser usado com alguns cuidados, pois em alguns casos ele pode n\u00e3o conseguir otimizar ou at\u00e9 piorar a performance. As fun\u00e7\u00f5es `map()` e `filter()` devem ser evitadas e m\u00f3dulos escritos em C, como o re (express\u00f5es regulares) devem ser marcados com a fun\u00e7\u00e3o `cannotcompile()` para que o Psyco os ignore. O m\u00f3dulo fornece formas de otimizar apenas determinadas partes do c\u00f3digo da aplica\u00e7\u00e3o, tal como a fun\u00e7\u00e3o `profile()`, que s\u00f3 otimiza as partes mais pesadas do aplicativo, e uma fun\u00e7\u00e3o `log()` que analisa a aplica\u00e7\u00e3o, para contornar estas situa\u00e7\u00f5es.\n", "\n", "Algumas dicas sobre otimiza\u00e7\u00e3o:\n", "\n", "+ Mantenha o c\u00f3digo simples.\n", "+ Otimize apenas o c\u00f3digo aonde a performance da aplica\u00e7\u00e3o \u00e9 realmente cr\u00edtica.\n", "+ Use ferramentas para identificar os gargalos no c\u00f3digo.\n", "+ Evite fun\u00e7\u00f5es recursivas.\n", "+ Use os recursos nativos da linguagem. As listas e dicion\u00e1rios do Python s\u00e3o muito otimizados.\n", "+ Use *List Comprehensions* ao inv\u00e9s de la\u00e7os para processar listas usando express\u00f5es simples.\n", "+ Evite fun\u00e7\u00f5es dentro de la\u00e7os. Fun\u00e7\u00f5es podem receber e devolver listas.\n", "+ Use geradores ao inv\u00e9s de fun\u00e7\u00f5es para grandes sequ\u00eancias de dados." ] }, { "cell_type": "code", "collapsed": false, "input": [], "language": "python", "metadata": {}, "outputs": [ { "html": [ "\n", "" ], "output_type": "pyout", "prompt_number": 1, "text": [ "" ] } ], "prompt_number": 1 } ], "metadata": {} } ] }