{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# M2.P2 Message Passing Interface (MPI)" ] }, { "attachments": { "0a8e7611-1dc0-482d-9e24-e88e538518ab.png": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHQAAAB0CAIAAADb+IFwAAAgAElEQVR4nOy9aZBk13Um9p1779tzr32v6q5e0A00upsACJgkAIISV5GSSM7QsuSZMSfssMNhhx3+YUeMI/TD4XDYCodlzdjaKFqERImrSIoQCC4SCRIkSIBoAI2lq2vprq6ta8nK/eXb7uIfr6rQIAmKlAhSETM3KgoZmV2Z750899zvfOc7B6SUwj+ZZQiKGABjTJIkQghbCAIAMAAAHfy7W/6GACCOYyJyHAdAkiRhGFYqlZ/jhf/oRf/UjCtBILrVoJ1Od3d3d3Nt3RgDYwAIxl3XDYLA87yZuWnGGGPMGKO1NsYwxjjnWutf4I3kS/yiL+AHVxonrufmZu2F/Xq9vrW+sbu7K7NMKaUymXuDbdsFz7c9t9fvDg0NjYyMCCGMMVmWWZb1i72Fo/VPznPjVLqODaDRbC0vL2+tb4RhCMBzXSIiA6VUlmVKKWZgGDGG8cmJ+fn5sbExIYTWOssyrXUeIn6x65+W55KB79gA9veb169du7680m61HMcpl8tR2BeHy2Jca01EhtH+/t7W1pYxRko5MTFh27YQot/v/wfj/ujVanUWFxc3NzdllgVB4LquZVkp5wCUUrnPMsaEEMwSpUo5TdOtrS0ppdZ6YmLCdd1isfgfYu6PWO1W59rKysb6ehzHrutyzqFNGidZlnHO84MrVeoompWrJWNMHMc7Ozu5/05PT3ue94u9i3z9kzPu5cuX6/V6HMeWZQkhpJRZkhpjOOee5xUKBdu2tdZRFEVRlKZpDsIsy0rTdHd3VwhRLBb/fTduHMe+7wNIkiSPj51OZ3FxcfXa9SAIhBBpmtq2TUSpzADcf//9udVyP200GtevX99fW6vVKmmaMsYKhUIeH/I3n5ubIyKlVBiGruvatp0/nz/4+axfmHF939daHx3r/X5/ZWXl2rVrjuNIKYkoCAIpZbPZLBaLZ8+e9X3f8/1iwTdAsVRwPU9pzYVotxr54QaAc54kSb1eV0q5rlutVj3PKxaLUsr8Q3+elsUvNixorTnnAMIwXFpaWl5ebjabxWJZCCGIlFL9KOJCTE1P337HHQCIkwHycyooBhPTU0GpePm5S2ma5vA2jxj9fj+KIs/zJicnp6enOeeWZcVxnD/4eR50vzDjHkWDXq+3uLi4vLzc7/dLpVIcJ7ZtK6U63a7ruidPnjx9+jRxAqCBKE211pZlcc4LhaBQCOq72zvE9vf30zTlnOdnoJTy5s2bSinO+dTUFBHl7/lzvsdfmHFzy4ZhuLi4uLi42Ov1PM/zPC/NVC8MlVK2587MzZ254/ZSsaCAXq+XaZX/oWEkYZSURqr5+XnHsqWUrVYriiLHcWzbdl230+nk9gUwOTmZp8g55Pi53eMvMiz0+/08GvR6vRzM5tu53mx4nnfixIn5+flisWCARGaFQuFWusYAXFiSsaLr0vS0lNIY02q1+v1+fnwxxuI4vnnzZm7W8fHxHMb9PG/wF2bcTqezsrKSRwPP8yzLyvPaTOpKpXL8+PHjJ+aDIDAAAVprBVPf32+328SZ53leIaj6RcG4Bsrl8vHjxznn169fb7VaWZbl3E3uqpubm7lNR0dH86D8c7vHn4dxDYF+yGMWFxevXbvWbDZLpZLnef04klnmed7Nm5sX3nDXxfPniIt2GDmeJxjCzPyvv/M7VxavLly9miTJ5OTkPffc89DbHrzrzgvC6FrBD0ql6Zm5XhilaSrTjIEMyPW8iMXNZnN9YysoV8oDg74Q+FEX8zqt1524SbLYdlyCiaLI8wIC9fvx8vLypUuXhBCWZTHGDFSSZTJNVZbeNn/s/PnzhepAO4zdwJfAV5949v/43d977oWXNRgTXKksDUNmiwt33nHPG97wv/2b/y5J4Aq4HAx4/plnry1ezbLEd739ZmN0fHJl9YZfKr/tne/oR1GWpnecnIdSjLFWq1WtVnM0/TrhX/7bv/3bP/M3PVqGYAmhIKXStu0wsDiJl5aWl5eXc4TLOc+yrN8PtTHlSmVibPSBtz7Y63RSqYJSSQJfe+LyH/7pX373uReZWzTCE37R8kuSWTpRzTDeqzeY7Y2MjlVLTi8ycZQcn5sql8uba2taayWVsB1tMDUzW65Wl1dvRP3+7NSUljLnJy3LSpIk/4Jfj9t/Xd701mVglDIOdzh4EsdLVxeXl5b263XXcSwhYFSc9JMkcW17amLi7B3nQNwt14JKTQLPLWz8wf/3p4995WtxJ840TxSlinHHK5RrrFCOo3Tlxvrv/F+/96W//buuhOORcF0FDI2M3nHhorAtL/BbrZYl2OzMVBrH25sbjKjRaDQaDQD8kAnK+fXX495f35hLBlEcuZ4PIE7ipcXFpaWlbjcs+IFgPE3SKAq1UqVSaWJiYmpqanBoeHN3f3B4IAW+/czV3/2DP/76t58mr1QcHIpShUym0lBihLDtoGy8wCKzu3Xjj//041LqD//WrxdstGOd9sJTZ87kyGz7xRdLlXK5UNzYuUlaH5+e2Vxbl1lSKpWUUo7jWJZ1WPf42a/X/UDL42wSx0uLi8uLS51223GcwHO0yqIoTJKkUCrOzM3Ozc0NDA0ZoDQ8kAAvLG3/uz95+NEvfx22X5uc5I4vk0xLpaVKlE6kBMi2PGHzytyJlavL//cf/Akx9p/91q8WXOa5lX6iZo/PR/3e2tpapVQWnHqtZtl1x4eHFl54Lsuy2dlZrXXOZ75+9/66G5dARz6bW9YWIooiYwwDSqXSxNTksWPHhkdGAfSVMZy+eWnl3/3+Hz3+5PcQlL1SrZeotNuCZUMIWBxKIQNA0rBIkVHwJmf2u53f+6OPtrq9//6/+s2AkBmqDg0P6KETp3ZqlbKWqtNoFX1PZ2m/38/RWKPRUEoNDw+/fmWh1924OTZYXlrpdsPcskZpo3SUxJVabWpmdmp6emB4yACZQZyZZ15a/dgnPvPVb35Ha6qOT0omom4MDUgDaHAOMNgeccCYxEhE6cDYiOsXdtZv/Pmn/sqyrN/49ffNDLoaEJyOz5+sloKtra1epzU/P58lEbSxbdu27e3t7fX19RMnTuT58esRdl934y4vLy8vL+/v7wdBELhOHMdaKtf32t1OoVSemp4eHBnSQGaQZFox9t/+D/9TI0ytoETCjTW0gVMqW7aXKq20VmkKpcCIuNBZgjgtjE7u794EUByfbkfdj/3lp20u/psPvz/NwC0UyxXH5Z1OJ47jWrXMOQHIS0H1en1nZ8dxnGKx6Pv+60EB/8ygWJqmeZaVJEnOwxpjlpaWnrn0bJqmtmXZtm0LzhiL4mh/f//snefvve9NpWqp3c+4xRXhmRev/pv/5X9fWt/JmOCWB2FpYpq4NEikVEobrUEEzkEwxjBGxrbTdgfFsuMHYDDa9MKw3Wzu7Xfecs8d3b4JPEaAYGxudqYQ+Fubm89dfvGhtz1ERNvb21LKOI673W7OTxpj0jRN05SIhBBEdERm/sPWz8xzcx7WGJNT4MaYhYWFq1ev5loNS4g0TZvtNicqlstjU9N3nDsvAWPg+lYGXF2rf+Zvvvz4d7+PQk2SMGAaTBPTP3xrpF8lCilWAZVEMci4wlGm/9LStXa7fe/d95ycHS/5XrsTSm3Gp2Zg9OXnXzTG9Pt9KaWUMpc3NJvN1dVVx3EqlUruv/mrQoh/pP7hZxZockHG0eGwuLi4sLCwtbWVs6iMIcuSMAwN0fjk1PkLF0rVIjm2IaTAizf2f/9PP/6pLz7W6YaSuIJQxDQjTTgkcDVIgzSgYcAMGDQZAIxzC2QDAuDc8bxKLQO7trH5P/7Pv/34d5+KAeG4wvVAAGFqZnp4eBhAfqwdlYq3trauXr26u7t7YBTG1C1lun/w+pl5rlIqzyCVUktLS1evXu10OnnJK03TNImyLCtVyhMTE1PT09XBWjdWtstTYHGr90cP/8XHP/sFFScjt93e6iW52wIAmCFGhgwh1y3lfkykASKAiFSSCT8gIbKo24+ywLVKA8NcJS9/4/Fn33Lfm+65Z2bQC5xSlMm0H86fui3KZKVS2d7eVkrlPG++4dbW1jjnjuMMDAwIIXKK5x9JUf7MjJtbNkmShYWF5eXlbrdr27bnecRZq9vr93qVSuXY/PH5+flydUABjsszYGG9+UcP/8WnvviY6qfOxDQPqqpf12AAA5gBAAKBwGAYDswKMvzgFcqr7IzDyjSZqB8aBxYjmWFoaGhy8ub+/u52dOfpE4ElIKw4S06ePAlgY2MjSZLcN3OKo16vb29vM8aSJJmYmMhjQk7A/4Nt8rPEH1rrq1evLiwsNJvNvOiSyizLMgCFQmF8fPz48eO1gUHGKIykAq6u1f/wox/7+Gc+F3d61ZNnvOrg1uqGJKFJaGKayBDlMeDH0ViMpf0kTSU5Loolzy9oRt1WD0Hx4t33tsPw//y93/vzT3xaA17gJ5myHYcxll+eUiqKorzC5jhOkiQ7Ozubm5v7+/v5ewvxj3K+n5nnaq0XFhaWlpbCMPQ8z3EcrbVUMk3TUqk0OT4+PT1dqdUAKMAQXly88fG/+utHHvtylmb+xJQSVqeXgDiYODyvGADQ4defP2k4AHN4ypFhjHMdJQrCcR2yBLeMShIQBsYnRsZGvvfNlx/53BdsmH/+gV+teLYf+HESW1yMjIxEUbS9vd1ut7XWSqmcysnjbx4QhoeH/5H8709t3CMy9OgO8wdXrlxZWlrabzaKxWIQBEkUSyldx2k19icnx+fnjw0Mjhigl5qMiFzxhw9/8tGvf2u/EU6cPUe2v3FlGUQj5+7c2dr5wY80DKQBBhz+zp8xzBCEECnLiDGjddLtxToCS6Gy97znPY6Hyy8toRv7lbIk1s3gcESJGii5taEhYTlCCGjT73W40SZLmbClNs1WJ4pTbch2vKGhgX+wZfHT8rlkkESRGwQA0iSxXcsAnX5vaWlpdXU1juMkzg6qhERhLwp7rbsu3H777bcHpWqrl3iFIAEuLTX++OFPfPlb386YLYkrEhocADNMEyQxQz/woYTckGRy4j1/Jv+ilQGUIkaB6ya9yGKZ7UStly599ptPVgqFD/+LD9949nuf+uLDY8ODf/bRz56eP/EbH3qvxbRFcDlTcXJt4eXVpasqzaQ2ClzbruFWmMQA5mYmT52cHx8ZJiBJkhwLHXFpPwn+/aljrusFkMZk2nYcgMVJvLKysryy0u31iLGgWGCMdTqddrvtuu7c3Nztt9+uYPpx4hWCFFjcij79yFe/9uQzCQUJc1PyJDkGAkbg4NSiH/rBwW9zy28cSniNATOMUyYTW3CtZL/bKp84NjpcaLf7UV+Vpo4PjIxu7O59/Zvf+9RnHu32U9tiQjAJCMs5duzY/NwxzkBGa5UppYkJYXsS2K43ri4udzq9NE0dx3FdlzGWK9J+wlPupzUug4HWIMEApKlcWlpZWVxp1JtpnAlmeY4rGMuShIHGx8bOnb/glwbIKXI3SIHlrd4fP/znn/7iI7v1hmZMgxliGmRAmkERNMtNd4AWYHJzMxwkFEcogplD/AAYMHCbp2ls2yJN47TTevOb3uQ6eO6ZS7ur68dmj41PTrQa0dLS2ssLy1HYb7b7sQIIJOCVy2Mz0+PTM7bngkgrSdCu69qW2+v1r9/YuHLlSq5CA0BEWZYdSUz+3vXTxVxDSGLpegJAmqjFpeXlpZUwDMvFolLKKJVEEYBapTo8PHzs2LGJ8fFGGHmBFwOr+/Ef/sUn/uxzX4hiTJw4u9fuApzAiGAIhijf9Ld+37e6KH7UY80Ao8EZYzAy5cJoGYPYA2+5X6X4/lNPY3d3bu54kWO/0QPzpmfmHcf77ne/DWa98a67R6ouAKdYOXP+TgXD6/v73TjLEtuyfc+LiWSarK6upUlERPn5lmOMNE1/EiDxUx9ori8AJAkWF5eWlhY73Y7j2J7n5QlYmspisTgzMzc3Nzc0NKQAN/BiYGmn9yd//olPPfJYlOpgfJo833RjGG6I5eHTkD5wxFsMmBfCb60n3hqOyRwkEhDCGAXBYDS0GhkfvXjn+V5Xb66vo1w+e/u5UOPlK9dB1rHjpwaHnKefeW75+lqc4v4H3lwrkOu5Ftzjp0/a6+vpjZudME6ikNmeZVmCUdRrbW9vG2OUUpOTk5xzznkURa+LcQ2QZVhcWlpcXOx02o5rW5ZIksQYlQOaWq02OTU1OjYCoJtBWbi6nfzRw5/4+Of+Okvk0Onbibkb19dYoXyUGuhXIq2GeXU/CaDpRwPdPDkGF8KysyT0PC9NYwY6f+7C+Kj75Uef2dhYG5ibueOOO/freOnlFRg+PjXFgN165/Enn1F2MbOcB++/Z9hFqk1lcMhwlirorZvtTpylioRlMXJdN0minZ0dy7J83x8aGsJPrDn7KcMCkEosLV9fWlrq9Lq2bVuWMErBqH4YFgqFsbGJqZnpwZFhDSiDVOOFpdYn/+YrX/jKt7IE1tisstxeLwO3DVkEdpTOGkYgAPxVx8AR2js07g9UxQ0BEJbwom7PLwe93V0u6O6Ld3vAE998fH9z49xd905NVnZ2e+sb23ahODV3TAKRsduReeryYiL+LmXinQ9cdI2xbFGq1o4dJ8752vrNdrtnUmU4Fw4pSXEc5/lbmqY5v/6T4N+f2nNXVlaWr63sNfYCz/U8J0kigg6CoL6/Pzk1dfzUyZGRUQ0kBlKDHHzk4U989clLzUZv8MRZ8vy91TVAjJ66bXt95xUPzb31yHN/aB0pH37Qiw0DY4wJZMpxvEaaOsSOzR4DsLmxgX5YrZa9AI1Gq9nuFAsDtaHBeh+RFrww0EjxjaefN7YzOT31xpNDzTSt2qJcrU1pHUWJSrM4TnPSMQcJrVYrTVOtteu6AwM/Ef59TeNmWeY4jjG53sAjoiRJlleuP3v5hSRNuQAxRaQcl3fb4W59587zF8+dvxgUS81+7PquISzcuPnwp//68e8+o7gfjE53wjSNFSoDMGJ7u84550TcGACMkWSkGQCOV8PHV0x55L23GJfATKZlZiy/nEbStNtvfPtDF87PfOlvX9zZ2UHg3nHuTDHAM888UygUlE6Gx4Y7Ea5v7YnyYFCt1et7Tzz3cvz/fuRD733o19/2xhhwgGKpcsftZ8aGhl964cWFhYXZ+ePctl3XlVLmOlel1PHjx8fHx/PmIcZYntExxnKL/f3GzbWuAHJ+Vmt948aNG2vXoyh0PNcWtsqSRrMOIAiCY0OD5y6cT7XJ4sz13QxY3Q0/9ehXHvnGEymVJQlNTBEB4hBmwYAAHNG1ZBiMPoi8t5jvyFXNYZD4Af2O7ftJ2B8oltKohSQ5c+ZM1cNLLy70wtAu+sfmZ6IUN3d3AJQqBS9wGiFScphX4H4JfhrJ6Nmr1zjpqfGRUxPDkyWfhG1YFgTe9MS4kVkrivI8MKfKckXa9evX8/pF/kz+EoAoim4Nx6+Jc/OCfq5FNMYsLy8vLi7u7OwA2raZ69nGqCiKtNajo6N3XLgYFCpk+8K1EuBaS3/kk5//y0e+dmNrX3JbcaFJgCwQBziIQCzvxQERiAwdLBABHPSqH8O4YdwQOyRxXvkBwIkhiQtFP+z2ELj33Hu3AZ5//oUwimuDtQsXzzfaZu3GRpapkeHawAhWVjeavX6ikBjiXoEK5V47evrFpYc/88UnnnupLqGIwfbcQnFicuz2O87Yjsj3ca5isSyr2+2urq6+/PLLrVYLQF5zOXCFVwfi1/RcY8xR2XlpaenKlSvNZpNxFvgOAUk/VEoVi8WRkZG5Y/MT49PNfur6dgKsdfCRv/jcxz/7aK/XHzp5Loy0OkoBcvc7xFOGjD50RgDMQB+9+MORl8jQEQv5ypJZDG6isKe6jbP33HX77QMvXsfaxrY0enhsdG7MvbRQr9f34ygaGRka9rFxcyvJUm14P04UKChVQ52B67/60jfSVBaK1XvvmB+0hRAlV/CgVJxtt/cbrWazmdcm8jaNOI4XFxeFEI7j+L6ft2wA+IFC8msa98iyuc+2223OueXYjud0Oq1+t+d73vT09LHjJ0bGpjTg+nYG3GjiI3/5+b/43Jd6PWWPzovScJbW85zqMBfP0SkMgzmgvPQhEU5EMEQ/EBnwQzj3KCwQtIAhm3bXV8DMh/7Z+ysWPvblr7baXWE5M7NzBljf3JBSI1YTY2MAOp226znEnMgoZEqDkfC4K0yy+fjTLxNz0sQ8cNeJAQHfD7RM50+c8re2pJSNRuNIkAkgSZLV1VUp5ezsbI7Pfli58+PQQpIkKysrKysrnU4nz6yzLJEJqTQTnNeqg9NTs+Pjk4zzSMFwrO7Ij37y8x//7CO9niyduANOcHN9i3yX8CNQi2YAYA58F4Qfz9q+1tKOw/tJgqR35g133P/AHZtdfPNb30qUchzn5G2nNbC4uGhbPpiYnZ4hYHtrLY57dslRhmfGADDaKGYHc7d1ttcfffwpJY1g7P4Lx4UFzngODOI4TtO01+vlwj3btoMgaDabi4uLAIIgKBQKeUHzVtrhNY0bx/Hy8vLS0lK32801P1JKpVTY7geBNzo8NjU1NTo6LriQgNK4utr41N987QuPfK3XTeyxOTjFTqLBHMO5MYbBAGAmP8qMPiRkzMF/NAHGECEv37Bbd/6Bt2qDQ589ytMYVJaEWRoeP3f6X/3L3xIuPvuJr9xYXwuKBSk7Z86eNcDLLy04juv65ZmZOQY09vfCXptcl1mBZQnOOTQUCe5wVhmVzfrjTz3PtTTxA285f2qk5BmgVqvlR8LGxkYURTklZllWzq9vbGwIIWZnZyuVyg/g39c07ubm5vr6eqvVyqN4GIZSSs9zol5ncmzk1Mn58YkZBcQaGWAsPPzpzz/y9e9sbTeHTtzOy8Pb69tgdnn+VPvmKnIi1rw6kJpbYyfTrzpbNV4p8hy5tiKDwyiN/NsSJk0ba7ZKHrzvfb/ylmNXNtNHHvlKHMnpqbGdnXBqYjYG1jZ2HD5UKPojQ4MasP0AQkhlwBTj3EiFVMZJhlarPDGOYrG9+NKjj3+3VCoMjY0GJY9LFARVa7VpqZMo3dm5mSSJ0rodxeVqzfcL+43GSy8vZFKfOXOmWAxuvUURhmHu0nEc4zDUXrly5YUXXsjPx3z0QX4mhu3WqZmpt953r1Wutdpdt1zMGK7tqH/7kY999dvfy4RfnRmIJJJWm5Wqmuz2XtO2CzBSa621NMYcHazEORERcYAZY7RSWipo5QbC6FRrrVWqMgmZQEloCQaoFFkKLUHgnNm27bFsGJ2P/dlHZk/MfGdp9//5w092uig4A7tre+Mjo6dH+Re+sSK1VW9uTU8PFwui3sXlpWvB8GSqSSqtMpXIEMUiGXJHxtpb27B47bZz/U7jz7/87eud7F994D2/dtdUV8EyGBoeLPulF194/trSYhTFXrEUdvuGkeMHcZIuXlvVjJ89c6YUuAT0+7GUUhRzQusWbLC6uppHA9/3XddVSsVxbIwpFotFb+D0ifk4TuFpv1yMgJc204c/88XPfO0J4xQUc7QWmjgMY8xwZogobbcBAw5ijHMSglucE5n8PZVSRsFoDZMT4Tq+tACh4bgI/FLBLwSe5/qOxbVKXFuUgqBWKQwO1oYGByqVSsXmb7/r/FDNu1wPH33sa9fXtsJIQUoo9fZ3vkMD19a3Ii0THZcG/doAX6+HGfGEhAY049AExmEIBKUMCkXHtjS3tR3ASp+/vv3Rz36pQG9/w9zkRE0oAIzPzhxjBktLSwCU1oa4xQW3KMmy9a0tKeWZk6cGB8q+7yplBIAoivJeUABra2tLS0t7e3t5W0EuTs6yzHXdsbGxyYmx6dm5KE6NY0fAtTY++dgTj33vhQ4vDo/MZoaUMjLTUkqdJVL1oSVUBihk0mgpZSZ1FmsJI7ktOIzLuG1x38knU7i+Zd35Sx/yLV7w/GIpKJVKpULgB45ji5PHpziBC1gMjAOAMTAarsCVTfzbj33mW89d7saQwhieuWXrg//yXTdCPLv0XExxrLrDY7Wig42NVWMMIABNIBAZMCJuCGmnB9fSoCjNhOWwgcFeP3ryqe8HcYt/6NcG7z1pAHA+MDZgOXY/TXb366RipTUR+a4dGdVuNHutlsOF4POVSolzEvm0kzxZ2N7eXlpaarfbhUIhz+q63W6uUxoaGpqZmZmcnFRgVuBHwOXl1sc+/+inH/tGJ0zLE3O7q9cABqMBmd83QQM6cBzLsgoFr1AsVAp+qeB7rnAFm5medAQPXLdSKlRK5XK5XCoUXEs4WjmMWZblOMxxYd1C+R/V0Y4OOsnw5ae6n/ziVx6/9DwveIWyH+3slgeKZ04dGyzhi996cXn1OgkOJUeGhjmwcWPDaGY4A3FzUDRiACfAcA7DslTCKNd1fL/ASYRx+K2nnpsYGQmKlZPTw4MeJKFQK548e5teWDD1ersXZlnmeZ7vemSQJcm15RXfczzvpONYgnOe+2y/3881Mnm3vVKq1+tFUZRXxY8dOzYyMsI4bzQ6Xrm0eGP7E5/8q0e+9HdRJAeGxwIeTZ0ccwQv+lYlcAYKfiXwAt92hDU7e8y2bd93S8WgWPQD3/Uc2AxxP7UtHlj8QP1xaLKjQy7X2qhDVidUUIDSSDJ0Q9Xphe1ut5dav/vRR5Y29rSolnx/d3srazfPXnjLv/7wfwrg+89c7nYSmweAP1Ib48D2xp7R3DAiYgwMlJ+azBC8ak1KqbJUa50p9PoJtHGCspTpl554utWLP/ieX7r/4nEhwDRqQwNTySyzrXRzox/GZODaduD42rL39+sbGxvFYjEfriEAKKVWVla2trbyHoFut1sul/Nu5YmJifn5+ZwnTpLE913BoZJosOT+8/f90tnzdxeq1c3tm3dduGhxBBZ8hxdt23OQD5NINRgDB8yhBTWggcC38WqzJhJRJn1PGKCv0O7KZrvT6vY6/ShO9eb2dj+RzW5vryDvYgcAACAASURBVL5/c3v35u7OfqPV7qMwdCYhZgu+d3NLri2XJyd++cG33HvKffy5/YUr17lxlSTXLQ4PjQPotCOtoQxjxIiJHPPlo3PyBJcxYVlcqyxNE2ZgCWdo+sT2i89/4SuPu649WCndfnwoYNDajEyOM0HK6JubW1EvijJpC0cI5jhOs9nc3d0dHBw8sGy32wUwODiYS3r6/f7AwIAxxrbtarU6PDycY2Ottec5gD4+Xv0X739XuTboeZYGDI4RkFPeDDCA0koarTWMZppzJg52dO6GEmjERhmSyqRSRv2s0+v3+3E/U9965tl+qsIwbLXbjXar2e70wijOUhBThjQIROCCcy4qo+Wa3Q+7ghHP4kCHU+fP/NqvvOtt95zmwBf/6m92Nhq2V2w2WsWgPDw6EQOxgjYMYESciHI5T45gZL8Px+GCK6MBCMthRErrCA4bm9H1zb978tJgtVArv+vkeCGMkornDg8Pcs4tLm6ubfU6Pa0yDV4sFNrdVrvdTtNU5NioVCodP358eHg4n3nyA3VjrXXeSe44VtTvK5VVy6VyuaKBrXpdGVMdqDmM53s5yy3IuALXHAxQQKjRaMmbe7s7u/W9RrPbj26sb0WZDPtJqxe1W2Gr3emGSSylFrZhREScMc6JMQbmMtdlIJNlWRpluUQXGpwzYjazfFtUfTE3M/n+X3n3r77jogI+9eiV71+63IlktWAnUX+wOlgYED1Acq3osNZBIDI6z12Y5kHg+Y5W6HdaMMb1fBiT9uN2P508cTqpVvYuP/mNb3/vof/o7hPjBUNIVOJya3RoBFIlYV8lqcoUGdi2nWVZHMdZlgkc9rXkAuCjAVJxHAdBkMPbfKBBTqa5fkEDEtBABpQHBw2ggC6ggP0EW7v9rZ3Gdr3ZaPejOL26sJBlWRyl/TiKkjjO0kxqZdCPEwky4IYsRoJbNV5xHEa9bouMMkYZqWSiyCQwkuks7raqtdKxsdHJidHx4aGx4YGh4cFKsXDh3G37u7uI0+Nzk0UXGfCJR77/6b/5asx5dWIwyiS6+1Onzk+OYHWvf3VjxfJ9TXZ+HmoCKJf6kTG6H4cA467DDJTRZGAFBVGobG7vuzq1h8ejOL1+Y/1NF2Y555xTIhNfOONj4zJV3Uan0Wvk5bVarba6unrnnXeKJEn4AZ6nvE02l/cUCoUjz80hsFIqlko49tLa3l6zkxraanTWd3ZbYRxL/cKVxVSbROo41Yk0qUJmyGgqFgpSyizLkiRLskxKlX95GtxoDa2gpNKUEQNxQFV8q+jwWnVwZKg2MTI8MjwwWCn6nvWme04JBhtgBG5gcVgCnEEA03PDHMi/8m9fXn/i2edfWN1AMKCIhUmI0dq5c6cJWLq2eGN9jdVO59mzJhjS+qAdHprytFAbggZjBoaggajZYp5bciud7m4YdsNuV0r4QghQL+rCUgW3UC6XD7MEwzkzhLzPWOQ9yACMMXk0yJ20Xq/n6fORTFxrrcn+/tLOJ7/w1e89+7xx/IQ5e50wTJXhljEHeZhRSmt9KCpg9StXQQwWgy3IsT1XuK6wOWdkAserlEojQ0NjI6MjIyOD1VrJE8dGqz4zjuP4rlXw4VkHBLo2IEAQ+C2gwgDpoVkz4Mvfu/H5v3v86y8t9xK4g6UwTsx+c2p28sE33usA1xeuk+YEyxAHAKaPxsO9Uu7IWU0ikxNL4AjbTjHwHd7TRiUZtBEEj4gAo3WuVwqCwLYFYwdFwZy+McaI3LLdbndvby8MQ611kiT9fh/AyMjI9PR07sv50BnbRgZncXX7qaeeR1AORiYTYmS5vhu093dzTsZm8BhZwliMc06j912wbbsQ+OVyMFApVspBOXBdR1y444xFZAnmWcK14FiwLbiA/2oCXwJKGWOMLQ6qmerQrARIYDM0/Yy2b3aeevalrz/1/RdubLTItiaPCzuIOyEyedeZ287P+Zt1c/XKjVptsgGRE0T6oCAC3EraGaYBZpghotw/Al8Y2Wu20n44US1OjAz7/ODcDlzPdVwCut1unKVgxBjLZCa1EkIwxgSANE3zkTG7u7v5aJQ4jj3Py4OD7/tH8/wAnJqvHD9+vDY8GoN8i1zGuUMFW49PDbkWqwbOYNkfqpYHq5VKseC5zqnbTgshHEsEnvA9uAI2QAC/BVoAMEqbVEPLRKe5xFNw50DFxsmAIkABzRStdtrqha1Ot95otvr9y4srrU66txPu7Hb2o1jaBadYdXy/s73HUnl8dvZ9D761ADz29OXnX7iRuWVwl2Ahp4RIg8yBGu3QuDCU1zgIjBmUq163sdut3yzAnD15/PSxeQswGaKkXyz4ufVWV1fb3Q4JAkeSpLlm3bZtobU+qg7lcrNcDNzpdLIsywng8fFx13WzLMskAoEH7j4ndJwZuEHRCfyg4Lm2ffeFczZHYFPBFUUPDh3sXH1oxCN/zJ1ub39PCOF5nm+7AIgzcGYgFFwNxEA7RjuMO72oF2eR1N968qlGL9zea+82ms1uGCZpmqkMGjZLE8Ok77mlyvAwLN6Ko872LlZXB6dH3/mmux9845ACLr300k4qmSMY8TwiHLit0YYOlZN5XDgSUwEMWgCy1xaQt58+8cC9982MVRmQRClnlN/J6urq+vp6v9+3LMvkwM6YIAiCIDgICwMDA+VyeXd3NxeqE9Hc3Fwu2sl9OUcUNjSBPXRh7r5TY0xwblulIL+IVyRzAFINLSENAO0wzUgB0Iak0anSmTLKmOrAUJ597QPNtmm0O91O2E/1TjNq95N6s7l1c2f95vbNnd3d/VY/iouDwxI800ZqAILsglv0LJd3+zc5h2OKglydpSqVIom8rFeZGvrlN97xa2+/zyd85tvPPrl0WU8MKSoxaXHNKS+IUA4XdI7Lcm8FWK5WIRBR1mvtW0bdftvJD7znHb/0pjOVAiBhceZ6dtLvr66trVy/1u50QMQ4N0BOyJRKpWIxEK1Wq1Kp2LY9NTWVu3C5XBZCVKvVKIo458ViEYcaSte1ms12pVgc9H0A/TAUCeOcut2O7wf5gFZFAswiK6eKGQMDhAIUkAAhQ2SQKDz2zZf6qWp2w716a2t3f2d7d6/R7ETZflcpbjPBBbe4JYQoW8O1MlG/H1uOV3JdMJZmWZYpJY3uJw6RoDTr7XUbXYQhc8X42PDEePUtb7j41jffe8fJUsvgc1/52sryNX7H/SpiWh/AMOQlZ4CBAVqDIQdnhjGASJNhls6Sxuagb954Zu7dD52ZKEAAWYLAEyA0293F5ZXdvb28qcawV8poeVfQa+pzj8pBOU5QSuWVCM8LGGNktMoSmSWMc9vzQEyCKSDNj6DDHHevh3qz3en3tuqtje3m2k5ra7/T6sX9RK5t3MyLDkRkDOUfJI0oDIwpxkgbraWWSsvUyAxacQInQ1rJJIn6PZ0lAASXs9PlSoFP1AbmRsdmxobGB2sD5aLv2rXB0VpVxMCffPG7f/boV1e7ifYHoUQeqoRizDCuc/0kFEFqBUcAgMwYI5co6vbM3o0Lo+y/+NC73/32Xy4HYBoOP9imjz32t1JlcdjPXdDzPGMQRf12q3Hx4vlz584Vi8XXrER0u92cwcnxbx6hDSCllkrZFhfcEa4FMAUmgfX9sB6m67ut1Zv7O61uO4zb3X437CwuL8ZGxYoSLTLjKmZDuIaL2vgJrbXJUi0zozSgOWMg3mzWISWSGDKDVmCGC+FxJP1urVSYGBmcHJscHRwYrFWrlXLBF7PTA2WfD5aDYZdZAAf4YYITAd+4dOPJZ17aaSZkFRl5mhTMgeu+Cn+BOYGftNswKSt4Numk02YyHRnwf/O9b3rbG28fLSBKpOMcmGt1bStTstVqJ1HseZ53KBnJsvS2226bmJgIggA/psyTR4OjdVBBIDICCmhHYaZVEATS0IuLK0+9sPDVJ55uZ6yVUSejZl92IwlNsK1SqaaMVgyZIQ1BzOLcYgz1mxuQKdIEMgUMCdiCO5wHSWoJ5gVWqRgMVcsjA5XBSqXk2WdOHi94Tq0cVItBpcCLPqzDKM8P4YfKTKqU1sggbB/PLOx/4a+/dunZBcm8QlBKmRspCZ3l5KU+BCsH3W7SAES2VxJW1NxVzfrxmfGHLlz4tV9511SFAfAOLbtybe3pp5/mxJIoJiLP8wjo9/vGmFqtdvbs2Vqtku/7H1f9zYWo+QrDsNVq9fqh7XsnTp8qekE3DrUhRlQbGRPLmws3thoxpaLgDU2URocsw1JNQrAk7BidSSmVSqGUSRNjQFDotMGNY4tiyakEXrXolwPfs/idp+fLrletlYcGaiMDtWqFlRzYhwo9BnAgU0bFGbTRjAxRqmGUJuKWLSxXAGDAd15uff6rTzz5/GI3IX94kNlBZrjFWKY0gRs6jLsgAwJIddpBsVDyrKRTl82dyUrw7je94UPvuX/w1dO5r9/YuHTpUqPRKPhBPquLiHrdbi4wPXFivlarOY6VW+81jbu0tNRoNPJhaJZlGWO63W6n01EqCxz77G1nKm6QGmOA0Yr/vvfc30fhO5eXvvfi8l6zxyKeMdvkfeadPUCCGXDjcO4K5nAIZh769V8uuFatVBgZqAxXywPlQtF3fWGmqjy/pqMh5Tmd1ml1iBmLkSWYZVm+Q4y4gsijfE7+psCuxG4DzQi//8lHXl7dqCvLGRggv9jPdGK04zgqE4B5BXcBMEQwsLjLKW7Ue7s3ZgcK73/ngx94z/0nRhCFmRNY+cXsbO+98Nxz9fpetVrRSrmew2B63W6WZaOjI6dPnpybm5JK52UzY8xrGjfv5d3f38/b9XLFmOM4BOvqCy/Fnd6xE/NBodRPdcocz6ffes9FP6htbu7cXLsCXWeDQ5XKoOc6dnWk4ovhgcrM2NDc5Oj02PDIQLXkM8HhWPBdBIf+mP8IgIBUaqnUQSxiRESVSim/sPyoTHJiE6inaCfYb/RubO0t31hfWN24vrG72wl7ickgRGXYuIWeRBrH4AKHE4sN6OB7Q/7NmFolyNrNcHdtxMX7Hnzjhz/w0HgFBBh2kAivXL+xeGUhny1ZLpfjKJJS9sK+Ump0dPS2226bnJzMK6daKwAHGdqPXPkUWs75zZs388IwY8x3PdcW9d29brOh0+Tc+YvVQtEQehK2wIPnZil+4K4T0ynYyPjU+ORUpeCdnAlsBp/DE/AtOHjlpvThrRmFLJFplkiZFQoFTSDOuW0xOgAeCugBGdCM0Oym7W6v0Wo3251OmPztd54JU90Oo3YYh6lMwTR3DPclhxUUINwkU0pJuA6BxXHEiEyOcImB8pYrzYxUaRdJ88RE+V33XfyP3/3W6QpsIA6TcuAAWLl246UXXmi327Zt25aVpSlpE4d9Y8zk5OTpk6cmJycti6TUnLPcbRljPw6K5RPlNjY2dnZ2Op1O/i+ZNoWCz0AgfWxm9vY7zzlBeb/ZsYKScCCBdooohevB4+CA1hAGds65GEXQpJUmFieZIgFihjgRMUaCHTSc5P0cCoiBToRWH2GqllfXtvdb19Y2V7du7u63u2HYj9NYGqdQU+CGmCKmiCtiGsIwDiLH93UmszCEsMrlssqy3n6deY4mgDhIHHa1ZbaK0931Idf86gN3/df/yftPDJDuK93vlmsVw7C6cfM73/lOPiqLMdYPQwCCWM4uXrhwYf7YDAApoY1kDEddr69pXKVUrh9pNBpbW1uNRkNrLWyrWhkISsVSwXUdyz6UTkqtJUiCa0N59whnTAAcMKkkraAljOKccSFATINSI0iQOQTFOS6SwHoPOy21tbm9sb2zXW9u1+tbe639bq8bS0kiY0KDKxKKSFOu+hc4+NB8N7yqjS1vb6XDx4aUQsxtS8HRmhG3LEZpbw/7m56V/OcffOeH3//QtANKpZXEgVcA8LePfz1SWb8X9vv9vMUOSvf7/Xq9fvHixTvPnSsWi0bpfACflJJbr/BOrxkW8hGejLFqteo4Tt5rbDteLJXrupYAtIQ50HlorQQXgh0238Eok2qZKqWNFo7jMe7g8MzJt3numI0u6q1ot9Haqzfq+81mlD6zsNqJZS8Mk0xKw1Jj+okKM1koD0iWm5VpcNAhXfEqGc+hPCd/knKiNhfvMACGtBcEYacJGYugzI1KWm1Kw2rF+833vvuX7j0z5EAZ5SENCi7iZP3GepYlzXY77kee5+UES9QL0zQ9e/bs1NRUqVQSgikQ5wwEIYS5hWP7ceLnowc5cU5EBrAPmRfGLBhGZAjgRHGSMUsIIQikVWaUZIBmFrlBeGjNZor1nX4vUsy2n770fDvs7zXbu41ms9VrdsNurx9mhvxqIo3Wmrhluw63HbjMNogNaZDOA+VBWzDBGLBc6a9hDqxJrwilmMkx7MGfEKCZlDCGO7bPWdioo7E3NT14/xvO/eb77ht28vCVuLYFmM31zUvPPZsJivshEbmuzaCjsGtgBgarF86fq9VqgrODTwcBZIzCLRWyH2fcvHT2ildorTUyqR3H4QxSZmSMEHn+ynw3AJExUFpqSQSyuKWE+/zqzSsbu2s7+72MNWOzttPc2K13+0kYxanMkixTBsQdJhz4RcmF61eE0koZAIZzyUhrnalDHRTlrcCH18QMoF6Rob2i3s0ZWwIBefQ4uGee9tp+oViw3ajZUvs3h4rOu+49/8F3v3nUgUqV5qnHbQazdm316sJiq9sjhzuOk+PZbqdziGdPDA0NMcZgoJXSWjPGjNZZllnOK8ryH5dEHJ16R5peLY1NWoicLj7UaxAEoDUMgTEILogJwIHKssxs3Nz75vee+cq3n9mNjD88Q151P0xNJ+S1mhaSmCLBmeUa4WomJChLFYyBAYzJtIIBtAYUWYcXTeooqmlow/Rhm8pB45U5JGQP5L90iJgJMBzc5ST6jf3ezfXJsv/+d7z5g+948+lRRH1TFChyj6BvbK5ffvGl5m7L8lxFynFdMqbbbkspR0dHbzt1enZ2Np8xlZvIEgKM0Q8NxHlN4zLGfniCDpERDj/wkSxJk0RrKJhUm8WlFW5ZhUJhcHBwoFoiAohnaXb3G85tdOMXru9sXdtuxShVB0olr1/LsqQPrWCkIVKcQxOUgjTw/bx9L3c2YzRpY4zRKiMDhsONDw0wBqMOgG++PYn0AWdokLdPvMpbAO0XKnG7mW1vl13+3re+8V9/4K0zFRgFV5BncwDrGzvPPftCo96ybMEECeGkaRr3+3mDQk4d4LCtIR81dLAtci80ryhcf5zn5pRN3rNyJFA0SqssazX2t3e2ms1mkqbSQCrT7fUlyPf9yanxE3OzI8M1xkgwWbGdh+67KyFXfOPSM6utZrMJD2CccwcsZQacc7IscDsjSxvSSUiMsbwYJZVRGYyBMUS5gIPjQIdOecRn4DAEIwCCEQwM5pZmNpNXHjUo/58mkYxU1k/Hx8fffd/ZD73zTbMVuEAWJUHgQGO33nv5xcXdnabt2LZnZVlmaYrC0BgzMTFx+vTpqakpS1hSyrz69QOumk8m+Ps9N59AkvtvXm47OOIMtG032o1eHDVbrShNDJghVq5V2+1ut9fe22ED5UKx4HqeyxnCXmuyWHnvg+d6Kfb7Ty/X+6R97haUMlKSzFSWSaQKlkK+wZOeEUxZDuVyLiIQsXwPHQ650IRDgTSDMQeif8MAdhBnD3VlLO/ONCbXLQkjw80bFY89dPENH/7gA6er0AnCfrtaLsJgZ2v3+89d3trdsxzb9WxD2hD6/X6apgMDAydOnDg2dwxAPhOHCYGDc0jnck1OxC3rVUDwp51BdMTzhmH36sLClYWFKEoKhUIqszxFjqLItZ3Z2eljx47VqmUiGWYKdrEJ/PV31z/62a+8dGPfG5rgbiFMc3GqZrZrO0IrpCoUQhvS2tjmIBnOLaV5flhpcSDxz8usZEDpEX6BEThgDDSSSPiuzYXKNJMQRsgoM+2tUn/xv/yNd/+z9757sgqWgaWJ7wjG6flLl27u7uzWWwrGcX0SXOnMSLW9sf6GCxfPnz9fKpVwy2CWH+6gpFsCQr5+6qFt+V7ItUAGZAyklEmSWMISnMMYmWVSSiJmWTbn5FrEoIgJQbxYKnu+2wt7q2trxpACyPbIsrVSMs1AVPAdJWMAZMQBEUY40ECRPqADiPI5Qjrnsw6k/+zwJYA0mLaKBdntZP3/v7kra47rOs7dfZa7zL5gX2ZAgqRAWTJN2+XYcuIldiTZKjupRLETO5W48pCflx+QvOUplXKJoiVSXCQCBIGZwcwAGGBm7nK2PJwBLDtUSqII0v3E4gPB2+jbt0+fb5kIKSNi+eTUZmlD5L/56c133ni1vVh1aV4IeCQ5Au7tP/n40SeT6dQ6TYw5tJnKs1QZrdrr6+1Wa25uzt94+Vf56drn+Md/8YXpqb7d+BHCq4Azxra3t8/phB7P4zVqJ+OTrWubYRxzQA2wXoaf//C6BTeanDzsHWmnmZAyKqYk3OnYoAWIAUMEAJCIiI78utUhIqBD9OgKcM6h8zJj5JAc88B/h75zcAAAw8EKIBEiM+Pj7KTfWpj79vWtX77743YZyIHls8v6vb2927c/GBweEmfEJRGleabSlDERR/GNGzea9UYYhp9O6OcUln8WucHza3bOuT9faK0nk0mWZedMOKXUZDKZTKaFuFwolKUI8iwhxiPESqPZaDYfdzppbiaZRSZFEGnkYCFXhmi2IMMZ9cQ5jzbyVYpsdouIAGgAgDlCy8Cn3jdfIABmR4mMy9VC0UxGk8HjhSL91V987Rc/+fNXmoTWgXWhYASwv9e7detWfzAAJOTkyzPPcsH5wvz8pXbryuXNPyBFEsHZ6/v8k3tu7eTrFBHDMAzD0O/UkyTxSfe+OMa48TgnxktxIJlFnTAOpVCsrdUyKyZ5/qTbN5MMwmJcKFlkNs1QSkAkYAhEAAheTMQTLQUgB+KACOT8UYw5gZ4xPLvKZYACgEGqC1EMk9PTzs5SCX/2l9/8+7e+/fqi1LkpcAoICaDXHb5/69b+ficIIxGGDkFpnWcZETVqlauXL1+7eoU8wNC533/SP3dyv7g60zlZzzmllJQyCAIP9I2i6NGjRx62MzvVgNh9sq9ygzrdvLwSxgKccuhqEP3s+6+zID4cpXe3+3p8REEYSjGJQoOWZuQeC85LDPogwLORazbVWr80cL4nzJjEBA7JgSyW1eQ07e/XA3zr2zf++Wc/ulwDCZBDziEigEePHj/46P5wOAyCUIaRQ8iSPMvyQPJGrbK2sry0OC8Ey7LMOZzNs/8nCf9/fOHK9YQ0jx3xR2T/K/XYHG+m5cEPjDEA0tol02meTkPBq8UQGDpjlHNlIWpzdRkUp9Ps5PRUGU3IkJFB4zzmCp2/4gZ0gGSRA/g9ob88Q+Y8sksaYs7j+BABGQIIZwVoOx6ulNk7f3Hzl2+/8doiL4J1+qQgYgTs9nrv37o9HAyFEFyGzoF1kOQZQ1xYmL+yebm1vlYqFuHsE+nnUX+q+vwa58/Scz2b3QP0PBseAPz1Muc8SZI0TWewqDyba85Pk+np6MhaJaWIg1AKKbm0wCoSFldqk1O1v79/PDp2zgLDM/7fTKcJZohEvzjkfsEI4BAcOQeIBtnZ2hs8go45I1yeHneLMP3ezav/8nffu7HMuQI3HhbjiIAOj49/+z+/7XY7cRhLKXNlldKAoLQqRHG7vf7KlStRJAEwm05FGBKx8/Oqn1z94H8hyfU169uQf1/8zwaAKIr8tOBJgULINE2k5IyL02kyOjl1yErFspSBQDIWyhy+8spCtbayu7s96PeiQhwVSrm2aB0XATHKcm2RZBhZAEQvOIQcHEPDwQKBcQhRgEJ4upogS/lYjw8wGfz65z/8t3/87tUKmInlyaRcKICDx7t7tz/8sLPfDbiIothaax1yzvf2n7TW17759W9curwRCO63l5xz585onmdH1s8cxZ5Lcp8a5yOal+BCxOl0ejQ6KpUKjtAA5MYoDcZSKIJiFHNEZg0jJhAKlUIxLo+npzuPd4GHzjICZi1aQOQSGTN2pj0FAOgsgiJnAHODTlQb5nQCkzHJIBJoJ0eUjioy/9s33/jhN7c250OWQwlsJQrB4LhzcG97e3A8UnkeyBCRsixTWW7BLszPt9bXlleWilGAAGAsWOOMAWJPmWA/Xzw3LUffiz3ryruYZll2Mh5Zq5U1gJIHcZrr/YNDdJhl6tL6iogC4QwC2yxB5fuXEfLpeHK3OxCFORFWFDDtyCIzoJzRwBCdRbAEFsFY0gTGkkO/cgxkMSBzemROemuN6Otbm79+57sLJSgARKjLIQcN6cHxrdsf7Z4MU6ulDIUQVludK85YIQpffXVrfn6uWopnz+OvAKz9Mh5Uz61yfaNQSvmvqkdaS8EPj4apSoHxKC4xHiSTZDoZp9NxKQrLcQSc6SRDLkKCemNuvrn44MGOv2xDEsiCzFjwatqEgBbBIjov8+bQWSSTaFYs1YsRTI7Tg8cLsX3zz15/963vbs1LqXURdIkLBnCw0/3gd3c6B/0UtOMkhUDn8ixnSAvN5tr6WrvdrlQK4Dfq1lpnEQiREPmzFu7zS64fbM+HB98fqtXKYDjI81wrI0TAmbBaW5UZlafJhBGVopAY08YJwaoBtFbLuY0m4+n+fidXjkVFEIEBB5xg9qzOf74ckSPhUICyxSiEZDTufLIUu7/+3s13f/ydmysR06bKsUiCA3T3D967dXuvN3BSslgCJ2dsnmZgTb1SubTR2ry0EQYSGSHO3G85cWQciblnzSw8x7aQpqk/E5+f38IwDEN57dq1nSe7ne4gS8acKcGBKBRkd588yfNUGd1qXyqVYusgc1Al+NVPthCx1+vvHI/RZkFccJYpnfpthsclzi5viAAgKDB1enLa366R/tG3bvzqnR9cqYIEYE4XUICDbqd3673bg+MRRSEGwjLrtFW5JudKq5/W0wAACIRJREFUxeLS4vzS/FxYiK3KwYC2VltDRMjOhHi+hIXac6vc88nXjw1+x0HEqrUqF9JkWTpNbJ5xAimQGFlnx0k2STMRhqVCmRNj2lmLkkN5rimi0mmmDqc6AzRcOOeQESERckTuMDQsBAoABDfKjg7nC+zt79x49803vjLPCgA8T2IuwcLJwej92x90ekMKQ14IlTVKaaUVIdYrlfb6+qV2q1qpeuEdIHRowQIxL2AISmmiZy/d5yyrDQAzLrqbqQIR8uWF5fR0kkymo+NjcMoYbq2dW1jq9XrdgwGXjziTS4uLQRTFDI5Op1ercf3tzVTbzn/89/jwmCpNywLOA4vcOfL35V4wR0Cih49LZvSdr17/h5++cWMeUEOWnNSKMVg7eNJ773d3uoNDCiQPuLZK6SzLMgJXiMJms9Fur9fqNXBgkgmLI0BgyMFZYgwAtHW5Upw/u63486zc89uKc8MQz/zkjDcazUajqZTu9/t5riuVan8wkEEgAzmejE9PT4hToVgQUsSBMAZigutXGgvlZnd3+2TQr9UaylKeWqucFaGMI47OmqlMevHph//05o1//ZsfbzVRaCgClMIAADp7e/d3tnd6+5lRIpKMmMoynaZH/e7K4tzXXnvtlWtX40JktHbO8iBwDsCBA0CagYURQYgvVXwX64d2Hue9wjlQSnn9rdkKKs+VVoAoAxkEAWfCGYNAAUG5WC4Vosl48mh7u1pvUhjyQiwDcukoHR2404Mw7f7m7ZtvfevateU6U64EGHMAB71O5+6DBweHw6nKiZPgwjrrcoXWri4vttZWVleWK5UyETlwZ4h798xTwWfFhdvHnPMpwjD08y8R7ezseGL8uZN3p9MBgCRJVleXZSFCRAvUbsDb37+SORieHO0dPNAsUkBaa5uOA2aWm+VLcwu/eOfNtQgKAEKgB54Mu8M7H9zdP+hpZ6XgnHNnrNGaA8o49vjZer3u/3v+VbsgY8oLT+6nuYNxHC8vL2dZlmXZcDj0oB7v5+dxadPpWEhcXV0NAq5UjkI2Gfzg5pVQ8n//z/86SvPRZJrqNI5wc33xWzdf//pWez0CBiDATwjwZGf/3t2P+v0+E4IxQM6stZnKyEG1Wms0GouL8/4ACQBeadgfLC/CFe3CPSg9FQsAvF0LIo7H436/f+fOHU/tDsOQc55lmdaac6rXKleubLZabRIysWAZzwFGFm7dGw5OTvtHx3maFEK2tth89erGcozGQIBQIBAO+nuH779366DbJUGFSjm3KlW5UhlaVykXN9sb6+vrcbHoMal+teR3I1/SJuaz4uL90HD2fXDO+fNxsViMosi3Ao9PlVIKITzEuvOk47RBCxsbG0UZGADmQBD8YKsxcY1cASMQBMyBRBsAKGdixgXA3uPu3dt3RkdHYRgCkTEuSdNplnBOjUp5ZXlxbWW5Uqs6B8bNrmy9wt3FPfuFJ9dj/3xv9fgor56zsbERxzEi7u/v+0OzV/KTIu73j8A+REetVosFodAGHTFOBMDlDCntrFXJ2FhdKdUB4Hh48uDevW63GwgRRAXjdKZyLypVr9cvtdaXlxYqlRKgRSKy5Mfwi3avfhGV63srAHiX43NROe+E4dkWAGCMyTLVnF/od3t7+10iQmRLS0thFElBxoC21hpkCM7m6EwpDIEIwA77h/fu3BsOB8TAAmQqB0aAiIzFoVxYWFhrrdcqRbAmGY+jUvm8w/qvmWeBXYQ17YX33M+K8w/dcDi8e/fuw4cPnXP1evPo8CSUEYC1WtWq5c3NS+12Oyj+XgxtMk0YUhgFfuXa7x589NFHO9u7ABTHBeOccUCM7e7vrbVWr1/fWl9bKsURgHH+SWeY5xcRL80O/Dy5URQtLi6maer5F1EUMSaMUVrZ0clJp3sQFeIFWrAGeDAzcxGB9HDH0fHJ/fv3B4OBtVaEwqJTWmtrCPhaa3VpaalSLQVB4AGe7hz++KLiZSYXzuaz1dVVKeXHH3+8s7MjZGhBAwGXLFOqc9ABBkmWbWxsiIADgH97HcBup7fzyaOdnV1rLQ+kDAOl7VSlRBSJ4Pr1rUq11KjVOeOzJSI4elE16+OlJRfOMAA+vysrK3meZ1l2NDrWSnMmwyjikqVp2ul1p2lSKBUrlUrxjHrY7R8+ePjxwwf3CzKUoZBSWudSlQLYUqnUbDbX11aCQEjGAayxZmb4wvCpmugXFC+t5356/vV/8PPv3bsfjkajTOkoihiJVOVWac55GIharVavV0UQZqkaHh8d9Icno1GzVvf/WpInWutiMW61Wq21laWFRf+D/PfKgpspqD3FePGi4qUlF84q14PX+Rlo8OHD+51Op9PppZliUnAuERGtOT05CQIRysCCU8poY0hIKSVa0FpnOnfOlEqF1bXlK5cuzzXqMwS1tcoa6283BWck3MWcdJ8aL60tnONbz+vXH0PbrVYUhs6xvf1urgxnyIkZgHK57Iw2SllrnXZeH5A7TFWeaw0I1Wp1bW3lUnu9Wa8BOPBF4xwHdETACOmizJM/K15mz/Xz7/nI6T1tpYzm5xaniRpPkuPjkVeG9U73xjgCiKIImdBa57lOkoQ4Z4xFgWw2m+319Ua9AWDzLJeczRaIhOi15eFF9luAl9sWPiuIyFh4/Pjxw4ef9Ho9DzEplQp0Bnvx+3ittbF2OBisrK5uvfLKWmu1XCwhojHKOcc/ffpyf+yx9mLiZVbuUyNXJgiICJrNJgBUKqXBYHB8fOzNZM85MF4Lgoi+8Y2b9Xp9cXGxVCh6S1nnPHb4KTY0Lzj+5CrXeBlBj2KyMJ1Oe71ev98fDAZe/M2PGWEYeg+MG199zYNWwWuNaA1nbJmX+yDwJ5hcnAFg/axGAKCUGY/Hfq3jk+tponEcM/YHKZyxFYguaIX4ReN/AZVoNU3OeFrqAAAAAElFTkSuQmCC" } }, "cell_type": "markdown", "metadata": {}, "source": [ "![image.png](attachment:0a8e7611-1dc0-482d-9e24-e88e538518ab.png)\n", "\n", "The Message Passing Interface (MPI) is a standardized and portable message-passing system that defines the syntax and semantics of a core set of library routines useful to a wide range of users writing portable message-passing programs in C, C++, and Fortran.\n", "\n", "### Programming Model\n", "MPI's programming model features:\n", "\n", "- **Distributed Programming Model**: MPI primarily supports distributed computing, but also facilitates data parallel programming.\n", "- **Hardware Platforms**: Compatible with distributed, shared, and hybrid platforms.\n", "- **Explicit Parallelism**: The programmer is responsible for implementing all parallel constructs, providing fine-grained control over parallel execution.\n", "\n", "While the number of tasks in an MPI program is static during runtime, MPI-2 has addressed the need for dynamic task creation.\n", "\n", "### Benefits of MPI\n", "MPI provides several key benefits:\n", "\n", "- **Portability**: No need to modify your source code when porting applications to different platforms that support MPI.\n", "- **Standardization**: MPI is the only message-passing library that can be considered a standard, with broad support across virtually all HPC platforms.\n", "- **Functionality**: Over 115 routines are defined in MPI-1 alone, offering extensive capabilities for parallel programming.\n", "- **Availability**: Numerous implementations of MPI are available, both from vendors and in the public domain.\n", "\n", "### Checking the Version\n", "In this environment, MPI is typically pre-installed by the system administrators. MPI implementations may include (but are not limited to):\n", "\n", "- Open MPI\n", "- Intel MPI\n", "- MPICH2\n", "- SGI's MPT\n", "\n", "Before running MPI code, the appropriate MPI module (e.g., `module load openmpi`) must be loaded. You can check if MPI is correctly set up by running the following command:\n", "\n", "```bash\n", "$ mpiexec --version\n", "mpiexec (OpenRTE) 1.8.8\n", "```\n", "\n", "MPI Header files:\n", "C/C++: #include \n", "ForTran: include 'mpif.h'\n", "Compiler wrappers:\n", "Intel: (icc -lmpi) and (ifort -lmpi)\n", "GNU: mpicc, mpif77, mpif90, mpicxx\n", "\n", "Run the code below to see what version is intalled in this cluster:\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "mpicc is already installed.\n", "mpirun is already installed.\n", "MPIexec Version:\n", "mpiexec (OpenRTE) 4.0.3\n", "\n", "Report bugs to http://www.open-mpi.org/community/help/\n", "\n" ] } ], "source": [ "import subprocess\n", "import sys\n", "\n", "def check_install_mpi():\n", " # Check if mpicc is installed\n", " mpicc_check = subprocess.run([\"which\", \"mpicc\"], capture_output=True, text=True)\n", " \n", " if mpicc_check.returncode == 0:\n", " print(\"mpicc is already installed.\")\n", " else:\n", " print(\"mpicc is not installed. Installing MPI...\")\n", " # Install OpenMPI\n", " install_process = subprocess.run([\"sudo\", \"apt-get\", \"install\", \"-y\", \"openmpi-bin\", \"openmpi-common\", \"libopenmpi-dev\"], capture_output=True, text=True)\n", " \n", " if install_process.returncode == 0:\n", " print(\"MPI installed successfully.\")\n", " else:\n", " print(\"Error installing MPI.\")\n", " print(install_process.stderr)\n", " sys.exit(1)\n", "\n", " # Check if mpirun is installed\n", " mpirun_check = subprocess.run([\"which\", \"mpirun\"], capture_output=True, text=True)\n", " \n", " if mpirun_check.returncode == 0:\n", " print(\"mpirun is already installed.\")\n", " else:\n", " print(\"mpirun is not installed. Installing MPI...\")\n", " # Install OpenMPI\n", " install_process = subprocess.run([\"sudo\", \"apt-get\", \"install\", \"-y\", \"openmpi-bin\", \"openmpi-common\", \"libopenmpi-dev\"], capture_output=True, text=True)\n", " \n", " if install_process.returncode == 0:\n", " print(\"MPI installed successfully.\")\n", " else:\n", " print(\"Error installing MPI.\")\n", " print(install_process.stderr)\n", " sys.exit(1)\n", " \n", " # Check and print mpiexec version\n", " mpiexec_version = subprocess.run([\"mpiexec\", \"--version\"], capture_output=True, text=True)\n", " \n", " if mpiexec_version.returncode == 0:\n", " print(\"MPIexec Version:\")\n", " print(mpiexec_version.stdout)\n", " else:\n", " print(\"Error fetching mpiexec version.\")\n", " print(mpiexec_version.stderr)\n", "\n", "check_install_mpi()\n", "\n", "# Enable oversubcription in the cluster\n", "import os\n", "os.environ[\"OMPI_MCA_rmaps_base_oversubscribe\"] = \"1\"\n", "def run_srun(command):\n", " os.system(f\"srun --oversubscribe {command}\")\n" ] }, { "attachments": { "c789f932-6ec2-4254-9f85-ee74e8b74074.png": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVsAAAGrCAYAAABuXTT0AAAgAElEQVR4nO3df2zU953n8deXhSQz2ZQfMwrteesa625nIG3xOVbiauyc5NPG7ja9k+AabBmn6qo1XoiEUkG0KwRxPOKfBglxAg7Y9HKNsSBUQbtdGpmVYu3GMypdOQRogj27UvCZzKVUMxCaZIaSlO/9MfP9Mh7PjMcEPjOG50OywF9/v5/P+/sd++XP9zPjz1iqZo/21sq2npB9o06WtbDS5QAVZdufy1owKct+S28fmqp0OZgbq9IFFNXY2ytLBytdBlCVbtz4od55+aeVLgPlq86w/foPvqr7F01J0sq6L+up1m/K+8B9la4KqKjUtes6MXpO45O/zWz4w2e1eveVi5WtCuWqzlvz+xY+IUmrVnxF7/28v9LVAFXjJ5vX6pHv9ev8hQ+dn5OhSteE8iyodAGFWXWS9J2Wb1S6EKDq3Py5yPycYH6o0rDVIklMHQAF5PxcLKpkHZibag1bALirELYAYABhCwAGELYAYABhCwAGELYAYABhCwAGELYAYABhCwAGELYAYABhCwAGELYAYABhCwAGELYAYABhWyXC4bAsy1I0Gp22PRqNyrIshcNhSdLQ0JAsy5r20dDQoFgs5n7d2bdQH0NDt7bWdDqdVl9f34z6yjlmrn0659zQ0KDNmzdraGjolvoHqglhW0XWrVunkZGRadtGRka0bt26ads2bNigVCol27Zl27b27dun3bt3K51Omyz3jpmcnNThw4d15swZ7dmzR93d3ZUuCfjCCNsq8vjjjysejyuZTEqSksmk4vG4Hn/88ZLH+f1+Xb16ValUquy+wuGwnn/+eTU0NMiyLPX19U0La2ekXWy0nbt//mjaOfav/uqvdPXqVXe7Mzot1q7T1vr167V+/XqFw+Gio3Fn9FuodqAaEbZV5MEHH5QkTUxMuP/W1NTo4YcfLnnc2NiYFi9eLK/XO6f+zp07pzfffFMTExM6deqUTp8+LUluuNm2rYmJCe3du9f9BTCboaEhRaNRJRIJ9ff3a3x83P3arl27VFNTI9u2FYlEtGnTJnf6w9Hd3a3Dhw9rYGBA27dvL9hHLBbTwMCAJiYm3F8wu3btmtO5A6YRtlXkwQcfVGtrqzuVMDIyora2thn7HTx4UF6v1x3ZDQ4OaufOnfJ4PHPqLxQKyefzqba2Vs3NzZIyo8/R0VG330AgoCNHjsjn85XV5vvvv6+enh75fD4FAgGtXbtWUmaUHo1G3XYbGxvV3NysRCIxp5qlzC+XFStWqLa2Vh6PRz09PYrH44xuUdUI2yrT1NSkjz/+WB988IHOnz8vv98/Y5/8Odvh4eGywzBXfX39jG2pVEqTk5O3VHs6nVY8Hi+5T0tLiyzLktfr1cGDB2+5r9xfOC0tLZqcnJzTNApgGmFbZWprayVJb7zxhhYvXux+borX61Vd3dzetPX999+XJHk8HtXU1BTdb+nSpZqYmHB/Sdi2fctPfg0MDExr51Z/4QCmELZVxuPx6KGHHtKGDRvU2to656mB29F/TU2NO5URi8XU3t6uf/u3f5u236lTpzQ1NeVODzja2to0ODioZDKpWCym119/XZLk8/m0atUqHTt2zG23oaHhll7K1dTUpEgk4s73hsNhniRD1VtY6QIwU1tbm15//XU1NTVVpP8tW7boueeek2VZkqRIJKI///M/d78eCoW0du1aBYNBtbe3q729XZ988on7tVAoJL/fr/b2dn3/+98v2u7hw4cVCoXmXF8gENCOHTsUDAYlSe3t7RoaGjL+iwmYC6vSBRTU2NsvSy+88KOn1L/hu5WuBqgq/Qf/US/+3QnJ1os6fai/0vWgPEwjAIABhC0AGEDYAoABhC0AGEDYAoABhK2mL7rifBRbpnAubd7qcoaFxGIxdXV1ua9fdf4/nySTSXV1dc1YD+FW96u0+VInqgNhm5X7F0mpVErxePwLB+6dMtf1CgBUHmFbgMfj0XPPPTftr5Scv3iyLEsdHR3TRpW5C3oXGs0mk0l1dHTMGDU7f53V3t7utpm7dKCzDGEymdTmzZt19OhRdXd369e//vW0kW2h5Qad5QxzR+2FlkEsdxTvLKv4/PPPT7sGTj+dnZ1uvYWWUkyn09q2bZuOHj2qdevWKRaLlbXfkSNHpv11WCwW0/PPP6/Lly+rr69Phw4dch+X3Gtf7PHK77Ocu49yzyf/urLQOXIRtkXU1tZqxYoVSiQSbtjt27dPtm0rFApp27ZtSqfTikajGh0dVSqVUiKR0IkTJ6bdVjo/lD09Pe6ShbkhfunSJe3YsUPDw8NKJBLu0oG2bevw4cMaHByU1+vVnj171NnZqaGhIS1ZssRtPxaLadOmTYpEIgWXG4zH40qlUopEInr99dcVi8UUjUbd7alUSh9//HFZt8IHDx7U6tWr3Wuwf/9+92uPPPKIu93p3+l306ZNmpqa0s6dO9XZ2anXXntNgUCgrP2efPJJXb16VVNTU5IyK36tXr3a/Wux48ePu8tEvvrqq4rFYiUfr+PHj7vLPCYSCY2Ojs46HVNoachC5/NFlqbE3Y+wLYOzvqzz56FPP/20u1j3yMiIu4aBz+fTkSNHFAgE3GM9Ho8OHDjgLrji9/u1bNky9+vLly93V/YKBAI6efKke3w5C8IkEgk1NzersbHRXW7QGQ1LcmsLBoNauXLljOM9Ho9+8pOfTKu5GGcE7lyD8+fPu/04K4g5K3/19PTI4/G4SymOjY1Na6vc/Zw1FRKJhNLptM6ePTvtz5hzl3NsaWnR2NhYyccrv+0DBw6UnI4pd2nIL7o0Je5+hG0R+UsNnjx5Un6/X5ZlKRgManx8XB988MGsSwpK02/z/X7/tAW1ly1bNm0Zxdzb0JaWllnbzl+i0O/3a+nSpe7nhQI7FAqptbXVXaKw3Cfy6urqii5Q7vSTf92KrQRW7n5SZq2IkZERd3SbuxJasV9IhR6vRCLh/tKb663+bEtDfpGlKXFvIGyLSCQSsm3bDcL8NWTPnDmj1atXl1xSUMqMjMLhsCKRiHvrWmiEKWVC2XmXA+eWdTb5YZNIJHTlypVZj+vu7nafDBwdHS0rdHLXjC3WT/4SjcXWuC13PykzQo3H43rjjTemTSE4NRU6vtDj5Yzet2/f7j4We/funXUKpZylIW9laUrcWwjbAtLptHbv3q2WlhYFAgEFg0FNTk5Oe9sY50mX+vp6jY6OKp1Ou0+ElQqu4eHhaSPbUjUMDg7Oup/f73ff0sY5xnkHhmLy3zPsS1/6UsFFyvOdPHnSvUUfGRkp2I8zQh0cHFQ6ndbp06d16tSpGSuYlbuflLndr6mp0c9+9rMZX3eu/dTUlC5cuKCmpqaSj1f+S/IWL15c8tzLXRqy2NKUvCwMDsI2a8eOHe6tpdfrVU1NjfseWD6fT3v27NGmTZtkWZZeeukl7dmzRz6fT93d3aqpqZHX65Xf71dPT8+0ZQN9Pp96enrc29BPP/206NvBNDY2qq6uTn6/X9/61rf07W9/251r9Pv9unLlirq7u/XRRx+5xwQCAe3bt08tLS3uLf6WLVtKnuuaNWsUj8fdc129erUCgcCsrxvt7OzU3r17ZVmW4vF40X6c7V6vVy0tLdq3b58CgYC8Xq8WL17sPntf7n5SZiqhubl5xmLqzrUPBoPasWOHAoFAycdr48aNGhwcdKd0nDnfUq9d3rJli3u9gsGgtm7dqlAoVPB8cvdz6rndr7kGbp/G3n492mu/cOAXNsz6+7//e3tiYmLG9kgkYm/YsMFOpVIVqMq2BwYG7MOHD7ufp1Ipe8OGDXYkErltfRw6dMhOJBK3rb075YUDv7D1aK+txl6WV5xHGNnClU6ndenSJeNvxVOK8xrXeDyuNWvW3LF+nBEtrx7AncI7NcDl8XjU29tb8GvOOzCY5rx0rtztt8rn8+lHP/rRbWsPyMfIFgAMqNaw/UySUteuV7oOoOrk/Fx8Vsk6MDdVGrb2pCT9MvKbShcCVJ2bPxc2f0Uxj1TnnO31z9/S/Yt0/sKH+mbngP4y9HU9cN+iSlcFVFTq2nWdGD2n8cnfZjZc//ytylaEuajOd9eVpMbeXlk6WOkykPX/3pb+w6OVrgKOGzd+qHde/mmly0D5qjdsJenR3lrZ1hOyb9TJsqpzFH6vOH3oBTX2vljpMu5ptv25rAWTun79X/TuKxcrXQ6AO8OudAHAfFalT5ABwN2FsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbADCAsAUAAwhbFPMXkq5K+n7e9u9L+kjSk8YrAuYxq9IFoKpdk/S5pLQkv6SEpAck3Sfp/grWBQB3lb+R9AdJds7HNUl/W8miAOBulNbMsAUA3Ga5o1tGtQBwBzmjW0a1AHAHOaNbRrUAcActkvR/lHkVAgAAAAAAAAAAAAAApc2ftREe7a2VbT0h+0adLGthpcsBKsq2P5e1YFKW/ZbePjRV6XIwu/kRto29vbJ0sNJlAFXpxo0f6p2Xf1rpMlBa9Yft13/wVd2/aEqSVtZ9WU+1flPeB3i5J+5tqWvXdWL0nMYnf5vZ8IfPavXuKxcrWxVKqf7b8fsWPiFJq1Z8Re/9vL/S1QBV4yeb1+qR7/Xr/IUPnZ+ToUrXhOLmweLhVp0kfaflG5UuBKg6N38uMj8nqF7zIGy1SBJTB0ABOT8XiypZB2Y3H8IWAOY9whYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBs7wGxWExdXV1KJpMl94tGo+rr61M6nVY4HNbQ0My1qGOxmBoaGtz9HOl0Wn19fWpoaFAsFlMymVRHR4csy5r2EQ6HJUnJZFJdXV2KxWK392SBKkXYwhUKhXTgwAF5PJ6S+y1fvlwfffSRpqZuvs/g1NSUPvroIy1fvnzavpFIRLZty7ZtpVIpxeNxRaPRO1I/UM0I23uMM8rt6+tzR5tO+Dkj21deeUU7duzQ+vXrC45uly1bpscff1xjY2PutrGxMT3++ONatmxZ0b49Ho9qamo0OTl5+08MqHKE7T1ofHxcra2tsm1bAwMDGhwcnDYl0NnZqYGBAR0+fFjd3d0F26ivr9fo6KjS6bTS6bTOnj2rxx57rGS/yWRS0WhUdXW8gwvuPYTtPWj58uVqamqSJLW1td1SGytXrtTVq1c1NTXlTid87Wtfm7FfS0uLO4L2+/3q6elRKBS69eKBear6310Xt92yZcvk9/u/UBs+n0+rVq1SIpHQ5OSkHnrooYJzvZFIhHAFRNjiC2hra9M//MM/6Pe//716enoqXQ5Q1Qhb3DK/369/+qd/0pe//GUFg8FKlwNUNcIWBdXX12v9+vWSVPRJstraWjU3N6umpkY+n2/W1/EC9zKr0gXMqrG3X5ZeeOFHT6l/w3crXQ1QVfoP/qNe/LsTkq0XdfpQf6XrQXG8GgEADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcCA+RC2n0lS6tr1StcBVJ2cn4vPKlkHZjcPwtaelKRfRn5T6UKAqnPz5yLzc4LqVf3vrnv987d0/yKdv/Chvtk5oL8MfV0P3Leo0lUBFZW6dl0nRs9pfPK3mQ3XP3+rshVhNtX/7rqS1NjbK0sHK10GUJVu3Pih3nn5p5UuA6XNj7CVpEd7a2VbT8i+USfLqv4R+d3sw7df0FcefbHSZdzTbPtzWQsmdf36v+jdVy5WuhwAd4Zd6QKA+WYePEEGAPMfYQsABhC2AGAAYQsABhC2AGAAYQsABhC2AGAAYQsABhC2AGAAYQsABhC2AGAAYQtI3cqs95D7cUZS4Db2EZJ0QJJH0vZsn+Xufzt5su3O1n+hY0K3uYbb1d68QNgCGQcleZVZCc+StEnSc7r9YQcpLalPUrTShZhE2AKFJSQtViaAnZGYM+p1RpzO9qPZ7SFJPknDOftun6Wf/LbLGe0FlBl559ai7LH5NTq2Z7f/7+x53Ur/j+T0mzsyLtZvbttHsh8hTR/ZhnK+ln9HkXuet/tOwzjCFiisSdJVSSlJWyTFlRnx+iXVSWrM2fe97NdOS9opaTD7eVBSi0qHRG7bLZL2zbK/T1K/pHXZY+KS1mSP2Zdtw5vTtpQJxlC29n5JK2+x/zWS/mv2vJ7J7heQtCO7Lb/fNdl/vZL2Suos0m5n9uteSackPa1MID+nzB2Gc6fxdJHj5wUW4QYyNmQ/HCeVCam0pHDO9pSk/Pf7ej/7r3N77EhIulyiT58yIei0f1qZsPGXOCaozC+BqeznzrGh7LGns3UMKjOa9Umqz36ezH68Xkb/sQJ957YRUeYXkiRdyNbj9Nsjaamk1uzn6Wzbxd5t5aSkiex+o9l680U1z6cdGNkCGflzth3KhIo0/XY2pemhLE0P39xb6oSmjyKLieS1XVdi32Jfy9+ekHRF0gOSam5T/8XeVHJD9lg721adMkFe6jzy203lbUtL2qab0x/D2TbnLcIWKM25nX1JmRD2qvgIzadMOLTo5pTD+CztX1FmtGrlfAyV2L9Y4OVv9yszurymzDTB7ejfCU+Ppgf4jrzjO7J9ftF3/E1m27KUGX3v1Dx+wpKwBeamUTNHtsV0qPTINinpvG7ORToj6FJPUiUkrZBUm/18e/YjIak5W59HmVv5aLaPkeznvmwfa2+x/9Zs27XZGsayH7nz0tuVefJLykwJ9GSPmct1U7bWI5o+fxxXZsQ7LxG2QGnOPORhZW5n25QZyRW6RU5m93Vuyx/U7HOwu5QZJdrKzFu+pNJzkzFJA9l97eyxu7LbN2X7dm7Jd2X/deY7E5L2SPrZLfZ/WtKvsvsNZPvMryekzO1/WtLx7HEpSf9dxe8ICkkq86SZ0+52SfvncDxwV+ANHzFXAWWeCJvXL98CTCNsUQ7nya25vIYYQA7CFpgj5mwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwADCFgAMIGwBwICF7v8e7a2VbT0h+0adLGthiWNwr/vwbekrj/ZXugxUgG1/LmvBpCz7Lb19aKrS5cwnliSpsbdXlg5WuBYA88mNGz/UOy//tNJlzBeWvv6Dr+r+RVOStLLuy3qq9ZvyPnBfpesCUIVS167rxOg5jU/+NrPhD5/V6t1XLla2qvlhoe5b+IQkrVrxFb33c+4MAZT2k81r9cj3+nX+wofK5sdQpWuaDxZIVp0kfaflG5WuBcA8cTMvMvmB2S2QtEgSUwcAypaTF4sqWcd8wku/AMAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADCBsAcAAwhYADLgjYTs0NCTLsqZ99PX1KZ1O34nuyhYOhzU0dOfXOU4mk+ro6FA0Gr3tbcdiMXV1dSmZTN72tlFYOp1WX1/fHXk8y+V87/L4z193bGS7YcMGpVIp2bYt27ZVU1Oj5557ruKBC8yVx+PRgQMHFAqFKl0K5jFj0wgbN27U5OSkTp8+LenmaMEZ+RYaNQwNDSkcDkuaOVqMRqPuaDkWi6mhoUGWZamjo8P9rR8Oh9XZ2SnLsmaMaJ1jCvUbjUanjcpz++zq6lJXV5csy1JDQ4NisdiM89m2bVvBa+Dsc+jQIbde5xzza3fO16nBuQ75wuHwrNch/7y7urq0e/du9xyi0ajbV24/xdrLf+yca1tsu/NYOttzr1vuefb19amrq8v92q2ej3N8qX5zH+Pctgttzx3ZOv0V+951+mtoaNDzzz9f9HHLPbfcu75i9RaTW29uO7ntl9MO7jxjYevz+RQKhTQ5OSlJ2rVrlyQplUopEolo06ZNM74hmpqa9PHHHyudTiuRSGjJkiVKJBKSpMnJSbW2tiqVSmnz5s3aunWrbNtWKBTStm3b3G+6JUuWKJVKqbu72203Foupv79fb7755ozRSiwW08DAgCYmJmTbtg4fPqzBwUG3vaNHj+rZZ59VKpVSc3Ozjh07NuN8enp6dPLkyaLX4vTp0/rVr36lSCSi9evXq76+XqlUSnV1dRoeHlY6nda2bdvU09Mj27Y1MTGhSCQy4/qEw2HV19frwIED7nXYt29fweuQa3x8XA8//LBs29batWu1adMm7dmzZ1o/yWSyaHvHjx9XTU2NbNtWIpHQ6Oiokslk0e3RaFSDg4NKJBJun851279/v0KhkGzbVmtrq8bHxyWpZP+FzueTTz6RbdvaunWrdu/erXQ6XbTfZDKpvXv3uo9xT0+PhoeHi24v1F9ra6ts29bAwID7/RGLxfTqq69qYmJCb775ps6dO1fw8U8mk+rv79drr73m3vUdP3685HUqJPd7NZVKScp8H6bTae3evdu9dvv27SvZDsyoyBNk6XRa8XhcPT098ng8amxsVHNzs8bGxqbt5/f7dfHiRU1NTSmRSKirq0vnzp1TMpnU2bNn1dTUpEQioaVLl6qjo0OS9PTTT+vChQuamsq8pX1NTY08Ho/b5u9+9zv19/erv79fPp9vRm2BQEAnT55UIBCQJNXVTX+Lpfb2dgWDQXk8HrW2thY9nw0bNhQ9/9bWVnk8Hvn9fnV2dqqjo0Mej0c1NTWSbt62Or8g/H6/li1bNq2NV199VZLcfSYmJiRJwWDQvQ5Xr151fwhzLV++XE1NTZKk+vp6rV27VoFAYFo/5bbn8/l04MCBGdcyd3soFNLw8LC7T319vaRM6Jw/f15PP/20JKmjo0MrV668pfPJbePq1auampoq2m++7u7uab+MZ9uee/3a2trc7WNjY2ppaVEgEJDP51NPT0/B/iYmJrR48WLV1tZKkrZv367u7u6y683tb8WKFaqtrZXH41FPT4/i8fiMX0ihUEjbt28v2RbuPGNh6wSSlBn9OSNcSdOCJpfP59OqVauUSCT0r//6r/ra174mKfND+vvf/15+v1+JREJXrlxxj8kPpvxv2JMnT+qxxx6bEey5nNt6y7LU0tIy7Wt1dXXyer3TtuWfz2zyA7yQ3NtDv9/vjvgkuf+Px+PTbq1Pnjwpv98vy7IUDAY1Pj7u3gnkWrZsmfx+/6w1FGvPCaD82+hi2/OnF9avXy9JSiQSunz58pz7L/d8ivXr8/nU39+vdevWTbv9Lra93P7ef//9ktfTUex7pVi9pRw8eFBer9f9Xp2cnJRt29q5c2fB6SlUjrGwdQLJCavcwMkN4nxtbW1677339Mknn+irX/2q/vRP/1T//M//rJqaGvl8Pvn9fi1dutTdf7Yf4J6eHj3zzDM6ceJEwXmsaDSqaDTq3spFIpFZzy3/fL6oZDKpcDisSCTi3pI7Iz5JWrlypZ555hm1trZq//797vb8JyXPnDnjjtBvRan2tm/f7ta2d+9e91oW2n78+HFJcts6fPiwpMIj9nL7z3X58mU3hHMf/2L9Spk7mDNnzrjTF840ULHt5ZhtJOoo9r1Sqt5iBgYG3Otj27Y7Mvb5fBoeHpZt29q+fXvRKRiYYyxs9+/fr7q6OjU2NrojWWeu6/Tp0zp16pR7a5bL7/drZGREH3/8sbxerx577DGNjIy4t29+v19Xrlxx59aOHTvm3loV4/P59NRTT806j5VOpzU4ODjruTlTCrnnc/DgwVmPK9fw8PC0ka2jo6ND58+fVywWUzAYnPYE5NDQ0Bca0ZRqL/8ldIsXL5bf7y+6PVcymXSvqXPn4jwOuec5l/O5dOmSe6eSe2tdrN9CL5+qr68vur1cTU1N0+a8i33v+P3+aVNd4XB4xhNppY4v1J/TTl9fnz744INpTzRKM6fTYN4dC9vc2xvLshSPx7V79273Ad+yZYukzKiwpaVF+/btKzhqqa2t1ZIlS7R69Wp3ntO5rZQyP7B79uzRSy+95N667ty5c9ZvrDVr1igej894NUJjY6Pq6urk9/v1rW99S9/+9reLzhXmt+ecz+DgoLZu3VrehSrAme9raWmRZVn69NNP1dzcPOMW2ufz6dlnn9Xu3bvl9Xq1Z88ebdq0SZZl6aWXXtKePXsKzkuXW0Ox9jZu3KjBwUF3iqOnp6fk9o6ODk1OTsrr9aq7u1t//dd/7c4tbty40Z0y+d3vfueO4Ev1n2/lypUaHR2VZVkaHBx0H/9i/dbW1uqpp55yv5dGR0e1Zs0aBQKBgtvLFQgE9MwzzygYDKq7u1vt7e1F99uxY4eCwaD7s7Fly5aS16mcdpzv/T/7sz/Ts88+624Ph8PauHFj2eeBO6Wxt1+P9tovHPiFDVRSJBKx29vb7UQiUfYxExMTdmdn55yOMSGVStkbNmywDx8+XOlS7ogXDvzC1qO9thp7+ysdYfMFf66Lisl/Qsh5CdqtjsYrLfdJTa/Xq5qamoKvZsC9aWGlC8C9y3mJ24uF/V0AAArJSURBVIEDB265jUAgoCNHjtzGqm6d83phoBBGtgBgAGELAAYQtgBgAGELAAYQtgBgAGELAAYQtgBgAGELAAYQtgBgAGELAAYQtgBgAGELAAYQtgBgAGELAAYQtgBgAGELAAYskPSZJKWuXa9wKQDmi5y8+KySdcwnCyR7UpJ+GflNpWsBME/czItMfmB2C3X987d0/yKdv/Chvtk5oL8MfV0P3Leo0nUBqEKpa9d1YvScxid/m9lw/fO3KlvR/GFJkhp7e2XpYIVrATCf3LjxQ73z8k8rXcZ8Ybn/e7S3Vrb1hOwbdbIs3ggSGR++/YK+8uiLlS4DVcK2P5e1YFLXr/+L3n3lYqXLAe4mvF0scBvw0i8AMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwBQADCFsAMICwxb0qIOmMJDvvo/s293FEkk9SSNIBSZ4y97/dusvo/06oVL9Vh7DFveySpKAkK/vhl/SUMqEH3FaELXBTStJVZUJXyozKnBHvGWVC2KPMSO1odntImZHocM6+22fpx2nD2T9URm25fQzr5ug3d4Seu13Zdp3aV99C/8X2c0bgxdoo1m8oZ//8Wu96hC1wU62kFZISygRDjzLBa0l6XdLTOfu+l91+WtJOSYPZz4OSWlR6dLxFUjy7f4ukfbPs78n2Ec4eMyhpozJhtUfSS9nt0ex+nmx7+7Ltf0vSl26h/1L7rZQ0mv3aDmWuVal+l0l6VjfvJAYldZQ457sOYYt72XJJE7o52pqQNCAppkxwdUhKZvd9P+9Y5/O0pD5JQ9nPE5Iul+hzmTJBPpL9/LSkU7o5mi6kVtLibH3K9hXOHnNFmVGiJB1T5pdFbfZrp7Ltp5UJt7n075tlv0uSxrL/H8k5rli/+YZ085rdEwhb3Mvy52yd0aE08xb6cN6xkzn/z709Tigz6ptNJLt/StIGSXUl9vVLWlrG9tygL9XeXPovtt/lbH/5irVzWVK/pNey7d1zT5oRtkBha7L/epUJ4fVF9vMpM0fboptPso3P0vYVzQz5UqO8RPaY2bb7lRm5StN/Gdxq/3Otc7Z+Y5Iasu2MKjNNcc8gbIHZ+ZSZkyxHh0qPbC9LOq+b87/OE1ylniRLKDOCDWY/d15O9Ul2uzP3+bSkC5KmlJlyqJPUqMwI0qm/3P6Tt1CnSvT7nzTzZW35UzN3NcIWKGxYmdBIKTOa+1+SaiQ9kLdfUpl5Sed2+0HNPge7K9uWM0/8km5OXxSSlLRZmSeebGUCbJsyT15tlrRVN18RsE2ZudKkMvO6kew5nJb0+zn2P9c6nVoL9fvvkk4o84vDltQq6fgsbQH3FLvSBQB3A0a2AGAAYQsABhC2AGAAYQsABhC2AGAAYQsABhC2AGAAYQsABhC2yPcXyqzp+v287d+X9JGkJ41XBNwFrEoXgKp0TdLnyvzZp1+ZP7F8QNJ9ku6vYF0AcFf5G0l/0PT35rom6W8rWRQA3I3Smhm2AIDbLHd0y6gWAO4gZ3TLqBYA7iBndMuoFviCbr4a4dHeWtnWE7Jv1MmyFlawJlQL+8YCXf7372rZfzwh60/+WOlyUAVs+3NZCyZl2W/p7UNTlS5nPsmEbWNvrywdrHAtAOaTGzd+qHde/mmly5gvLH39B1/V/YumJGll3Zf1VOs35X3gvkrXBaAKpa5d14nRcxqf/G1mwx8+q9W7r1ysbFXzw0Ldt/AJSVq14it67+f9la4HQJX7yea1euR7/Tp/4UNl82O2d9yFpAWSVSdJ32n5RqVrATBP3MyLTH5gdgskLZLE1AGAsuXkxaJK1jGfsBANABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYRthUSjUVmWVfAjGo3e1r5isZi6urqUTCZv6fhkMqmuri7FYrEvXEs6nVZfX58aGhpmtDc0NCTLsjQ0lFmLOhwOz7g2HR0d7nmEw2F3X6DaEbYVEgqFZNu2bNvWwMCABgYG3M9DodBt7SsQCOjIkSPy+Xy3td0vIhgMamxszP08nU7r7NmzevLJJ6ftl3tdbNtWT0+P9u/fb7pc4AsjbKuQM/rLH+nGYjG1t7ervb1dHR0d+vWvf62uri7t3r1blmWpoaFB0WhUHR0dsixL4XDYPc4Z2UajUXV1damrq8s9xhlhJpNJ91jn+HQ6rW3btuno0aNat26dYrFY0fryR5pDQ0NuDfna2tp09uxZpdNpSdLUVOaNWlevXl3y2tTV1Skej7vHAfMFYVuFdu3apZqaGtm2rUgkok2bNrmBeOnSJe3YsUPDw8NasmSJxsfH9fDDD8u2ba1du1abNm3Snj17NDExoUgkUvDW/+jRo3r22WeVSqXU3NysY8eOuaHa09Mj27bd46emprRz5051dnbqtddeUyAQKFpfW1ubRkdHlU6nlU6nNTo6qra2toLnuHz5cl28eNEN2bGxMa1evVoPPfRQyWszMjKimpoaeTyeL3iVAbMI2yrjjD6dkGpsbFRzc7MSiYSkTEj5/X53/+XLl6upqUmSVF9fr7Vr1yoQCMjv92vZsmUF+2hvb1cwGJTH41Fra6skyePx6MCBA+ru7pakoseXqs/v9+vChQuamprS1NSUrl69qmAwWLAGv9+vVatWaWxszJ1CcM4j144dO6bN2cbjcW3ZsqWsawlUk4WVLgCFtbS0TPu8tbXVDcDcsM3/vBx1dXXyer0ztkej0Wn9lrqlL1RfY2OjVqxYoUQiocnJSa1atarkPHFbW5tGRkY0NTWlixcvFjyPgYEBbd++vZzTAqoaI9sqtHTpUk1MTEx7YsgZcd4pyWRS4XBYkUhEtm0rkUho5cqVc6rPGSmPjIzo/fffLzqF4AgGg4rH43rjjTdmDWZgviNsq4zP59OqVat07NgxSZknt5wnvkwaHh7W+Pj4nOtramrS66+/rmg0WnQKweGMrn/84x/PGszAfEfYVqEtW7YoHo/LsiwFg0Ft3br1tr8cLJ/P51NPT49aWlpkWZY+/fRTdy7W6/Vq8eLF7qsRStVXW1ur5uZmhUKhWUeqzkjYmUMG7maWGnv7ZemFF370lPo3fLfS9QCYB/oP/qNe/LsTkq0XdfpQf6XrmQ8Y2QKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABhA2AKAAYQtABiwQJb9B0lKXbte6VoAzBNuXmTzA7NboD/a5yXpl5HfVLoWAPOEmxf2gncrW8n8sVB/suAdSTp/4UOt+h8v6L/9l9V64L5Fla4LQBVKXbuuX7x1VrH/eymz4Y/W6cpWNH9YkqT//KMfy7JeksUcLoBy2H+UrB/r7UP/s9KVzBf/H+q40u7B4HwzAAAAAElFTkSuQmCC" } }, "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "Depending on the programming language you use, you will need to include the proper headed files and compile the program.\n", "\n", "\n", "**MPI Header files:** \n", "```C/C++: #include \n", "ForTran: include 'mpif.h'\n", "```\n", "\n", "**Compiler wrappers:**\n", "```Intel: (icc -lmpi) and (ifort -lmpi)\n", "GNU: mpicc, mpif77, mpif90, mpicxx\n", "```\n", "\n", "\n", "### MPI program stucture\n", "\n", "MPI uses objects called communicators and groups to define which collection of processes may communicate with each other. Most MPI routines require you to specify a communicator as an argument.\n", "\n", "![image.png](attachment:c789f932-6ec2-4254-9f85-ee74e8b74074.png)\n", "\n", "#### MPI program schema\n", "It is very important to learn the MPI program structure before we proceed to coding message passing programs. Following are the common functions used while developing an MPI code:\n", "\n", "\n", "**MPI Init:**\n", "\n", "``MPI_Init (&argc,&argv)\n", "MPI_INIT (ierr)\n", "MPI intialisation.\n", "``\n", "\n", "**MPI_Comm_size:**\n", "\n", "``MPI_Comm_size (comm,&size)\n", "MPI_COMM_SIZE (comm,size,ierr)\n", "``\n", "Determines the number of processes in the group associated with a communicator.\n", "\n", "**MPI_Comm_rank:**\n", "\n", "``\n", "MPI_Comm_rank (comm,&rank)\n", "MPI_COMM_RANK (comm,rank,ierr)\n", "``\n", "Determines the rank (task ID) of the calling process within the communicator. Value 0...p-1\n", "\n", "**MPI_Abort:**\n", "\n", "``\n", "MPI_Abort (comm,errorcode)\n", "MPI_ABORT (comm,errorcode,ierr)\n", "``\n", "Terminates all MPI processes associated with the communicator.\n", "\n", "**MPI_Finalize:**\n", "\n", "``\n", "MPI_Finalize ()\n", "MPI_FINALIZE (ierr)\n", "``\n", "Terminates the MPI execution environment. This function should be the last MPI routine called.\n", "\n", "Let's get introduced with the functions and environment veriables used in the code:\n", "\n", "- **MPI_Init**, all of MPI's global and internal variables are constructed.\n", "- **MPI_Comm_size** returns the size of a communicator and the built-in MPI_COMM_WORLD encloses all of the processes in the job, so this call should return the amount of processes that were requested for the job.\n", "- **MPI_Comm_rank** returns the rank of a process in a communicator. Each process inside of a communicator is assigned an incremental rank starting from zero.\n", "- **MPI_Finalize** is used to clean up the MPI environment. No more MPI calls can be made after this one.\n", "\n", "As you can see this is a C program, so we will need to compile it. \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## MPI Hello World Example\n", "\n", "In this exercise, we will write a basic MPI \"Hello World\" program and discuss how to run an MPI job across several processes. This exercise will cover the basics of initializing MPI, obtaining the number of processes and ranks, and printing a \"Hello World\" message from each process.\n", "\n", "### Introduction to MPI\n", "\n", "The Message Passing Interface (MPI) is a standardized and portable message-passing system designed to function on parallel computing architectures. MPI is commonly used in high-performance computing (HPC) environments to achieve parallelism across multiple processors.\n", "\n", "In this example, we will use MPI to run a simple \"Hello World\" program across multiple processors. Each processor will print its rank, the total number of processors, and the name of the processor it is running on. This will help you understand the basic MPI concepts and how parallelism is achieved in MPI programs.\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MPI program written to mpi_hello.c.\n" ] } ], "source": [ "import os\n", "import subprocess\n", "\n", "# Define the path for the MPI program file\n", "mpi_program_path = \"mpi_hello.c\"\n", "\n", "# Remove the existing file if it exists\n", "if os.path.exists(mpi_program_path):\n", " os.remove(mpi_program_path)\n", "\n", "# Create and write the MPI program\n", "mpi_program = \"\"\"\n", "#include \n", "#include \n", "#include \n", "#include \n", "#include \n", "\n", "int main(int argc, char** argv) {\n", " // Initialize the MPI environment\n", " MPI_Init(NULL, NULL);\n", "\n", " // Get the number of processes\n", " int world_size;\n", " MPI_Comm_size(MPI_COMM_WORLD, &world_size);\n", "\n", " // Get the rank of the process\n", " int world_rank;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);\n", "\n", " // Get the name of the processor\n", " char processor_name[MPI_MAX_PROCESSOR_NAME];\n", " int name_len;\n", " MPI_Get_processor_name(processor_name, &name_len);\n", "\n", " // Generate a unique random seed for each process based on rank\n", " srand(time(NULL) + world_rank);\n", "\n", " // Generate a random number for each process\n", " int random_value = rand() % 100;\n", "\n", " // Get the current time\n", " time_t current_time;\n", " time(¤t_time);\n", " char* time_string = ctime(¤t_time);\n", " time_string[strlen(time_string) - 1] = '\\\\0'; // Remove newline from time string\n", "\n", " // Print off a hello world message with dynamic content\n", " printf(\"Hello world from process %d out of %d . Random value: %d. Time: %s\\\\n\",\n", " world_rank, world_size, random_value, time_string);\n", "\n", " // Finalize the MPI environment.\n", " MPI_Finalize();\n", "}\n", "\"\"\"\n", "\n", "# Write the MPI program to a file\n", "with open(mpi_program_path, \"w\") as file:\n", " file.write(mpi_program)\n", "\n", "print(f\"MPI program written to {mpi_program_path}.\")\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now compile and execute the code." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Compiling the MPI program...\n", "Compilation successful, executable created.\n", "Running the MPI program...\n", "Program Output:\n", " Hello world from process 0 out of 4 . Random value: 93. Time: Sat Sep 7 12:24:21 2024\n", "Hello world from process 1 out of 4 . Random value: 87. Time: Sat Sep 7 12:24:21 2024\n", "Hello world from process 2 out of 4 . Random value: 25. Time: Sat Sep 7 12:24:21 2024\n", "Hello world from process 3 out of 4 . Random value: 25. Time: Sat Sep 7 12:24:21 2024\n", "\n" ] } ], "source": [ "import subprocess\n", "\n", "# Compile the MPI program using mpicc\n", "compile_command = f\"mpicc -o mpi_hello mpi_hello.c\"\n", "compile_process = subprocess.run(compile_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output and error (if any) after compilation attempt\n", "print(\"Compiling the MPI program...\")\n", "if compile_process.stdout:\n", " print(\"Output:\", compile_process.stdout)\n", "if compile_process.stderr:\n", " print(\"Error:\", compile_process.stderr)\n", "\n", "# Check if the executable was created\n", "executable_path = \"mpi_hello\"\n", "if os.path.exists(executable_path):\n", " print(\"Compilation successful, executable created.\")\n", "else:\n", " print(\"Compilation failed.\")\n", "\n", "# Run the compiled program using mpirun with 4 processes\n", "run_command = \"mpirun -np 4 ./mpi_hello\"\n", "run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output of the program execution\n", "print(\"Running the MPI program...\")\n", "if run_process.stdout:\n", " print(\"Program Output:\\n\", run_process.stdout)\n", "if run_process.stderr:\n", " print(\"Program Error:\\n\", run_process.stderr)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "After compiling and running the MPI \"Hello World\" program, you should observe output from each process. Each process reports its rank, the total number of processes, and the name of the processor it is running on.\n", "\n", "### Key Points:\n", "\n", "1. **MPI Initialization**: The program starts by initializing the MPI environment using `MPI_Init`. This step is crucial as it sets up the necessary environment for MPI functions.\n", "\n", "2. **Rank and Size**: Each process is assigned a unique rank, which can be used to identify it among the other processes. The total number of processes is referred to as the world size.\n", "\n", "3. **Processor Name**: Each process can retrieve the name of the processor it is running on, which is especially useful in a distributed system.\n", "\n", "4. **Parallel Execution**: By running the program with `mpirun -np 4 ./mpi_hello`, the program is executed in parallel across 4 processes. Each process runs independently and prints its information, demonstrating how MPI can be used to parallelize tasks.\n", "\n", "5. **Finalization**: The program concludes by calling `MPI_Finalize`, which cleans up the MPI environment and ends the parallel execution.\n", "\n", "This basic example provides a foundation for understanding how MPI works and how it can be used to develop parallel applications. In a more complex scenario, these principles can be extended to perform large-scale computations across many processors, significantly reducing execution time.\n" ] }, { "attachments": { "1e20047e-2be2-4a72-8991-ed7ef27df119.png": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIAAAAB3CAYAAADLqUhqAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAAWuSURBVHhe7Z29Ly1PGMfHISe5RMIJWi+VoFYSNSVKoZMo/QEoRY0SLUpaEp0eUQm1BIkgOTj3d79jH9b+zsue4bzsPN9PMndnd2d3bzyfnZmdnXNOw99/GKKWVLAkSqEAyqEAyqEAyqEAyvnVpwA+UFSXhoaGIOfOjwSQQxn42iIiuAjhJEA48OG84HBKUgbhQIeD7yJC2QKgeLFEqocEvVCKg5MAuVzOpmw2a+7v7837+3uwl9SCxsZG097ebtLptEmlUjZVRAAUFQHe3t7M7e2tyWQypqWlJShBasHT05O9ETs6OkxTU9OnAKUkQCzLfgwUCSAARGDwaw9igFoYMZH4xKUsAeTECPzr66vNk/oBMUFsQBwJUMa5BmC7X38gJhKfuDiNBJZ7EVIdXOISWwAGPHkUi5nsc2oCwktSP7jExqkJIP5AAZRDAZRDAZQTeygYxZBkxOnx8dG8vLyY7u7uoETl2dvbMxcXF8FafTMwMGCmpqaCtcpzc3Nj/vz5Y1pbW+1wMN4PFBsOlngmSoCVlRVzcnISrNU3IyMjZmlpKVirPK4CsAlQDgVQDgVQDgVQDgVQDgVQDgVQDgVQDgVQjtcCTE9P20QKQwGUwyZAOd4J0NnZaWZnZ83y8nKwxdj8+Pi4aW5uDrYQwSsBxsbGzObmppmYmDBDQ0PBVmPzc3Nzdl9PT0+wlQBvBBgeHjYLCwvBWn7wCRq8UmZN8IU3AoQ7e8/Pz/Zd/OTkpJmZmTHHx8fBng8J2DH8whsBwlX70dGROT8/t3nIsL6+bq6vr+06GBwcDHLEu04gQNCjnJ2dBTljent7gxzxRoBw0DEfL8r29rZtEiSRD7wR4PT0NMh99PrX1tZY1cfAGwG2trbsF1YIqObR49/Y2OAYQBG8agIWFxc/O39CV1fX5xgAe///x6tOoDz+IYUf/YA8/pUaK9CGVwIIqAXw6IcxgN3d3W8dRIwWYqSQfOClAAICDwHm5+e/9Q9GR0eDHPFCAHT29vf3bUI+CkQINwkcB/jCCwHwNWkCRgTz9fj5FJAfLwQIf2AUnT30+sNgPABtvxAeFtaOFwIcHBx8a+MR7J2dHTsPAANCaBYghnB4eBjkiDedwNXV1W8SIOAYEYy29+gLRB8RNePVx8PRzuMRD3MDwm8H0QnEyyDc+dGBIoAmIuafwZ4rThOSlI+He/v9AAgqyiPgpQKBp4e4xDkf4PcDkETgbQ2AyaHoDKJfUKrNL+cdQZzzATYBFaAcAWoNmwCSCCiAciiAciiAciiAciiAchL1GMivii2MinEAUhiOAxAnKIByKIByKIByKIByKIByKIByKIByKIByKIByKIByKIByEiVAX19fwfTb/PSclfg/VYJECXB1dfWZ8q3/JpU4Zz3CJkA53giQrymQ9XzLcNnoOojul2W+cvm2JwUvBMAfX5qCfAEKV+fRssWODVOonGwrdXy94nUNABCYMKXWC1GsXKFrJwFvBECAJFUTBL4W1/0t2AlUjhcC4O6TarjaVXH42kmEs4I9gbOCiRMUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDkUQDllCyATDApNNCC1wyU2sQVgwJNHsZjJPqcmAAdTiPrDJS5OTUD4QuFf7SS1QWIQjU0cYk8KBSiay+XsxFBc9O7u7nPyIakdiAtikslk7O8lIiapVKpkXHBc2QLIxTAj+OHhwUpweXlpxbAnpAxVQ+72/v5+G/y2tjY7M7jUjGDBxuvfP7EFACiOYGezWVsL4IcUsRQBpAypHBJYLHGn467Hj2ZimU6nY939gpMASAg4Ph+AhBpBtpPqIXc57nh8FgBJgl8xAYAcgiVEkLzgcEpSBuHgSj5818cNPnASQJBDGfDa4hJ44UcCRKEI1cUl4FF+VQCSPPgySDkUQDXG/AeIg+5zeqs9mgAAAABJRU5ErkJggg==" } }, "cell_type": "markdown", "metadata": {}, "source": [ "\n", "If you want to change the code you can do it from the left panel. Select your file inside folder mpi and modify it (do not forget to save it with File/save). \n", "Do not forget that you will need to compile it. You can reuse the previous code above or run the same commands directlyl in the terminal.\n", "\n", "You can launch a terminal directly from jupyter launcher or just from docker (if from docker you can sue the `bash` command from docker to get a proper bash terminal). \n", "\n", "To do it from jupyter, start a Terminal in the Launcher tab. You can use the terminal to launchany command, including slurm jobs via commands.\n", "\n", "![image15.png](attachment:1e20047e-2be2-4a72-8991-ed7ef27df119.png)\n", "\n", "Try to change thenumber of MPI tasks when you run mpirun `-n 4` You do not need tocompile the program again for this.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "## MPI Point to Point Communication\n", "MPI point-to-point operations typically involve message passing between two, and only two, different MPI tasks. One task is performing a send operation and the other task is performing a matching receive operation. Different types of send and receive routines:\n", "\n", "- Synchronous send\n", "- Blocking send / blocking receive\n", "- Non-blocking send / non-blocking receive\n", "- Buffered send\n", "- Combined send/receive\n", "- \"Ready\" send\n", "\n", "Any type of send routine can be paired with any type of receive routine.\n", "\n", "### MPI Send / Receive\n", "\n", "MPI point-to-point communication routines generally have an argument list that takes one of the following formats:\n", "\n", "``MPI_Send (&buf,count,datatype,dest,tag,comm)\n", "MPI_SEND (buf,count,datatype,dest,tag,comm,ierr)``\n", "\n", "**Buffer**: Program address space that references the data that is to be sent or received. In most cases, this is simply the variable name that is be sent/received. For C programs, this argument is passed by reference and usually must be prepended with an ampersand: &var1.\n", "\n", "**Data Count**: Indicates the number of data elements of a particular type to be sent.\n", "\n", "**Data Type**: For reasons of portability, MPI predefines its elementary data types.\n", "\n", "- `MPI_CHAR` – signed char\n", "- `MPI_INT` – signed int\n", "- `MPI_FLOAT` – float\n", "- `MPI_DOUBLE` – double\n", "\n", "You can also create your own derived data types.\n", "\n", "**Destination**: An argument to send routines that indicates the process where a message should be delivered. Specified as the rank of the receiving process.\n", "\n", "**Source**: An argument to receive routines that indicates the originating process of the message. Specified as the rank of the sending process. This may be set to the wild card MPI_ANY_SOURCE to receive a message from any task.\n", "\n", "**Tag**: Arbitrary non-negative integer assigned by the programmer to uniquely identify a message. Send and receive operations should match message tags. For a receive operation, the wild card MPI_ANY_TAG can be used to receive any message regardless of its tag. The MPI standard guarantees that int can be used as tags, but most implementations allow a much larger range than this.\n", "\n", "**Communicator**: Indicates the communication context, or set of processes for which the source or destination fields are valid. Unless the programmer is explicitly creating new communicators, the predefined communicator MPI_COMM_WORLD is usually used.\n", "\n", "**Status**: For a receive operation, indicates the source of the message and the tag of the message. In C, this argument is a pointer to a predefined structure MPI_Status (ex. stat.MPI_SOURCE stat.MPI_TAG). Additionally, the actual number of bytes received are obtainable from Status via the MPI_Get_count routine.\n", "\n", "**Request**: Used by non-blocking send and receive operations. Since non-blocking operations may return before the requested system buffer space is obtained, the system issues a unique \"request number\". The programmer uses this system assigned \"handle\" later (in a WAIT type routine) to determine completion of the non-blocking operation. In C, this argument is a pointer to a predefined structure MPI_Request.\n", "\n", "Blocking sends: `MPI_Send(buffer,count,type,dest,tag,comm)` Non-blocking sends: `MPI_Isend(buffer,count,type,dest,tag,comm,request)` Blocking receive: `MPI_Recv(buffer,count,type,source,tag,comm,status)` Non-blocking receive: `MPI_Irecv(buffer,count,type,source,tag,comm,request)`\n", "\n", "**MPI_Send**:Basic blocking send operation. Routine returns only after the application buffer in the sending task is free for reuse.\n", "\n", "``MPI_Send (&buf,count,datatype,dest,tag,comm)\n", "MPI_SEND (buf,count,datatype,dest,tag,comm,ierr)\n", "MPI_Recv (&buf,count,datatype,source,tag,comm,&status)\n", "MPI_RECV (buf,count,datatype,source,tag,comm,status,ierr)``\n", "\n", "**Synchronous blocking send**:Send a message and block until the application buffer in the sending task is free for reuse and the destination process has started to receive the message.\n", "\n", "``MPI_Ssend (&buf,count,datatype,dest,tag,comm)\n", "MPI_SSEND (buf,count,datatype,dest,tag,comm,ierr)``\n", "\n", "**Buffered blocking send**:permits the programmer to allocate the required amount of buffer space into which data can be copied until it is delivered. Insulates against the problems associated with insufficient system buffer space.\n", "\n", "``MPI_Bsend (&buf,count,datatype,dest,tag,comm)\n", "MPI_BSEND (buf,count,datatype,dest,tag,comm,ierr)``\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## MPI Ping Pong Example\n", "\n", "In this exercise, we will write an MPI program that demonstrates the basic communication between two processes using MPI's `send` and `receive` functions. This is often referred to as the \"Ping Pong\" example because one process sends a message to another, which then sends it back, creating a back-and-forth communication pattern.\n", "\n", "### Introduction to MPI Send and Receive\n", "\n", "MPI's `send` and `receive` functions are fundamental for point-to-point communication in parallel applications. These functions allow processes to exchange data in a distributed system, enabling coordination and collaboration between different parts of a parallel program.\n", "\n", "In this example, we will implement a simple Ping Pong program where two processes exchange a message back and forth a certain number of times. This will help you understand the basics of message-passing in MPI and how data can be transferred between processes.\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MPI program written to mpi_pingpong.c.\n" ] } ], "source": [ "import os\n", "\n", "# Define the path for the MPI program file\n", "mpi_program_path = \"mpi_pingpong.c\"\n", "\n", "# Remove the existing file if it exists\n", "if os.path.exists(mpi_program_path):\n", " os.remove(mpi_program_path)\n", "\n", "# Create and write the MPI program\n", "mpi_program = \"\"\"\n", "#include \n", "#include \n", "\n", "#define PING_PONG_LIMIT 10\n", "\n", "int main(int argc, char** argv) {\n", " MPI_Init(NULL, NULL);\n", "\n", " int world_rank;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);\n", "\n", " int world_size;\n", " MPI_Comm_size(MPI_COMM_WORLD, &world_size);\n", "\n", " if (world_size != 2) {\n", " fprintf(stderr, \"World size must be two for %s\\\\n\", argv[0]);\n", " MPI_Abort(MPI_COMM_WORLD, 1);\n", " }\n", "\n", " int ping_pong_count = 0;\n", " int partner_rank = (world_rank + 1) % 2;\n", "\n", " while (ping_pong_count < PING_PONG_LIMIT) {\n", " if (world_rank == ping_pong_count % 2) {\n", " // Increment the ping_pong_count before you send it\n", " ping_pong_count++;\n", " MPI_Send(&ping_pong_count, 1, MPI_INT, partner_rank, 0, MPI_COMM_WORLD);\n", " printf(\"Process %d sent ping_pong_count %d to process %d\\\\n\", world_rank, ping_pong_count, partner_rank);\n", " } else {\n", " MPI_Recv(&ping_pong_count, 1, MPI_INT, partner_rank, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);\n", " printf(\"Process %d received ping_pong_count %d from process %d\\\\n\", world_rank, ping_pong_count, partner_rank);\n", " }\n", " }\n", "\n", " MPI_Finalize();\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Write the MPI program to a file\n", "with open(mpi_program_path, \"w\") as file:\n", " file.write(mpi_program)\n", "\n", "print(f\"MPI program written to {mpi_program_path}.\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now compiling and executing the code." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Compiling the MPI program...\n", "Compilation successful, executable created.\n", "Running the MPI program...\n", "Program Output:\n", " Process 0 sent ping_pong_count 1 to process 1\n", "Process 1 received ping_pong_count 1 from process 0\n", "Process 1 sent ping_pong_count 2 to process 0\n", "Process 0 received ping_pong_count 2 from process 1\n", "Process 0 sent ping_pong_count 3 to process 1\n", "Process 1 received ping_pong_count 3 from process 0\n", "Process 1 sent ping_pong_count 4 to process 0\n", "Process 0 received ping_pong_count 4 from process 1\n", "Process 0 sent ping_pong_count 5 to process 1\n", "Process 1 received ping_pong_count 5 from process 0\n", "Process 1 sent ping_pong_count 6 to process 0\n", "Process 0 received ping_pong_count 6 from process 1\n", "Process 0 sent ping_pong_count 7 to process 1\n", "Process 1 received ping_pong_count 7 from process 0\n", "Process 1 sent ping_pong_count 8 to process 0\n", "Process 0 received ping_pong_count 8 from process 1\n", "Process 0 sent ping_pong_count 9 to process 1\n", "Process 1 received ping_pong_count 9 from process 0\n", "Process 1 sent ping_pong_count 10 to process 0\n", "Process 0 received ping_pong_count 10 from process 1\n", "\n" ] } ], "source": [ "import subprocess\n", "\n", "# Compile the MPI program using mpicc\n", "compile_command = f\"mpicc -o mpi_pingpong mpi_pingpong.c\"\n", "compile_process = subprocess.run(compile_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output and error (if any) after compilation attempt\n", "print(\"Compiling the MPI program...\")\n", "if compile_process.stdout:\n", " print(\"Output:\", compile_process.stdout)\n", "if compile_process.stderr:\n", " print(\"Error:\", compile_process.stderr)\n", "\n", "# Check if the executable was created\n", "executable_path = \"mpi_pingpong\"\n", "if os.path.exists(executable_path):\n", " print(\"Compilation successful, executable created.\")\n", "else:\n", " print(\"Compilation failed.\")\n", "\n", "# Run the compiled program using mpirun with 2 processes\n", "run_command = f\"mpirun --oversubscribe -np 2 ./mpi_pingpong\"\n", "run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output of the program execution\n", "print(\"Running the MPI program...\")\n", "if run_process.stdout:\n", " print(\"Program Output:\\n\", run_process.stdout)\n", "if run_process.stderr:\n", " print(\"Program Error:\\n\", run_process.stderr)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "After compiling and running the MPI Ping Pong program, you should observe a series of messages being sent back and forth between two processes.\n", "\n", "### Key Points:\n", "\n", "1. **MPI Initialization**: The program begins by initializing the MPI environment with `MPI_Init`. This sets up the necessary environment for MPI communication.\n", "\n", "2. **Process Rank and World Size**: Each process obtains its rank using `MPI_Comm_rank` and the total number of processes using `MPI_Comm_size`. In this example, the program requires exactly two processes to demonstrate the Ping Pong communication.\n", "\n", "3. **Ping Pong Logic**: The processes take turns sending and receiving a message (an integer representing the ping-pong count). The process with rank 0 starts the exchange, sending the message to process 1. The message is then sent back and forth until the ping-pong count reaches a predefined limit.\n", "\n", "4. **MPI Send and Receive**: The `MPI_Send` function is used to send the message, and `MPI_Recv` is used to receive it. These functions are fundamental for point-to-point communication in MPI. The exchange of messages demonstrates how data can be transferred between processes in a parallel program.\n", "\n", "5. **Finalization**: The program concludes with `MPI_Finalize`, which cleans up the MPI environment and ends the parallel execution.\n", "\n", "### Observing the Communication:\n", "\n", "Each process prints a message each time it sends or receives the ping-pong count. This output allows you to trace the communication between the two processes, showing how they work together to increment and transfer the count until the limit is reached.\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## MPI Synchronous vs. Asynchronous Communication\n", "\n", "In this example, we will explore the difference between synchronous and asynchronous communication in MPI. \n", "\n", "### Synchronous Communication\n", "In synchronous communication, the `MPI_Ssend` function is used for sending messages. This function will not return until the matching `MPI_Recv` has begun, ensuring that both the sender and receiver are synchronized during the communication. \n", "\n", "### Asynchronous Communication\n", "In contrast, asynchronous communication uses `MPI_Isend` and `MPI_Irecv`, which return immediately without waiting for the corresponding receive or send operation to begin. This allows the program to overlap communication with computation, potentially improving performance in parallel applications.\n", "\n", "We will compare these two approaches by measuring the time taken to send and receive messages between two processes.\n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MPI program written to mpi_sync_async.c.\n" ] } ], "source": [ "import os\n", "\n", "# Define the path for the MPI program file\n", "mpi_program_path = \"mpi_sync_async.c\"\n", "\n", "# Remove the existing file if it exists\n", "if os.path.exists(mpi_program_path):\n", " os.remove(mpi_program_path)\n", "\n", "# Create and write the MPI program\n", "mpi_program = \"\"\"\n", "#include \n", "#include \n", "#include \n", "\n", "#define MESSAGE_SIZE 1000000\n", "\n", "void synchronous_communication(int world_rank, int partner_rank) {\n", " int *message = (int*)malloc(MESSAGE_SIZE * sizeof(int));\n", " if (world_rank == 0) {\n", " double start_time = MPI_Wtime();\n", " MPI_Ssend(message, MESSAGE_SIZE, MPI_INT, partner_rank, 0, MPI_COMM_WORLD);\n", " double end_time = MPI_Wtime();\n", " printf(\"Synchronous send time: %f seconds\\\\n\", end_time - start_time);\n", " } else if (world_rank == 1) {\n", " MPI_Recv(message, MESSAGE_SIZE, MPI_INT, partner_rank, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);\n", " }\n", " free(message);\n", "}\n", "\n", "void asynchronous_communication(int world_rank, int partner_rank) {\n", " int *message = (int*)malloc(MESSAGE_SIZE * sizeof(int));\n", " MPI_Request request;\n", " if (world_rank == 0) {\n", " double start_time = MPI_Wtime();\n", " MPI_Isend(message, MESSAGE_SIZE, MPI_INT, partner_rank, 0, MPI_COMM_WORLD, &request);\n", " MPI_Wait(&request, MPI_STATUS_IGNORE);\n", " double end_time = MPI_Wtime();\n", " printf(\"Asynchronous send time: %f seconds\\\\n\", end_time - start_time);\n", " } else if (world_rank == 1) {\n", " MPI_Irecv(message, MESSAGE_SIZE, MPI_INT, partner_rank, 0, MPI_COMM_WORLD, &request);\n", " MPI_Wait(&request, MPI_STATUS_IGNORE);\n", " }\n", " free(message);\n", "}\n", "\n", "int main(int argc, char** argv) {\n", " MPI_Init(NULL, NULL);\n", "\n", " int world_rank;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);\n", "\n", " int world_size;\n", " MPI_Comm_size(MPI_COMM_WORLD, &world_size);\n", "\n", " if (world_size != 2) {\n", " fprintf(stderr, \"World size must be two for %s\\\\n\", argv[0]);\n", " MPI_Abort(MPI_COMM_WORLD, 1);\n", " }\n", "\n", " int partner_rank = (world_rank + 1) % 2;\n", "\n", " // Perform synchronous communication\n", " synchronous_communication(world_rank, partner_rank);\n", "\n", " // Perform asynchronous communication\n", " asynchronous_communication(world_rank, partner_rank);\n", "\n", " MPI_Finalize();\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Write the MPI program to a file\n", "with open(mpi_program_path, \"w\") as file:\n", " file.write(mpi_program)\n", "\n", "print(f\"MPI program written to {mpi_program_path}.\")\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Compiling the MPI program...\n", "Compilation successful, executable created.\n", "Running the MPI program...\n", "Program Output:\n", " Synchronous send time: 0.002307 seconds\n", "Asynchronous send time: 0.001881 seconds\n", "\n" ] } ], "source": [ "import subprocess\n", "\n", "# Compile the MPI program using mpicc\n", "compile_command = f\"mpicc -o mpi_sync_async mpi_sync_async.c\"\n", "compile_process = subprocess.run(compile_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output and error (if any) after compilation attempt\n", "print(\"Compiling the MPI program...\")\n", "if compile_process.stdout:\n", " print(\"Output:\", compile_process.stdout)\n", "if compile_process.stderr:\n", " print(\"Error:\", compile_process.stderr)\n", "\n", "# Check if the executable was created\n", "executable_path = \"mpi_sync_async\"\n", "if os.path.exists(executable_path):\n", " print(\"Compilation successful, executable created.\")\n", "else:\n", " print(\"Compilation failed.\")\n", "\n", "# Run the compiled program using mpirun with 2 processes\n", "run_command = f\"mpirun --oversubscribe -np 2 ./mpi_sync_async\"\n", "run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output of the program execution\n", "print(\"Running the MPI program...\")\n", "if run_process.stdout:\n", " print(\"Program Output:\\n\", run_process.stdout)\n", "if run_process.stderr:\n", " print(\"Program Error:\\n\", run_process.stderr)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "After running the MPI program, you should observe the time taken for synchronous and asynchronous communication between the two processes.\n", "\n", "### Synchronous Communication\n", "\n", "- **Synchronous Send (`MPI_Ssend`)**: This function ensures that the sending process waits until the matching receive operation begins. The time reported by the program for this operation includes both the communication time and the waiting time until the receive operation starts.\n", "\n", "### Asynchronous Communication\n", "\n", "- **Asynchronous Send (`MPI_Isend`) and Receive (`MPI_Irecv`)**: These functions allow the sending and receiving operations to proceed without waiting for each other. The send operation returns immediately, allowing the program to perform other computations while the data is being sent. The communication time reported by the program may be shorter than that for synchronous communication, especially if the communication and computation overlap effectively.\n", "\n", "### Performance Comparison\n", "\n", "By comparing the time taken for synchronous and asynchronous communication, you can observe the potential performance benefits of using asynchronous communication, particularly in scenarios where computation and communication can be overlapped.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## MPI Point-to-Point vs. Collective Communication and Custom Data Types\n", "\n", "In this example, we will explore the difference between point-to-point and collective communication in MPI and introduce the concept of custom data types.\n", "\n", "### Point-to-Point Communication\n", "Point-to-point communication involves direct communication between two MPI processes. This includes `MPI_Send` and `MPI_Recv` for sending and receiving messages, respectively.\n", "\n", "### Collective Communication\n", "Collective communication involves communication among all processes in a communicator. Examples include `MPI_Bcast`, which broadcasts a message from one process to all other processes, and `MPI_Gather`, which gathers messages from all processes to a single process.\n", "\n", "### Custom Data Types\n", "MPI provides the ability to define custom data types, allowing complex data structures to be sent and received in a single MPI communication operation. This can be particularly useful when dealing with structured data such as arrays of structs.\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "MPI program written to mpi_point_to_point_collective.c.\n" ] } ], "source": [ "import os\n", "\n", "# Define the path for the MPI program file\n", "mpi_program_path = \"mpi_point_to_point_collective.c\"\n", "\n", "# Remove the existing file if it exists\n", "if os.path.exists(mpi_program_path):\n", " os.remove(mpi_program_path)\n", "\n", "# Create and write the MPI program\n", "mpi_program = \"\"\"\n", "#include \n", "#include \n", "#include \n", "\n", "// Define a custom data type for a struct containing an integer and a double\n", "typedef struct {\n", " int id;\n", " double value;\n", "} custom_data_t;\n", "\n", "void define_custom_data_type(MPI_Datatype* custom_data_type) {\n", " int lengths[2] = {1, 1};\n", " MPI_Aint offsets[2];\n", " offsets[0] = offsetof(custom_data_t, id);\n", " offsets[1] = offsetof(custom_data_t, value);\n", " MPI_Datatype types[2] = {MPI_INT, MPI_DOUBLE};\n", " MPI_Type_create_struct(2, lengths, offsets, types, custom_data_type);\n", " MPI_Type_commit(custom_data_type);\n", "}\n", "\n", "void point_to_point_communication(int world_rank, int world_size, MPI_Datatype custom_data_type) {\n", " custom_data_t data;\n", " if (world_rank == 0) {\n", " data.id = 1;\n", " data.value = 3.14;\n", " MPI_Send(&data, 1, custom_data_type, 1, 0, MPI_COMM_WORLD);\n", " printf(\"Process 0 sent data to Process 1: id = %d, value = %f\\\\n\", data.id, data.value);\n", " } else if (world_rank == 1) {\n", " MPI_Recv(&data, 1, custom_data_type, 0, 0, MPI_COMM_WORLD, MPI_STATUS_IGNORE);\n", " printf(\"Process 1 received data from Process 0: id = %d, value = %f\\\\n\", data.id, data.value);\n", " }\n", "}\n", "\n", "void collective_communication(int world_rank, int world_size, MPI_Datatype custom_data_type) {\n", " custom_data_t data;\n", " data.id = world_rank;\n", " data.value = world_rank * 2.0;\n", " \n", " custom_data_t* gathered_data = NULL;\n", " if (world_rank == 0) {\n", " gathered_data = (custom_data_t*)malloc(world_size * sizeof(custom_data_t));\n", " }\n", "\n", " MPI_Gather(&data, 1, custom_data_type, gathered_data, 1, custom_data_type, 0, MPI_COMM_WORLD);\n", "\n", " if (world_rank == 0) {\n", " printf(\"Process 0 gathered data from all processes:\\\\n\");\n", " for (int i = 0; i < world_size; i++) {\n", " printf(\"Process %d: id = %d, value = %f\\\\n\", i, gathered_data[i].id, gathered_data[i].value);\n", " }\n", " free(gathered_data);\n", " }\n", "}\n", "\n", "int main(int argc, char** argv) {\n", " MPI_Init(NULL, NULL);\n", "\n", " int world_rank;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);\n", "\n", " int world_size;\n", " MPI_Comm_size(MPI_COMM_WORLD, &world_size);\n", "\n", " if (world_size < 2) {\n", " fprintf(stderr, \"World size must be greater than one for %s\\\\n\", argv[0]);\n", " MPI_Abort(MPI_COMM_WORLD, 1);\n", " }\n", "\n", " MPI_Datatype custom_data_type;\n", " define_custom_data_type(&custom_data_type);\n", "\n", " // Perform point-to-point communication\n", " point_to_point_communication(world_rank, world_size, custom_data_type);\n", "\n", " // Perform collective communication\n", " collective_communication(world_rank, world_size, custom_data_type);\n", "\n", " MPI_Type_free(&custom_data_type);\n", " MPI_Finalize();\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Write the MPI program to a file\n", "with open(mpi_program_path, \"w\") as file:\n", " file.write(mpi_program)\n", "\n", "print(f\"MPI program written to {mpi_program_path}.\")\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Comile and execute the program" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Compiling the MPI program...\n", "Compilation successful, executable created.\n", "Running the MPI program...\n", "Program Output:\n", " Process 0 sent data to Process 1: id = 1, value = 3.140000\n", "Process 1 received data from Process 0: id = 1, value = 3.140000\n", "Process 0 gathered data from all processes:\n", "Process 0: id = 0, value = 0.000000\n", "Process 1: id = 1, value = 2.000000\n", "Process 2: id = 2, value = 4.000000\n", "Process 3: id = 3, value = 6.000000\n", "\n" ] } ], "source": [ "import subprocess\n", "\n", "# Compile the MPI program using mpicc\n", "compile_command = f\"mpicc -o mpi_point_to_point_collective mpi_point_to_point_collective.c\"\n", "compile_process = subprocess.run(compile_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output and error (if any) after compilation attempt\n", "print(\"Compiling the MPI program...\")\n", "if compile_process.stdout:\n", " print(\"Output:\", compile_process.stdout)\n", "if compile_process.stderr:\n", " print(\"Error:\", compile_process.stderr)\n", "\n", "# Check if the executable was created\n", "executable_path = \"mpi_point_to_point_collective\"\n", "if os.path.exists(executable_path):\n", " print(\"Compilation successful, executable created.\")\n", "else:\n", " print(\"Compilation failed.\")\n", "\n", "# Run the compiled program using mpirun with 4 processes\n", "run_command = f\"mpirun --oversubscribe -np 4 ./mpi_point_to_point_collective\"\n", "run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", "\n", "# Print the output of the program execution\n", "print(\"Running the MPI program...\")\n", "if run_process.stdout:\n", " print(\"Program Output:\\n\", run_process.stdout)\n", "if run_process.stderr:\n", " print(\"Program Error:\\n\", run_process.stderr)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Explanation of the Output\n", "\n", "### Point-to-Point Communication\n", "In this part of the program, process 0 sends a custom data structure containing an integer and a double to process 1. The program uses `MPI_Send` and `MPI_Recv` for this communication, which are typical point-to-point communication functions. The output will show that process 1 successfully received the data sent by process 0.\n", "\n", "### Collective Communication\n", "Here, each process prepares a custom data structure containing its rank and a value derived from its rank. The program uses `MPI_Gather` to collect these structures from all processes into a single array on process 0. This is an example of collective communication, where data from all processes is gathered into one process. The output will show the data from all processes gathered together on process 0.\n", "\n", "### Custom Data Types\n", "The custom data type used in this example allows the program to send and receive a complex structure (a struct containing an integer and a double) in a single MPI communication operation. This simplifies the code and makes the communication more efficient, especially when dealing with structured data.\n", "\n", "By running this program, you will see the practical differences between point-to-point and collective communication and understand how custom data types can be used to handle complex data structures in MPI.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to change the code you can do it from the left panel. Select your file inside folder mpi and modify it (do not forget to save it with File/save). Do not forget that you will need to compile it. You can reuse the previous code above or run the same commands directlyl in the terminal.\n", "\n", "You can launch a terminal directly from jupyter launcher or just from docker (if from docker you can sue the bash command from docker to get a proper bash terminal).\n", "\n", "To do it from jupyter, start a Terminal in the Launcher tab. You can use the terminal to launchany command, including slurm jobs via commands.\n", "\n", "\n", "This is the end of the MPI assignment! \n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.6" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": false, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false } }, "nbformat": 4, "nbformat_minor": 4 }