{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "___\n", "\n", " \n", " \n", "
\n", "
\n", " \n", "
Content Copyright by Pierian Data and xDM Consulting
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Exercices de Web Scraping \n", "\n", "## Réalisez les tâches suivantes" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tâche : Importer toutes les bibliothèques que vous pensez nécessaires pour \"scraper\" un site Web**" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "# VOTRE CODE ICI" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tâche : Utiliser les bibliothèques requests et BeautifulSoup pour vous connecter à http://quotes.toscrape.com/ et lire le code HMTL de la page de garde**" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# VOTRE CODE ICI" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": true }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'\\n\\n\\n\\t\\n\\tQuotes to Scrape\\n \\n \\n\\n\\n
\\n
\\n
\\n

\\n Quotes to Scrape\\n

\\n
\\n
\\n

\\n \\n Login\\n \\n

\\n
\\n
\\n \\n\\n
\\n
\\n\\n
\\n “The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”\\n by Albert Einstein\\n (about)\\n \\n
\\n Tags:\\n \\n \\n change\\n \\n deep-thoughts\\n \\n thinking\\n \\n world\\n \\n
\\n
\\n\\n
\\n “It is our choices, Harry, that show what we truly are, far more than our abilities.”\\n by J.K. Rowling\\n (about)\\n \\n
\\n Tags:\\n \\n \\n abilities\\n \\n choices\\n \\n
\\n
\\n\\n
\\n “There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”\\n by Albert Einstein\\n (about)\\n \\n
\\n Tags:\\n \\n \\n inspirational\\n \\n life\\n \\n live\\n \\n miracle\\n \\n miracles\\n \\n
\\n
\\n\\n
\\n “The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”\\n by Jane Austen\\n (about)\\n \\n
\\n Tags:\\n \\n \\n aliteracy\\n \\n books\\n \\n classic\\n \\n humor\\n \\n
\\n
\\n\\n
\\n “Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”\\n by Marilyn Monroe\\n (about)\\n \\n
\\n Tags:\\n \\n \\n be-yourself\\n \\n inspirational\\n \\n
\\n
\\n\\n
\\n “Try not to become a man of success. Rather become a man of value.”\\n by Albert Einstein\\n (about)\\n \\n
\\n Tags:\\n \\n \\n adulthood\\n \\n success\\n \\n value\\n \\n
\\n
\\n\\n
\\n “It is better to be hated for what you are than to be loved for what you are not.”\\n by André Gide\\n (about)\\n \\n
\\n Tags:\\n \\n \\n life\\n \\n love\\n \\n
\\n
\\n\\n
\\n “I have not failed. I've just found 10,000 ways that won't work.”\\n by Thomas A. Edison\\n (about)\\n \\n
\\n Tags:\\n \\n \\n edison\\n \\n failure\\n \\n inspirational\\n \\n paraphrased\\n \\n
\\n
\\n\\n
\\n “A woman is like a tea bag; you never know how strong it is until it's in hot water.”\\n by Eleanor Roosevelt\\n (about)\\n \\n
\\n Tags:\\n \\n \\n misattributed-eleanor-roosevelt\\n \\n
\\n
\\n\\n
\\n “A day without sunshine is like, you know, night.”\\n by Steve Martin\\n (about)\\n \\n
\\n Tags:\\n \\n \\n humor\\n \\n obvious\\n \\n simile\\n \\n
\\n
\\n\\n \\n
\\n
\\n \\n

Top Ten tags

\\n \\n \\n love\\n \\n \\n \\n inspirational\\n \\n \\n \\n life\\n \\n \\n \\n humor\\n \\n \\n \\n books\\n \\n \\n \\n reading\\n \\n \\n \\n friendship\\n \\n \\n \\n friends\\n \\n \\n \\n truth\\n \\n \\n \\n simile\\n \\n \\n \\n
\\n
\\n\\n
\\n \\n\\n'" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tâche : Récupérer tous les noms des auteurs de la première page**" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# VOTRE CODE ICI" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'Albert Einstein',\n", " 'André Gide',\n", " 'Eleanor Roosevelt',\n", " 'J.K. Rowling',\n", " 'Jane Austen',\n", " 'Marilyn Monroe',\n", " 'Steve Martin',\n", " 'Thomas A. Edison'}" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "authors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tâche : Créer une liste de toutes les citations de la première page**" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# VOTRE CODE ICI" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['“The world as we have created it is a process of our thinking. It cannot be changed without changing our thinking.”',\n", " '“It is our choices, Harry, that show what we truly are, far more than our abilities.”',\n", " '“There are only two ways to live your life. One is as though nothing is a miracle. The other is as though everything is a miracle.”',\n", " '“The person, be it gentleman or lady, who has not pleasure in a good novel, must be intolerably stupid.”',\n", " \"“Imperfection is beauty, madness is genius and it's better to be absolutely ridiculous than absolutely boring.”\",\n", " '“Try not to become a man of success. Rather become a man of value.”',\n", " '“It is better to be hated for what you are than to be loved for what you are not.”',\n", " \"“I have not failed. I've just found 10,000 ways that won't work.”\",\n", " \"“A woman is like a tea bag; you never know how strong it is until it's in hot water.”\",\n", " '“A day without sunshine is like, you know, night.”']" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "citations" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tâche : inspecter le site et utiliser Beautiful Soup pour extraire les 10 premiers Tags afichés en haut et à droite de la page d'accueil (par.ex. Love,Inspirational,Life, etc...). \n", "INDICE : cherchez le marqueur qui se cache derrière chaque citation, essayer d'identifier une classe uniquement présente en haut à droite de la page, cherchez ausssi du côté du span**" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# VOTRE CODE ICI" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "love\n", "\n", "\n", "inspirational\n", "\n", "\n", "life\n", "\n", "\n", "humor\n", "\n", "\n", "books\n", "\n", "\n", "reading\n", "\n", "\n", "friendship\n", "\n", "\n", "friends\n", "\n", "\n", "truth\n", "\n", "\n", "simile\n", "\n" ] } ], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Tâche: Remarquez qu'il y a plus d'une page et que les pages suivantes ressemblent à ceci http://quotes.toscrape.com/page/2/. Utilisez ce que vous savez sur les boucles for et la concaténation de chaînes pour parcourir toutes les pages et obtenir la liste des auteurs présents sur le site Web. Gardez à l'esprit qu'il existe de nombreuses façons d'y parvenir, notez également que vous devrez d'une manière ou d'une autre déterminer comment vérifier que votre boucle est sur la dernière page avec des citations. À des fins de débogage, je vous fait remarquer qu'il n'y a que 10 pages, donc la dernière est http://quotes.toscrape.com/page/10/, mais essayez de créer une boucle suffisamment robuste pour qu'elle n'aie pas besoin de connaître le nombre de pages à l'avance, vous pouvez par exemple utiliser try/except pour cela. C'est à vous de jouer !**" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "# VOTRE CODE ICI" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.7" } }, "nbformat": 4, "nbformat_minor": 2 }