{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Web-Scraping - pokemon\n", "\n", "Il faut récupérer automatiquement des images de pokémon depuis le site [pokemondb.net](https://pokemondb.net/pokedex/national)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Pour cet exercice, nous vous demandons d'obtenir 1) les informations personnelles des 721 pokemons sur le site internet [pokemondb.net](https://pokemondb.net/pokedex/national). Les informations que nous aimerions obtenir au final pour les pokemons sont celles contenues dans 4 tableaux :\n", "\n", "- Pokédex data\n", "- Training\n", "- Breeding\n", "- Base stats\n", "\n", "Pour exemple : [Pokemon Database](https://pokemondb.net/pokedex/nincada).\n", "\n", "2) Nous aimerions que vous récupériez également les images de chacun des pokémons et que vous les enregistriez dans un dossier (indice : utilisez les modules request et [shutil](https://docs.python.org/3/library/shutil.html))\n", "_pour cette question ci, il faut que vous cherchiez de vous même certains éléments, tout n'est pas présent dans le TD_." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Correction" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import collections\n", "import urllib.request\n", "import bs4\n", "import pandas as pd\n", "\n", "# pour le site que nous utilisons, le user agent de python 3 n'est pas bien passé :\n", "# on le change donc pour celui de Mozilla\n", "\n", "req = urllib.request.Request(\n", " \"http://pokemondb.net/pokedex/national\", headers={\"User-Agent\": \"Mozilla/5.0\"}\n", ")\n", "html = urllib.request.urlopen(req).read()\n", "page = bs4.BeautifulSoup(html, \"lxml\")\n", "\n", "# récupérer la liste des noms de pokémon\n", "\n", "liste_pokemon = []\n", "for pokemon in page.findAll(\"span\", {\"class\": \"infocard-lg-img\"}):\n", " pokemon = pokemon.find(\"a\").get(\"href\").replace(\"/pokedex/\", \"\")\n", " liste_pokemon.append(pokemon)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fonction pour obtenir les caractéristiques de pokemons" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0 bulbasaur\n", "1 ivysaur\n", "2 venusaur\n", "3 charmander\n", "4 charmeleon\n", "5 charizard\n", "6 squirtle\n" ] }, { "data": { "text/html": [ "
\n", " | National № | \n", "name | \n", "Type | \n", "Species | \n", "Height | \n", "Weight | \n", "Abilities | \n", "Local № | \n", "EV yield | \n", "Catch rate | \n", "... | \n", "Growth Rate | \n", "Egg Groups | \n", "Gender | \n", "Egg cycles | \n", "HP | \n", "Attack | \n", "Defense | \n", "Sp. Atk | \n", "Sp. Def | \n", "Speed | \n", "
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
0 | \n", "0001 | \n", "bulbasaur | \n", "Grass Poison | \n", "Seed Pokémon | \n", "0.7 m (2′04″) | \n", "6.9 kg (15.2 lbs) | \n", "1. OvergrowChlorophyll (hidden ability) | \n", "0001 (Red/Blue/Yellow)0226 (Gold/Silver/Crysta... | \n", "1 Sp. Atk | \n", "45 (5.9% with PokéBall, full HP) | \n", "... | \n", "Medium Slow | \n", "Grass, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884–5,140 steps) | \n", "45 | \n", "49 | \n", "49 | \n", "65 | \n", "65 | \n", "45 | \n", "
1 | \n", "0002 | \n", "ivysaur | \n", "Grass Poison | \n", "Seed Pokémon | \n", "1.0 m (3′03″) | \n", "13.0 kg (28.7 lbs) | \n", "1. OvergrowChlorophyll (hidden ability) | \n", "0002 (Red/Blue/Yellow)0227 (Gold/Silver/Crysta... | \n", "1 Sp. Atk, 1 Sp. Def | \n", "45 (5.9% with PokéBall, full HP) | \n", "... | \n", "Medium Slow | \n", "Grass, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884–5,140 steps) | \n", "60 | \n", "62 | \n", "63 | \n", "80 | \n", "80 | \n", "60 | \n", "
2 | \n", "0003 | \n", "venusaur | \n", "Grass Poison | \n", "Seed Pokémon | \n", "2.0 m (6′07″) | \n", "100.0 kg (220.5 lbs) | \n", "1. OvergrowChlorophyll (hidden ability) | \n", "0003 (Red/Blue/Yellow)0228 (Gold/Silver/Crysta... | \n", "2 Sp. Atk, 1 Sp. Def | \n", "45 (5.9% with PokéBall, full HP) | \n", "... | \n", "Medium Slow | \n", "Grass, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884–5,140 steps) | \n", "80 | \n", "82 | \n", "83 | \n", "100 | \n", "100 | \n", "80 | \n", "
3 | \n", "0004 | \n", "charmander | \n", "Fire | \n", "Lizard Pokémon | \n", "0.6 m (2′00″) | \n", "8.5 kg (18.7 lbs) | \n", "1. BlazeSolar Power (hidden ability) | \n", "0004 (Red/Blue/Yellow)0229 (Gold/Silver/Crysta... | \n", "1 Speed | \n", "45 (5.9% with PokéBall, full HP) | \n", "... | \n", "Medium Slow | \n", "Dragon, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884–5,140 steps) | \n", "39 | \n", "52 | \n", "43 | \n", "60 | \n", "50 | \n", "65 | \n", "
4 | \n", "0005 | \n", "charmeleon | \n", "Fire | \n", "Flame Pokémon | \n", "1.1 m (3′07″) | \n", "19.0 kg (41.9 lbs) | \n", "1. BlazeSolar Power (hidden ability) | \n", "0005 (Red/Blue/Yellow)0230 (Gold/Silver/Crysta... | \n", "1 Sp. Atk, 1 Speed | \n", "45 (5.9% with PokéBall, full HP) | \n", "... | \n", "Medium Slow | \n", "Dragon, Monster | \n", "87.5% male, 12.5% female | \n", "20 (4,884–5,140 steps) | \n", "58 | \n", "64 | \n", "58 | \n", "80 | \n", "65 | \n", "80 | \n", "
5 rows × 22 columns
\n", "