Add trafilatura

book/machinelearning/getdata.ipynb

@@ -206,6 +206,48 @@
     "\n",
     "df = clevercsv.read_dataframe('imdb.csv')"
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Powerful Web Text Gathering with `trafilatura`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Do you need a powerful text extractor on the web?\n",
+    "\n",
+    "Try `trafilatura`!\n",
+    "\n",
+    "It’s a Python package for Web Crawling, Downloads and Scraping of text, metadata and comments from websites.\n",
+    "\n",
+    "Trafilatura supports different output formats like JSON, XML and CSV."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install trafilatura"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from trafilatura import fetch_url, extract\n",
+    "\n",
+    "downloaded = fetch_url('https://github.com/adbar/trafilatura')\n",
+    "\n",
+    "result = extract(downloaded, output_format=\"xml\")"
+   ]
   }
  ],
  "metadata": {