This repository contains the materials for D-Lab's Python Text Analysis. Prior experience with Python Fundamentals, Python Data Wrangling, and Python Machine Learning Fundamentals is assumed.
In this workshop, we'll focus fundamental approaches to applying computational
methods to text in Python. We'll cover some of the major packages used in natural language processing, including scikit-learn, nltk, spacy, and gensim.
This workshop is divided into the following parts:
- Part 1: Preprocessing Text. How do we standardize and clean text documents?
- Part 2: Bag-of-words Representations. How do we convert text into a representation that we can operate on computationally?
- Part 3: Topic Modeling. How do we identify topics within a corpus of documents?
- Part 4: Word Embeddings. How can we use neural networks to create meaningful representations of words?
The first two parts are taught as a joint series. Parts 3 and 4 can be attended "a la carte"; however, prior knowledge of Parts 1 and 2 is assumed.
Anaconda is a useful package management software that allows you to run Python and Jupyter notebooks very easily. Installing Anaconda is the easiest way to make sure you have all the necessary software to run the materials for this workshop. Complete the following steps:
-
Download and install Anaconda (Python 3.8 distribution). Click "Download" and then click 64-bit "Graphical Installer" for your current operating system.
-
Download the Python-Text-Analysis workshop materials:
- Click the green "Code" button in the top right of the repository information.
- Click "Download Zip".
- Extract this file to a folder on your computer where you can easily access it (we recommend Desktop).
- Optional: if you're familiar with
git, you can instead clone this repository by opening a terminal and enteringgit clone [email protected]:dlab-berkeley/Python-Text-Analysis.git.
Now that you have all the required software and materials, you need to run the code:
-
Open the Anaconda Navigator application. You should see the green snake logo appear on your screen. Note that this can take a few minutes to load up the first time.
-
Click the "Launch" button under "Jupyter Notebooks" and navigate through your file system to the
Python-Text-Analysisfolder you downloaded above. -
Go to the
lessonsfolder and find the notebook corresponding to the workshop you are attending. -
Press Shift + Enter (or Ctrl + Enter) to run a cell.
-
You will need to install additional packages depending on which workshop you are attending.
Note that all of the above steps can be run from the terminal, if you're familiar with how to interact with Anaconda in that fashion. However, using Anaconda Navigator is the easiest way to get started if this is your first time working with Anaconda.
If you do not have Anaconda installed and the materials loaded on your workshop by the time it starts, we strongly recommend using the UC Berkeley Datahub to run the materials for these lessons. You can access the DataHub by clicking the following button:
The DataHub downloads this repository, along with any necessary packages, and allows you to run the materials in a Jupyter notebook that is stored on UC Berkeley's servers. No installation is necessary from your end - you only need an internet browser and a CalNet ID to log in. By using the DataHub, you can save your work and come back to it at any time. When you want to return to your saved work, just go straight to DataHub, sign in, and you click on the Python-Text-Analysis folder.
If you don't have a Berkeley CalNet ID, you can still run these lessons in the cloud, by clicking this button:
By using this button, however, you cannot save your work.
- CTAWG (Computational Text Analsysis Working Group) website
- Lectures from Stanford's NLP class
- Info 256 - Applied NLP class by David Bamman
D-Lab works with Berkeley faculty, research staff, and students to advance data-intensive social science and humanities research. Our goal at D-Lab is to provide practical training, staff support, resources, and space to enable you to use R for your own research applications. Our services cater to all skill levels and no programming, statistical, or computer science backgrounds are necessary. We offer these services in the form of workshops, one-to-one consulting, and working groups that cover a variety of research topics, digital tools, and programming languages.
Visit the D-Lab homepage to learn more about us. You can view our calendar for upcoming events, learn about how to utilize our consulting and data services, and check out upcoming workshops.
Here are other Python workshops offered by the D-Lab:
- Computational Text Analysis in Python
- Introduction to Machine Learning in Python
- Introduction to Artificial Neural Networks in Python
- Fairness and Bias in Machine Learning
- Pratik Sachdeva
- Ben Gebre-Medhin
- Laura Nelson
- Teddy Roland
- Geoff Bacon
- Caroline Le Pennec-Caldichoury
These materials have evolved over a number of years. They were first developed for the D-Lab by Laura Nelson & Teddy Roland, with contributions and revisions made by Ben Gebre-Medhin, Geoff Bacon and updated by Caroline Le Pennec-Caldichoury. They were heavily revamped by Pratik Sachdeva in 2022.