- Understand when and why you might train your own model from scratch versus use a pre-trained model or transfer learning.
- Learn about the Google “Quick, Draw!” dataset.
- Understand how to work with image data for training your own model.
- Data and Documentation
- Video tutorial: Replaying Drawings with node server
- Video tutorial: Replaying Drawings with Google Web API
- Preparing Data as Images for Doodle Classifer Part 1
- Preparing Data as Images for Doodle Classifer Part 2
- Displaying MNIST (no web editor version)
- Displaying Quick, Draw! 28x28 images
- Animating Quick, Draw! paths
- Exploring and Visualizing an Open Global Dataset by Google Research
- Letter collages by Deborah Schmidt
- Face tracking experiment by Neil Mendoza
- Faces of Humanity by Tortue
- Scribbling Speech by Xinyue Yang
- How do you draw a circle? by Quartz
- Machine Learning for Visualization - Talk / article by Ian Johnson
- MegaPixels: Faces curated by Tactical Tech, design and development by Adam Harvey
- Watch What Neural Networks See by Gene Kogan
- Interactive Node-Link Visualizations of Convolutional Neural Networks
- Recognizing Human Facial Expressions With Machine Learning by Angelica Perez
- Learn to train an image classifier (no convolutional layers) with ml5.js.
- Learn the distinction between different types of layers of a neural network, specifically “What is a convolutional layer?”
- Learn to feed the input of a graphics canvas into a machine learning model.
- Original 1998 "LetNet5" paper: "Gradient-Based Learning Applied to Document Recognition" by Y. Lecun, L. Bottou, Y. Bengio, P. Haffner
- How computers got shockingly good at recognizing images by Timothy B. Lee
- Image Kernels Explained Visually by Victor Powell
- An Intuitive Explanation of Convolutional Neural Networks by Ujjwal Karn
- A visual and intuitive understanding of deep learning, CNNs (0:00 - 9:40) by Octavio Good
- Convolution p5.js demo
- Training a model with
ml5.neuralNetwork()and Google Quick, Draw! images - Classifying Drawings with ml5's DoodleNet (model trained by @yining1023)
- @ellennickles summary of Datasheets for Datasets paper.
- A Duke study recorded thousands of students’ faces by Jake Satisky
- Atlanta Asks Google Whether It Targeted Black Homeless People by Jack Nicas, NY Times
- An Intuitive Explanation of Convolutional Neural Networks by Ujjwal Karn
- AI ‘Emotion Recognition’ can’t be Trusted by James Vincent
- Spend a few minutes playing Quick, Draw! and reviewing the documentation of the Quick, Draw! dataset. Put yourself in the shoes of the researchers at Google collecting the data and answer the following questions from the Datasheets for Datasets to the best of your ability. If you prefer, find another dataset to analyze (the datasets described in the above readings are good choices as well.)
- Does the dataset contain data that might be considered confidential?
- Does the dataset contain data that, if viewed directly, might be offensive, insulting, threatening, or might otherwise cause anxiety?
- Does the dataset identify any subpopulations of people (e.g., by age, gender)?
- Were individuals (e.g. players of Quick, Draw!) notified about the data collection?
- Did these individuals in question consent to the collection and use of their data?
- If consent was obtained, were the consenting individuals provided with a mechanism to revoke their consent in the future or for certain uses?
- Are there tasks for which the dataset should not be used?
For this week's coding exercise you can build on top of your neural network assignment from last week. Make sure to bring in the latest ml5 build to your code. Here are some ideas (note I am not suggesting you do all of them, this is just a list to consider, invent your own!)
- If your sketch was not working, try fixing bugs with the new ml5.js version.
- If "classification" is more appropriate for your sketch, switch your
taskto'classification'. - Consider adding functionality to save training data or save the model (reloading it later). A working example of saving/loading is now available in the week 5 material.
Instead of building on your previous assignment, you can also choose to create a new sketch inspired by the examples around classifying drawings and convolutional neural networks. For example:
- Can you train your own image classifier with
ml5.neuralNetwork()? You can build upon the doodle classification example. This example assumes the image data is loaded via binary files. How might you work with actual image files? - Can you invent a user interaction around classifying drawings building on the ml5 DoodleNet classification example.