Skip to content

Latest commit

 

History

History
86 lines (56 loc) · 3.12 KB

README.md

File metadata and controls

86 lines (56 loc) · 3.12 KB
Raw

Project 3: Feature Selection + Classification

Domain and Data

You're working as a data scientist with a research firm. You're firm is bidding on a big project that will involve working with thousands or possibly tens of thousands of features. You know it will be impossible to use conventional feature selection techniques. You propose that a way to win the contract is to demonstrate a capacity to identify relevant features using machine learning. Your boss says, "Great idea. Write it up." You figure that working with a synthetic dataset such as Madelon is an excellent way to demonstrate your abilities.

Requirement

This work must be done on AWS.

Problem Statement

Your challenge here is to develop a series of models for two purposes:

  1. for the purposes of identifying relevant features.
  2. for the purposes of generating predictions from the model.

Solution Statement

Your final product will consist of:

  1. A prepared report
  2. A series of Jupyter notebooks to be used to control your pipelines

Tasks

Data Manipulation

You should do substantive work on at least six subsets of the data.

  • 3 sets of 10% of the data from the UCI Madelon set
  • 3 sets of 10% of the data from the Madelon set made available by your instructors
Prepared Report

Your report should:

  1. be a pdf
  2. include EDA of each subset
    • EDA needs may be different depending upon subset or your approach to a solution
  3. present results from Step 1: Benchmarking
  4. present results from Step 2: Identify Salient Features
  5. present results from Step 3: Feature Importances
  6. present results from Step 4: Build Model
Jupyter Notebook, EDA
  • perform EDA on each set as you see necessary
Jupyter Notebook, Step 1 - Benchmarking
  • build pipeline to perform a naive fit for each of the base model classes:
    • logistic regression
    • decision tree
    • k nearest neighbors
    • support vector classifier
  • in order to do this, you will need to set a high C value in order to perform minimal regularization, in the case of logistic regression and support vector classifier.
Jupyter Notebook, Step 2 - Identify Features
  • Build feature selection pipelines using at least three different techniques
  • NOTE: these pipelines are being used for feature selection not prediction
Jupyter Notebook, Step 3 - Feature Importance
  • Use the results from step 2 to discuss feature importance in the dataset
  • Considering these results, develop a strategy for building a final predictive model
  • recommended approaches:
    • Use feature selection to reduce the dataset to a manageable size then use conventional methods
    • Use dimension reduction to reduce the dataset to a manageable size then use conventional methods
    • Use an iterative model training method to use the entire dataset
Jupyter Notebook, Step 4 - Build Model
  • Implement your final model
  • (Optionally) use the entire data set

Requirements

  • Many Jupyter Notebooks
  • A written report of your findings that detail the accuracy and assumptions of your model.

Suggestions

  • Document everything.