follow the insrtuction
Group Project
Mostafa Rezaei
Big Data (Introduction to Data Science)
General information
The group project gives you the opportunity to practice many of the skills we learned in the class.
It includes 5 steps:
Step 1: Find and describe a data set
Find a publicly available data set. The data set should not be from the UCI ML Repository or any other
data set commonly used in ML competitions. Open Data Initiative websites are also good places to find
data sets, for example:
•
•
•
Once you find a dataset, you should figure out:
• the individual or organization that created it;
• the purpose of its creation;
• its terms of use;
• how the data was collected and the sampling procedure;
• the definitions of the variables and their units.
Step 2: Perform an initial EDA
Perform an initial EDA, where you create plots and group summaries to understand the variation of each
variable, including typical values, clusters, outliers, missing values, etc.
Step 3: Perform an in-depth EDA
In this step, you should perform an in-depth EDA in order to discover interesting covariations and patterns
in the data.
As discussed in the class, you should go through an iterative cycle of asking questions about the data and
finding answers to the questions using data transformation and visualization. Investigate the answers you
obtain with curiosity and skepticism and follow-up with further (more detailed) questions.
1
Step 4: Build a prediction model
In the final step, you should think about an interesting prediction problem using your dataset. Think about
the details of the prediction model, including
• the response variable and predictor variables;
• the evaluation metrics;
• how you will conduct CV to estimate the out-of-sample performance of the model and to tune the
hyper-parameters of the model.
Step 5: Present your findings!
The value of your research is limited if you keep it to yourself. So in this step you will polish your most
interesting findings in a presentations for the world to see. See below for details.
Deliverables
1. Create two plots that visually represent your most notable findings from your EDA:
• The plots should be created using ggplot2
• Ensuring they are polished and self-contained with meaningful titles, subtitles, labels, and captions
• Save each plot separately as an PNG or PDF file
• For preparing plots for communication, see
2. Build a prediction model and calculate its out-of-sample performance
• Specify the response and predictor variables
• Specify the evaluation metrics used
• Specify how you perform CV to obtain an estimate of its out-of-sample performance and to tune
its hyper-parameters
3. Provide a R Notebook file (with an extension .Rmd) containing your code and code outputs, such as
plots and tables
• Output a HTML file from the notebook, ensuring it correctly displays all code and outputs
• Use minimal comments in your code and follow the Tidyverse style guide:
org/index.html
• On the first line of the R Notebook, briefly list the contributions of each team member
4. Record a 5-minute presentation of your work
• Create a 5-slide presentation
• Slide 1: Contextual information about the dataset
• Slide 2: A detailed description of the dataset
• Slides 3 and 4: The two plots showcasing your primary EDA findings
• Slide 5: Information and results of your trained prediction model
• Your recorded presentation should not exceed 5 minutes
Submission details
• Upload your work on the dedicated assignment for the group project on BB
• Only one person per group needs to submit their group’s work
• Compress all your files into a ZIP file, containing
2
– the 2 PNG or PDF files of the EDA plots
– Your R Notebook, with an extension .Rmd
– The HTML file created from your R Notebook
– Your presentation file in PDF format
– The recording of your presentation
3
- General information
- Step 1: Find and describe a data set
- Step 2: Perform an initial EDA
- Step 3: Perform an in-depth EDA
- Step 4: Build a prediction model
- Step 5: Present your findings!
- Deliverables
- Submission details
