One paragraph project proposal, a final project

The final project requires each student to perform data analysis/data science tasks of a data set selected by the student self-using Python. Each student will produce a Project Proposal, and a Final Project .ipynb Jupyter Notebook file (which is a written report detailing the analysis techniques and the findings of the project + Python codes)

Data Set: The data set may be your own or may be obtained from an external source. The newer the data set is, the better. The data set should have enough features (ideally more than 5 features) and observations (ideally more than 700 observations) so you have enough data to perform the necessary analyses. Here are some links where you may find good datasets:
• https://toolbox.google.com/datasetsearch
• https://aws.amazon.com/fr/datasets/
• https://www.kaggle.com/datasets
• http://archive.ics.uci.edu/ml/index.php

Analyses: You must use Python that we learn and practice in this course to conduct all your analyses. In your analyses, you should cover at least three of the analyses topics we talk about in this course. However, covering 4 to 5 topics is recommended in order to gain excellent points in Completeness and Thoroughness (Please see GRADING on the last page). They include:
• Data preprocessing: how to clean, aggregate, match the raw data sets and how to transform, clean different variables
• Descriptive analysis: summary/descriptive statistics for your data
• Data visualization: different tables, charts, graphs to help the audience better understand your analyses and your findings
• Statistical analysis: hypothesis testing and testing for differences between groups and for predictive relationships
• Predictive analysis: predictive models (such as linear regression)
• Design a tool that contains one function or multiple functions imported from one .py file to hide your code. You should also include an instruction for other users to follow.

Citation format: You must include the proper citation in both the final project presentation and notebook file. You are allowed to use the citation formatting that you prefer for this project. Your project may be checked by the tool for plagiarism. If you fail to use proper citation, or the project you submit contains more than 30% exact wording from other sources, you may receive a failing score for your project.

DETAILED GUIDELINE

1. One paragraph Project Proposal – Due Date: Wednesday, 3/30/2022, 6 pm (New York Time)

You will need to turn in a 1–page paper (SINGLE spaced) proposal (Word/PDF). The proposal should have the following information:
• Data: 1 paragraph telling the data source, and important features of the data, links to the data set, sample size, what was sampled, year collected, covariates you will use, and why you choose this data set
• List your initial analysis plan

2. Final Project Python files

Each student will need to turn in your Jupyter Notebook file and any supporting materials (i.e. .py Python file if you design a tool) called “YourFullName _final_project”. The Jupite Notebook file should contain (1) the data analysis/data science code you build (execute all of them) (2) the explanation of why you conduct such tasks (3) findings based on the tasks. They should be well organized as a report document with clear headers and sections. I have offered one way to organize your Jupyter Notebook below, but feel free to do what works best for you. You need to make sure that the organization is easy to follow. • Introduction/background—A brief (approximately one page) general description of the problem, including:
o Why the problem is of interest—you might refer to previous studies using this, or similar
data sets.

• Data/Data Preprocessing:
o Information about the data set (e.g, the source of the data, the year the data got collected, how many observations, number of variables, etc.)

o If you make any change to the raw data set, you should describe how you change it and why you change it as part of data preprocessing (e.g. you delete the observations with missing values, you select random samples in the data set, etc.)

• Exploratory analyses —This section should provide the reader (me) with graphical and numerical summaries and results of the data, paying special attention to summaries that provide evidence for the results you’ve mentioned in the introduction.
o Descriptive statistics of the variables you are interested in
o Data visualizations of the variables you are interested in
o You should explain both the processes and the meanings of conducting the descriptive statistics and visualizations

• Methods —The methods section should expand the description of the methods used. Topics that should be covered in this section include:
o If appropriate, explicitly define any tests/models used in your analysis (e.g., significance tests, linear regression, etc.). Make sure to state why you think the method(s) is (are) appropriate for the data.
o Discuss and evaluate the assumptions of the method(s) used. If your data does not quite conform to the assumptions, make note of it, and discuss the implications.
o State the hypotheses you are testing and also state which testing procedure(s) you are using. For example, “I perform a hypothesis test to determine whether the mean response vector varies by display shelf, with the p–value statistic.”

• Detailed Results—This section expands the explanation of the results, and includes, where appropriate, tables and figures providing evidence for the conclusions you’ve stated. You can also report any secondary results you’ve found.

• Discussion—Summarize the findings one last time, paying close attention to the limitations of the analysis. You can share thoughts with the reader about how you might expand the study, improve on the model you’ve used, and what are the long–term implications of the findings.

• Application of your package tool – User instruction + code