Run in Google Colab

View on GitHub

Why Visualization?¶

In the class, we explored the Anscombe's Quartet. This dataset is composed of four different datasets that have the same statistical properties. However, when we visualize the data, we can see that they are quite different. Illustrate this concept by yourself using the Anscombe's Quartet dataset.

In addition to the Anscombe's dataset and without giving too much away, let's also explore a very particular dataset for a simple statistics exercise.

This notebook can be submitted together with assignment 3 for bonus points.

Save it as a HTML or PDF file and submit it on the assignment page on Canvas.

Anscombe's Quartet dataset¶

In [ ]:

import pandas as pd
import numpy as np

# Load the dataset
data = pd.read_csv('../../Datasets/Anscombe_quartet_data.csv')
# The ../../ are needed to go back two levels in the directory structure.
# Note that the path is relative to the location of the notebook file.

import pandas as pd import numpy as np # Load the dataset data = pd.read_csv('../../Datasets/Anscombe_quartet_data.csv') # The ../../ are needed to go back two levels in the directory structure. # Note that the path is relative to the location of the notebook file.

Explore the dataset and calculate the mean and standard deviation for each dataset.

In [ ]:

# Explore here

# Explore here

Also calculate the correlation between the x123 and y1,y2,y3 variables. Also do the same for the x4 and y4 variables.

In [ ]:

# Calculate correlations here

# Calculate correlations here

Now, showcase the advantages of visualization by plotting the data. You can use any visualization library you want.

For instance in matplotlib you can use the following code to plot data:

import matplotlib.pyplot as plt # Import the library

fig = plt.figure(figsize=(12, 6)) # Create a figure object
x1 = [2,4,6,8,10] # Example x values
y1 = [4,4,4,8,8] # Example y values
plt.plot(x1, y1, 'o') # Plot the data
plt.show() # Show the plot

Check out the documentation on how to create a matrix of 2x2 plots with the data from the Anscombe's Quartet dataset. Check out subplots in matplotlib documentation

In [ ]:

# Plot the data here

# Plot the data here

Simple statistics exercise¶

Load the data, each row contains for one person the number of steps that this person took on a particular day (steps), the body mass index (bmi) and gender (male or female). Assume that both traits are normally distributed for males and for females. Consider the following (alternative, not null) hypotheses:

a) There is a difference in the mean number of steps between women and men.

b) The correlation coefficient between steps and bmi is negative for women.

c) The correlation coefficient between steps and bmi is positive for men.

Think about which test to use and calculate the corresponding P-value.

Hint: You can use the scipy.stats library to calculate correlation and the p-value.

Tip: Check out the pearsonr function.

Which other conclusions can you draw from the data?

If you are not familiar to statistics or want to try something else, you can instead (or also) explore the dataset and try to find other interesting insights. Feel free to explore it!

Write a very short report (50-150 words) with your findings and submit it together within the notebook.

Important: If possible, try not to share your findings with your colleagues. This is an individual assignment. All students that submit this part of the assignment will receive bonus points.

In [ ]:

data = pd.read_csv('../../Datasets/data9b.csv')
data.head()

data = pd.read_csv('../../Datasets/data9b.csv') data.head()

In [ ]:

# Your code here

# Your code here

SHORT REPORT HERE