{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "cfb06845",
   "metadata": {},
   "source": [
    "# Lesson 14 activity: advanced statistics\n",
    "\n",
    "## Learning objectives\n",
    "\n",
    "This activity will help you to:\n",
    "\n",
    "1. Apply appropriate statistical tests to real-world data\n",
    "2. Interpret hypothesis test results and p-values\n",
    "3. Compare multiple groups using statistical methods\n",
    "4. Make data-driven conclusions based on statistical evidence"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6cd8294",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Import the required libraries and load the weather dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2daa4e14",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy import stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc8e909d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the weather dataset\n",
    "url = 'https://media.githubusercontent.com/media/gperdrizet/fullstack-2605/refs/heads/main/data/weather.csv'\n",
    "df = pd.read_csv(url)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d04c9648",
   "metadata": {},
   "source": [
    "## Exercise 1: Humidity Differences Among Weather Conditions\n",
    "\n",
    "**Objective**: Determine if humidity is significantly different among different weather conditions.\n",
    "\n",
    "**Background**: \n",
    "The weather dataset contains humidity measurements for different weather conditions (Sunny, Rainy, Cloudy, etc.). We want to know if these weather conditions are associated with significantly different humidity levels.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. **Explore the data**:\n",
    "   - Check how many unique weather conditions are in the dataset\n",
    "   - Calculate the mean and standard deviation of humidity for each weather condition\n",
    "   - Create a boxplot showing the humidity distribution for each weather condition\n",
    "   - Based on the visualization, do you think the humidity levels differ significantly?\n",
    "\n",
    "2. **Choose an appropriate statistical test**:\n",
    "   - Since we're comparing humidity across multiple groups (weather conditions), what test should we use?\n",
    "   - Hint: Consider ANOVA (Analysis of Variance) or Kruskal-Wallis test\n",
    "   - State your null and alternative hypotheses\n",
    "\n",
    "3. **Check assumptions** (if using ANOVA):\n",
    "   - Are the sample sizes roughly equal across groups?\n",
    "   - Use a visualization or test to check if the data appears roughly normally distributed\n",
    "   - Check for equal variances across groups (you can use a simple visual check or Levene's test)\n",
    "\n",
    "4. **Perform the statistical test**:\n",
    "   - Conduct the chosen test using scipy.stats\n",
    "   - Report the test statistic and p-value\n",
    "   - Using α = 0.05, determine whether to reject or accept the null hypothesis\n",
    "\n",
    "5. **Interpret the results**:\n",
    "   - What does your test result tell you about humidity and weather conditions?\n",
    "   - If you rejected the null hypothesis, which weather conditions appear to have different humidity levels? (Use the boxplot and descriptive statistics to support your conclusion)\n",
    "   - What are the practical implications of this finding?\n",
    "\n",
    "**Bonus**: \n",
    "- If you found a significant difference, perform post-hoc pairwise comparisons to determine which specific weather conditions differ from each other\n",
    "- Consider whether Bonferroni correction should be applied to the pairwise comparisons"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d09c795",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb71c456",
   "metadata": {},
   "source": [
    "## Exercise 2: Testing for Association Between Weather Condition and Wind Strength\n",
    "\n",
    "**Objective**: Determine if there is a significant association (non-random relationship) between weather condition and wind strength.\n",
    "\n",
    "**Background**: \n",
    "If weather conditions and wind strength are independent, we would expect days to be randomly distributed across all combinations of weather condition and wind strength. However, certain weather conditions might be more likely to occur with certain wind strengths (e.g., rainy days might be associated with stronger winds). We can use a chi-square test of independence to test this.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. **Explore the data**:\n",
    "   - Check the unique values in both `weather_condition` and `wind_strength` columns\n",
    "   - Count how many days fall into each category for both variables\n",
    "   - Create a contingency table (crosstab) showing the count of days for each combination of weather condition and wind strength\n",
    "   - Use `pd.crosstab(df['weather_condition'], df['wind_strength'])` to create this table\n",
    "\n",
    "2. **Visualize the relationship**:\n",
    "   - Create a heatmap of the contingency table to visualize the distribution of days\n",
    "   - You can use `plt.imshow()` or `sns.heatmap()` if seaborn is available\n",
    "   - Based on the visualization, do certain weather conditions appear to be associated with certain wind strengths?\n",
    "\n",
    "3. **State your hypotheses**:\n",
    "   - **Null hypothesis**: Weather condition and wind strength are independent (days are randomly distributed across combinations)\n",
    "   - **Alternative hypothesis**: Weather condition and wind strength are associated (days are NOT randomly distributed)\n",
    "\n",
    "4. **Perform a chi-square test of independence**:\n",
    "   - Use `scipy.stats.chi2_contingency()` on your contingency table\n",
    "   - This test compares the observed frequencies to expected frequencies if the variables were independent\n",
    "   - Report the chi-square statistic, p-value, and degrees of freedom\n",
    "\n",
    "5. **Interpret the results**:\n",
    "   - Using α = 0.05, determine whether to reject or accept the null hypothesis\n",
    "   - If you reject the null hypothesis, what does this mean in practical terms?\n",
    "\n",
    "6. **Draw conclusions**: Are weather condition and wind strength independent or associated?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbeed790",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91899a7f",
   "metadata": {},
   "source": [
    "## Exercise 3: Testing for Relationship Between Humidity and Pressure\n",
    "\n",
    "**Objective**: Determine if there is a significant linear relationship between humidity (%) and atmospheric pressure (hPa).\n",
    "\n",
    "**Background**: \n",
    "When working with two continuous variables, we can test whether they are correlated. Correlation measures the strength and direction of a linear relationship. In meteorology, understanding relationships between variables like humidity and pressure can help improve weather predictions.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. **Explore the data**:\n",
    "   - Check for any missing values in `humidity_percent` and `pressure_hpa`\n",
    "   - Calculate basic descriptive statistics (mean, std, min, max) for both variables\n",
    "   - Create a scatter plot of humidity vs. pressure\n",
    "   - Based on the visualization, do you think there's a relationship? If so, is it positive or negative?\n",
    "\n",
    "2. **Choose an appropriate correlation test**:\n",
    "   - Since we have two continuous variables, we can use Pearson or Spearman correlation\n",
    "   - Pearson correlation measures linear relationships and assumes normality\n",
    "   - Spearman correlation measures monotonic relationships and is more robust\n",
    "   - Check the distributions of both variables (histograms or Q-Q plots) to help decide\n",
    "   - State your null and alternative hypotheses\n",
    "\n",
    "3. **Perform the correlation test**:\n",
    "   - Calculate both Pearson and Spearman correlation coefficients\n",
    "   - Use `scipy.stats.pearsonr()` and `scipy.stats.spearmanr()`\n",
    "   - Report the correlation coefficient (r) and p-value for each test\n",
    "   - Using α = 0.05, determine whether the correlation is statistically significant\n",
    "\n",
    "4. **Interpret the correlation coefficient**:\n",
    "   - What is the direction of the relationship (positive or negative)?\n",
    "   - What is the strength of the relationship? (Hint: |r| < 0.3 weak, 0.3-0.7 moderate, > 0.7 strong)\n",
    "   - Calculate the coefficient of determination (r²) - what percentage of variance in one variable is explained by the other?\n",
    "\n",
    "5. **Draw conclusions**:\n",
    "   - Is there a statistically significant relationship between humidity and pressure?\n",
    "   - Is the relationship strong enough to be practically useful?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8a46a6dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}