{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4c8a6a87",
   "metadata": {},
   "source": [
    "# Lesson 12 activity: linear algebra & basic statistics\n",
    "\n",
    "## Learning objectives\n",
    "\n",
    "This activity will help you to:\n",
    "\n",
    "1. Apply concepts from linear algebra to gain meaningful insight from data\n",
    "2. Understand statistical data types\n",
    "3. Use measures of shape to describe distributions\n",
    "4. Apply covariance and correlation to describe relationships between variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f6d51d8",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "Import the required libraries and load the weather dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8526de0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9eb120c9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the weather dataset\n",
    "url = 'https://media.githubusercontent.com/media/gperdrizet/fullstack-2605/refs/heads/main/data/weather.csv'\n",
    "df = pd.read_csv(url)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02075c94",
   "metadata": {},
   "source": [
    "## Exercise 1: linear algebra - finding similar days\n",
    "\n",
    "**Objective**: Apply concepts from linear algebra to gain meaningful insight from data.\n",
    "\n",
    "In linear algebra, we can treat each row of data as a vector and measure how similar different vectors are. This is useful for finding patterns - for example, finding days with similar weather conditions.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. Select only the numeric columns from the dataset: `temperature_c`, `rainfall_inches`, `humidity_percent`, and `pressure_hpa`\n",
    "\n",
    "2. Extract the **first row** (day 0) as a reference vector\n",
    "\n",
    "3. For each row in the dataset, calculate the **cosine similarity** to the first row using:\n",
    "   $$\\text{similarity} = \\frac{\\mathbf{a} \\cdot \\mathbf{b}}{\\|\\mathbf{a}\\| \\|\\mathbf{b}\\|}$$\n",
    "   - You can use `np.dot()` for the dot product\n",
    "   - You can use `np.linalg.norm()` to calculate vector magnitudes\n",
    "   \n",
    "4. Add the similarity scores as a new column to the dataframe\n",
    "\n",
    "5. **Sort** the dataframe by similarity (highest to lowest) so the most similar days to day 0 appear first\n",
    "\n",
    "6. Display the **top 10 most similar days** including their similarity scores and weather conditions\n",
    "\n",
    "7. **Interpret**: Look at the top similar days - do they have similar temperature, humidity, pressure values? Does this make sense?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32528c05",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d2903e3",
   "metadata": {},
   "source": [
    "## Exercise 2: data types and visualization\n",
    "\n",
    "**Objective**: Understand statistical data types and visualize interactions between variables.\n",
    "\n",
    "Understanding data types is crucial for choosing appropriate statistical methods and visualizations. In this exercise, you'll identify data types and explore how numeric variables interact with categorical ones.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. **Identify data types**: For `humidity_percent` and `pressure_hpa`, determine what type of data they are:\n",
    "   - Are they **interval** data (no true zero) or **ratio** data (has true zero)?\n",
    "   - For each variable, explain your reasoning:\n",
    "     - Does zero mean \"none\" or \"absence of the quantity\"?\n",
    "     - Are ratios meaningful? (e.g., is 100% humidity \"twice\" 50% humidity?)\n",
    "     - Can the value go below zero?\n",
    "\n",
    "2. **Create visualizations**: Choose an appropriate plot type to show how `humidity_percent` and `pressure_hpa` vary across different `weather_condition` categories\n",
    "   - Consider options like: box plots, violin plots, bar plots with error bars, or scatter plots with color coding\n",
    "   - Create **one plot** that effectively shows the relationship between both numeric variables and the weather condition\n",
    "   - You might use a single plot with subplots, or find a creative way to show all three variables together\n",
    "\n",
    "3. **Interpret your visualization**:\n",
    "   - Which weather condition tends to have the highest humidity?\n",
    "   - Which weather condition tends to have the lowest pressure?\n",
    "   - Do you see clear differences between weather conditions?\n",
    "   - Does this pattern make sense from a meteorological perspective?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70878eef",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6d42e77",
   "metadata": {},
   "source": [
    "## Exercise 3: analyzing distribution skewness\n",
    "\n",
    "**Objective**: Use measures of shape to describe distributions.\n",
    "\n",
    "Skewness describes the asymmetry of a distribution. Understanding skewness helps you choose appropriate statistical methods and understand the nature of your data.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. Calculate the **skewness** for all four numeric variables:\n",
    "   - `temperature_c`\n",
    "   - `rainfall_inches`\n",
    "   - `humidity_percent`\n",
    "   - `pressure_hpa`\n",
    "\n",
    "2. Identify which variable has:\n",
    "   - The **greatest skew** (furthest from zero)\n",
    "   - The **least skew** (closest to zero, most symmetric)\n",
    "   - Print these findings with their skewness values\n",
    "\n",
    "3. Create **two histograms** (side by side) showing only these two variables:\n",
    "   - One histogram for the most skewed variable\n",
    "   - One histogram for the least skewed variable\n",
    "   - For each histogram:\n",
    "     - Add vertical lines showing the mean (in red) and median (in green)\n",
    "     - Include the skewness value in the title\n",
    "     - Use appropriate bin sizes\n",
    "\n",
    "4. **Interpret** your findings:\n",
    "   - Why does the most skewed variable have the sign that it does? (Think about the real-world meaning)\n",
    "   - For the skewed distribution, how do the mean and median compare? Why?\n",
    "   - What does the skewness tell you about typical vs extreme values for this variable?\n",
    "   - Why is the least skewed variable more symmetric?\n",
    "   - **Bonus**: Explain why skewness matters when choosing between mean and median as a measure of central tendency."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9cbbe0e1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "134687d5",
   "metadata": {},
   "source": [
    "## Exercise 4: exploring relationships with correlation\n",
    "\n",
    "**Objective**: Apply covariance and correlation to describe relationships between variables.\n",
    "\n",
    "Weather variables often have meaningful relationships. Some pairs of variables are strongly related while others have little relationship at all.\n",
    "\n",
    "**Tasks**:\n",
    "\n",
    "1. Calculate the **correlation matrix** for all four numeric variables:\n",
    "   - `temperature_c`\n",
    "   - `rainfall_inches`\n",
    "   - `humidity_percent`\n",
    "   - `pressure_hpa`\n",
    "\n",
    "2. Identify the pair of variables with:\n",
    "   - The **strongest correlation** (highest absolute value, whether positive or negative)\n",
    "   - The **weakest correlation** (closest to zero)\n",
    "   - Print both pairs with their correlation coefficients\n",
    "   - Note whether the strongest correlation is positive or negative\n",
    "\n",
    "3. Create **two scatter plots** (side by side or in separate figures):\n",
    "   - One for the strongest correlation pair\n",
    "   - One for the weakest correlation pair\n",
    "   - For each plot:\n",
    "     - Include the correlation coefficient in the title\n",
    "     - Add appropriate axis labels\n",
    "\n",
    "4. **Interpret** your findings:\n",
    "   - What is the strongest relationship? Does it make meteorological sense?\n",
    "   - Is this strongest correlation positive or negative? What does that mean in real-world terms?\n",
    "   - What is the weakest relationship? Why might these variables have little correlation?\n",
    "   - Compare the scatter plots: How does the pattern differ between strong and weak correlations?\n",
    "   - Based on the correlation strengths, which relationship is more predictable?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bd995489",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Your code here"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}