**This is the first project that i'm trying to finish to brush up on my analysis skills and specifically pandas' skills**

So the first one is gonna involve a weather dataset which i will try to analyse and therefore finish all the tasks that came with it.

import pandas as pd
import numpy as np

filepath = "C:\\Users\\abdelilah\\Desktop\\Python Projects\\Weather\\"
raw_data = pd.read_csv(filepath + "Weather Data.csv", delimiter = ',')
raw_data.head()

	Date/Time	Temp_C	Dew Point Temp_C	Rel Hum_%	Wind Speed_km/h	Visibility_km	Press_kPa	Weather
0	1/1/2012 0:00	-1.8	-3.9	86	4	8.0	101.24	Fog
1	1/1/2012 1:00	-1.8	-3.7	87	4	8.0	101.24	Fog
2	1/1/2012 2:00	-1.8	-3.4	89	7	4.0	101.26	Freezing Drizzle,Fog
3	1/1/2012 3:00	-1.5	-3.2	88	6	4.0	101.27	Freezing Drizzle,Fog
4	1/1/2012 4:00	-1.5	-3.3	88	7	4.8	101.23	Fog

First Task: find all the unique values in the "Wind Speed" column

#pd.unique finds all the unique values in Series object then sorts them and appends them to an array
pd.unique(raw_data["Wind Speed_km/h"])

array([ 4,  7,  6,  9, 15, 13, 20, 22, 19, 24, 30, 35, 39, 32, 33, 26, 44,
       43, 48, 37, 28, 17, 11,  0, 83, 70, 57, 46, 41, 52, 50, 63, 54,  2],
      dtype=int64)

#file.value_counts() on the other hand creates a table with all the unique values and their count
raw_data["Wind Speed_km/h"].value_counts()

9     830
11    791
13    735
15    719
7     677
17    666
19    616
6     609
20    496
4     474
22    439
24    374
0     309
26    242
28    205
30    161
32    139
33     85
35     53
37     45
39     24
41     22
44     14
48     13
43     13
46     11
52      7
57      5
50      4
2       2
70      1
54      1
83      1
63      1
Name: Wind Speed_km/h, dtype: int64

Second Task: find how many times the weather was clear

#this is an unconventional way to find the count of clear weather where i created a list that appends c whenever we meet the
#condition so at the end we just have to count the length of that list
clear_count = len([i for i in raw_data["Weather"] if i == "Clear"])
clear_count

1326

#this an easier way to get the number of times Clear was mentioned and with it all the other weather conditions 
raw_data["Weather"].value_counts()

Mainly Clear                               2106
Mostly Cloudy                              2069
Cloudy                                     1728
Clear                                      1326
Snow                                        390
Rain                                        306
Rain Showers                                188
Fog                                         150
Rain,Fog                                    116
Drizzle,Fog                                  80
Snow Showers                                 60
Drizzle                                      41
Snow,Fog                                     37
Snow,Blowing Snow                            19
Rain,Snow                                    18
Thunderstorms,Rain Showers                   16
Haze                                         16
Drizzle,Snow,Fog                             15
Freezing Rain                                14
Freezing Drizzle,Snow                        11
Freezing Drizzle                              7
Snow,Ice Pellets                              6
Freezing Drizzle,Fog                          6
Snow,Haze                                     5
Snow Showers,Fog                              4
Rain,Snow,Ice Pellets                         4
Freezing Rain,Fog                             4
Freezing Fog                                  4
Moderate Snow                                 4
Freezing Drizzle,Haze                         3
Thunderstorms,Rain Showers,Fog                3
Thunderstorms,Rain                            3
Rain,Haze                                     3
Drizzle,Snow                                  2
Moderate Snow,Blowing Snow                    2
Freezing Rain,Haze                            2
Rain Showers,Snow Showers                     2
Thunderstorms                                 2
Drizzle,Ice Pellets,Fog                       1
Thunderstorms,Rain,Fog                        1
Rain Showers,Fog                              1
Rain,Ice Pellets                              1
Thunderstorms,Heavy Rain Showers              1
Freezing Rain,Snow Grains                     1
Rain,Snow Grains                              1
Rain,Snow,Fog                                 1
Thunderstorms,Moderate Rain Showers,Fog       1
Moderate Rain,Fog                             1
Freezing Rain,Ice Pellets,Fog                 1
Snow Pellets                                  1
Name: Weather, dtype: int64

Third Task: Find the number of times where the speed was exactly 4 km/h

#I already did this when I got all the values in the wind speed column with all their counts, so now to easily get the data 
#i can categorize it, so i'll add every value and its count to a new DataFrame

#the first step is adding the value_counts() output to a DataFrame
wind_speed_count = pd.DataFrame(raw_data["Wind Speed_km/h"].value_counts())

#then we have to turn the column that has the speed values into an official column since it all got inverted into an index, 
#and meanwhile rename the second to its appropriate name
wind_speed_count.reset_index(inplace=True)
wind_speed_count = wind_speed_count.rename(columns = {'index':'Speed', 'Wind Speed_km/h':'Count'})

#now we sort according to the Speed column
wind_speed_count = wind_speed_count.sort_values('Speed')
wind_speed_count

	Speed	Count
12	0	309
29	2	2
9	4	474
7	6	609
4	7	677
0	9	830
1	11	791
2	13	735
3	15	719
5	17	666
6	19	616
8	20	496
10	22	439
11	24	374
13	26	242
14	28	205
15	30	161
16	32	139
17	33	85
18	35	53
19	37	45
20	39	24
21	41	22
24	43	13
22	44	14
25	46	11
23	48	13
28	50	4
26	52	7
31	54	1
27	57	5
33	63	1
30	70	1
32	83	1

#finally we get the number of times 4km/h was mentioned
#i can also use loc()
fourkmh_count = wind_speed_count[wind_speed_count["Speed"] == 4]
fourkmh_count["Count"]

9    474
Name: Count, dtype: int64

Fouth Task: Find all the null values

#here i am getting all the null values in each column, by going through the columns with a loop and using the isnull()✓
cols = []
for i in raw_data:
    cols.append(i)
for name in cols:
    print(raw_data[name].isnull().sum())

0
0
0
0
0
0
0
0

Fifth Task: Rename the column "Weather" to "Weather Condition"

#all i did was using the rename() function with its parameter columns and then giving it as arument the new name
raw_data = raw_data.rename(columns = {'Weather':'Weather Condition'})
raw_data.head(2)

	Date/Time	Temp_C	Dew Point Temp_C	Rel Hum_%	Wind Speed_km/h	Visibility_km	Press_kPa	Weather Condition
0	1/1/2012 0:00	-1.8	-3.9	86	4	8.0	101.24	Fog
1	1/1/2012 1:00	-1.8	-3.7	87	4	8.0	101.24	Fog

Sixth Task: Find the mean Visibility

#we have two way we can do this, the 1st manual traditional way, in which we sum all the values and devide them on the lenght
print("Mean is:",sum(raw_data["Visibility_km"])/len(raw_data["Visibility_km"]))

Mean is: 27.66444672131151

#and the 2nd way involves using the library "statistics" and directly implement the function mean()
from statistics import mean

#later we can optionaly import the fractions module to output the mean in a fraction
#from fractions import Fraction as fr
print("Mean is:",mean(raw_data["Visibility_km"]))

Mean is: 27.664446721311474

Seventh Task: Find the standard deviation of Pressure

#this is straightforward, we're just gonna use the different functions provided by the statistics module
from statistics import stdev

print("Standard Deviation is:", stdev(raw_data["Press_kPa"]))

Standard Deviation is: 0.8440047459486483

Eight Task: Find the variance of Relative Humidity

from statistics import variance 

print("Variance is:", variance(raw_data["Rel Hum_%"]))

Variance is: 286.24855019850196

Nineth Task: Find every instance where Snow was recorded

snow_count = 0
for i in raw_data["Weather Condition"]:
    if "Snow" in i:
        snow_count += 1
print("Snow was recorded {} times in the Dataset".format(snow_count))

Snow was recorded 583 times in the Dataset

Tenth Task: Find every instance where the Wind Speed was above 24 and Visibility is 25

wind_vis = 0
for i in range(0, len(raw_data["Visibility_km"])):
    if raw_data.at[i, "Visibility_km"] == 25 and raw_data.at[i, "Wind Speed_km/h"] > 24: 
        wind_vis += 1
print("Wind Speed was above 24 at the same time Visibility was 25 {} times in the Dataset".format(wind_vis))

Wind Speed was above 24 at the same time Visibility was 25 308 times in the Dataset

#this is a great way to retrieve a value from a cell in a dataframe, first uses the row's index and actual names of the 
#column's index, but the second uses instead the number of that index
print(raw_data.at[2, "Visibility_km"])
print(raw_data.iat[2, 5])

4.0
4.0

---

Eleventh Task: Find the Mean Value for each column against each 'Weather Condition'

#what i got from this task is i have to get the mean value for each column but they'd have to be in the same weather condition
#so my logic here is to seperate the table i have into 50 different ones, each containing the data for that weather condition
#it's a little bit of work which might be done easily
#so i'll do this the "hard" way and figure out later an easy one
#i think i came up with a way to easily do this, which doesn't involve creating so many tables, actually two ways
'''1st involves sorting the conditions till every column is sorted according to the alphabetical order of the conditions, 
here i'll be able to measure the mean for the consecutive indices of that condition which might need me to note down each 
condition with its first and last index'''
'''2nd one has to do with adding a condition to my table, meaning i'll calculate the mean but just if that row has the same 
condition'''

"2nd one has to do with adding a condition to my table, meaning i'll calculate the mean but just if that row has the same \ncondition"

raw_data["Weather Condition"].nunique()

50

cond_values = pd.DataFrame(raw_data["Weather Condition"].value_counts())
cond_values.reset_index(inplace=True)
cond_values.rename(columns = {'index':'Condition', 'Weather Condition':'Count'})
cond_values

	index	Weather Condition
0	Mainly Clear	2106
1	Mostly Cloudy	2069
2	Cloudy	1728
3	Clear	1326
4	Snow	390
5	Rain	306
6	Rain Showers	188
7	Fog	150
8	Rain,Fog	116
9	Drizzle,Fog	80
10	Snow Showers	60
11	Drizzle	41
12	Snow,Fog	37
13	Snow,Blowing Snow	19
14	Rain,Snow	18
15	Thunderstorms,Rain Showers	16
16	Haze	16
17	Drizzle,Snow,Fog	15
18	Freezing Rain	14
19	Freezing Drizzle,Snow	11
20	Freezing Drizzle	7
21	Snow,Ice Pellets	6
22	Freezing Drizzle,Fog	6
23	Snow,Haze	5
24	Snow Showers,Fog	4
25	Rain,Snow,Ice Pellets	4
26	Freezing Rain,Fog	4
27	Freezing Fog	4
28	Moderate Snow	4
29	Freezing Drizzle,Haze	3
30	Thunderstorms,Rain Showers,Fog	3
31	Thunderstorms,Rain	3
32	Rain,Haze	3
33	Drizzle,Snow	2
34	Moderate Snow,Blowing Snow	2
35	Freezing Rain,Haze	2
36	Rain Showers,Snow Showers	2
37	Thunderstorms	2
38	Drizzle,Ice Pellets,Fog	1
39	Thunderstorms,Rain,Fog	1
40	Rain Showers,Fog	1
41	Rain,Ice Pellets	1
42	Thunderstorms,Heavy Rain Showers	1
43	Freezing Rain,Snow Grains	1
44	Rain,Snow Grains	1
45	Rain,Snow,Fog	1
46	Thunderstorms,Moderate Rain Showers,Fog	1
47	Moderate Rain,Fog	1
48	Freezing Rain,Ice Pellets,Fog	1
49	Snow Pellets	1

cond_values = cond_values.sort_values(by = 'index')
cond_values.head(5)

	index	Weather Condition
3	Clear	1326
2	Cloudy	1728
11	Drizzle	41
9	Drizzle,Fog	80
38	Drizzle,Ice Pellets,Fog	1

cond_name = list(cond_values['index'])
cond_count = list(cond_values['Weather Condition'])
for i in range(50):
    print(cond_name[i], "\t :: \t", cond_count[i])

Clear 	 :: 	 1326
Cloudy 	 :: 	 1728
Drizzle 	 :: 	 41
Drizzle,Fog 	 :: 	 80
Drizzle,Ice Pellets,Fog 	 :: 	 1
Drizzle,Snow 	 :: 	 2
Drizzle,Snow,Fog 	 :: 	 15
Fog 	 :: 	 150
Freezing Drizzle 	 :: 	 7
Freezing Drizzle,Fog 	 :: 	 6
Freezing Drizzle,Haze 	 :: 	 3
Freezing Drizzle,Snow 	 :: 	 11
Freezing Fog 	 :: 	 4
Freezing Rain 	 :: 	 14
Freezing Rain,Fog 	 :: 	 4
Freezing Rain,Haze 	 :: 	 2
Freezing Rain,Ice Pellets,Fog 	 :: 	 1
Freezing Rain,Snow Grains 	 :: 	 1
Haze 	 :: 	 16
Mainly Clear 	 :: 	 2106
Moderate Rain,Fog 	 :: 	 1
Moderate Snow 	 :: 	 4
Moderate Snow,Blowing Snow 	 :: 	 2
Mostly Cloudy 	 :: 	 2069
Rain 	 :: 	 306
Rain Showers 	 :: 	 188
Rain Showers,Fog 	 :: 	 1
Rain Showers,Snow Showers 	 :: 	 2
Rain,Fog 	 :: 	 116
Rain,Haze 	 :: 	 3
Rain,Ice Pellets 	 :: 	 1
Rain,Snow 	 :: 	 18
Rain,Snow Grains 	 :: 	 1
Rain,Snow,Fog 	 :: 	 1
Rain,Snow,Ice Pellets 	 :: 	 4
Snow 	 :: 	 390
Snow Pellets 	 :: 	 1
Snow Showers 	 :: 	 60
Snow Showers,Fog 	 :: 	 4
Snow,Blowing Snow 	 :: 	 19
Snow,Fog 	 :: 	 37
Snow,Haze 	 :: 	 5
Snow,Ice Pellets 	 :: 	 6
Thunderstorms 	 :: 	 2
Thunderstorms,Heavy Rain Showers 	 :: 	 1
Thunderstorms,Moderate Rain Showers,Fog 	 :: 	 1
Thunderstorms,Rain 	 :: 	 3
Thunderstorms,Rain Showers 	 :: 	 16
Thunderstorms,Rain Showers,Fog 	 :: 	 3
Thunderstorms,Rain,Fog 	 :: 	 1

cols.remove('Date/Time')
cols

['Temp_C',
 'Dew Point Temp_C',
 'Rel Hum_%',
 'Wind Speed_km/h',
 'Visibility_km',
 'Press_kPa',
 'Weather']

sorted_data = raw_data.sort_values(by = "Weather Condition")

sorted_data

	Date/Time	Temp_C	Dew Point Temp_C	Rel Hum_%	Wind Speed_km/h	Visibility_km	Press_kPa	Weather Condition
7103	10/22/2012 23:00	6.4	4.4	87	0	25.0	101.34	Clear
4203	6/24/2012 3:00	14.9	9.1	68	0	25.0	101.02	Clear
4204	6/24/2012 4:00	14.4	9.0	70	11	48.3	101.04	Clear
4205	6/24/2012 5:00	14.0	9.2	73	6	48.3	101.04	Clear
4206	6/24/2012 6:00	17.8	10.5	62	4	48.3	101.04	Clear
...	...	...	...	...	...	...	...	...
5842	8/31/2012 10:00	19.7	17.8	89	20	24.1	100.54	Thunderstorms,Rain Showers
5108	7/31/2012 20:00	22.4	18.7	80	35	9.7	100.64	Thunderstorms,Rain Showers,Fog
4323	6/29/2012 3:00	19.5	16.1	81	7	9.7	99.71	Thunderstorms,Rain Showers,Fog
4761	7/17/2012 9:00	22.9	21.3	91	17	9.7	99.84	Thunderstorms,Rain Showers,Fog
4757	7/17/2012 5:00	20.6	18.6	88	19	4.8	100.08	Thunderstorms,Rain,Fog

8784 rows × 8 columns

#what i need to do now is calculate the mean for each condition and each variable and put it in its place 

import numpy as np

array_data = sorted_data.to_numpy()

array_data = array_data.transpose()

array_data = np.delete(array_data, (0), axis = 0)

array_data = np.delete(array_data, (-1), axis = 0)

array_data

array([[6.4, 14.9, 14.4, ..., 19.5, 22.9, 20.6],
       [4.4, 9.1, 9.0, ..., 16.1, 21.3, 18.6],
       [87, 68, 70, ..., 81, 91, 88],
       [0, 0, 11, ..., 7, 17, 19],
       [25.0, 25.0, 48.3, ..., 9.7, 9.7, 4.8],
       [101.34, 101.02, 101.04, ..., 99.71, 99.84, 100.08]], dtype=object)

print(*cond_count)

1326 1728 41 80 1 2 15 150 7 6 3 11 4 14 4 2 1 1 16 2106 1 4 2 2069 306 188 1 2 116 3 1 18 1 1 4 390 1 60 4 19 37 5 6 2 1 1 3 16 3 1

mean_table = np.empty((6, 50))
for i in range(6):
    for j in range(50):
        if j == 0:
            mean_table[i, j] = np.mean(array_data[i,[0, cond_count[0]]])
        else :
            mean_table[i, j] = np.mean(array_data[i,[cond_count[j-1], cond_count[j]]])

**IT'S SURPRISING AND AMAZING HOW MUCH TIME IT TOOK NUMPY TO CALCULATE THE MEAN FOR EACH ROW AND COLUMN, IT'S SOO FREAKING FAST COMPARED TO PANADAS**

mean_table_t = mean_table.transpose()

mean_table_t

array([[ 11.9  ,   8.25 ,  79.   ,   9.5  ,  25.   , 100.94 ],
       [ 15.05 ,   3.55 ,  50.   ,  16.   ,  25.   , 101.09 ],
       [ 10.   ,  -3.2  ,  41.5  ,  13.   ,  25.   , 101.725],
       [ 11.35 ,   3.25 ,  57.5  ,  16.5  ,  25.   , 101.53 ],
       [ 15.15 ,   8.5  ,  64.5  ,  10.   ,  25.   , 101.135],
       [ 14.65 ,   9.05 ,  69.   ,   5.5  ,  36.65 , 101.03 ],
       [  9.6  ,   3.75 ,  67.   ,  10.   ,  48.3  , 101.37 ],
       [  2.5  ,  -3.5  ,  64.5  ,  15.5  ,  36.65 , 101.415],
       [ 11.2  ,   2.55 ,  56.5  ,  14.5  ,  36.65 , 101.08 ],
       [ 13.35 ,   4.55 ,  56.5  ,  11.   ,  36.65 , 101.045],
       [  9.25 ,   3.85 ,  69.   ,  10.5  ,  36.65 , 101.05 ],
       [ 19.9  ,  13.3  ,  66.5  ,   8.5  ,  36.65 , 101.025],
       [ 21.8  ,  13.95 ,  61.   ,   7.5  ,  36.65 , 101.025],
       [ 12.6  ,   4.65 ,  58.   ,   5.   ,  36.65 , 101.365],
       [ 12.6  ,   4.65 ,  58.   ,   5.   ,  36.65 , 101.365],
       [ 16.1  ,   9.75 ,  66.   ,   7.5  ,  48.3  , 101.04 ],
       [ 14.65 ,   9.05 ,  69.   ,   5.5  ,  36.65 , 101.03 ],
       [ 14.9  ,   9.1  ,  68.   ,   0.   ,  25.   , 101.02 ],
       [ 10.   ,   4.05 ,  66.5  ,   6.5  ,  25.   , 101.425],
       [  2.8  ,  -2.8  ,  67.   ,  20.5  ,  25.   , 101.515],
       [  7.7  ,   2.25 ,  68.5  ,  14.   ,  25.   , 101.11 ],
       [ 16.35 ,   9.8  ,  65.   ,   2.   ,  36.65 , 101.03 ],
       [ 16.1  ,   9.75 ,  66.   ,   7.5  ,  48.3  , 101.04 ],
       [  2.8  ,  -1.9  ,  71.5  ,   9.   ,  36.65 , 101.4  ],
       [  2.7  ,  -2.45 ,  69.5  ,  10.   ,  25.   , 101.975],
       [  2.9  ,  -2.25 ,  69.5  ,  11.   ,  25.   , 102.025],
       [  3.25 ,  -1.65 ,  70.5  ,   4.5  ,  25.   , 101.44 ],
       [ 14.65 ,   9.05 ,  69.   ,   5.5  ,  36.65 , 101.03 ],
       [ 15.95 ,  10.6  ,  70.5  ,  12.   ,  36.2  , 100.925],
       [ 15.75 ,  10.7  ,  72.   ,   9.5  ,  36.2  , 100.925],
       [ 14.45 ,   9.15 ,  70.5  ,   3.   ,  36.65 , 101.03 ],
       [ 16.25 ,   8.9  ,  62.   ,   3.   ,  25.   , 101.02 ],
       [ 16.25 ,   8.9  ,  62.   ,   3.   ,  25.   , 101.02 ],
       [ 14.9  ,   9.1  ,  68.   ,   0.   ,  25.   , 101.02 ],
       [ 16.35 ,   9.8  ,  65.   ,   2.   ,  36.65 , 101.03 ],
       [ 20.65 ,   9.3  ,  49.5  ,  17.   ,  48.3  , 100.545],
       [ 19.2  ,   8.6  ,  52.5  ,  15.   ,  36.65 , 100.535],
       [ 12.7  ,   3.9  ,  56.   ,  10.   ,  36.65 , 101.355],
       [ 14.15 ,   4.6  ,  53.   ,  12.   ,  48.3  , 101.365],
       [ 18.05 ,   9.45 ,  57.   ,   2.   ,  36.65 , 101.02 ],
       [  6.6  ,  -3.35 ,  48.5  ,   7.5  ,  24.55 , 101.395],
       [  7.45 ,  -1.45 ,  53.   ,  11.   ,  36.2  , 101.41 ],
       [ 12.25 ,   5.35 ,  63.   ,  11.   ,  36.65 , 101.045],
       [  9.45 ,   3.75 ,  67.5  ,  13.   ,  36.65 , 101.05 ],
       [ 14.65 ,   9.05 ,  69.   ,   5.5  ,  36.65 , 101.03 ],
       [ 14.9  ,   9.1  ,  68.   ,   0.   ,  25.   , 101.02 ],
       [ 14.45 ,   9.15 ,  70.5  ,   3.   ,  36.65 , 101.03 ],
       [  9.55 ,   4.1  ,  69.   ,   9.5  ,  36.65 , 101.435],
       [  9.55 ,   4.1  ,  69.   ,   9.5  ,  36.65 , 101.435],
       [ 14.45 ,   9.15 ,  70.5  ,   3.   ,  36.65 , 101.03 ]])

mean_df = pd.DataFrame(mean_table_t, 
                       columns = [ 
                           'Temp Mean', 
                           'DPT Mean', 
                           'Humidity Mean',
                           'Wind Speed Mean',
                           'Visisbility Mean',
                           'Pressure Mean'], 
                       index = cond_name)

mean_df.head(5)

	Temp Mean	DPT Mean	Humidity Mean	Wind Speed Mean	Visisbility Mean	Pressure Mean
Clear	11.90	8.25	79.0	9.5	25.0	100.940
Cloudy	15.05	3.55	50.0	16.0	25.0	101.090
Drizzle	10.00	-3.20	41.5	13.0	25.0	101.725
Drizzle,Fog	11.35	3.25	57.5	16.5	25.0	101.530
Drizzle,Ice Pellets,Fog	15.15	8.50	64.5	10.0	25.0	101.135

What i did here was turn the sorted dataframe i had into a numpy array, that way i could calculate the mean faster, which surprised me seeing how freaking fast it was, that array was empty and shaped according to the number of conditions and variables i had. Then i used the count i got from the value_counts() that i applied on the "Weather" column to get the unique values and their count. From here i sorted my dataframe based on the same column which grouped each weather condition and appanded each to a list and , thus allowing me to calculate the mean faster using the count as an index

---

Twelfth Task: Find The Min and Max for Variable according to their Weather Condition

max_min_df = pd.DataFrame(columns = ["Condition",
                                     'Temp Max',
                                     'Temp Min',
                                     'DPT Max',
                                     'DPT Min',
                                     'Humidity Max',
                                     'Humidity Min',
                                     'Wind Speed Max',
                                     'Wind Speed Min',
                                     'Visibility Max',
                                     'Visibility Min',
                                     'Pressure Max',
                                     'Pressure Min',
                                    ])
max_min_df.Condition = cond_name
max_min_df.head(2)

	Condition	Temp Max	Temp Min	DPT Max	DPT Min	Humidity Max	Humidity Min	Wind Speed Max	Wind Speed Min	Visibility Max	Visibility Min	Pressure Max	Pressure Min
0	Clear	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
1	Cloudy	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

max_table = np.empty((6, 50))
for i in range(0,6):
    for j in range(0,50):
        if j == 0:
            max_table[i, j] = np.max(array_data[i,[0, cond_count[0]]])
        else :
            max_table[i, j] = np.max(array_data[i,[cond_count[j-1], cond_count[j]]])

max_table_t = max_table.transpose()

min_table = np.empty((6, 50))
for i in range(0,6):
    for j in range(0,50):
        if j == 0:
            min_table[i, j] = np.min(array_data[i,[0, cond_count[0]]])
        else :
            min_table[i, j] = np.min(array_data[i,[cond_count[j-1], cond_count[j]]])

min_table_t = min_table.transpose()

max_df = pd.DataFrame(max_table_t, 
                      columns = ['Temp Min',
                  'DPT Min',
                  'Humidity Min',
                  'Wind Speed Min',
                  'Visibility Min',
                  'Pressure Min'],
                     index = cond_name)

max_df

	Temp Min	DPT Min	Humidity Min	Wind Speed Min	Visibility Min	Pressure Min
Clear	17.4	12.1	87.0	19.0	25.0	101.34
Cloudy	17.4	12.1	71.0	19.0	25.0	101.64
Drizzle	12.7	-1.4	54.0	13.0	25.0	101.81
Drizzle,Fog	15.4	7.9	61.0	20.0	25.0	101.81
Drizzle,Ice Pellets,Fog	15.4	9.1	68.0	20.0	25.0	101.25
Drizzle,Snow	14.9	9.1	70.0	11.0	48.3	101.04
Drizzle,Snow,Fog	14.4	9.0	70.0	11.0	48.3	101.70
Fog	4.8	-1.5	65.0	22.0	48.3	101.70
Freezing Drizzle	22.2	10.6	65.0	22.0	48.3	101.13
Freezing Drizzle,Fog	22.2	10.6	65.0	15.0	48.3	101.06
Freezing Drizzle,Haze	14.0	9.2	73.0	15.0	48.3	101.06
Freezing Drizzle,Snow	25.8	17.4	73.0	11.0	48.3	101.04
Freezing Fog	25.8	17.4	62.0	11.0	48.3	101.04
Freezing Rain	17.8	10.5	62.0	6.0	48.3	101.69
Freezing Rain,Fog	17.8	10.5	62.0	6.0	48.3	101.69
Freezing Rain,Haze	17.8	10.5	70.0	11.0	48.3	101.04
Freezing Rain,Ice Pellets,Fog	14.9	9.1	70.0	11.0	48.3	101.04
Freezing Rain,Snow Grains	14.9	9.1	68.0	0.0	25.0	101.02
Haze	14.9	9.1	68.0	13.0	25.0	101.83
Mainly Clear	5.1	-1.0	69.0	28.0	25.0	101.83
Moderate Rain,Fog	14.9	9.1	69.0	28.0	25.0	101.20
Moderate Snow	17.8	10.5	68.0	4.0	48.3	101.04
Moderate Snow,Blowing Snow	17.8	10.5	70.0	11.0	48.3	101.04
Mostly Cloudy	14.4	9.0	73.0	11.0	48.3	101.76
Rain	14.2	7.9	73.0	13.0	25.0	102.19
Rain Showers	14.2	7.9	73.0	13.0	25.0	102.19
Rain Showers,Fog	14.9	9.1	73.0	9.0	25.0	101.86
Rain Showers,Snow Showers	14.9	9.1	70.0	11.0	48.3	101.04
Rain,Fog	17.5	12.2	71.0	13.0	48.3	101.04
Rain,Haze	17.5	12.2	73.0	13.0	48.3	101.04
Rain,Ice Pellets	14.9	9.2	73.0	6.0	48.3	101.04
Rain,Snow	17.6	9.1	68.0	6.0	25.0	101.02
Rain,Snow Grains	17.6	9.1	68.0	6.0	25.0	101.02
Rain,Snow,Fog	14.9	9.1	68.0	0.0	25.0	101.02
Rain,Snow,Ice Pellets	17.8	10.5	68.0	4.0	48.3	101.04
Snow	23.5	10.5	62.0	30.0	48.3	101.04
Snow Pellets	23.5	9.1	68.0	30.0	48.3	101.02
Snow Showers	14.9	9.1	68.0	20.0	48.3	101.69
Snow Showers,Fog	17.8	10.5	62.0	20.0	48.3	101.69
Snow,Blowing Snow	18.3	10.5	62.0	4.0	48.3	101.04
Snow,Fog	18.3	8.4	52.0	15.0	25.0	101.79
Snow,Haze	20.0	12.2	61.0	15.0	48.3	101.79
Snow,Ice Pellets	20.0	12.2	65.0	15.0	48.3	101.06
Thunderstorms	14.4	9.0	70.0	15.0	48.3	101.06
Thunderstorms,Heavy Rain Showers	14.9	9.1	70.0	11.0	48.3	101.04
Thunderstorms,Moderate Rain Showers,Fog	14.9	9.1	68.0	0.0	25.0	101.02
Thunderstorms,Rain	14.9	9.2	73.0	6.0	48.3	101.04
Thunderstorms,Rain Showers	14.0	9.2	73.0	13.0	48.3	101.83
Thunderstorms,Rain Showers,Fog	14.0	9.2	73.0	13.0	48.3	101.83
Thunderstorms,Rain,Fog	14.9	9.2	73.0	6.0	48.3	101.04

min_df = pd.DataFrame(min_table_t, 
                      columns = ['Temp Min',
                  'DPT Min',
                  'Humidity Min',
                  'Wind Speed Min',
                  'Visibility Min',
                  'Pressure Min'],
                     index = cond_name)

min_df

	Temp Min	DPT Min	Humidity Min	Wind Speed Min	Visibility Min	Pressure Min
Clear	6.4	4.4	71.0	0.0	25.0	100.54
Cloudy	12.7	-5.0	29.0	13.0	25.0	100.54
Drizzle	7.3	-5.0	29.0	13.0	25.0	101.64
Drizzle,Fog	7.3	-1.4	54.0	13.0	25.0	101.25
Drizzle,Ice Pellets,Fog	14.9	7.9	61.0	0.0	25.0	101.02
Drizzle,Snow	14.4	9.0	68.0	0.0	25.0	101.02
Drizzle,Snow,Fog	4.8	-1.5	64.0	9.0	48.3	101.04
Fog	0.2	-5.5	64.0	9.0	25.0	101.13
Freezing Drizzle	0.2	-5.5	48.0	7.0	25.0	101.03
Freezing Drizzle,Fog	4.5	-1.5	48.0	7.0	25.0	101.03
Freezing Drizzle,Haze	4.5	-1.5	65.0	6.0	25.0	101.04
Freezing Drizzle,Snow	14.0	9.2	60.0	6.0	25.0	101.01
Freezing Fog	17.8	10.5	60.0	4.0	25.0	101.01
Freezing Rain	7.4	-1.2	54.0	4.0	25.0	101.04
Freezing Rain,Fog	7.4	-1.2	54.0	4.0	25.0	101.04
Freezing Rain,Haze	14.4	9.0	62.0	4.0	48.3	101.04
Freezing Rain,Ice Pellets,Fog	14.4	9.0	68.0	0.0	25.0	101.02
Freezing Rain,Snow Grains	14.9	9.1	68.0	0.0	25.0	101.02
Haze	5.1	-1.0	65.0	0.0	25.0	101.02
Mainly Clear	0.5	-4.6	65.0	13.0	25.0	101.20
Moderate Rain,Fog	0.5	-4.6	68.0	0.0	25.0	101.02
Moderate Snow	14.9	9.1	62.0	0.0	25.0	101.02
Moderate Snow,Blowing Snow	14.4	9.0	62.0	4.0	48.3	101.04
Mostly Cloudy	-8.8	-12.8	70.0	7.0	25.0	101.04
Rain	-8.8	-12.8	66.0	7.0	25.0	101.76
Rain Showers	-8.4	-12.4	66.0	9.0	25.0	101.86
Rain Showers,Fog	-8.4	-12.4	68.0	0.0	25.0	101.02
Rain Showers,Snow Showers	14.4	9.0	68.0	0.0	25.0	101.02
Rain,Fog	14.4	9.0	70.0	11.0	24.1	100.81
Rain,Haze	14.0	9.2	71.0	6.0	24.1	100.81
Rain,Ice Pellets	14.0	9.1	68.0	0.0	25.0	101.02
Rain,Snow	14.9	8.7	56.0	0.0	25.0	101.02
Rain,Snow Grains	14.9	8.7	56.0	0.0	25.0	101.02
Rain,Snow,Fog	14.9	9.1	68.0	0.0	25.0	101.02
Rain,Snow,Ice Pellets	14.9	9.1	62.0	0.0	25.0	101.02
Snow	17.8	8.1	37.0	4.0	48.3	100.05
Snow Pellets	14.9	8.1	37.0	0.0	25.0	100.05
Snow Showers	10.5	-1.3	44.0	0.0	25.0	101.02
Snow Showers,Fog	10.5	-1.3	44.0	4.0	48.3	101.04
Snow,Blowing Snow	17.8	8.4	52.0	0.0	25.0	101.00
Snow,Fog	-5.1	-15.1	45.0	0.0	24.1	101.00
Snow,Haze	-5.1	-15.1	45.0	7.0	24.1	101.03
Snow,Ice Pellets	4.5	-1.5	61.0	7.0	25.0	101.03
Thunderstorms	4.5	-1.5	65.0	11.0	25.0	101.04
Thunderstorms,Heavy Rain Showers	14.4	9.0	68.0	0.0	25.0	101.02
Thunderstorms,Moderate Rain Showers,Fog	14.9	9.1	68.0	0.0	25.0	101.02
Thunderstorms,Rain	14.0	9.1	68.0	0.0	25.0	101.02
Thunderstorms,Rain Showers	5.1	-1.0	65.0	6.0	25.0	101.04
Thunderstorms,Rain Showers,Fog	5.1	-1.0	65.0	6.0	25.0	101.04
Thunderstorms,Rain,Fog	14.0	9.1	68.0	0.0	25.0	101.02

max_min_df = pd.concat((min_df, max_df), axis = 1)

max_min_df

	Temp Min	DPT Min	Humidity Min	Wind Speed Min	Visibility Min	Pressure Min	Temp Min	DPT Min	Humidity Min	Wind Speed Min	Visibility Min	Pressure Min
Clear	6.4	4.4	71.0	0.0	25.0	100.54	17.4	12.1	87.0	19.0	25.0	101.34
Cloudy	12.7	-5.0	29.0	13.0	25.0	100.54	17.4	12.1	71.0	19.0	25.0	101.64
Drizzle	7.3	-5.0	29.0	13.0	25.0	101.64	12.7	-1.4	54.0	13.0	25.0	101.81
Drizzle,Fog	7.3	-1.4	54.0	13.0	25.0	101.25	15.4	7.9	61.0	20.0	25.0	101.81
Drizzle,Ice Pellets,Fog	14.9	7.9	61.0	0.0	25.0	101.02	15.4	9.1	68.0	20.0	25.0	101.25
Drizzle,Snow	14.4	9.0	68.0	0.0	25.0	101.02	14.9	9.1	70.0	11.0	48.3	101.04
Drizzle,Snow,Fog	4.8	-1.5	64.0	9.0	48.3	101.04	14.4	9.0	70.0	11.0	48.3	101.70
Fog	0.2	-5.5	64.0	9.0	25.0	101.13	4.8	-1.5	65.0	22.0	48.3	101.70
Freezing Drizzle	0.2	-5.5	48.0	7.0	25.0	101.03	22.2	10.6	65.0	22.0	48.3	101.13
Freezing Drizzle,Fog	4.5	-1.5	48.0	7.0	25.0	101.03	22.2	10.6	65.0	15.0	48.3	101.06
Freezing Drizzle,Haze	4.5	-1.5	65.0	6.0	25.0	101.04	14.0	9.2	73.0	15.0	48.3	101.06
Freezing Drizzle,Snow	14.0	9.2	60.0	6.0	25.0	101.01	25.8	17.4	73.0	11.0	48.3	101.04
Freezing Fog	17.8	10.5	60.0	4.0	25.0	101.01	25.8	17.4	62.0	11.0	48.3	101.04
Freezing Rain	7.4	-1.2	54.0	4.0	25.0	101.04	17.8	10.5	62.0	6.0	48.3	101.69
Freezing Rain,Fog	7.4	-1.2	54.0	4.0	25.0	101.04	17.8	10.5	62.0	6.0	48.3	101.69
Freezing Rain,Haze	14.4	9.0	62.0	4.0	48.3	101.04	17.8	10.5	70.0	11.0	48.3	101.04
Freezing Rain,Ice Pellets,Fog	14.4	9.0	68.0	0.0	25.0	101.02	14.9	9.1	70.0	11.0	48.3	101.04
Freezing Rain,Snow Grains	14.9	9.1	68.0	0.0	25.0	101.02	14.9	9.1	68.0	0.0	25.0	101.02
Haze	5.1	-1.0	65.0	0.0	25.0	101.02	14.9	9.1	68.0	13.0	25.0	101.83
Mainly Clear	0.5	-4.6	65.0	13.0	25.0	101.20	5.1	-1.0	69.0	28.0	25.0	101.83
Moderate Rain,Fog	0.5	-4.6	68.0	0.0	25.0	101.02	14.9	9.1	69.0	28.0	25.0	101.20
Moderate Snow	14.9	9.1	62.0	0.0	25.0	101.02	17.8	10.5	68.0	4.0	48.3	101.04
Moderate Snow,Blowing Snow	14.4	9.0	62.0	4.0	48.3	101.04	17.8	10.5	70.0	11.0	48.3	101.04
Mostly Cloudy	-8.8	-12.8	70.0	7.0	25.0	101.04	14.4	9.0	73.0	11.0	48.3	101.76
Rain	-8.8	-12.8	66.0	7.0	25.0	101.76	14.2	7.9	73.0	13.0	25.0	102.19
Rain Showers	-8.4	-12.4	66.0	9.0	25.0	101.86	14.2	7.9	73.0	13.0	25.0	102.19
Rain Showers,Fog	-8.4	-12.4	68.0	0.0	25.0	101.02	14.9	9.1	73.0	9.0	25.0	101.86
Rain Showers,Snow Showers	14.4	9.0	68.0	0.0	25.0	101.02	14.9	9.1	70.0	11.0	48.3	101.04
Rain,Fog	14.4	9.0	70.0	11.0	24.1	100.81	17.5	12.2	71.0	13.0	48.3	101.04
Rain,Haze	14.0	9.2	71.0	6.0	24.1	100.81	17.5	12.2	73.0	13.0	48.3	101.04
Rain,Ice Pellets	14.0	9.1	68.0	0.0	25.0	101.02	14.9	9.2	73.0	6.0	48.3	101.04
Rain,Snow	14.9	8.7	56.0	0.0	25.0	101.02	17.6	9.1	68.0	6.0	25.0	101.02
Rain,Snow Grains	14.9	8.7	56.0	0.0	25.0	101.02	17.6	9.1	68.0	6.0	25.0	101.02
Rain,Snow,Fog	14.9	9.1	68.0	0.0	25.0	101.02	14.9	9.1	68.0	0.0	25.0	101.02
Rain,Snow,Ice Pellets	14.9	9.1	62.0	0.0	25.0	101.02	17.8	10.5	68.0	4.0	48.3	101.04
Snow	17.8	8.1	37.0	4.0	48.3	100.05	23.5	10.5	62.0	30.0	48.3	101.04
Snow Pellets	14.9	8.1	37.0	0.0	25.0	100.05	23.5	9.1	68.0	30.0	48.3	101.02
Snow Showers	10.5	-1.3	44.0	0.0	25.0	101.02	14.9	9.1	68.0	20.0	48.3	101.69
Snow Showers,Fog	10.5	-1.3	44.0	4.0	48.3	101.04	17.8	10.5	62.0	20.0	48.3	101.69
Snow,Blowing Snow	17.8	8.4	52.0	0.0	25.0	101.00	18.3	10.5	62.0	4.0	48.3	101.04
Snow,Fog	-5.1	-15.1	45.0	0.0	24.1	101.00	18.3	8.4	52.0	15.0	25.0	101.79
Snow,Haze	-5.1	-15.1	45.0	7.0	24.1	101.03	20.0	12.2	61.0	15.0	48.3	101.79
Snow,Ice Pellets	4.5	-1.5	61.0	7.0	25.0	101.03	20.0	12.2	65.0	15.0	48.3	101.06
Thunderstorms	4.5	-1.5	65.0	11.0	25.0	101.04	14.4	9.0	70.0	15.0	48.3	101.06
Thunderstorms,Heavy Rain Showers	14.4	9.0	68.0	0.0	25.0	101.02	14.9	9.1	70.0	11.0	48.3	101.04
Thunderstorms,Moderate Rain Showers,Fog	14.9	9.1	68.0	0.0	25.0	101.02	14.9	9.1	68.0	0.0	25.0	101.02
Thunderstorms,Rain	14.0	9.1	68.0	0.0	25.0	101.02	14.9	9.2	73.0	6.0	48.3	101.04
Thunderstorms,Rain Showers	5.1	-1.0	65.0	6.0	25.0	101.04	14.0	9.2	73.0	13.0	48.3	101.83
Thunderstorms,Rain Showers,Fog	5.1	-1.0	65.0	6.0	25.0	101.04	14.0	9.2	73.0	13.0	48.3	101.83
Thunderstorms,Rain,Fog	14.0	9.1	68.0	0.0	25.0	101.02	14.9	9.2	73.0	6.0	48.3	101.04

---

Thriteenth Task: Show all the Records where the Weather Condition is Fog

#these function is used to filter specific individual strings 
fog_alone_df = sorted_data[sorted_data["Weather Condition"] == "Fog"]

#and this is used to filter a list of strings so we'll use it to to check all the strings that have Fog in them
fog_ex_df = sorted_data[sorted_data["Weather Condition"].isin(["Fog", "Snow,Fog"])]

fog_alone_df.equals(fog_ex_df)

False

foggy_conds = [i for i in cond_name if "Fog" in i]

foggy_conds

['Drizzle,Fog',
 'Drizzle,Ice Pellets,Fog',
 'Drizzle,Snow,Fog',
 'Fog',
 'Freezing Drizzle,Fog',
 'Freezing Fog',
 'Freezing Rain,Fog',
 'Freezing Rain,Ice Pellets,Fog',
 'Moderate Rain,Fog',
 'Rain Showers,Fog',
 'Rain,Fog',
 'Rain,Snow,Fog',
 'Snow Showers,Fog',
 'Snow,Fog',
 'Thunderstorms,Moderate Rain Showers,Fog',
 'Thunderstorms,Rain Showers,Fog',
 'Thunderstorms,Rain,Fog']

fog_df = sorted_data[sorted_data["Weather Condition"].isin(foggy_conds)]
fog_df

	Date/Time	Temp_C	Dew Point Temp_C	Rel Hum_%	Wind Speed_km/h	Visibility_km	Press_kPa	Weather Condition
1824	3/17/2012 0:00	3.8	3.5	98	4	1.6	102.05	Drizzle,Fog
6899	10/14/2012 11:00	6.7	6.4	98	11	3.2	101.11	Drizzle,Fog
6900	10/14/2012 12:00	6.7	6.3	97	13	2.4	101.00	Drizzle,Fog
6907	10/14/2012 19:00	9.6	9.1	97	15	9.7	100.48	Drizzle,Fog
1818	3/16/2012 18:00	3.8	3.2	96	7	4.8	101.71	Drizzle,Fog
...	...	...	...	...	...	...	...	...
4758	7/17/2012 6:00	19.6	18.5	93	15	3.2	100.01	Thunderstorms,Moderate Rain Showers,Fog
5108	7/31/2012 20:00	22.4	18.7	80	35	9.7	100.64	Thunderstorms,Rain Showers,Fog
4323	6/29/2012 3:00	19.5	16.1	81	7	9.7	99.71	Thunderstorms,Rain Showers,Fog
4761	7/17/2012 9:00	22.9	21.3	91	17	9.7	99.84	Thunderstorms,Rain Showers,Fog
4757	7/17/2012 5:00	20.6	18.6	88	19	4.8	100.08	Thunderstorms,Rain,Fog

426 rows × 8 columns

---

Fourteenth Task: Find all instances when Weather is Clear or Visibility is above 40.

clear_conds = [i for i in cond_name if "Clear" in i]

#here we pulled the number of times these conditions were met 
wea_vis = 0
for i in range(0, len(sorted_data["Visibility_km"])):
    if sorted_data.at[i, "Visibility_km"] > 40 and "Clear" in sorted_data.at[i, "Weather Condition"] : 
        wea_vis += 1
print("Weather was Clear at the same time Visibility was above 40 {} times in the Dataset".format(wea_vis))

Weather was Clear at the same time Visibility was above 40 1184 times in the Dataset

wea_vis_df = sorted_data[sorted_data["Weather Condition"].isin(clear_conds)]

wea_vis_df = wea_vis_df[wea_vis_df["Visibility_km"] > 40]

wea_vis_df.shape

(1184, 8)

---

Fifteenth Task: Find all instances when :
A. 'Weather is Clear' and 'Relative Humidity is greater than 50'
or
B. 'Visibility is above 40'

wea_rel_df = sorted_data[sorted_data["Weather Condition"].isin(clear_conds)]

wea_rel_df = wea_rel_df[wea_rel_df["Rel Hum_%"] > 50]

wea_rel_df.shape

(2604, 8)

vis_df = sorted_data[sorted_data["Visibility_km"] > 40]

vis_df.shape

(2014, 8)