\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Cross Column Statistics\n",
"\n",
"Amazon SageMaker Autopilot calculates Pearson’s correlation between columns in your dataset.\n",
"Removing highly correlated columns can reduce overfitting and training time.\n",
"Pearson’s correlation is in the range [-1, 1] where 0 implies no correlation, 1 implies perfect correlation,\n",
"and -1 implies perfect inverse correlation.\n",
"\n",
"No correlation scores were calculated because the calculation took too long to complete.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Anomalous Rows\n",
"\n",
"Anomalous rows are detected using the Isolation forest algorithm on a sample of\n",
"**7110**\n",
"randomly chosen\n",
"rows after basic preprocessing. The isolation forest algorithm associates an anomaly score to each row of the dataset\n",
"it is trained on. Rows with negative anomaly scores are usually considered anomalous and rows with positive anomaly\n",
"scores are considered non-anomalous. When investigating an anomalous row, look for any unusual values -\n",
"in particular any that might have resulted from errors in the gathering and processing of data.\n",
"Deciphering whether a row is indeed anomalous, contains errors, or is in fact valid requires domain knowledge and\n",
"application of business logic.\n",
"\n",
"There were no anomalous rows found in the dataset.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Missing Values\n",
"Within the data sample, the following columns contained missing values, such as: `nan`, white spaces, or empty fields.\n",
"\n",
"SageMaker Autopilot will attempt to fill in missing values using various techniques. For example,\n",
"missing values can be replaced with a new 'unknown' category for `Categorical` features\n",
"and missing `Numerical` values can be replaced with the **mean** or **median** of the column.\n",
"\n",
"We found **0 of the 2** of the columns contained missing values.\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Cardinality\n",
"For `String` features, it is important to count the number of unique values to determine whether to treat a feature as `Categorical` or `Text`\n",
"and then processes the feature according to its type.\n",
"\n",
"For example, SageMaker Autopilot counts the number of unique entries and the number of unique words.\n",
"The following string column would have **3** total entries, **2** unique entries, and **3** unique words.\n",
"\n",
"| | String Column |\n",
"|-------|-------------------|\n",
"| **0** | \"red blue\" |\n",
"| **1** | \"red blue\" |\n",
"| **2** | \"red blue yellow\" |\n",
"\n",
"If the feature is `Categorical`, SageMaker Autopilot can look at the total number of unique entries and transform it using techniques such as one-hot encoding.\n",
"If the field contains a `Text` string, we look at the number of unique words, or the vocabulary size, in the string.\n",
"We can use the unique words to then compute text-based features, such as Term Frequency-Inverse Document Frequency (tf-idf).\n",
"\n",
"**Note:** If the number of unique values is too high, we risk data transformations expanding the dataset to too many features.\n",
"In that case, SageMaker Autopilot will attempt to reduce the dimensionality of the post-processed data,\n",
"such as by capping the number vocabulary words for tf-idf, applying Principle Component Analysis (PCA), or other dimensionality reduction techniques.\n",
"\n",
"The table below shows **2 of the 2** columns ranked by the number of unique entries.\n",
"\n",
"
💡 Suggested Action Items\n",
"\n",
"- Verify the number of unique values of a feature is as expected.\n",
" One explanation for unexpected number of unique values could be multiple encodings of a value.\n",
" For example `US` and `U.S.` will count as two different words.\n",
" You could correct the error at the data source or pre-process your dataset in your S3 bucket.\n",
"- If the number of unique values seems too high for Categorical variables,\n",
" investigate if multiple unique values can be grouped into a smaller set of possible values.\n",
"
\n",
"\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Number of Unique Entries | \n",
" Number of Unique Words (if Text) | \n",
"
\n",
" \n",
" \n",
" \n",
" | sentiment | \n",
" 3 | \n",
" n/a | \n",
"
\n",
" \n",
" | review_body | \n",
" 7108 | \n",
" 17986 | \n",
"
\n",
" \n",
"
\n",
"
"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Descriptive Stats\n",
"For each of the input features that has at least one numeric value, several descriptive statistics are computed from the data sample.\n",
"\n",
"SageMaker Autopilot may treat numerical features as `Categorical` if the number of unique entries is sufficiently low.\n",
"For `Numerical` features, we may apply numerical transformations such as normalization, log and quantile transforms,\n",
"and binning to manage outlier values and difference in feature scales.\n",
"\n",
"We found **1 of the 2** columns contained at least one numerical value.\n",
"The table below shows the **1** columns which have the largest percentage of numerical values.\n",
"Percentage of outliers is calculated only for columns which Autopilot detected to be of numeric type. Percentage of outliers is\n",
"not calculated for the target column.\n",
"\n",
"
💡 Suggested Action Items\n",
"\n",
"- Investigate the origin of the data field. Are some values non-finite (e.g. infinity, nan)?\n",
" Are they missing or is it an error in data input?\n",
"- Missing and extreme values may indicate a bug in the data collection process.\n",
" Verify the numerical descriptions align with expectations.\n",
" For example, use domain knowledge to check that the range of values for a feature meets with expectations.\n",
"
\n",
"\n",
"\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" % of Numerical Values | \n",
" Mean | \n",
" Median | \n",
" Min | \n",
" Max | \n",
"
\n",
" \n",
" \n",
" \n",
" | sentiment | \n",
" 100.0% | \n",
" 0.0 | \n",
" 0.0 | \n",
" -1.0 | \n",
" 1.0 | \n",
"
\n",
" \n",
"
\n",
"
\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definitions\n",
"\n",
"### Feature types\n",
"\n",
"**Numeric:** Numeric values, either floats or integers. For example: age, income. When training a machine learning model, it is assumed that numeric values are ordered and a distance is defined between them. For example, 3 is closer to 4 than to 10 and 3 < 4 < 10.\n",
"\n",
"**Categorical:** The column entries belong to a set of unique values that is usually much smaller than number of rows in the dataset. For example, a column from datasets with 100 rows with the unique values \"Dog\", \"Cat\" and \"Mouse\". The values could be numeric, textual, or combination of both. For example, \"Horse\", \"House\", 8, \"Love\" and 3.1 are\n",
"all valid values and can be found in the same categorical column. When manipulating column of categorical values, a machine learning model does not assume that they are ordered or that distance function is defined on them, even if all of the values are numbers.\n",
"\n",
"**Binary:** A special case of categorical column for which the cardinality of the set of unique values is 2.\n",
"\n",
"**Text:** A text column that contains many non-numeric unique values, often a human readable text. In extreme cases, all the elements of the column are unique, so no two entries are the same.\n",
"\n",
"**Datetime:** This column contains date and/or time information.\n",
"\n",
"### Feature statistics\n",
"\n",
"**Prediction power:** Prediction power of a column (feature) is a measure of how useful it is for predicting the target variable. It is measured using a stratified split into 80%/20% training and validation folds. We fit a model for each feature separately on the training fold after applying minimal feature pre-processing and measure prediction performance on the validation data. The scores are normalized to the range [0,1]. A higher prediction power score near 1 indicate that a column is more useful for predicting the target on its own. A lower score near 0 indicate that a column contains little useful information for predicting the target on their own. Although it is possible that a column that is uninformative on its own can be useful in predicting the target when used in tandem with other features, a low score usually indicates the feature is redundant. A score of 1 implies perfect predictive abilities, which often indicates an error called target leakage. The cause is typically a column present in dataset that is hard or impossible to obtain at prediction time, such as a duplicate of the target.\n",
"\n",
"**Outliers:** Outliers are detected using two statistics that are robust to outliers: median and robust standard deviation (RSTD). RSTD is derived by clipping the feature values to the range [5 percentile, 95 percentile] and calculating the standard deviation of the clipped vector. All values larger than median + 5 * RSTD or smaller than median - 5 * RSTD are considered to be outliers.\n",
"\n",
"**Skew:** Skew measures the symmetry of the distribution and is defined as the third moment of the distribution divided by the third power of the standard deviation. The skewness of the normal distribution or any other symmetric distribution is zero. Positive values imply that the right tail of the distribution is longer than the left tail. Negative values imply that the left tail of the distribution is longer than the right tail. As a thumb rule, a distribution is considered skewed when the absolute value of the skew is larger than 3.\n",
"\n",
"**Kurtosis:** Pearson's kurtosis measures the heaviness of the tail of the distribution and is defined as the fourth moment of the distribution divided by the fourth power of the standard deviation. The kurtosis of the normal distribution is 3. Thus, kurtosis values lower than 3 imply that the distribution is more concentrated around the mean and the tails are lighter than the tails of the normal distribution. Kurtosis values higher than 3 imply heavier tails than the normal distribution or that the data contains outliers.\n",
"\n",
"**Missing Values:** Empty strings and strings composed of only white spaces are considered missing.\n",
"\n",
"**Valid values:**\n",
"\n",
"* **Numeric features / regression target:** All values that could be casted to finite floats are valid. Missing values are not valid.\n",
"* **Categorical / binary / text features / classification target:** All values that are not missing are valid.\n",
"* **Datetime features:** All values that could be casted to datetime object are valid. Missing values are not valid.\n",
"\n",
"**Invalid values:** values that are either missing or that could not be casted to the desired type. See the definition of valid values for more information"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "conda_python3",
"language": "python",
"name": "conda_python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}