skimpy logo *A light weight tool for creating summary statistics from dataframes.* [![PyPI](https://img.shields.io/pypi/v/skimpy.svg)](https://pypi.org/project/skimpy/) [![Status](https://img.shields.io/pypi/status/skimpy.svg)](https://pypi.org/project/skimpy/) [![Python Version](https://img.shields.io/pypi/pyversions/skimpy)](https://pypi.org/project/skimpy) [![License](https://img.shields.io/pypi/l/skimpy)](https://opensource.org/licenses/MIT) [![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff) [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit) [![Tests](https://github.com/aeturrell/skimpy/workflows/Tests/badge.svg)](https://github.com/aeturrell/skimpy/actions?workflow=Tests) [![Codecov](https://codecov.io/gh/aeturrell/skimpy/branch/main/graph/badge.svg)](https://codecov.io/gh/aeturrell/skimpy) [![Read the documentation at https://aeturrell.github.io/skimpy/](https://img.shields.io/badge/Go%20to%20the%20docs-purple?style=flat)](https://aeturrell.github.io/skimpy/) [![Downloads](https://img.shields.io/pepy/dt/skimpy.svg)](https://pepy.tech/projects/skimpy) [![Source](https://img.shields.io/badge/source%20code-github-lightgrey?style=for-the-badge)](https://github.com/aeturrell/skimpy)
![Linux](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black) ![macOS](https://img.shields.io/badge/mac%20os-000000?style=for-the-badge&logo=macos&logoColor=F0F0F0) ![Windows](https://img.shields.io/badge/Windows-0078D6?style=for-the-badge&logo=windows&logoColor=white) **skimpy** is a light weight tool that provides summary statistics about variables in **pandas** or **Polars** data frames within the console or your interactive Python window. Think of it as a super-charged version of **pandas**' `df.describe()`. [You can find the documentation here](https://aeturrell.github.io/skimpy/). ## Quickstart `skim` a **pandas** or **polars** dataframe and produce summary statistics within the console using: ```python from skimpy import skim skim(df) ``` where `df` is a **pandas** or **polars** dataframe. If you need to a dataset to try *skimpy* out on, you can use the built-in test **Pandas** data frame: ```python from skimpy import generate_test_data, skim df = generate_test_data() skim(df) ```
╭──────────────────────────────────────────────── skimpy summary ─────────────────────────────────────────────────╮
│          Data Summary                     Data Types                   Categories                               │
│ ┏━━━━━━━━━━━━━━━━━━━┳━━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┓ ┏━━━━━━━━━━━━━━━━━━━━━━━┓                       │
│ ┃ Dataframe         ┃ Values ┃ ┃ Column Type          ┃ Count ┃ ┃ Categorical Variables ┃                       │
│ ┡━━━━━━━━━━━━━━━━━━━╇━━━━━━━━┩ ┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━┩ ┡━━━━━━━━━━━━━━━━━━━━━━━┩                       │
│ │ Number of rows    │ 1000   │ │ float64              │ 3     │ │ class                 │                       │
│ │ Number of columns │ 13     │ │ category             │ 2     │ │ location              │                       │
│ └───────────────────┴────────┘ │ datetime64[us]       │ 2     │ └───────────────────────┘                       │
│                                │ date32[day][pyarrow] │ 2     │                                                 │
│                                │ int64                │ 1     │                                                 │
│                                │ bool                 │ 1     │                                                 │
│                                │ string               │ 1     │                                                 │
│                                │ timedelta64[us]      │ 1     │                                                 │
│                                └──────────────────────┴───────┘                                                 │
│                                                     number                                                      │
│ ┏━━━━━━━━━┳━━━━━━┳━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━┓  │
│ ┃ column  ┃ NA   ┃ NA %  ┃ mean      ┃ sd      ┃ p0         ┃ p25     ┃ p50        ┃ p75    ┃ p100  ┃ hist   ┃  │
│ ┡━━━━━━━━━╇━━━━━━╇━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━┩  │
│ │ length  │    0 │     0 │    0.5016 │  0.3597 │  1.573e-06 │   0.134 │     0.4976 │ 0.8602 │     1 │ █▃▃▃▄█ │  │
│ │  width  │    0 │     0 │     2.037 │   1.929 │   0.002057 │   0.603 │      1.468 │  2.953 │ 13.91 │  █▃▁   │  │
│ │  depth  │    0 │     0 │     10.02 │   3.208 │          2 │       8 │         10 │     12 │    20 │ ▁▄█▆▃▁ │  │
│ │   rnd   │  118 │  11.8 │  -0.01977 │   1.002 │     -2.809 │ -0.7355 │ -0.0007736 │ 0.6639 │ 3.717 │ ▁▄█▅▁  │  │
│ └─────────┴──────┴───────┴───────────┴─────────┴────────────┴─────────┴────────────┴────────┴───────┴────────┘  │
│                                                    category                                                     │
│ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┓  │
│ ┃ column                      ┃ NA         ┃ NA %            ┃ ordered                 ┃ unique              ┃  │
│ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━┩  │
│ │            class            │          0 │               0 │ False                   │                   2 │  │
│ │          location           │          1 │             0.1 │ False                   │                   5 │  │
│ └─────────────────────────────┴────────────┴─────────────────┴─────────────────────────┴─────────────────────┘  │
│                                                      bool                                                       │
│ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┓  │
│ ┃ column                          ┃ true             ┃ true rate                      ┃ hist                 ┃  │
│ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━┩  │
│ │            booly_col            │              516 │                           0.52 │        █    █        │  │
│ └─────────────────────────────────┴──────────────────┴────────────────────────────────┴──────────────────────┘  │
│                                                    datetime                                                     │
│ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓  │
│ ┃ column                       ┃ NA    ┃ NA %     ┃ first              ┃ last              ┃ frequency       ┃  │
│ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩  │
│ │           datetime           │     0 │        0 │     2018-01-31     │    2101-04-30     │       ME        │  │
│ │       datetime_no_freq       │     3 │      0.3 │     1992-01-05     │    2023-03-04     │       NA        │  │
│ └──────────────────────────────┴───────┴──────────┴────────────────────┴───────────────────┴─────────────────┘  │
│                                                      date                                                       │
│ ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━┓  │
│ ┃ column                            ┃ NA    ┃ NA %    ┃ first            ┃ last             ┃ frequency      ┃  │
│ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━┩  │
│ │           datetime.date           │     0 │       0 │ 2018-01-31       │ 2101-04-30       │       ME       │  │
│ │       datetime.date_no_freq       │     0 │       0 │ 1992-01-05       │ 2023-03-04       │       NA       │  │
│ └───────────────────────────────────┴───────┴─────────┴──────────────────┴──────────────────┴────────────────┘  │
│                                                timedelta64[ns]                                                  │
│ ┏━━━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓  │
│ ┃ column         ┃ NA   ┃ NA %    ┃ mean                   ┃ median                 ┃ max                    ┃  │
│ ┡━━━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩  │
│ │   time diff    │    5 │     0.5 │        8 days 00:05:47 │        6 days 00:00:00 │       26 days 00:00:00 │  │
│ └────────────────┴──────┴─────────┴────────────────────────┴────────────────────────┴────────────────────────┘  │
│                                                     string                                                      │
│ ┏━━━━━━━━┳━━━━┳━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┓  │
│ ┃        ┃    ┃      ┃            ┃           ┃            ┃           ┃ chars per  ┃ words per ┃ total      ┃  │
│ ┃ column ┃ NA ┃ NA % ┃ shortest   ┃ longest   ┃ min        ┃ max       ┃ row        ┃ row       ┃ words      ┃  │
│ ┡━━━━━━━━╇━━━━╇━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━┩  │
│ │  text  │  6 │  0.6 │  How are   │  Indeed,  │  How are   │   What    │       31.1 │       5.8 │       5761 │  │
│ │        │    │      │    you?    │  it was   │    you?    │ weather!  │            │           │            │  │
│ │        │    │      │            │ the most  │            │           │            │           │            │  │
│ │        │    │      │            │ outrageou │            │           │            │           │            │  │
│ │        │    │      │            │    sly    │            │           │            │           │            │  │
│ │        │    │      │            │  pompous  │            │           │            │           │            │  │
│ │        │    │      │            │   cat I   │            │           │            │           │            │  │
│ │        │    │      │            │ have ever │            │           │            │           │            │  │
│ │        │    │      │            │   seen.   │            │           │            │           │            │  │
│ └────────┴────┴──────┴────────────┴───────────┴────────────┴───────────┴────────────┴───────────┴────────────┘  │
╰────────────────────────────────────────────────────── End ──────────────────────────────────────────────────────╯
It is recommended that you set your datatypes before using **skimpy** (for example converting any text columns to pandas string datatype), as this will produce richer statistical summaries. However, the `skim()` function will try and guess what the datatypes of your columns are. ## Requirements You can find a full list of requirements in the [pyproject.toml](https://github.com/aeturrell/skimpy/blob/main/pyproject.toml) file. You can try this package out right now in your browser using this [Google Colab notebook](https://colab.research.google.com/gist/aeturrell/7bf183c559dc1d15ab7e7aaac39ea0ed/skimpy_demo.ipynb) (requires a Google account). Note that the Google Colab notebook uses the latest package released on PyPI (rather than the development release). ## Installation You can install the latest release of *skimpy* via [pip](https://pip.pypa.io/) from [PyPI](https://pypi.org/): ```bash $ pip install skimpy ``` To install the development version from git, use: ```bash $ pip install git+https://github.com/aeturrell/skimpy.git ``` For development, see [contributing](docs/contributing/contributing.qmd). ## License Distributed under the terms of the [MIT license](https://opensource.org/licenses/MIT), *skimpy* is free and open source software. ## Issues If you encounter any problems, please [file an issue](https://github.com/aeturrell/skimpy/issues) along with a detailed description. ## Credits This project was generated from [\@cjolowicz](https://github.com/cjolowicz)\'s [Hypermodern Python Cookiecutter](https://github.com/cjolowicz/cookiecutter-hypermodern-python) template. **skimpy** was inspired by the R package [**skimr**](https://docs.ropensci.org/skimr/articles/skimr.html) and by exploratory Python packages including [**ydata_profiling**](https://docs.profiling.ydata.ai) and [**dataprep**](https://dataprep.ai/), from which the `clean_columns` function comes. This package would not have been possible without the [**Rich**](https://github.com/Textualize/rich) package. The package is built with [uv](https://docs.astral.sh/uv/), while the documentation is built with [Quarto](https://quarto.org/) and [great-docs](https://posit-dev.github.io/great-docs) (a Python package). Tests are run with [nox](https://nox.thea.codes/en/stable/). Using **skimpy** in your paper? Let us know by raising an issue beginning with "citation" and we'll add it to this page.