# Publicly Available Datasets For Electric Load Forecasting
A (hopefully eventually) complete listing of the most popular electric LF datasets
### Why?
We found it difficult to find suitable datasets in the flood of information.
So we came up with the idea of doing a proper search and making the results available to the public.
### What?
Based on a sample set of representative publications, relevant, publicly accessible data sets were extracted, structured and analyzed.
The details of the search can be found in the scientific publication: [https://doi.org/10.15488/17659](https://doi.org/10.15488/17659)
### Improvements? π€
We are happy about any kind of cooperation, feedback or extension to make the list even more valuable for other scientists.
So feel free to expand the list and initiate a pull request.
# The list
| ID | Abbrev | Name | Domain1 | Resolution2 | Features3 | Duration4 | Spanned years | Horizons5 | Regions6 | Type7 | Links | Access8 |
| -------------- | ---------------- | ------------------------------------------------------------------------------------------------------------- |--------------------| ---------------------- | -------------------- | -------------------- | ------------- | -------------------- | ------------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ |
| 1 | ISO-NE | New England Independent System Operator | S | 60 | E | 108 | 2003-2014 | ββοΈβοΈβ | βοΈ | π¦ | [πLink](https://www.iso-ne.com/isoexpress/web/reports/pricing/-/tree/zone-info) | π |
| 2 | NYISO | New York Independent System Operator | S | 5 | E | 264 | 2001-2023 | βοΈβοΈβοΈβ | βοΈ | π¦ | [πLink](http://mis.nyiso.com/public/P-58Blist.htm) | π |
| 3 | PJM | PJM Hourly Energy Consumption | S | 60 | E | 240 | 1998-2018 | ββοΈβοΈβοΈ | βοΈ | π¦ | [πLink](https://www.kaggle.com/datasets/robikscube/hourly-energy-consumption?resource=download) | π |
| 4 | CIF | CIF 2016 competition dataset | ? | d,m,y | Undef. | 8-909 | unknown | βββοΈβοΈ | β | π¦ | [πLink](https://irafm.osu.cz/cif2015/main.php?c=Static&page=download) | π |
| 5 | GEFCOM14 | GEFCom 2014 | S | 60 | E, W, T, PV | 10 | 2021 | ββοΈββ | β | π¦ | [πLink](https://www.dropbox.com/s/pqenrr2mcvl0hk9/GEFCom2014.zip?dl=0&file_subpath=%2FGEFCom2014+Data) | π |
| 6 | EUNITE | EUNITE 2001 | S | 30 | E, T, H | 24 | 1997-1999 | ββοΈβοΈβ | β | π¦ | [πLink](https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/regression.html) | π |
| 7 | ENTSO-E | ENTSO-E electric load dataset | S | 60 | E | <=288 | till 2015 | ββοΈβοΈβοΈ | βοΈ | π¦ | [πLink](https://www.entsoe.eu/publications/statistics-and-data/) | π |
| 299 | EWELD | Large-Scale Industrial and Commercial Load Dataset in Extreme Weather Events | I | 15 | E, W, xW | <=74 | 2016-2022 | βοΈβοΈβοΈβοΈ | βοΈ (386) | π¦ | [πLink](https://www.nature.com/articles/s41597-023-02503-6) | π |
| 289 | WPuQ | Electrical single-family house and heat pump load | R | <1 | E | 30 | 2018-2020 | ββοΈβοΈβ | βοΈ (38) | π¦ | [πLink](https://www.nature.com/articles/s41597-022-01156-1) | π |
| 329 | PanETESA | Panama ETESA | S | 60 | E, W, H | 66 | 2015-2020 | ββοΈβοΈβοΈ | β | π¦ | [πLink](https://www.kaggle.com/datasets/ernestojaguilar/shortterm-electricity-load-forecasting-panama) | π |
| 389 | REFIT | REFIT: Electrical Load Measurements | R | 8sec | E | 20 | 2013-2015 | βοΈβοΈβοΈβ | βοΈ(20) | π¦ | [πLink1](https://doi.org/10.15129/31da3ece-f902-4e95-a093-e0a9536983c4) [πLink2](https://doi.org/10.5281/zenodo.5063428) | π |
| 399 | ECD-UY | household electricity consumption dataset of Uruguay | S, R | 1-15 | E | 11-23 | 2019-2020 | βοΈβοΈββ | βοΈ(9) | π¦ | [πLink1](https://www.nature.com/articles/s41597-022-01122-x) [πLink2](https://doi.org/10.6084/m9.figshare.c.5428608.v1) | π |
| 409 | IDEAL | IDEAL UK Household Energy Dataset 255 | R | 1-12sec | E, W, T | 23 | 2019-2020 | βοΈβοΈβοΈβ | βοΈ(255) | π¦ | [πLink1](https://www.nature.com/articles/s41597-021-00921-y) [πLink2](https://doi.org/10.7488/ds/2836) | π |
| 419 | HANOI-Res | Residential Apartments Dataset Hanoi, Vietnam (CAMaRSEC Project) | R | 15 | E, W, T | 12 | 2020-2021 | βοΈβοΈββ | βοΈ(49) | π¦ | [πLink1](https://data.mendeley.com/datasets/s9wkdww94w/2) [πLink2](https://doi.org/10.17632/s9wkdww94w.2) | π |
| 429 | UK-DALE | UK Domestic Appliance Level Electricity (UKERC EDC), Disaggregated (6s) and aggregated (1s) | R | 1-6sec | E | 5-53 | 2012-2017 | βοΈβοΈβοΈβοΈ | βοΈ(5) | π¦ | [πLink1](https://data.ukedc.rl.ac.uk/browse/edc/efficiency/residential/EnergyConsumption/Domestic/UK-DALE-2017/ReadMe_DALE-2017.html) [πLink2](http://www.nature.com/articles/sdata20157) [πLink3](https://doi.org/10.1038/sdata.2015.7) | π |
| 449 | ELMAS | Hourly electrical load profiles (18 aggregated curves: 1 for each industrial sector) [no individual profiles] | I | 60 | E, T | 12 | 2018 | ββοΈββ | β | π¦ | [πLink1](https://doi.org/10.6084/m9.figshare.23889780) | π |
| 8 | LCL | LCL Load Dataset (London Households) | R | 30 | E | 12 | 2013 | ββοΈββ | β | π | [πLink](https://data.london.gov.uk/dataset/smartmeter-energy-use-data-in-london-households) | π |
| 9 | SET | Energy Consumption Dataset for Milano/Trento | S | 10 | E | <1 | 2013 | βοΈβββ | β | π | [πLink](https://www.nature.com/articles/sdata201555) | π |
| 10 | BDG-Proj | Building Data Genome Project | S | 60 | E | 12 | unknown | ββοΈββ | βοΈ | π | [πLink](https://github.com/buds-lab/the-building-data-genome-project) | π |
| 349 | BDG-Proj2 | Building Data Genome Project 2 (BDG2) | R | 60 | E | 24 | 2016-2017 | ββοΈβοΈβ | βοΈ (1636) | π | [πLink](https://github.com/buds-lab/building-data-genome-project-2) | π |
| 11 | IHPC | Individual Household power consumption | S | 1 | E | 48 | 2006-2010 | βοΈβοΈβοΈβοΈ | β | π | [πLink](https://archive.ics.uci.edu/dataset/235/individual+household+electric+power+consumption) | π |
| 12 | GEFCOM12 | GEFCom 2012 | S | 60 | E, W, T | 42 | 2004-2008 | ββοΈβοΈβ | β | π | [πLink](https://www.kaggle.com/c/global-energy-forecasting-competition-2012-load-forecasting/) | π |
| 13 | OPSD-TS | Open Power System Data TS | S | 15-60 | E, PV, W | 148 | 2005-2019 | βοΈβοΈβοΈβοΈ | βοΈ | π | [πLink](https://doi.org/10.25832/time_series/2019-06-05) | π |
| 279 | OPSD-HH | Open Power System Data Household Data | R, I | 1-60 | E, PV | diff | 2012-2019 | βοΈβοΈβοΈβοΈ | βοΈ | π | [πLink](https://data.open-power-system-data.org/household_data/) | π |
| 14 | ELD | ElectricityLoadDiagrams20112014 | S | 15 | E | 36 | 2011-2014 | βοΈβοΈβοΈβοΈ | β | π | [πLink1](https://archive.ics.uci.edu/dataset/321/electricityloaddiagrams20112014) [πLink2](https://doi.org/10.24432/C58C86) | π |
| 15 | ENERTALK | ENERTALK Dataset Korea (household) | S | 15 hz | E | 12 | 2016 | βοΈβοΈββ | β | π | [πLink](https://www.nature.com/articles/s41597-019-0212-5) | π |
| 16 | S-TSO | Spanish Transmission Service operator (TSO) | H | 60 | >25 | 24 | 2017-2018 | ββοΈβοΈβ | β | π | [πLink](https://www.kaggle.com/datasets/nicholasjhana/energy-consumption-generation-prices-and-weather) | π |
| 269 | CER | CER Smart Metering Project | R,I | 30 | E | 18 | 2009-2010 | ββοΈβοΈβ | βοΈ(5237) | π | [πLink](https://www.ucd.ie/issda/data/commissionforenergyregulationcer/) | π§ |
| 309 | DEDDIAG | domestic electricity demand dataset (individual appliances in Germany) | R | 1Hz | E | 2-44 | 2011-2014 | βοΈβοΈβοΈβ | βοΈ(14) | π | [πLink1]( https://www.nature.com/articles/s41597-021-00963-2) [πLink2](https://figshare.com/articles/dataset/DEDDIAG_a_domestic_electricity_demand_dataset_of_individual_appliances_in_Germany/13615073) | π |
| 319 | AusSmartGrid | Electricity Use Interval Reading | R | 60 | E | ? | 2010-2014 | ββοΈβοΈβ | βοΈ | π | [πLink](https://data.gov.au/data/dataset/smart-grid-smart-city-customer-trial-data/resource/b71eb954-196a-4901-82fd-69b17f88521e) | π |
| 359 | UK-GRID | Electricity consumption UK 2009-2024 | S | 30 | E | 180 | 2009-2024 | ββοΈβοΈβοΈ | β | π | [πLink](https://www.kaggle.com/datasets/albertovidalrod/electricity-consumption-uk-20092022) | π |
| 369 | HoustonRes | Houston Residential power usage (one house) | R | 60 | E, W | 49 | 2016-2020 | ββοΈβοΈβ | β | π | [πLink](https://www.kaggle.com/datasets/srinuti/residential-power-usage-3years-data-timeseries) | π |
| 379 | CU-BEMS-Bangkok | Bangkok CU-BEMS, smart building energy and IAQ data | R | 1 | E, W | 18 | 2018-2019 | βοΈβοΈβοΈβ | β | π | [πLink](https://www.kaggle.com/datasets/claytonmiller/cubems-smart-building-energy-and-iaq-data) | π |
| 439 | 5359 VEA loadd | 5359 industrial VEA load profiles | I | 15 | E | 12 | 2016 | βοΈβοΈββ | βοΈ(5359) | π | [πLink](https://zenodo.org/records/13910298) | π |
| 459 | Germ-Industry-16 | 20 industrial load profiles for german plants | I | 15 | E | 12 | 2016 | βοΈβοΈββ | β | π | [πLink](https://doi.org/10.5281/zenodo.3899018) | π |
| 469 | Germ-Industry-17 | 30 industrial load profiles for german plants | I | 15 | E | 12 | 2017 | βοΈβοΈββ | β | π | [πLink](https://doi.org/10.5281/zenodo.3899018) | π |
| 17 | RTE-France | RTE France | S | 30 | E | 12 | 2012-2020 | ββοΈββ | βοΈ | π | [πLink](https://www.rte-france.com/en/eco2mix/download-indicators) | π |
| 18 | AEMO | Australian Energy market operator | H | 60 | E | 12 | 2013 | ββοΈββ | βοΈ | π | [πLink](https://aemo.com.au/energy-systems/electricity/national-electricity-market-nem/data-nem/network-data) | π |
| 19 | IESO-O | IESO Ontario | H | 60 | E, P | 20+ | 2022-2023 | ββοΈβοΈβ | β | π | [πLink](https://www.ieso.ca/en/Power-Data/Data-Directory) | π |
| 20 | AESO | Alberta Electric Sys. Op. Electrical Load Dataset | S | 60 | E | 132 | 2005-2016 | ββοΈβοΈβοΈ | β | π | [πLink](https://www.aeso.ca/market/market-and-system-reporting/data-requests/) | π |
| 21 | PPS | Polish power system | S | 15-60 | E | 120+ | 2013- now | βοΈβοΈβοΈβοΈ | β | π | [πLink](https://www.pse.pl/web/pse-eng/data/polish-power-system-operation/load-of-polish-power-system) | π |
| 22 | AUSGRID | Ausgrid: Distribution zone substation | S | 15 | E | 204 | 2005-2022 | βοΈβοΈβοΈβοΈ | βοΈ(>100) | π | [πLink](https://www.ausgrid.com.au/Industry/Our-Research/Data-to-share/Distribution-zone-substation-data) | π |
| 23 | KPX | KPX Korea | H | 5 | E | 240 | 2003-now | βοΈβοΈβοΈβοΈ | β | π | [πLink](https://epsis.kpx.or.kr/epsisnew/selectEkgeEpsMepRealChart.do?menuId=030300) | π |
| 24 | ADMIE | Independent Electricity Transmission Operator | S | 60 | E | 120+ | 2011-now | ββοΈβοΈβοΈ | βοΈ | π | [πLink](https://www.admie.gr/en/market/market-statistics/detail-data) | π |
| 25 | Pecan | Pecan Street dataset | S | 15 | E, W | 24 | 2017-2018 | βοΈβοΈβοΈβ | βοΈ | π | [πLink](https://dataport.pecanstreet.org/) | π |
| 339 | Cal-ISO | California ISO Hourly Load Data | S | 60 | E | 100+ | 2014-now | ββοΈβοΈβοΈ | βοΈ | π | [πLink1](https://www.caiso.com/generation-transmission/resource-adequacy#Historical) [πLink2](https://bigdata.seas.gwu.edu/data-set-15-california-iso-load-data-set/) | π |
#### Legend
1Domain: Either system level load (S), residential load (R) or Industry (I)
2Resolution: In minutes, if not other stated (d=day, m=month, y=year, hz=1sec)
3Features: Electricity (E), Weather (W), Extreme Weather Events, e.g. heat periods and taifune (xW), Temperature (T), Photovoltaic production (PV), Holiday features (H), Price (P)
4Duration: in number of months
5Forecasting-Horizons for modeling applicable: Very Short Term (VST), Short Term (ST), Medium Long Term (MT), Long Term (LT)
6Dataset records multiple regions / consumers separately (e.g. buildings, cities, countries) or disaggregated single loads available. Numbers in brackets indicate the number of regions / consumers / loads
7Type: Either π¦ = a collection (accumulation of datasets), π=a file or achive or π=a data platform / API
8Access: Either π = can be accessed directly (no login, no request), π§ = written application / request has to be sent first
9 not part of the original Paper, added later (only here)
*for further details, take a look at the publication below ‡οΈ*
# In a Rush? Use Our Python-Package:
[](https://pypi.org/project/padelf/)
[](https://pypi.org/project/padelf/)
[](https://opensource.org/licenses/MIT)
[](https://github.com/LSB-dev/PADELF-PIP)
**Installation**
```bash
pip install padelf
```
**Use**
```python
import padelf
# Load a dataset - one line, sensible defaults
df = padelf.get_dataset("OPSD")
# show some lines
print(df.head())
```
**Output**
```
consumption_kW DE_wind_onshore_generation_actual
datetime
2015-01-01 00:00:00+00:00 41209.0 7568.0
2015-01-01 01:00:00+00:00 40029.0 7666.0
2015-01-01 02:00:00+00:00 38891.0 7637.0
```
See [padelf-pip](https://github.com/LSB-dev/PADELF-PIP) for more details.
# Overwhelmed? Use Our Interactive Search Tool:
Finding the right dataset for your task can be hard.
Use our [PADELF Search](https://padelf.ipa.fraunhofer.de/) Online-Dashboard to filter the above table on-the-fly.
Simply specify your required filters and get the subset that is useful for you.
# How to cite
If this work has helped you with your scientific work, we would appreciate a proper mention. β€οΈ
Our citation recommendation is:
```
Baur, L.; Chandramouli, V.; Sauer, A.: Publicly Available Datasets For Electric Load Forecasting β An Overview. In: Herberger, D.; HΓΌbner, M. (Eds.): Proceedings of the CPSL 2024. Hannover : publish-Ing., 2024, S. 1-12. DOI: https://doi.org/10.15488/17659
```
BibTeX entry
```
@inproceedings{baur2024datasets,
author = {Baur, Lukas and Chandramouli, Vignesh and Sauer, Alexander},
title = {Publicly Available Datasets For Electric Load Forecasting β An Overview},
booktitle = {Proceedings of the CPSL 2024},
editor = {Herberger, D. and HΓΌbner, M.},
location = {Hannover},
publisher = {publish-Ing.},
year = {2024},
pages = {1--12},
doi = {10.15488/17659}
}
```
# How to contribute
See how to contribute in the [CONTRIBUTING.md](https://github.com/LSB-dev/Publicly-Available-Datasets-For-Electric-Load-Forecasting/blob/main/CONTRIBUTING.md)
# Acknowledgements
π° We'd like to thank the German Federal Ministry of Economic Affairs and Climate Action (**BMWK**) and the project supervision of the Project Management JΓΌlich (**PtJ**) for the project βFlexGUIdeβ which allowed for the work.
π‘ We would also like to thank an **anonymous reviewer** who suggested publishing the datasets not only in the above-mentioned publication but also as a repository.
π¨βπ We would like to thank K. Kunkel, whose master's thesis contributed greatly to the expansion of the initial dataset collection.
𧨠We'd like to thank G. Schmid, who contributed towards the overall vision to path a way through the dataset jungle by working on an interactive search dashboard and the pip-package.