{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Basic core"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This module contains all the basic functions we need in other modules of the fastai library (split with [`torch_core`](/torch_core.html#torch_core) that contains the ones requiring pytorch). Its documentation can easily be skipped at a first read, unless you want to know what a given function does."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [],
"source": [
"from fastai.gen_doc.nbdoc import *\n",
"from fastai.core import * "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Global constants"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`default_cpus = min(16, num_cpus())` "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Check functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> ifnone(`a`:`Any`, `b`:`Any`) → `Any`\n",
"\n",
"`a` if `a` is not None, otherwise `b`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(ifnone)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> is_listy(`x`:`Any`) → `bool`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(is_listy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check if `x` is a `Collection`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> is_tuple(`x`:`Any`) → `bool`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(is_tuple)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check if `x` is a `tuple`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Collection related functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> arrays_split(`mask`:`ndarray`, `arrs`:`NPArrayableList`) → `SplitArrayList`\n",
"\n",
"Given `arrs` is [a,b,...] and `mask`index - return[(a[mask],a[~mask]),(b[mask],b[~mask]),...]. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(arrays_split)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> extract_kwargs(`names`:`StrList`, `kwargs`:`KWArgs`)\n",
"\n",
"Extract the keys in `names` from the `kwargs`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(extract_kwargs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> get_chunk_length(`data`:`Union`\\[`Path`, `str`, `DataFrame`, `TextFileReader`\\], `chunksize`:`Optional`\\[`int`\\]=`None`) → `int`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(get_chunk_length, doc_string=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return the number of chunks we will have when opening the `DataFrame` in `csv_name` with `chunksize`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> get_total_length(`csv_name`:`PathOrStr`, `chunksize`:`int`) → `int`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(get_total_length, doc_string=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return the total length we will have when opening the `DataFrame` in `csv_name` with `chunksize`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> idx_dict(`a`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(idx_dict)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a dictionary value to index from `a`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'a': 0, 'b': 1, 'c': 2}"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"idx_dict(['a','b','c'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> listify(`p`:`OptListOrItem`=`None`, `q`:`OptListOrItem`=`None`)\n",
"\n",
"Make `p` same length as `q` "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(listify)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> random_split(`valid_pct`:`float`, `arrs`:`NPArrayableList`) → `SplitArrayList`\n",
"\n",
"Randomly split `arrs` with `valid_pct` ratio. good for creating validation set. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(random_split)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> series2cat(`df`:`DataFrame`, `col_names`)\n",
"\n",
"Categorifies the columns `col_names` in `df`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(series2cat)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> uniqueify(`x`:`Series`) → `List`\n",
"\n",
"Return unique values of `x` "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(uniqueify)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return the unique elements in `x`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Files management and downloads"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> download_url(`url`:`str`, `dest`:`str`, `overwrite`:`bool`=`False`, `pbar`:`ProgressBar`=`None`, `show_progress`=`True`)\n",
"\n",
"Download `url` to `dest` unless it exists and not `overwrite`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(download_url)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> find_classes(`folder`:`Path`) → `FilePathList`\n",
"\n",
"List of label subdirectories in imagenet-style `folder`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(find_classes)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> join_path(`fname`:`PathOrStr`, `path`:`PathOrStr`=`'.'`) → `Path`\n",
"\n",
"Return `Path(path)/Path(fname)`, `path` defaults to current dir. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(join_path)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> join_paths(`fnames`:`FilePathList`, `path`:`PathOrStr`=`'.'`) → `FilePathList`\n",
"\n",
"Join `path` to every file name in `fnames`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(join_paths)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> loadtxt_str(`path`:`PathOrStr`) → `ndarray`\n",
"\n",
"Return `ndarray` of `str` of lines of text from `path`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(loadtxt_str)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> maybe_copy(`old_fnames`:`Collection`\\[`PathOrStr`\\], `new_fnames`:`Collection`\\[`PathOrStr`\\])\n",
"\n",
"Copy the `old_fnames` to `new_fnames` location if `new_fnames` don't exist or are less recent. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(maybe_copy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Others"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> ItemBase()\n",
"\n",
"All transformable dataset items use this type. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(ItemBase, title_level=3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> camel2snake(`name`:`str`) → `str`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(camel2snake)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Format `name` by removing capital letters from a class-style name and separates the subwords with underscores."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'device_data_loader'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"camel2snake('DeviceDataLoader')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> even_mults(`start`:`float`, `stop`:`float`, `n`:`int`) → `ndarray`\n",
"\n",
"Build evenly stepped schedule from `start` to `stop` in `n` steps. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(even_mults)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> noop(`x`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(noop)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return `x`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> num_cpus() → `int`\n",
"\n",
"Get number of cpus "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(num_cpus)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> partition(`a`:`Collection`, `sz`:`int`) → `List`\\[`Collection`\\]\n",
"\n",
"Split iterables `a` in equal parts of size `sz` "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(partition)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"partition_by_cores[source]
\n",
"\n",
"> partition_by_cores(`a`:`Collection`, `n_cpus`:`int`) → `List`\\[`Collection`\\]\n",
"\n",
"Split data in `a` equally among `n_cpus` cores "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(partition_by_cores)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Undocumented Methods - Methods moved below this line will intentionally be hidden"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## New Methods - Please document or move to the undocumented section"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> range_of(`x`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(range_of)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> to_int(`b`:`Any`) → `Union`\\[`int`, `List`\\[`int`\\]\\]\n",
"\n",
"Convert `b` to an int or list of ints (if [`is_listy`](/core.html#is_listy)); raises exception if not convertible "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(to_int)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> arange_of(`x`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(arange_of)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
"jekyll": {
"keywords": "fastai",
"summary": "Basic helper functions for the fastai library",
"title": "core"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}