{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Basic core"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This module contains all the basic functions we need in other modules of the fastai library (split with [`torch_core`](/torch_core.html#torch_core) that contains the ones requiring pytorch). Its documentation can easily be skipped at a first read, unless you want to know what a given function does."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [],
"source": [
"from fastai.gen_doc.nbdoc import *\n",
"from fastai.core import * "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Global constants"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`default_cpus = min(16, num_cpus())` "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Check functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> has_arg(`func`, `arg`) → `bool`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(has_arg)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check if `func` accepts `arg` as an argument."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> ifnone(`a`:`Any`, `b`:`Any`) → `Any`\n",
"\n",
"`a` if `a` is not None, otherwise `b`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(ifnone)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> is1d(`a`:`Collection`) → `bool`\n",
"\n",
"Returns True if a collection is one dimensional "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(is1d)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> is_listy(`x`:`Any`) → `bool`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(is_listy)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check if `x` is a `Collection`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> is_tuple(`x`:`Any`) → `bool`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(is_tuple)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check if `x` is a `tuple`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> try_int(`o`:`Any`) → `Any`\n",
"\n",
"Try to conver `o` to int, default to `o` if not possible. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(try_int)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Collection related functions"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> arange_of(`x`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(arange_of)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return the numpy array of the range of the same length as `x`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> arrays_split(`mask`:`ndarray`, `arrs`:`NPArrayableList`) → `SplitArrayList`\n",
"\n",
"Given `arrs` is [a,b,...] and `mask`index - return[(a[mask],a[~mask]),(b[mask],b[~mask]),...]. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(arrays_split)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> df_names_to_idx(`names`:`Union`\\[`int`, `Collection`\\[`int`\\], `str`, `StrList`\\], `df`:`DataFrame`)\n",
"\n",
"Return the column indexes of `names` in `df`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(df_names_to_idx)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> extract_kwargs(`names`:`StrList`, `kwargs`:`KWArgs`)\n",
"\n",
"Extract the keys in `names` from the `kwargs`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(extract_kwargs)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> idx_dict(`a`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(idx_dict)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a dictionary value to index from `a`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"{'a': 0, 'b': 1, 'c': 2}"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"idx_dict(['a','b','c'])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> index_row(`a`:`Collection`, `idxs`:`Collection`\\[`int`\\]) → `Any`\n",
"\n",
"Return the slice of `a` corresponding to `idxs`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(index_row)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> listify(`p`:`OptListOrItem`=`None`, `q`:`OptListOrItem`=`None`)\n",
"\n",
"Make `p` same length as `q` "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(listify)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> random_split(`valid_pct`:`float`, `arrs`:`NPArrayableList`) → `SplitArrayList`\n",
"\n",
"Randomly split `arrs` with `valid_pct` ratio. good for creating validation set. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(random_split)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> range_of(`x`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(range_of)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Make a `range` of the same size as `x`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> series2cat(`df`:`DataFrame`, `col_names`)\n",
"\n",
"Categorifies the columns `col_names` in `df`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(series2cat)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> to_int(`b`:`Any`) → `Union`\\[`int`, `List`\\[`int`\\]\\]\n",
"\n",
"Convert `b` to an int or list of ints (if [`is_listy`](/core.html#is_listy)); raises exception if not convertible "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(to_int)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> uniqueify(`x`:`Series`) → `List`\n",
"\n",
"Return unique values of `x` "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(uniqueify)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Files management and downloads"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> download_url(`url`:`str`, `dest`:`str`, `overwrite`:`bool`=`False`, `pbar`:`ProgressBar`=`None`, `show_progress`=`True`, `chunk_size`=`1048576`, `timeout`=`4`)\n",
"\n",
"Download `url` to `dest` unless it exists and not `overwrite`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(download_url)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> find_classes(`folder`:`Path`) → `FilePathList`\n",
"\n",
"List of label subdirectories in imagenet-style `folder`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(find_classes)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> join_path(`fname`:`PathOrStr`, `path`:`PathOrStr`=`'.'`) → `Path`\n",
"\n",
"Return `Path(path)/Path(fname)`, `path` defaults to current dir. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(join_path)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> join_paths(`fnames`:`FilePathList`, `path`:`PathOrStr`=`'.'`) → `FilePathList`\n",
"\n",
"Join `path` to every file name in `fnames`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(join_paths)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> loadtxt_str(`path`:`PathOrStr`) → `ndarray`\n",
"\n",
"Return `ndarray` of `str` of lines of text from `path`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(loadtxt_str)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> save_texts(`fname`:`PathOrStr`, `texts`:`StrList`)\n",
"\n",
"Save in `fname` the content of `texts`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(save_texts)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Data block API"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> ItemBase(`data`:`Any`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(ItemBase, title_level=3, doc_string=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The base clase for all items. Must have a [`data`](/tabular.data.html#tabular.data) field that will be used when collating in mini-batches."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> Category(`data`, `obj`) :: [`ItemBase`](/core.html#ItemBase)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(Category, doc_string=False, title_level=3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a [`Category`](/core.html#Category) with an `obj` of index [`data`](/tabular.data.html#tabular.data) in a certain classes list. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"class MultiCategory[source]
\n",
"\n",
"> MultiCategory(`data`, `obj`, `raw`) :: [`ItemBase`](/core.html#ItemBase)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(MultiCategory, doc_string=False, title_level=3)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a [`MultiCategory`](/core.html#MultiCategory) with an `obj` that is a collection of labels. [`data`](/tabular.data.html#tabular.data) corresponds to the one-hot encoded labels and `raw` is a list of associated string."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Others"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> camel2snake(`name`:`str`) → `str`"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(camel2snake)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Format `name` by removing capital letters from a class-style name and separates the subwords with underscores."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'device_data_loader'"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"camel2snake('DeviceDataLoader')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> even_mults(`start`:`float`, `stop`:`float`, `n`:`int`) → `ndarray`\n",
"\n",
"Build evenly stepped schedule from `start` to `stop` in `n` steps. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(even_mults)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> func_args(`func`) → `bool`\n",
"\n",
"Return the arguments of `func`. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(func_args)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> noop(`x`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(noop)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Return `x`."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> num_cpus() → `int`\n",
"\n",
"Get number of cpus "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(num_cpus)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> one_hot(`x`:`Collection`\\[`int`\\], `c`:`int`)\n",
"\n",
"One-hot encode the target. "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(one_hot)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> partition(`a`:`Collection`, `sz`:`int`) → `List`\\[`Collection`\\]\n",
"\n",
"Split iterables `a` in equal parts of size `sz` "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(partition)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"partition_by_cores[source]
\n",
"\n",
"> partition_by_cores(`a`:`Collection`, `n_cpus`:`int`) → `List`\\[`Collection`\\]\n",
"\n",
"Split data in `a` equally among `n_cpus` cores "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(partition_by_cores)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Undocumented Methods - Methods moved below this line will intentionally be hidden"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> show(`ax`:`Axes`, `kwargs`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(ItemBase.show)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> apply_tfms(`tfms`:`Collection`, `kwargs`)"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(ItemBase.apply_tfms)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## New Methods - Please document or move to the undocumented section"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"hide_input": true
},
"outputs": [
{
"data": {
"text/markdown": [
"\n",
"\n",
"> array(`a`, `args`, `kwargs`) → `ndarray`\n",
"\n",
"Same as `np.array` but also handles generators "
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"show_doc(array)"
]
}
],
"metadata": {
"jekyll": {
"keywords": "fastai",
"summary": "Basic helper functions for the fastai library",
"title": "core"
},
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 2
}