{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Torch Core" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This module contains all the basic functions we need in other modules of the fastai library (split with [`core`](/core.html#core) that contains the ones not requiring pytorch). Its documentation can easily be skipped at a first read, unless you want to know what a given fuction does." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.gen_doc.nbdoc import *\n", "from fastai.torch_core import * " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Global constants" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`AdamW = partial(optim.Adam, betas=(0.9,0.99))`
[source]
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`bn_types = (nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d)`
[source]
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`default_device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')`
[source]
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions that operate conversions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

flatten[source]

\n", "\n", "> flatten(`m`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(flatten_model, full_name='flatten')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Flattens all the layers of `m`. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

model2half[source]

\n", "\n", "> model2half(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n", "\n", "Convert `model` to half precision except the batchnorm layers. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(model2half)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

np2model_tensor[source]

\n", "\n", "> np2model_tensor(`a`)\n", "\n", "Tranform numpy array `a` to a tensor of the same type. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(np2model_tensor)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

requires_grad[source]

\n", "\n", "> requires_grad(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `b`:`Optional`\\[`bool`\\]=`None`) → `Optional`\\[`bool`\\]" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(requires_grad, doc_string=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If `b` is None, returns the [`requires_grad`](/torch_core.html#requires_grad) state of the first layer of `m`. Otherwise, sets `requires_grad=b` in all children of `m`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

tensor[source]

\n", "\n", "> tensor(`x`:`Any`, `rest`) → `Tensor`\n", "\n", "Like `torch.as_tensor`, but handle lists too, and can pass multiple vector elements directly " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(tensor)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Ensures `x` is a torch `Tensor`." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_data[source]

\n", "\n", "> to_data(`b`:`ItemsList`)\n", "\n", "Recursively map lists of items in `b ` to their wrapped data. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(to_data)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_detach[source]

\n", "\n", "> to_detach(`b`:`Tensors`, `cpu`:`bool`=`True`)\n", "\n", "Recursively detach lists of tensors in `b `, puts them on the CPU if `cpu=True`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(to_detach)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_device[source]

\n", "\n", "> to_device(`b`:`Tensors`, `device`:[`device`](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device))\n", "\n", "Ensure `b` is on `device`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(to_device)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_half[source]

\n", "\n", "> to_half(`b`:`Collection`\\[`Tensor`\\]) → `Collection`\\[`Tensor`\\]" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(to_half, doc_string=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Put the input of the batch `b` in half precision." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_np[source]

\n", "\n", "> to_np(`x`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(to_np)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Convert `x` to a numpy array." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions to deal with model initialization" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

apply_init[source]

\n", "\n", "> apply_init(`m`, `init_func`:`LayerFunc`)\n", "\n", "Initialize all non-batchnorm layers of `m` with `init_func`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(apply_init)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

apply_leaf[source]

\n", "\n", "> apply_leaf(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `f`:`LayerFunc`)\n", "\n", "Apply `f` to children of `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(apply_leaf)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

cond_init[source]

\n", "\n", "> cond_init(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `init_func`:`LayerFunc`)\n", "\n", "Initialize the non-batchnorm layers of `m` with `init_func` " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(cond_init)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

in_channels[source]

\n", "\n", "> in_channels(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `List`\\[`int`\\]\n", "\n", "Return the shape of the first weight layer in `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(in_channels)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions to get information of a model" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

children[source]

\n", "\n", "> children(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `ModuleList`\n", "\n", "Get children of `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(children)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

first_layer[source]

\n", "\n", "> first_layer(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n", "\n", "Retrieve first layer in a module `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(first_layer)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

last_layer[source]

\n", "\n", "> last_layer(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n", "\n", "Retrieve last layer in a module `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(last_layer)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

num_children[source]

\n", "\n", "> num_children(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `int`\n", "\n", "Get number of children modules in module `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(num_children)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

range_children[source]

\n", "\n", "> range_children(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `Iterator`\\[`int`\\]\n", "\n", "Return iterator of len of children of `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(range_children)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

trainable_params[source]

\n", "\n", "> trainable_params(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `ParamList`\n", "\n", "Return list of trainable params in `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(trainable_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions to deal with BatchNorm layers" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

bn2float[source]

\n", "\n", "> bn2float(`module`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n", "\n", "If `module` is batchnorm don't use half precision. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(bn2float)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

set_bn_eval[source]

\n", "\n", "> set_bn_eval(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module))\n", "\n", "Set bn layers in eval mode for all recursive children of `m`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(set_bn_eval)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

split_bn_bias[source]

\n", "\n", "> split_bn_bias(`layer_groups`:`ModuleList`) → `ModuleList`\n", "\n", "Sort each layer in `layer_groups` into batchnorm (`bn_types`) and non-batchnorm groups. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(split_bn_bias)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Other functions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

calc_loss[source]

\n", "\n", "> calc_loss(`y_pred`:`Tensor`, `y_true`:`Tensor`, `loss_func`:`LossFunction`)\n", "\n", "Calculate loss between `y_pred` and `y_true` using `loss_class` and `bs`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(calc_loss)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

data_collate[source]

\n", "\n", "> data_collate(`batch`:`ItemsList`) → `Tensor`\n", "\n", "Convert `batch` items to tensor data. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(data_collate)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

model_type[source]

\n", "\n", "> model_type(`dtype`)\n", "\n", "Return the torch type corresponding to `dtype`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(model_type)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

np_address[source]

\n", "\n", "> np_address(`x`:`ndarray`) → `int`\n", "\n", "Address of `x` in memory " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(np_address)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

split_model[source]

\n", "\n", "> split_model(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `splits`:`Collection`\\[`Union`\\[[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `ModuleList`\\]\\], `want_idxs`:`bool`=`False`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(split_model, doc_string=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Splits the `model` according to the layer in `splits`. If `splits` are layers, the model is split at those (not included) sequentially. If `want_idxs` is True, the corresponding indexes are returned. If `splits` are lists of layers, the model is split according to those." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

split_model_idx[source]

\n", "\n", "> split_model_idx(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `idxs`:`Collection`\\[`int`\\]) → `ModuleList`\n", "\n", "Split `model` according to the indices in `idxs`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(split_model_idx)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

trange_of[source]

\n", "\n", "> trange_of(`x`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(trange_of)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Return a tensor from a range that has the same length as `x`." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Undocumented Methods - Methods moved below this line will intentionally be hidden" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/markdown": [ "

tensor__array__[source]

\n", "\n", "> tensor__array__(`dtype`=`None`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(tensor__array__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## New Methods - Please document or move to the undocumented section" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

init_default[source]

\n", "\n", "> init_default(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `func`:`LayerFunc`=`'kaiming_normal_'`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(init_default)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

log_uniform[source]

\n", "\n", "> log_uniform(`low`, `high`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `FloatOrTensor`\n", "\n", "Draw 1 or shape=`size` random floats from uniform dist: min=log(`low`), max=log(`high`). " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(log_uniform)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

grab_idx[source]

\n", "\n", "> grab_idx(`x`, `i`, `batch_first`:`bool`=`True`)" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(grab_idx)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

uniform_int[source]

\n", "\n", "> uniform_int(`low`:`int`, `high`:`int`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `IntOrTensor`\n", "\n", "Generate int or tensor `size` of ints between `low` and `high` (included). " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(uniform_int)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

to_cpu[source]

\n", "\n", "> to_cpu(`b`:`ItemsList`)\n", "\n", "Recursively map lists of tensors in `b ` to the cpu. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(to_cpu)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

logit[source]

\n", "\n", "> logit(`x`:`Tensor`) → `Tensor`\n", "\n", "Logit of `x`, clamped to avoid inf " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(logit)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

class FloatItem[source]

\n", "\n", "> FloatItem(`obj`) :: [`ItemBase`](/core.html#ItemBase)\n", "\n", "All transformable dataset items use this type. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(FloatItem)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

logit_[source]

\n", "\n", "> logit_(`x`:`Tensor`) → `Tensor`\n", "\n", "Inplace logit of `x`, clamped to avoid inf " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(logit_)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

rand_bool[source]

\n", "\n", "> rand_bool(`p`:`float`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `BoolOrTensor`\n", "\n", "Draw 1 or shape=`size` random booleans (True occuring probability `p`). " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(rand_bool)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

one_param[source]

\n", "\n", "> one_param(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `Tensor`" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(one_param)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "

uniform[source]

\n", "\n", "> uniform(`low`:`Number`, `high`:`Number`=`None`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `FloatOrTensor`\n", "\n", "Draw 1 or shape=`size` random floats from uniform dist: min=`low`, max=`high`. " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "show_doc(uniform)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] } ], "metadata": { "jekyll": { "keywords": "fastai", "summary": "Basic functions using pytorch", "title": "torch_core" }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 2 }