{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Torch Core" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This module contains all the basic functions we need in other modules of the fastai library (split with [`core`](/core.html#core) that contains the ones not requiring pytorch). Its documentation can easily be skipped at a first read, unless you want to know what a given fuction does." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [], "source": [ "from fastai.gen_doc.nbdoc import *\n", "from fastai.torch_core import * " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Global constants" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`AdamW = partial(optim.Adam, betas=(0.9,0.99))`
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`bn_types = (nn.BatchNorm1d, nn.BatchNorm2d, nn.BatchNorm3d)` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "`default_device = torch.device('cuda') if torch.cuda.is_available() else torch.device('cpu')` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Functions that operate conversions" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "hide_input": true }, "outputs": [ { "data": { "text/markdown": [ "flatten
[source]flatten
(`m`)"
],
"text/plain": [
"model2half
[source]model2half
(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n",
"\n",
"Convert `model` to half precision except the batchnorm layers. "
],
"text/plain": [
"np2model_tensor
[source]np2model_tensor
(`a`)\n",
"\n",
"Tranform numpy array `a` to a tensor of the same type. "
],
"text/plain": [
"requires_grad
[source]requires_grad
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `b`:`Optional`\\[`bool`\\]=`None`) → `Optional`\\[`bool`\\]"
],
"text/plain": [
"tensor
[source]tensor
(`x`:`Any`, `rest`) → `Tensor`\n",
"\n",
"Like `torch.as_tensor`, but handle lists too, and can pass multiple vector elements directly "
],
"text/plain": [
"to_data
[source]to_data
(`b`:`ItemsList`)\n",
"\n",
"Recursively map lists of items in `b ` to their wrapped data. "
],
"text/plain": [
"to_detach
[source]to_detach
(`b`:`Tensors`, `cpu`:`bool`=`True`)\n",
"\n",
"Recursively detach lists of tensors in `b `, puts them on the CPU if `cpu=True`. "
],
"text/plain": [
"to_device
[source]to_device
(`b`:`Tensors`, `device`:[`device`](https://pytorch.org/docs/stable/tensor_attributes.html#torch-device))\n",
"\n",
"Ensure `b` is on `device`. "
],
"text/plain": [
"to_half
[source]to_half
(`b`:`Collection`\\[`Tensor`\\]) → `Collection`\\[`Tensor`\\]"
],
"text/plain": [
"to_np
[source]to_np
(`x`)"
],
"text/plain": [
"apply_init
[source]apply_init
(`m`, `init_func`:`LayerFunc`)\n",
"\n",
"Initialize all non-batchnorm layers of `m` with `init_func`. "
],
"text/plain": [
"apply_leaf
[source]apply_leaf
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `f`:`LayerFunc`)\n",
"\n",
"Apply `f` to children of `m`. "
],
"text/plain": [
"cond_init
[source]cond_init
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `init_func`:`LayerFunc`)\n",
"\n",
"Initialize the non-batchnorm layers of `m` with `init_func` "
],
"text/plain": [
"in_channels
[source]in_channels
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `List`\\[`int`\\]\n",
"\n",
"Return the shape of the first weight layer in `m`. "
],
"text/plain": [
"children
[source]children
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `ModuleList`\n",
"\n",
"Get children of `m`. "
],
"text/plain": [
"first_layer
[source]first_layer
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n",
"\n",
"Retrieve first layer in a module `m`. "
],
"text/plain": [
"last_layer
[source]last_layer
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n",
"\n",
"Retrieve last layer in a module `m`. "
],
"text/plain": [
"num_children
[source]num_children
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `int`\n",
"\n",
"Get number of children modules in module `m`. "
],
"text/plain": [
"range_children
[source]range_children
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `Iterator`\\[`int`\\]\n",
"\n",
"Return iterator of len of children of `m`. "
],
"text/plain": [
"trainable_params
[source]trainable_params
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `ParamList`\n",
"\n",
"Return list of trainable params in `m`. "
],
"text/plain": [
"bn2float
[source]bn2float
(`module`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → [`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)\n",
"\n",
"If `module` is batchnorm don't use half precision. "
],
"text/plain": [
"set_bn_eval
[source]set_bn_eval
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module))\n",
"\n",
"Set bn layers in eval mode for all recursive children of `m`. "
],
"text/plain": [
"split_bn_bias
[source]split_bn_bias
(`layer_groups`:`ModuleList`) → `ModuleList`\n",
"\n",
"Sort each layer in `layer_groups` into batchnorm (`bn_types`) and non-batchnorm groups. "
],
"text/plain": [
"calc_loss
[source]calc_loss
(`y_pred`:`Tensor`, `y_true`:`Tensor`, `loss_func`:`LossFunction`)\n",
"\n",
"Calculate loss between `y_pred` and `y_true` using `loss_class` and `bs`. "
],
"text/plain": [
"data_collate
[source]data_collate
(`batch`:`ItemsList`) → `Tensor`\n",
"\n",
"Convert `batch` items to tensor data. "
],
"text/plain": [
"model_type
[source]model_type
(`dtype`)\n",
"\n",
"Return the torch type corresponding to `dtype`. "
],
"text/plain": [
"np_address
[source]np_address
(`x`:`ndarray`) → `int`\n",
"\n",
"Address of `x` in memory "
],
"text/plain": [
"split_model
[source]split_model
(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `splits`:`Collection`\\[`Union`\\[[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `ModuleList`\\]\\], `want_idxs`:`bool`=`False`)"
],
"text/plain": [
"split_model_idx
[source]split_model_idx
(`model`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `idxs`:`Collection`\\[`int`\\]) → `ModuleList`\n",
"\n",
"Split `model` according to the indices in `idxs`. "
],
"text/plain": [
"trange_of
[source]trange_of
(`x`)"
],
"text/plain": [
"tensor__array__
[source]tensor__array__
(`dtype`=`None`)"
],
"text/plain": [
"init_default
[source]init_default
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module), `func`:`LayerFunc`=`'kaiming_normal_'`)"
],
"text/plain": [
"log_uniform
[source]log_uniform
(`low`, `high`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `FloatOrTensor`\n",
"\n",
"Draw 1 or shape=`size` random floats from uniform dist: min=log(`low`), max=log(`high`). "
],
"text/plain": [
"grab_idx
[source]grab_idx
(`x`, `i`, `batch_first`:`bool`=`True`)"
],
"text/plain": [
"uniform_int
[source]uniform_int
(`low`:`int`, `high`:`int`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `IntOrTensor`\n",
"\n",
"Generate int or tensor `size` of ints between `low` and `high` (included). "
],
"text/plain": [
"to_cpu
[source]to_cpu
(`b`:`ItemsList`)\n",
"\n",
"Recursively map lists of tensors in `b ` to the cpu. "
],
"text/plain": [
"logit
[source]logit
(`x`:`Tensor`) → `Tensor`\n",
"\n",
"Logit of `x`, clamped to avoid inf "
],
"text/plain": [
"class
FloatItem
[source]FloatItem
(`obj`) :: [`ItemBase`](/core.html#ItemBase)\n",
"\n",
"All transformable dataset items use this type. "
],
"text/plain": [
"logit_
[source]logit_
(`x`:`Tensor`) → `Tensor`\n",
"\n",
"Inplace logit of `x`, clamped to avoid inf "
],
"text/plain": [
"rand_bool
[source]rand_bool
(`p`:`float`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `BoolOrTensor`\n",
"\n",
"Draw 1 or shape=`size` random booleans (True occuring probability `p`). "
],
"text/plain": [
"one_param
[source]one_param
(`m`:[`Module`](https://pytorch.org/docs/stable/nn.html#torch.nn.Module)) → `Tensor`"
],
"text/plain": [
"uniform
[source]uniform
(`low`:`Number`, `high`:`Number`=`None`, `size`:`Optional`\\[`List`\\[`int`\\]\\]=`None`) → `FloatOrTensor`\n",
"\n",
"Draw 1 or shape=`size` random floats from uniform dist: min=`low`, max=`high`. "
],
"text/plain": [
"