This ontology models classes and relationships describing deep learning networks, their component layers and activation functions, as well as potential biases.
Artificial Intelligence Ontology
2024-06-11
definition
description
license
title
part of
has part
An abstract layer object representing an RNN cell that is the base class for implementing RNN cells with custom behavior.
AbstractRNNCell
An abstract layer object representing an RNN cell that is the base class for implementing RNN cells with custom behavior.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AbstractRNNCell
A layer that applies an activation function to an output.
Activation Layer
A layer that applies an activation function to an output.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Activation
A type of machine learning focused on methods that interactively query a user or another information source to label new data points with the desired outputs.
Query Learning
Active Learning
A type of machine learning focused on methods that interactively query a user or another information source to label new data points with the desired outputs.
https://en.wikipedia.org/wiki/Active_learning_(machine_learning)
A use and interpretation bias occurring when systems/platforms get training data from their most active users rather than less active or inactive users.
Activity Bias
A use and interpretation bias occurring when systems/platforms get training data from their most active users rather than less active or inactive users.
GTP-4o with Seppala et al. 2017
https://en.wikipedia.org/wiki/Interpretive_bias
A regularization layer that applies an update to the cost function based on input activity.
ActivityRegularization Layer
A regularization layer that applies an update to the cost function based on input activity.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ActivityRegularization
A pooling layer that applies a 1D adaptive average pooling over an input signal composed of several input planes.
AdaptiveAvgPool1D
AdaptiveAvgPool1d
AdaptiveAvgPool1D Layer
A pooling layer that applies a 1D adaptive average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 2D adaptive average pooling over an input signal composed of several input planes.
AdaptiveAvgPool2D
AdaptiveAvgPool2d
AdaptiveAvgPool2D Layer
A pooling layer that applies a 2D adaptive average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 3D adaptive average pooling over an input signal composed of several input planes.
AdaptiveAvgPool3D
AdaptiveAvgPool3d
AdaptiveAvgPool3D Layer
A pooling layer that applies a 3D adaptive average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 1D adaptive max pooling over an input signal composed of several input planes.
AdaptiveMaxPool1D
AdaptiveMaxPool1d
AdaptiveMaxPool1D Layer
A pooling layer that applies a 1D adaptive max pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 2D adaptive max pooling over an input signal composed of several input planes.
AdaptiveMaxPool2D
AdaptiveMaxPool2d
AdaptiveMaxPool2D Layer
A pooling layer that applies a 2D adaptive max pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 3D adaptive max pooling over an input signal composed of several input planes.
AdaptiveMaxPool3D
AdaptiveMaxPool3d
AdaptiveMaxPool3D Layer
A pooling layer that applies a 3D adaptive max pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A merging layer that adds a list of inputs taking as input a list of tensors all of the same shape.
Add Layer
A merging layer that adds a list of inputs taking as input a list of tensors all of the same shape.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Add
A layer that adds inputs from one or more other layers to cells or neurons of a target layer.
Addition Layer
An attention layer that implements additive attention also known as Bahdanau-style attention.
AdditiveAttention Layer
An attention layer that implements additive attention also known as Bahdanau-style attention.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AdditiveAttention
A regularization layer that applies Alpha Dropout to the input keeping mean and variance of inputs to ensure self-normalizing property.
AlphaDropout Layer
A regularization layer that applies Alpha Dropout to the input keeping mean and variance of inputs to ensure self-normalizing property.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AlphaDropout
A processing bias arising when the distribution over prediction outputs is skewed compared to the prior distribution of the prediction target.
Amplification Bias
A processing bias arising when the distribution over prediction outputs is skewed compared to the prior distribution of the prediction target.
GTP-4o with Seppala et al. 2017
https://royalsocietypublishing.org/doi/10.1098/rspb.2019.0165#d1e5237
A cognitive bias characterized by the influence of a reference point or anchor on decisions leading to insufficient adjustment from that anchor point.
Anchoring Bias
A cognitive bias characterized by the influence of a reference point or anchor on decisions leading to insufficient adjustment from that anchor point.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An individual bias occurring when users rely on automation as a heuristic replacement for their own information seeking and processing.
Annotator Reporting Bias
An individual bias occurring when users rely on automation as a heuristic replacement for their own information seeking and processing.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A network based on a collection of connected units called artificial neurons modeled after biological neurons.
ANN
NN
Artificial Neural Network
A network based on a collection of connected units called artificial neurons modeled after biological neurons.
https://en.wikipedia.org/wiki/Artificial_neural_network
A supervised learning method focused on a rule-based approach for discovering interesting relations between variables in large databases.
Association Rule Learning
A supervised learning method focused on a rule-based approach for discovering interesting relations between variables in large databases.
https://en.wikipedia.org/wiki/Association_rule_learning
A layer that implements dot-product attention also known as Luong-style attention.
Attention Layer
A layer that implements dot-product attention also known as Luong-style attention.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Attention
An unsupervised pretrained network that learns efficient codings of unlabeled data by training to ignore insignificant data and regenerate input from encoding.
AE
Layers: Input, Hidden, Matched Output-Input
Auto Encoder Network
An unsupervised pretrained network that learns efficient codings of unlabeled data by training to ignore insignificant data and regenerate input from encoding.
https://en.wikipedia.org/wiki/Autoencoder
A bias characterized by over-reliance on automated systems leading to attenuated human skills.
Automation Complaceny
Automation Complacency Bias
A bias characterized by over-reliance on automated systems leading to attenuated human skills.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A model that describes the variance of the current error term as a function of the previous periods' error terms, capturing volatility clustering. Used for time series data.
ARCH
Autoregressive Conditional Heteroskedasticity
A model that includes lagged values of both the dependent variable and one or more independent variables, capturing dynamic relationships over time. Used in time series analysis.
ARDL
Autoregressive Distributed Lag
A model which combines autoregression (AR), differencing (I), and moving average (MA) components. Used for analyzing and forecasting time series data.
ARIMA
Autoregressive Integrated Moving Average
A language model that generates text sequentially predicting one token at a time based on the previously generated tokens excelling at natural language generation tasks by modeling the probability distribution over sequences of tokens.
Autoregressive Language Model
generative language model
sequence-to-sequence model
Autoregressive Language Model
A model that combines autoregressive (AR) and moving average (MA) components to represent time series data, suitable for stationary series without the need for differencing.
ARMA
Autoregressive Moving Average
A cognitive bias characterized by a mental shortcut where easily recalled information is overweighted in judgment and decision-making.
Availability Bias
Availability Heuristic
Availability Heuristic Bias
A cognitive bias characterized by a mental shortcut where easily recalled information is overweighted in judgment and decision-making.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A merging layer that averages a list of inputs element-wise taking as input a list of tensors all of the same shape.
Average Layer
A merging layer that averages a list of inputs element-wise taking as input a list of tensors all of the same shape.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Average
A pooling layer that performs average pooling for temporal data.
AvgPool1D
AvgPool1d
AveragePooling1D Layer
A pooling layer that performs average pooling for temporal data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AveragePooling1D
A pooling layer that performs average pooling for spatial data.
AvgPool2D
AvgPool2d
AveragePooling2D Layer
A pooling layer that performs average pooling for spatial data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AveragePooling2D
A pooling layer that performs average pooling for 3D data (spatial or spatio-temporal).
AvgPool3D
AveragePooling3D Layer
A pooling layer that performs average pooling for 3D data (spatial or spatio-temporal).
https://www.tensorflow.org/api_docs/python/tf/keras/layers/AveragePooling3D
A pooling layer that applies a 1D average pooling over an input signal composed of several input planes.
AvgPool1D
AvgPool1d
AvgPool1D Layer
A pooling layer that applies a 1D average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 2D average pooling over an input signal composed of several input planes.
AvgPool2D
AvgPool2d
AvgPool2D Layer
A pooling layer that applies a 2D average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 3D average pooling over an input signal composed of several input planes.
AvgPool3D
AvgPool3d
AvgPool3D Layer
A pooling layer that applies a 3D average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
An input layer that receives values from another layer.
Backfed Input Layer
A batch normalization layer that applies Batch Normalization over a 2D or 3D input.
BatchNorm1D
BatchNorm1d
BatchNorm1D Layer
A batch normalization layer that applies Batch Normalization over a 2D or 3D input.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A batch normalization layer that applies Batch Normalization over a 4D input.
BatchNorm2D
BatchNorm2d
BatchNorm2D Layer
A batch normalization layer that applies Batch Normalization over a 4D input.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A batch normalization layer that applies Batch Normalization over a 5D input.
BatchNorm3D
BatchNorm3d
BatchNorm3D Layer
A batch normalization layer that applies Batch Normalization over a 5D input.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A normalization layer that normalizes its inputs applying a transformation that maintains the mean close to 0 and the standard deviation close to 1.
BatchNormalization Layer
A normalization layer that normalizes its inputs applying a transformation that maintains the mean close to 0 and the standard deviation close to 1.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/BatchNormalization
A network that is a probabilistic graphical model representing variables and their conditional dependencies via a directed acyclic graph.
Bayesian Network
A network that is a probabilistic graphical model representing variables and their conditional dependencies via a directed acyclic graph.
https://en.wikipedia.org/wiki/Bayesian_network
An individual bias characterized by systematic distortions in user behavior across platforms or contexts or across users represented in different datasets.
Behavioral Bias
An individual bias characterized by systematic distortions in user behavior across platforms or contexts or across users represented in different datasets.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others.
Bias
A systematic error introduced into sampling or testing by selecting or encouraging one outcome or answer over others.
GTP-4o with Seppala et al. 2017
https://www.merriam-webster.com/dictionary/bias
A machine learning task focused on methods that simultaneously cluster the rows and columns of a matrix to identify submatrices with coherent patterns.
Block Clustering
Co-clustering
Joint Clustering
Two-mode Clustering
Two-way Clustering
Biclustering
A machine learning task focused on methods that simultaneously cluster the rows and columns of a matrix to identify submatrices with coherent patterns.
https://en.wikipedia.org/wiki/Biclustering
A recurrent layer that is a bidirectional wrapper for RNNs.
Bidirectional Layer
A recurrent layer that is a bidirectional wrapper for RNNs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Bidirectional
A transformer language model such as BERT that uses the transformer architecture to build deep bidirectional representations by predicting masked tokens based on their context.
BERT
Bidirectional Transformer LM
Bidirectional Transformer Language Model
A transformer language model such as BERT that uses the transformer architecture to build deep bidirectional representations by predicting masked tokens based on their context.
https://arxiv.org/abs/1810.04805
https://en.wikipedia.org/wiki/BERT_(language_model)
A machine learning task focused on methods that classify elements into two groups based on a classification rule.
Binary Classification
A machine learning task focused on methods that classify elements into two groups based on a classification rule.
https://en.wikipedia.org/wiki/Binary_classification
A symmetrically connected network that is a type of stochastic recurrent neural network and Markov random field.
BM
Sherringtonâ€“Kirkpatrick model with external field
stochastic Hopfield network with hidden units
stochastic Ising-Lenz-Little model
Layers: Backfed Input, Probabilistic Hidden
Boltzmann Machine Network
A symmetrically connected network that is a type of stochastic recurrent neural network and Markov random field.
https://en.wikipedia.org/wiki/Boltzmann_machine
A layer that performs categorical data preprocessing operations.
Categorical Features Preprocessing Layer
A layer that performs categorical data preprocessing operations.
https://keras.io/guides/preprocessing_layers/
A categorical features preprocessing layer that encodes integer features providing options for condensing data into a categorical encoding.
CategoryEncoding Layer
A categorical features preprocessing layer that encodes integer features providing options for condensing data into a categorical encoding.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/CategoryEncoding
A probabilistic graphical model used to encode assumptions about the data-generating process.
Casaul Bayesian Network
Casaul Graph
DAG
Directed Acyclic Graph
Path Diagram
Causal Graphical Model
A probabilistic graphical model used to encode assumptions about the data-generating process.
https://en.wikipedia.org/wiki/Causal_graph
A large language model that only attends to previous tokens in the sequence when generating text modeling the probability distribution autoregressively from left-to-right or causally.
Causal Large Language Model
autoregressive
unidirectional
Causal LLM
An image preprocessing layer that crops the central portion of images to a target size.
CenterCrop Layer
An image preprocessing layer that crops the central portion of images to a target size.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/CenterCrop
A supervised learning task focused on methods that distinguish and distribute kinds of "things" into different groups.
Classification
A supervised learning task focused on methods that distinguish and distribute kinds of "things" into different groups.
https://en.wikipedia.org/wiki/Classification_(general_theory)
The process of removing noise inconsistencies and irrelevant information from data to enhance its quality and prepare it for analysis or further processing.
Data Cleansing
Standardization
Data cleaning
Text normalization
Cleaning
A machine learning task focused on methods that group a set of objects such that objects in the same group are more similar to each other than to those in other groups.
Cluster analysis
Clustering
A machine learning task focused on methods that group a set of objects such that objects in the same group are more similar to each other than to those in other groups.
https://en.wikipedia.org/wiki/Cluster_analysis
A systematic deviation from rational judgment and decision-making including adaptive mental shortcuts known as heuristics.
Cognitive Bias
A systematic deviation from rational judgment and decision-making including adaptive mental shortcuts known as heuristics.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A large language model that is trained to understand and recombine the underlying compositional structures in language enabling better generalization to novel combinations and out-of-distribution examples.
Compositional Generalization Large Language Model
out-of-distribution generalization
systematic generalization
Compositional Generalization LLM
A bias caused by differences between results and facts in the process of data analysis (including the source of data the estimator chose) and analysis methods.
Statistical Bias
Computational Bias
A bias caused by differences between results and facts in the process of data analysis (including the source of data the estimator chose) and analysis methods.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A merging layer that concatenates a list of inputs taking as input a list of tensors all of the same shape except for the concatenation axis.
Concatenate Layer
A merging layer that concatenates a list of inputs taking as input a list of tensors all of the same shape except for the concatenation axis.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Concatenate
A use and interpretation bias due to the use of a system outside its planned domain of application causing performance gaps between laboratory settings and the real world.
Concept Drift
Concept Drift Bias
A use and interpretation bias due to the use of a system outside its planned domain of application causing performance gaps between laboratory settings and the real world.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A cognitive bias characterized by the tendency to prefer information that confirms existing beliefs influencing the search for interpretation of and recall of information.
Confirmation Bias
A cognitive bias characterized by the tendency to prefer information that confirms existing beliefs influencing the search for interpretation of and recall of information.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A bias arising when an algorithm or platform provides users a venue to express their biases occurring from either side in a digital interaction.
Consumer Bias
A bias arising when an algorithm or platform provides users a venue to express their biases occurring from either side in a digital interaction.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A use and interpretation bias arising from structural lexical semantic and syntactic differences in user-generated content.
Content Production Bias
A use and interpretation bias arising from structural lexical semantic and syntactic differences in user-generated content.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A deep neural network that learns sequential tasks without forgetting knowledge from preceding tasks and without access to old task data during new task training.
Incremental Learning
Life-Long Learning
Continual Learning
A deep neural network that learns sequential tasks without forgetting knowledge from preceding tasks and without access to old task data during new task training.
https://paperswithcode.com/task/continual-learning
A large language model that continually acquires new knowledge and skills over time without forgetting previously learned information allowing the model to adapt and expand its capabilities as new data becomes available.
CL-Large Language Model
Continual Learning Large Language Model
catastrophic forgetting
lifelong learning
Continual Learning LLM
A deep neural network self-supervised learning approach that learns to distinguish between similar and dissimilar data samples.
Contrastive Learning
A deep neural network self-supervised learning approach that learns to distinguish between similar and dissimilar data samples.
https://arxiv.org/abs/2202.14037
A large language model that is trained to pull semantically similar samples closer together and push dissimilar samples apart in the representation space learning high-quality features useful for downstream tasks.
Contrastive Learning LLM
Representation learning
Contrastive Learning LLM
A large language model that allows for explicit control over certain attributes of the generated text such as style tone topic or other desired characteristics through conditioning or specialized training objectives.
Controllable Large Language Model
conditional generation
guided generation
Controllable LLM
A convolutional layer that implements a 1D Convolutional LSTM similar to an LSTM but with convolutional input and recurrent transformations.
ConvLSTM1D Layer
A convolutional layer that implements a 1D Convolutional LSTM similar to an LSTM but with convolutional input and recurrent transformations.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ConvLSTM1D
A convolutional layer that implements a 2D Convolutional LSTM similar to an LSTM but with convolutional input and recurrent transformations.
ConvLSTM2D Layer
A convolutional layer that implements a 2D Convolutional LSTM similar to an LSTM but with convolutional input and recurrent transformations.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ConvLSTM2D
A convolutional layer that implements a 3D Convolutional LSTM similar to an LSTM but with convolutional input and recurrent transformations.
ConvLSTM3D Layer
A convolutional layer that implements a 3D Convolutional LSTM similar to an LSTM but with convolutional input and recurrent transformations.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ConvLSTM3D
A layer that implements 1D convolution (e.g. temporal convolution).
Conv1D Layer
Conv1d
Convolution1D
Convolution1d
nn.Conv1d
Convolution1D Layer
A layer that implements 1D convolution (e.g. temporal convolution).
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1D
A layer that implements transposed 1D convolution sometimes called deconvolution.
Conv1DTranspose Layer
ConvTranspose1d
Convolution1DTranspose
Convolution1dTranspose
nn.ConvTranspose1d
Convolution1DTranspose Layer
A layer that implements transposed 1D convolution sometimes called deconvolution.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv1DTranspose
A layer that implements 2D convolution (e.g. spatial convolution over images).
Conv2D Layer
Conv2d
Convolution2D
Convolution2d
nn.Conv2d
Convolution2D Layer
A layer that implements 2D convolution (e.g. spatial convolution over images).
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D
A layer that implements transposed 2D convolution
Conv2DTranspose Layer
ConvTranspose2d
Convolution2DTranspose
Convolution2dTranspose
nn.ConvTranspose2d
Convolution2DTranspose Layer
A layer that implements transposed 2D convolution
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2DTranspose
A layer that implements 3D convolution (e.g. spatial convolution over volumes).
Conv3D Layer
Conv3d
Convolution3D
Convolution3d
nn.Conv3d
Convolution3D Layer
A layer that implements 3D convolution (e.g. spatial convolution over volumes).
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv3D
A layer that implements transposed 3D convolution
Conv3DTranspose Layer
ConvTranspose3d
Convolution3DTranspose
Convolution3dTranspose
nn.ConvTranspose3d
Convolution3DTranspose Layer
A layer that implements transposed 3D convolution
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv3DTranspose
A layer that contains a set of filters (or kernels) parameters of which are to be learned throughout the training.
Convolutional Layer
A layer that contains a set of filters (or kernels) parameters of which are to be learned throughout the training.
https://www.sciencedirect.com/topics/engineering/convolutional-layer#:~:text=A%20convolutional%20layer%20is%20the,and%20creates%20an%20activation%20map.
A layer that crops along the time dimension (axis 1) for 1D input.
Cropping1D Layer
A layer that crops along the time dimension (axis 1) for 1D input.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Cropping1D
A layer that crops along spatial dimensions (i.e. height and width) for 2D input.
Cropping2D Layer
A layer that crops along spatial dimensions (i.e. height and width) for 2D input.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Cropping2D
A layer that crops along spatial dimensions (depth
Cropping3D Layer
A layer that crops along spatial dimensions (depth
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Cropping3D
A LLM that performs well across a wide range of domains without significant loss in performance, facilitated by advanced domain adaptation techniques.
Domain-General LLM
cross-domain transfer
domain adaptation
Cross-Domain LLM
A training strategy in machine learning where models are trained on data in a meaningful order starting with simpler examples and gradually increasing the complexity to improve learning efficiency and model performance.
Sequential Learning
Structured Learning
Complexity grading
Sequential learning
Curriculum Learning
A large language model that is trained by presenting learning examples in a meaningful order from simple to complex mimicking the learning trajectory followed by humans.
Curriculum Learning LLM
Learning progression
Curriculum Learning LLM
A technique used to increase the diversity and quantity of training data by applying various transformations such as rotation scaling flipping and cropping to existing data samples enhancing the robustness and performance of machine learning models.
Data Enrichment
Data Expansion
Paraphrasing
Synonym replacement
Data Augmentation
A use and interpretation bias where testing many hypotheses in a dataset may yield apparent statistical significance even when results are nonsignificant.
Data Dredging
Data Dredging Bias
A use and interpretation bias where testing many hypotheses in a dataset may yield apparent statistical significance even when results are nonsignificant.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
Techniques used to improve the quality diversity and volume of data available for training machine learning models such as data augmentation synthesis and enrichment to enhance model robustness and accuracy.
DataEnhancement
A selection and sampling bias arising from adding synthetic or redundant data samples to a dataset.
Data Generation Bias
A selection and sampling bias arising from adding synthetic or redundant data samples to a dataset.
GTP-4o with Seppala et al. 2017
https://en.wikipedia.org/wiki/Selection_bias
A machine learning task focused on methods that replace missing data with substituted values.
Data Imputation
A machine learning task focused on methods that replace missing data with substituted values.
https://en.wikipedia.org/wiki/Imputation_(statistics)
The process of cleaning transforming and organizing raw data into a suitable format for analysis and modeling ensuring the quality and relevance of the data for machine learning tasks.
Data Assembly
Data Curation
Data Processing
Data Preparation
A LLM that generates natural language descriptions from structured data sources like tables, graphs, and knowledge bases, requiring grounding in meaning representations.
Data-to-Text LLM
Meaning representation
Data-to-Text LLM
A machine learning model that uses a tree-like model of decisions and their possible consequences including chance event outcomes resource costs and utilities.
Decision Tree
A machine learning model that uses a tree-like model of decisions and their possible consequences including chance event outcomes resource costs and utilities.
https://en.wikipedia.org/wiki/Decision_tree
A large language model that uses a decoder-only architecture consisting of only a decoder trained to predict the next token in a sequence given the previous tokens.
Decoder LLM
A large language model that uses a decoder-only architecture consisting of only a decoder trained to predict the next token in a sequence given the previous tokens.
https://www.practicalai.io/understanding-transformer-model-architectures/#:~:text=Encoder
A deep neural network that uses deconvolution for unsupervised construction of hierarchical image representations.
DN
Layers: Input, Kernel, Convolutional/Pool, Output
Deconvolutional Network
A deep neural network that uses deconvolution for unsupervised construction of hierarchical image representations.
https://ieeexplore.ieee.org/document/5539957
A deep neural network that combines deep learning and active learning to maximize model performance while annotating the fewest samples possible.
DeepAL
Deep Active Learning
A deep neural network that combines deep learning and active learning to maximize model performance while annotating the fewest samples possible.
https://arxiv.org/pdf/2009.00236.pdf
An unsupervised pretrained network composed of multiple layers of latent variables that learns to probabilistically reconstruct inputs and perform classification.
DBN
Layers: Backfed Input, Probabilistic Hidden, Hidden, Matched Output-Input
Deep Belief Network
An unsupervised pretrained network composed of multiple layers of latent variables that learns to probabilistically reconstruct inputs and perform classification.
https://en.wikipedia.org/wiki/Deep_belief_network
An autoencoder network that learns interpretable disentangled image representations through convolution and de-convolution layers trained with the stochastic gradient variational Bayes algorithm.
DCIGN
Layers: Input, Kernel, Convolutional/Pool, Probabilistic Hidden, Convolutional/Pool, Kernel, Output
Deep Convolutional Inverse Graphics Network
A deep neural network specialized for analyzing visual imagery using shared-weight architecture and translation-equivariant feature maps.
CNN
ConvNet
Convolutional Neural Network
DCN
Layers: Input, Kernel, Convolutional/Pool, Hidden, Output
Deep Convolutional Network
A deep neural network specialized for analyzing visual imagery using shared-weight architecture and translation-equivariant feature maps.
https://en.wikipedia.org/wiki/Convolutional_neural_network
A deep neural network that processes information in one directionâ€”from input nodes through hidden nodes to output nodesâ€”without cycles or loops.
DFF
MLP
Multilayer Perceptoron
Layers: Input, Hidden, Output
Deep Feed-Forward Network
A deep neural network that processes information in one directionâ€”from input nodes through hidden nodes to output nodesâ€”without cycles or loops.
https://en.wikipedia.org/wiki/Feedforward_neural_network
An artificial neural network characterized by multiple hidden layers between the input and output layers.
DNN
Deep Neural Network
A deep neural network that relaxes the hypothesis that training data must be independent and identically distributed with test data to address insufficient training data.
Deep Transfer Learning
A deep neural network that relaxes the hypothesis that training data must be independent and identically distributed with test data to address insufficient training data.
https://arxiv.org/abs/1808.01974
An autoencoder network trained to reconstruct the original undistorted input from a partially corrupted input.
DAE
Denoising Autoencoder
Layers: Noisy Input, Hidden, Matched Output-Input
Denoising Auto Encoder
An autoencoder network trained to reconstruct the original undistorted input from a partially corrupted input.
https://doi.org/10.1145/1390156.1390294
A layer that produces a dense tensor based on given feature columns.
DenseFeatures Layer
A layer that produces a dense tensor based on given feature columns.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/DenseFeatures
A layer that is a regular densely-connected neural network layer.
Dense Layer
A layer that is a regular densely-connected neural network layer.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dense
A bias arising when systems are used as decision aids for humans since the human intermediary may act on predictions in ways that are typically not modeled in the system.
Deployment Bias
A bias arising when systems are used as decision aids for humans since the human intermediary may act on predictions in ways that are typically not modeled in the system.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A layer that performs depthwise 1D convolution
DepthwiseConv1D Layer
A layer that performs depthwise 1D convolution
https://www.tensorflow.org/api_docs/python/tf/keras/layers/DepthwiseConv1D
A layer that performs depthwise 2D convolution
DepthwiseConv2D Layer
A layer that performs depthwise 2D convolution
https://www.tensorflow.org/api_docs/python/tf/keras/layers/DepthwiseConv2D
A selection and sampling bias characterized by systematic differences between groups in how outcomes are determined potentially over- or underestimating effect size.
Detection Bias
A selection and sampling bias characterized by systematic differences between groups in how outcomes are determined potentially over- or underestimating effect size.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A large language model that is optimized for engaging in multi-turn conversations understanding context and generating relevant coherent responses continuously over many dialogue turns.
Dialogue Large Language Model
conversational AI
multi-turn dialogue
Dialogue LLM
A large language model that has an architecture amenable to full end-to-end training via backpropagation without relying on teacher forcing or unlikelihood training objectives.
Differentiable Large Language Model
end-to-end training
fully backpropagable
Differentiable LLM
A machine learning task focused on the process of transforming data from a high-dimensional space into a lower-dimensional space while retaining meaningful properties of the original data.
Dimension Reduction
Dimensionality Reduction
A machine learning task focused on the process of transforming data from a high-dimensional space into a lower-dimensional space while retaining meaningful properties of the original data.
https://en.wikipedia.org/wiki/Dimensionality_reduction
A preprocessing layer which buckets continuous features by ranges.
Discretization Layer
A preprocessing layer which buckets continuous features by ranges.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Discretization
The process of training a smaller model to replicate the behavior of a larger model aiming to compress the knowledge into a more compact form without significant loss of performance.
Purification
Refining
Knowledge compression
Teacher-student model
Distillation
The process of training a smaller model to replicate the behavior of a larger model aiming to compress the knowledge into a more compact form without significant loss of performance.
https://doi.org/10.48550/arXiv.2105.13093
A LLM which is pre-trained on a broad corpus and then fine-tuned on domain-specific data to specialize its capabilities for particular domains or applications, like scientific literature or code generation.
Domain-Adapted Large Language Model
domain robustness
transfer learning
Domain-Adapted LLM
A layer that computes a dot product between samples in two tensors.
Dot Layer
A layer that computes a dot product between samples in two tensors.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dot
A regularization layer that applies Dropout to the input
Dropout Layer
A regularization layer that applies Dropout to the input
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Dropout
A cognitive bias in which people with low ability in an area overestimate that ability. Often measured by comparing self-assessment with objective performance.
Dunning-Kruger Effect
Dunning-Kruger Effect Bias
A cognitive bias in which people with low ability in an area overestimate that ability. Often measured by comparing self-assessment with objective performance.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A model that allows for time-varying correlations between different time series, used in financial econometrics to model and forecast covariances.
DCC
Dynamic Conditional Correlation
An activation function that is x if x > 0 and alpha * (exp(x) - 1) if x < 0 where alpha controls the value to which an ELU saturates for negative net inputs.
ELU
Exponential Linear Unit
ELU Function
An activation function that is x if x > 0 and alpha * (exp(x) - 1) if x < 0 where alpha controls the value to which an ELU saturates for negative net inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/elu
An activation layer that applies the Exponential Linear Unit (ELU) function element-wise.
ELU Layer
An activation layer that applies the Exponential Linear Unit (ELU) function element-wise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ELU
A recurrent neural network with a recurrent hidden layer and sparsely connected hidden neurons that learns output weights to produce temporal patterns.
ESN
Layers: Input, Recurrent, Output
Echo State Network
A recurrent neural network with a recurrent hidden layer and sparsely connected hidden neurons that learns output weights to produce temporal patterns.
https://en.wikipedia.org/wiki/Echo_state_network#:~:text=The%20echo%20state%20network%20(ESN,are%20fixed%20and%20randomly%20assigned
A selection and sampling bias occurring when an inference about an individual is made based on their group membership.
Ecological Fallacy
Ecological Fallacy Bias
A selection and sampling bias occurring when an inference about an individual is made based on their group membership.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A layer that turns positive integers (indexes) into dense vectors of fixed size.
Embedding Layer
A layer that turns positive integers (indexes) into dense vectors of fixed size.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Embedding
A large language model that integrates language with other modalities like vision audio and robotics to enable grounded language understanding in real-world environments.
Embodied Large Language Model
multimodal grounding
Embodied LLM
A use and interpretation bias resulting from the use and reliance on algorithms across new or unanticipated contexts.
Emergent Bias
A use and interpretation bias resulting from the use and reliance on algorithms across new or unanticipated contexts.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
The LLM introduced in the "Attention Is All You Need" paper. The encoder processes the input sequence to generate a hidden representation summarizing the input information, while the decoder uses this hidden representation to generate the desired output sequence.
Encoder-Decoder LLM
The LLM introduced in the "Attention Is All You Need" paper. The encoder processes the input sequence to generate a hidden representation summarizing the input information, while the decoder uses this hidden representation to generate the desired output sequence.
https://www.practicalai.io/understanding-transformer-model-architectures/#:~:text=Encoder
A large language model that uses an encoder-only architecture to encode the input sequence into a fixed-length representation which is then used as input to a classifier or regressor for prediction.
Encoder LLM
A large language model that uses an encoder-only architecture to encode the input sequence into a fixed-length representation which is then used as input to a classifier or regressor for prediction.
https://www.practicalai.io/understanding-transformer-model-architectures/#:~:text=Encoder
A LLM which models the explicit probability density over token sequences using an energy function, rather than an autoregressive factorization. This can improve modeling of long-range dependencies and global coherence.
Energy-Based Large Language Model
energy scoring
explicit density modeling
Energy-Based LLM
A type of machine learning focused on methods that use multiple learning algorithms to achieve better predictive performance than any of the constituent algorithms alone.
Ensemble Learning
A type of machine learning focused on methods that use multiple learning algorithms to achieve better predictive performance than any of the constituent algorithms alone.
https://en.wikipedia.org/wiki/Ensemble_learning
A processing bias characterized by the effect of variables' uncertainties (or errors more specifically random errors) on the uncertainty of a function based on them.
Error Propagation
Error Propagation Bias
A processing bias characterized by the effect of variables' uncertainties (or errors more specifically random errors) on the uncertainty of a function based on them.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A large language model that is trained to uphold certain ethical principles values or rules in its language generation to increase safety and trustworthiness.
Ethical Large Language Model
constituitional AI
value alignment
Ethical LLM
A selection and sampling bias arising when testing populations do not equally represent user populations or when inappropriate performance metrics are used.
Evaluation Bias
A selection and sampling bias arising when testing populations do not equally represent user populations or when inappropriate performance metrics are used.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A large language model that applies principles of evolutionary computation to optimize its structure and parameters evolving over time to improve performance.
Evolutionary Language Model
evolutionary algorithms
genetic programming
Evolutionary LLM
A selection and sampling bias occurring when specific groups of user populations are excluded from testing and analysis.
Exclusion Bias
A selection and sampling bias occurring when specific groups of user populations are excluded from testing and analysis.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A large language model that is designed to provide insights into its decision-making process making it easier for users to understand and trust the model's outputs by incorporating mechanisms for interpreting and explaining its predictions in human-understandable terms.
Explainable Language Model
XAI LLM
interpretability
model understanding
Explainable LLM
An activation function that is the mathematical function denoted by f(x)=exp or e^{x}.
Exponential Function
An activation function that is the mathematical function denoted by f(x)=exp or e^{x}.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/exponential
A model that combines exponential smoothing with state space modeling, allowing for the inclusion of both trend and seasonal components. Used in forecasting.
ETS
Exponential Smoothing State Space Model
A feedback network with randomly assigned hidden nodes that are not updated during training.
ELM
Layers: Input, Hidden, Output
Extreme Learning Machine
A feedback network with randomly assigned hidden nodes that are not updated during training.
https://en.wikipedia.org/wiki/Extreme_Learning_machine
A language model that views each word as a vector of multiple factors such as part-of-speech morphology and semantics to improve language modeling.
Factorized Language Model
Factored Language Model
A language model that views each word as a vector of multiple factors such as part-of-speech morphology and semantics to improve language modeling.
https://en.wikipedia.org/wiki/Factored_language_model
A large language model that decomposes the full language modeling task into multiple sub-components or experts that each focus on a subset of the information enabling more efficient scaling.
Factorized Large Language Model
Factorized Learning Assisted with Large Language Model
Conditional masking
Product of experts
Factorized LLM
A large language model that decomposes the full language modeling task into multiple sub-components or experts that each focus on a subset of the information enabling more efficient scaling.
https://doi.org/10.48550/arXiv.2403.12556
The process of transforming raw data into a set of measurable characteristics that can be used as input for machine learning algorithms enhancing the ability to make accurate predictions.
Attribute Extraction
Feature Isolation
Semantic embeddings
Syntactic information
Feature Extraction
A large language model that is trained in a decentralized manner across multiple devices or silos without directly sharing private data enabling collaborative training while preserving data privacy and security.
Federated Large Language Model
decentralized training
privacy-preserving
Federated LLM
A deep neural network trained across decentralized edge devices or servers holding local data samples without exchanging them.
Federated Learning
A deep neural network trained across decentralized edge devices or servers holding local data samples without exchanging them.
https://en.wikipedia.org/wiki/Federated_learning
A use and interpretation bias occurring when an algorithm learns from user behavior and feeds that behavior back into the model.
Feedback Loop Bias
A use and interpretation bias occurring when an algorithm learns from user behavior and feeds that behavior back into the model.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An artificial neural network that refines its representations iteratively based on feedback from previous outputs.
FBN
Layers: Input, Hidden, Output, Hidden
Feedback Network
A regression analysis model in which the model parameters are fixed or non-random quantities.
FEM
Fixed Effects Model
A regression analysis model in which the model parameters are fixed or non-random quantities.
https://en.wikipedia.org/wiki/Fixed_effects_model
A layer that flattens the input
Flatten Layer
A layer that flattens the input
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Flatten
A pooling layer that applies a 2D fractional max pooling over an input signal composed of several input planes.
FractionalMaxPool2D
FractionalMaxPool2d
FractionalMaxPool2D Layer
A pooling layer that applies a 2D fractional max pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies a 3D fractional max pooling over an input signal composed of several input planes.
FractionalMaxPool3D
FractionalMaxPool3d
FractionalMaxPool3D Layer
A pooling layer that applies a 3D fractional max pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A mathematical rule that gives the value of a dependent variable corresponding to specified values of independent variables.
Function
A mathematical rule that gives the value of a dependent variable corresponding to specified values of independent variables.
https://www.sciencedirect.com/topics/mathematics/mathematical-function
A bias arising when biased results are reported to support or satisfy the funding agency or financial supporter of a research study.
Funding Bias
A bias arising when biased results are reported to support or satisfy the funding agency or financial supporter of a research study.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An activation function that computes x * P(X <= x) where P(X) ~ N(0 1) weighting inputs by their value rather than gating inputs by their sign as in ReLU.
GELU
Gaussian Error Linear Unit
GELU Function
An activation function that computes x * P(X <= x) where P(X) ~ N(0 1) weighting inputs by their value rather than gating inputs by their sign as in ReLU.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/gelu
A layer that processes one step within the whole time sequence input for a GRU layer.
GRUCell Layer
A layer that processes one step within the whole time sequence input for a GRU layer.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GRUCell
A recurrent layer that implements the Gated Recurrent Unit architecture.
GRU Layer
A recurrent layer that implements the Gated Recurrent Unit architecture.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GRU
A long short-term memory network that is a gating mechanism in recurrent neural networks similar to LSTMs but with fewer parameters and no output gate.
GRU
Layers: Input, Memory Cell, Output
Gated Recurrent Unit
A long short-term memory network that is a gating mechanism in recurrent neural networks similar to LSTMs but with fewer parameters and no output gate.
https://en.wikipedia.org/wiki/Gated_recurrent_unit
A regularization layer that applies multiplicative 1-centered Gaussian noise.
GaussianDropout Layer
A regularization layer that applies multiplicative 1-centered Gaussian noise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GaussianDropout
A regularization layer that applies additive zero-centered Gaussian noise.
GaussianNoise Layer
A regularization layer that applies additive zero-centered Gaussian noise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GaussianNoise
A model that incorporates lagged conditional variances, allowing for more flexibility in modeling time-varying volatility.
GARCH
Generalized Autoregressive Conditional Heteroskedasticity
A deep neural network that learns novel classes from few samples per class, preventing catastrophic forgetting of base classes and ensuring classifier calibration.
GFSL
Generalized Few-shot Learning
A deep neural network that learns novel classes from few samples per class, preventing catastrophic forgetting of base classes and ensuring classifier calibration.
https://paperswithcode.com/paper/generalized-and-incremental-few-shot-learning/review/
A machine learning model that generalizes linear regression by relating the linear model to the response variable via a link function and allowing the variance of each measurement to be a function of its predicted value.
GLM
Generalized Linear Model
A machine learning model that generalizes linear regression by relating the linear model to the response variable via a link function and allowing the variance of each measurement to be a function of its predicted value.
https://en.wikipedia.org/wiki/Generalized_linear_model
An unsupervised pretrained network framework where two neural networks contest in a game to generate new data with the same statistics as the training set.
GAN
Layers: Backfed Input, Hidden, Matched Output-Input, Hidden, Matched Output-Input
Generative Adversarial Network
An unsupervised pretrained network framework where two neural networks contest in a game to generate new data with the same statistics as the training set.
https://en.wikipedia.org/wiki/Generative_adversarial_network
A LLM which incorporates a generative adversarial network (GAN) into its training process, using a discriminator network to provide a signal for generating more realistic and coherent text. This adversarial training can improve the quality and diversity of generated text.
GAN-Large Language Model
Generative Adversarial Network-Augmented Large Language Model
adversarial training
text generation
Generative Adversarial Network-Augmented LLM
A large language model that is trained to understand and model basic physics causality and common sense about how the real world works.
Generative Commonsense Large Language Model
World Model
causal modeling
physical reasoning
Generative Commonsense LLM
A large language model that is trained to understand and model basic physics causality and common sense about how the real world works.
https://arxiv.org/abs/2306.12672
A language model that enables users to engage in an interactive dialogue with an LLM providing feedback to guide and refine the generated outputs iteratively.
Generative Language Interface
Interactive generation
Generative Language Interface
A pooling layer that performs global average pooling operation for temporal data.
GlobalAvgPool1D
GlobalAveragePooling1D Layer
A pooling layer that performs global average pooling operation for temporal data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling1D
A pooling layer that performs global average pooling operation for spatial data.
GlobalAvgPool2D
GlobalAveragePooling2D Layer
A pooling layer that performs global average pooling operation for spatial data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling2D
A pooling layer that performs global average pooling operation for 3D data.
GlobalAvgPool3D
GlobalAveragePooling3D Layer
A pooling layer that performs global average pooling operation for 3D data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalAveragePooling3D
A pooling layer that performs global max pooling operation for temporal data.
GlobalMaxPool1D
GlobalMaxPooling1D Layer
A pooling layer that performs global max pooling operation for temporal data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalMaxPool1D
A pooling layer that performs global max pooling operation for spatial data.
GlobalMaxPool2D
GlobalMaxPooling2D Layer
A pooling layer that performs global max pooling operation for spatial data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalMaxPool2D
A pooling layer that performs global max pooling operation for 3D data.
GlobalMaxPool3D
GlobalMaxPooling3D Layer
A pooling layer that performs global max pooling operation for 3D data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/GlobalMaxPool3D
A deep neural network that operates directly on graph structures utilizing structural information.
GCN
Layers: Input, Hidden, Hidden, Output
Graph Convolutional Network
A deep neural network that operates directly on graph structures utilizing structural information.
https://arxiv.org/abs/1609.02907
A graph convolutional network that generates goal-directed graphs using reinforcement learning and optimizing for rewards and adversarial loss.
GPCN
Layers: Input, Hidden, Hidden, Policy, Output
Graph Convolutional Policy Network
A graph convolutional network that generates goal-directed graphs using reinforcement learning and optimizing for rewards and adversarial loss.
https://arxiv.org/abs/1806.02473
A language model that operates over structured inputs or outputs represented as graphs enabling reasoning over explicit relational knowledge representations during language tasks.
Graph LM
Structured representations
Graph Language Model
A language model that operates over structured inputs or outputs represented as graphs enabling reasoning over explicit relational knowledge representations during language tasks.
https://arxiv.org/abs/2401.07105
A bias characterized by favoring members of one's in-group over out-group members expressed in evaluation resource allocation and other ways.
In-group Favoritism
In-group bias
In-group preference
In-groupâ€“out-group Bias
Intergroup bias
Group Bias
A bias characterized by favoring members of one's in-group over out-group members expressed in evaluation resource allocation and other ways.
GTP-4o with Seppala et al. 2017
https://en.wikipedia.org/wiki/In-group_favoritism
A normalization layer that applies Group Normalization over a mini-batch of inputs.
GroupNorm
GroupNorm Layer
A normalization layer that applies Group Normalization over a mini-batch of inputs.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A psychological phenomenon where people in a group make non-optimal decisions due to a desire to conform or fear of dissent.
Groupthink
Groupthink Bias
A psychological phenomenon where people in a group make non-optimal decisions due to a desire to conform or fear of dissent.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An activation function that is a faster approximation of the sigmoid activation using a piecewise linear approximation.
Hard Sigmoid Function
An activation function that is a faster approximation of the sigmoid activation using a piecewise linear approximation.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/hard_sigmoid
A categorical features preprocessing layer which hashes and bins categorical features.
Hashing Layer
A categorical features preprocessing layer which hashes and bins categorical features.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Hashing
A layer located between the input and output that performs nonlinear transformations of the inputs entered into the network.
Hidden Layer
A layer located between the input and output that performs nonlinear transformations of the inputs entered into the network.
https://deepai.org/machine-Learning-glossary-and-terms/hidden-layer-machine-Learning
A classification task focused on methods that group things according to a hierarchy.
Hierarchical Classification
A classification task focused on methods that group things according to a hierarchy.
https://en.wikipedia.org/wiki/Hierarchical_classification
A clustering method that builds a hierarchy of clusters.
HCL
Hierarchical Clustering
A clustering method that builds a hierarchy of clusters.
https://en.wikipedia.org/wiki/Hierarchical_clustering
A language model that represents language at multiple levels of granularity learning hierarchical representations that capture both low-level patterns and high-level abstractions.
Hierarchical LM
multi-scale representations
Hierarchical Language Model
A language model that represents language at multiple levels of granularity learning hierarchical representations that capture both low-level patterns and high-level abstractions.
https://doi.org/10.1016/j.ipm.2024.103698
A bias characterized by long-standing biases encoded in society over time distinct from biases in historical description or interpretation.
Historical Bias
A bias characterized by long-standing biases encoded in society over time distinct from biases in historical description or interpretation.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A symmetrically connected network that is a type of recurrent artificial neural network serving as a content-addressable memory system.
HN
Ising model of a neural network
Isingâ€“Lenzâ€“Little model
Layers: Backfed input
Hopfield Network
A symmetrically connected network that is a type of recurrent artificial neural network serving as a content-addressable memory system.
https://en.wikipedia.org/wiki/Hopfield_network
A use and interpretation bias where individuals perceive benign or ambiguous behaviors as hostile.
Hostile Attribution Bias
A use and interpretation bias where individuals perceive benign or ambiguous behaviors as hostile.
GTP-4o with Seppala et al. 2017
https://en.wikipedia.org/wiki/Interpretive_bias
A systematic error in human thought based on heuristic principles leading to simplified judgmental operations.
Human Bias
A systematic error in human thought based on heuristic principles leading to simplified judgmental operations.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An individual bias that arises when users depend on automated systems as heuristic substitutes for their own information-seeking and processing efforts.
Human Reporting Bias
An individual bias that arises when users depend on automated systems as heuristic substitutes for their own information-seeking and processing efforts.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A layer that performs image data preprocessing augmentations.
Image Augmentation Layer
A layer that performs image data preprocessing augmentations.
https://keras.io/guides/preprocessing_layers/
A layer that performs image data preprocessing operations.
Image Preprocessing Layer
A layer that performs image data preprocessing operations.
https://keras.io/guides/preprocessing_layers/
An individual bias characterized by unconscious beliefs attitudes feelings associations or stereotypes that affect information processing decision-making and actions.
Confirmatory Bias
Implicit Bias
An individual bias characterized by unconscious beliefs attitudes feelings associations or stereotypes that affect information processing decision-making and actions.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A language model that uses an energy function to score entire sequences instead of factorizing probabilities autoregressively better capturing global properties and long-range dependencies.
Implicit LM
Energy-based models
Token-level scoring
Implicit Language Model
A language model that uses an energy function to score entire sequences instead of factorizing probabilities autoregressively better capturing global properties and long-range dependencies.
https://arxiv.org/pdf/2303.16189
A deep neural network trained on a base set of classes and then presented with novel classes, each with few labeled examples.
IFSL
Incremenetal Few-shot Learning
A deep neural network trained on a base set of classes and then presented with novel classes, each with few labeled examples.
https://arxiv.org/abs/1810.07218
A persistent point of view or limited list of such points of view applied by an individual.
Individual Bias
A persistent point of view or limited list of such points of view applied by an individual.
GTP-4o with Seppala et al. 2017
https://develop.consumerium.org/wiki/Individual_bias
A processing bias arising when machine learning applications generate inputs for other machine learning algorithms passing on any existing bias.
Inherited Bias
A processing bias arising when machine learning applications generate inputs for other machine learning algorithms passing on any existing bias.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A layer composed of artificial input neurons that brings the initial data into the system for further processing by subsequent layers.
Input Layer
A layer composed of artificial input neurons that brings the initial data into the system for further processing by subsequent layers.
https://www.techopedia.com/definition/33262/input-layer-neural-networks#:~:text=Explains%20Input%20Layer-,What%20Does%20Input%20Layer%20Mean%3F,for%20the%20artificial%20neural%20network.
A layer to be used as an entry point into a Network (a graph of layers).
InputLayer Layer
A layer to be used as an entry point into a Network (a graph of layers).
https://www.tensorflow.org/api_docs/python/tf/keras/layers/InputLayer
A layer that specifies the rank
InputSpec Layer
A layer that specifies the rank
https://www.tensorflow.org/api_docs/python/tf/keras/layers/InputSpec
A normalization layer that applies Instance Normalization over a 2D (unbatched) or 3D (batched) input.
InstanceNorm1D
InstanceNorm1d
InstanceNorm1d Layer
A normalization layer that applies Instance Normalization over a 2D (unbatched) or 3D (batched) input.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A normalization layer that applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension).
InstanceNorm2D
InstanceNorm2d
InstanceNorm2d
A normalization layer that applies Instance Normalization over a 4D input (a mini-batch of 2D inputs with additional channel dimension).
https://pytorch.org/docs/stable/nn.html#normalization-layers
A normalization layer that applies Instance Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension).
InstanceNorm3D
InstanceNorm3d
InstanceNorm3d Layer
A normalization layer that applies Instance Normalization over a 5D input (a mini-batch of 3D inputs with additional channel dimension).
https://pytorch.org/docs/stable/nn.html#normalization-layers
A bias exhibited at the level of entire institutions where practices or norms result in the favoring or disadvantaging of certain social groups.
Institutional Bias
A bias exhibited at the level of entire institutions where practices or norms result in the favoring or disadvantaging of certain social groups.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A LLM which is fine-tuned to follow natural language instructions accurately and safely, learning to map from instructions to desired model behavior in a more controlled and principled way.
Instruction-Tuned Large Language Model
constitutional AI
natural language instructions
Instruction-Tuned LLM
A categorical features preprocessing layer that maps integer features to contiguous ranges.
IntegerLookup Layer
A categorical features preprocessing layer that maps integer features to contiguous ranges.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/IntegerLookup
An individual bias where users interpret algorithmic outputs according to their internalized biases and views.
Interpretation Bias
An individual bias where users interpret algorithmic outputs according to their internalized biases and views.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A layer that obtains the dot product of input values or subsets of input values.
Kernel Layer
A machine learning that groups objects by a plurality vote of its neighbors, assigning each object to the class most common among its k nearest neighbors.
K-NN
KNN
K-nearest Neighbor Algorithm
A machine learning that groups objects by a plurality vote of its neighbors, assigning each object to the class most common among its k nearest neighbors.
https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
A classification and clustering that classifies objects by a plurality vote of its neighbors, assigning each object to the class most common among its k nearest neighbors.
K-NN
KNN
K-nearest Neighbor Classification Algorithm
A classification and clustering that classifies objects by a plurality vote of its neighbors, assigning each object to the class most common among its k nearest neighbors.
https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
An regression analysis that assigns the average of the values of k nearest neighbors to objects.
K-NN
KNN
K-nearest Neighbor Regression Algorithm
An regression analysis that assigns the average of the values of k nearest neighbors to objects.
https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm
A LLM which incorporates external knowledge sources or knowledge bases into the model architecture, enabling it to generate more factually accurate and knowledge-aware text.
Knowledge-Grounded Large Language Model
factual grounding
knowledge integration
Knowledge-Grounded LLM
The process by which knowledge is passed from one entity such as a person organization or system to another facilitating learning and adaptation in the receiving entity through various methods such as teaching training or data exchange.
Inductive Transfer
Skill Acquisition
Adaptation
Pretrained models
Knowledge Transfer
The process by which knowledge is passed from one entity such as a person organization or system to another facilitating learning and adaptation in the receiving entity through various methods such as teaching training or data exchange.
https://doi.org/10.1016/j.knosys.2015.01.010
A network that is an unsupervised technique producing a low-dimensional representation of high-dimensional data preserving topological structure.
KN
SOFM
SOM
Self-Organizing Feature Map
Self-Organizing Map
Layers: Input, Hidden
Kohonen Network
A network that is an unsupervised technique producing a low-dimensional representation of high-dimensional data preserving topological structure.
https://en.wikipedia.org/wiki/Self-organizing_map
A pooling layer that applies 1D power-average pooling over an input signal composed of several input planes.
LPPool1D
LPPool1d
LPPool1D Layer
A pooling layer that applies 1D power-average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that applies 2D power-average pooling over an input signal composed of several input planes.
LPPool2D
LPPool2d
LPPool2D Layer
A pooling layer that applies 2D power-average pooling over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A layer that processes one step within the whole time sequence input for an LSTM layer.
LSTMCell Layer
A layer that processes one step within the whole time sequence input for an LSTM layer.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTMCell
A recurrent layer that implements the Long Short-Term Memory architecture.
LSTM Layer
A recurrent layer that implements the Long Short-Term Memory architecture.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM
A layer that wraps arbitrary expressions as a Layer object.
Lambda Layer
A layer that wraps arbitrary expressions as a Layer object.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Lambda
A large language model that supports interactive semantic parsing enabling users to provide feedback and corrections to dynamically refine and update the language model.
Language Interface LLM
Interactive learning
Language Interface LLM
A model designed to predict the next word in a sequence or assign probabilities to sequences of words in natural language.
Language Model
Language Model
A model designed to predict the next word in a sequence or assign probabilities to sequences of words in natural language.
https://en.wikipedia.org/wiki/Language_model
A language model consisting of a neural network with many parameters (typically billions of weights or more) trained on large quantities of unlabeled text using self-supervised learning or semi-supervised learning.
LLM
Large Language Model
A language model consisting of a neural network with many parameters (typically billions of weights or more) trained on large quantities of unlabeled text using self-supervised learning or semi-supervised learning.
https://en.wikipedia.org/wiki/Large_language_model
A regression analysis method that performs both variable selection and regularization to enhance prediction accuracy and interpretability.
Lasso Regression
A regression analysis method that performs both variable selection and regularization to enhance prediction accuracy and interpretability.
https://en.wikipedia.org/wiki/Lasso_(statistics)
A structure or network topology in a deep learning model that takes information from previous layers and passes it to the next layer.
Layer
A structure or network topology in a deep learning model that takes information from previous layers and passes it to the next layer.
https://en.wikipedia.org/wiki/Layer_(deep_learning)
The base class from which all layers inherit.
Layer Layer
The base class from which all layers inherit.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Layer
A normalization layer that applies Layer Normalization over a mini-batch of inputs.
LayerNorm
LayerNorm Layer
A normalization layer that applies Layer Normalization over a mini-batch of inputs.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A normalization layer that applies Layer Normalization over the inputs.
LayerNormalization Layer
A normalization layer that applies Layer Normalization over the inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LayerNormalization
A batch normalization layer that lazily initializes the num_features argument from the input size for 1D data.
LazyBatchNorm1D
LazyBatchNorm1d
LazyBatchNorm1D Layer
A batch normalization layer that lazily initializes the num_features argument from the input size for 1D data.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A batch normalization layer that lazily initializes the num_features argument from the input size for 2D data.
LazyBatchNorm2D
LazyBatchNorm2d
LazyBatchNorm2D Layer
A batch normalization layer that lazily initializes the num_features argument from the input size for 2D data.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A batch normalization layer that lazily initializes the num_features argument from the input size for 3D data.
LazyBatchNorm3D
LazyBatchNorm3d
LazyBatchNorm3D Layer
A batch normalization layer that lazily initializes the num_features argument from the input size for 3D data.
https://pytorch.org/docs/stable/nn.html#normalization-layers
An instance normalization layer that lazily initializes the num_features argument from the input size for 1D data.
LazyInstanceNorm1D
LazyInstanceNorm1d
LazyInstanceNorm1d Layer
An instance normalization layer that lazily initializes the num_features argument from the input size for 1D data.
https://pytorch.org/docs/stable/nn.html#normalization-layers
An instance normalization layer that lazily initializes the num_features argument from the input size for 2D data.
LazyInstanceNorm2D
LazyInstanceNorm2d
LazyInstanceNorm2d Layer
An instance normalization layer that lazily initializes the num_features argument from the input size for 2D data.
https://pytorch.org/docs/stable/nn.html#normalization-layers
An instance normalization layer that lazily initializes the num_features argument from the input size for 3D data.
LazyInstanceNorm3D
LazyInstanceNorm3d
LazyInstanceNorm3d Layer
An instance normalization layer that lazily initializes the num_features argument from the input size for 3D data.
https://pytorch.org/docs/stable/nn.html#normalization-layers
An activation layer that applies the leaky rectified linear unit function element-wise.
LeakyReLU Layer
An activation layer that applies the leaky rectified linear unit function element-wise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LeakyReLU
A regression analysis which approximates the solution of overdetermined systems by minimizing the sum of the squares of the residuals.
Least-squares Analysis
A regression analysis which approximates the solution of overdetermined systems by minimizing the sum of the squares of the residuals.
https://en.wikipedia.org/wiki/Least_squares
A large language model that continually acquires new knowledge over time without forgetting previously learned information maintaining a balance between plasticity and stability.
Continual Learning LLM
Forever Learning
Lifelong Learning LLM
Catastrophic forgetting
Plasticity-Stability balance
Lifelong Learning LLM
An activation function that has the form f(x) = a + bx.
Linear Function
An activation function that has the form f(x) = a + bx.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/linear
A regression analysis model that is a linear approach for modeling the relationship between a scalar response and one or more explanatory variables.
Linear Regression
A regression analysis model that is a linear approach for modeling the relationship between a scalar response and one or more explanatory variables.
https://en.wikipedia.org/wiki/Linear_regression
A use and interpretation bias arising when network attributes obtained from user connections activities or interactions misrepresent true user behavior.
Linking Bias
A use and interpretation bias arising when network attributes obtained from user connections activities or interactions misrepresent true user behavior.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A network that is a type of reservoir computer turning time-varying input into spatio-temporal activation patterns.
LSM
Layers: Input, Spiking Hidden, Output
Liquid State Machine Network
A network that is a type of reservoir computer turning time-varying input into spatio-temporal activation patterns.
https://en.wikipedia.org/wiki/Liquid_state_machine
A normalization layer that applies local response normalization over an input signal composed of several input planes.
LocalResponseNorm
LocalResponseNorm Layer
A normalization layer that applies local response normalization over an input signal composed of several input planes.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A locally-connected layer for 1D inputs where each patch of the input is convolved with a different set of filters.
LocallyConnected1D Layer
A locally-connected layer for 1D inputs where each patch of the input is convolved with a different set of filters.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LocallyConnected1D
A locally-connected layer for 2D inputs where each patch of the input is convolved with a different set of filters.
LocallyConnected2D Layer
A locally-connected layer for 2D inputs where each patch of the input is convolved with a different set of filters.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LocallyConnected2D
A layer that works similarly to the Convolution1D layer, except that weights are unshared, that is, a different set of filters is applied at each different patch of the input.
Locally-connected Layer
A layer that works similarly to the Convolution1D layer, except that weights are unshared, that is, a different set of filters is applied at each different patch of the input.
https://faroit.com/keras-docs/1.2.2/layers/local/
A regression analysis model that estimates the probability of an event occurring by modeling the log-odds of the event as a linear combination of one or more independent variables.
Logistic Regression
A regression analysis model that estimates the probability of an event occurring by modeling the log-odds of the event as a linear combination of one or more independent variables.
https://en.wikipedia.org/wiki/Logistic_regression
A recurrent neural network with feedback connections that processes entire sequences of data.
LSTM
Layers: Input, Memory Cell, Output
Long Short Term Memory
A recurrent neural network with feedback connections that processes entire sequences of data.
https://en.wikipedia.org/wiki/Long_short-term_memory
An individual bias occurring when automation leads to humans being unaware of their situation making them unprepared to assume control in cooperative systems.
Loss Of Situational Awareness Bias
An individual bias occurring when automation leads to humans being unaware of their situation making them unprepared to assume control in cooperative systems.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A LLM which is optimized for performance in scenarios with limited data, computational resources, or for languages with sparse datasets.
Low-Resource Language Model
low-resource languages
resource-efficient
Low-Resource LLM
A field of inquiry devoted to understanding and building methods that learn from data to improve performance on a set of tasks.
Machine Learning
A field of inquiry devoted to understanding and building methods that learn from data to improve performance on a set of tasks.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A dimensionality reduction method based on the assumption that observed data lie on a low-dimensional manifold embedded in a higher-dimensional space.
Manifold Learning
A dimensionality reduction method based on the assumption that observed data lie on a low-dimensional manifold embedded in a higher-dimensional space.
https://arxiv.org/abs/2011.01307
A network that is a stochastic model describing a sequence of possible events where the probability of each event depends only on the previous event's state.
MC
MP
Markov Process
Layers: Probalistic Hidden
Markov Chain
A network that is a stochastic model describing a sequence of possible events where the probability of each event depends only on the previous event's state.
https://en.wikipedia.org/wiki/Markov_chain
A language model that is trained to predict randomly masked tokens in a sequence based on the remaining unmasked tokens allowing it to build deep bidirectional representations that can be effectively transferred to various NLP tasks via fine-tuning.
Masked Language Model
bidirectional encoder
denoising autoencoder
Masked Language Model
A layer that masks a sequence by using a mask value to skip timesteps.
Masking Layer
A layer that masks a sequence by using a mask value to skip timesteps.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Masking
An input layer with a shape corresponding to that of the output layer.
Matched Input-Output Layer
A pooling layer that performs max pooling operation for temporal data.
MaxPool1D
MaxPool1d
MaxPooling1D
MaxPooling1d
MaxPooling1D Layer
A pooling layer that performs max pooling operation for temporal data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool1D
A pooling layer that performs max pooling operation for spatial data.
MaxPool2D
MaxPool2d
MaxPooling2D
MaxPooling2d
MaxPooling2D Layer
A pooling layer that performs max pooling operation for spatial data.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D
A pooling layer that performs max pooling operation for 3D data (spatial or spatio-temporal).
MaxPool3D
MaxPool3d
MaxPooling3D
MaxPooling3d
MaxPooling3D Layer
A pooling layer that performs max pooling operation for 3D data (spatial or spatio-temporal).
https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool3D
A pooling layer that computes a partial inverse of MaxPool1d.
MaxUnpool1D
MaxUnpool1d
MaxUnpool1D Layer
A pooling layer that computes a partial inverse of MaxPool1d.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that computes a partial inverse of MaxPool2d.
MaxUnpool2D
MaxUnpool2d
MaxUnpool2D Layer
A pooling layer that computes a partial inverse of MaxPool2d.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A pooling layer that computes a partial inverse of MaxPool3d.
MaxUnpool3D
MaxUnpool3d
MaxUnpool3D Layer
A pooling layer that computes a partial inverse of MaxPool3d.
https://pytorch.org/docs/stable/nn.html#pooling-layers
A merging layer that computes the maximum (element-wise) of a list of inputs.
Maximum Layer
A merging layer that computes the maximum (element-wise) of a list of inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Maximum
A selection and sampling bias arising when features and labels are proxies for desired quantities potentially leading to differential performance.
Measurement Bias
A selection and sampling bias arising when features and labels are proxies for desired quantities potentially leading to differential performance.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A LLM which incorporates external writable and readable memory components, allowing it to store and retrieve information over long contexts.
Memory-Augmented Large Language Model
external memory
Memory-Augmented LLM
A LLM which incorporates external writable and readable memory components, allowing it to store and retrieve information over long contexts.
https://arxiv.org/abs/2306.07174
A layer of cells, each with an internal state or weights.
Memory Cell Layer
A layer of cells, each with an internal state or weights.
https://doi.org/10.1162/neco.1997.9.8.1735
A layer used to merge a list of inputs.
Merging Layer
A layer used to merge a list of inputs.
https://www.tutorialspoint.com/keras/keras_merge_layer.htm
A machine learning that automatically learns from metadata about machine learning experiments.
Meta-Learning
A machine learning that automatically learns from metadata about machine learning experiments.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A LLM which is trained in a way that allows it to quickly adapt to new tasks or datasets through only a few examples or fine-tuning steps, leveraging meta-learned priors about how to efficiently learn.
Meta-Learning Large Language Model
few-shot adaptation
learning to learn
Meta-Learning LLM
A deep neural network that learns a representation function mapping objects into an embedded space.
Distance Metric Learning
Metric Learning
A deep neural network that learns a representation function mapping objects into an embedded space.
https://paperswithcode.com/task/metric-learning
A merging layer that computes the minimum (element-wise) of a list of inputs.
Minimum Layer
A merging layer that computes the minimum (element-wise) of a list of inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Minimum
A LLM which dynamically selects and combines outputs from multiple expert submodels, allowing for efficient scaling by conditionally activating only a subset of model components for each input.
Mixture-of-Experts Large Language Model
MoE Large Language Model
conditional computation
model parallelism
Mixture-of-Experts LLM
A LLM which dynamically selects and combines outputs from multiple expert submodels, allowing for efficient scaling by conditionally activating only a subset of model components for each input.
https://proceedings.mlr.press/v162/du22c.html
A bias occurring when modal interfaces confuse human operators causing actions appropriate for a different mode but incorrect for the current situation.
Mode Confusion Bias
A bias occurring when modal interfaces confuse human operators causing actions appropriate for a different mode but incorrect for the current situation.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An abstract representation of a complex system generally assembled as a set of logical mathematical or conceptual properties to simulate or understand the system's behavior.
Model
Model
An abstract representation of a complex system generally assembled as a set of logical mathematical or conceptual properties to simulate or understand the system's behavior.
https://en.wikipedia.org/wiki/Mathematical_model
Techniques aimed at making models more efficient such as knowledge distillation.
Computational Efficiency
Model Optimization
Model Efficiency
Techniques aimed at making models more efficient such as knowledge distillation.
https://doi.org/10.1145/3578938
A processing bias introduced when using data to select a single "best" model from many or when an explanatory variable has a weak relationship with the response variable.
Model Selection Bias
A processing bias introduced when using data to select a single "best" model from many or when an explanatory variable has a weak relationship with the response variable.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A modular large language model that consists of multiple specialized components or skills that can be dynamically composed and recombined to solve complex tasks mimicking the modular structure of human cognition.
Modular Large Language Model
component skills
skill composition
Modular LLM
A modular large language model that consists of multiple specialized components or skills that can be dynamically composed and recombined to solve complex tasks mimicking the modular structure of human cognition.
https://arxiv.org/abs/2302.11529v2
A language model that consists of multiple specialized components or skills that can be dynamically composed and recombined to solve complex tasks mimicking the modular structure of human cognition.
Modular LM
Modular Language Model
A language model that consists of multiple specialized components or skills that can be dynamically composed and recombined to solve complex tasks mimicking the modular structure of human cognition.
https://arxiv.org/abs/2302.11529v2
An attention layer that allows the model to attend to information from different representation subspaces.
MultiHeadAttention Layer
An attention layer that allows the model to attend to information from different representation subspaces.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/MultiHeadAttention
A LLM which is trained jointly on multiple language tasks simultaneously, learning shared representations that transfer across tasks.
Multi-Task Large Language Model
transfer learning
Multi-Task LLM
A machine learning task focused on methods that classify instances into one of three or more classes.
Multinomial Classification
Multiclass Classification
A machine learning task focused on methods that classify instances into one of three or more classes.
https://en.wikipedia.org/wiki/Multiclass_classification
A dimensionality reduction method that translates information about the pairwise distances among a set of objects or individuals into a configuration of points mapped into an abstract Cartesian space.
MDS
Multidimensional Scaling
A dimensionality reduction method that translates information about the pairwise distances among a set of objects or individuals into a configuration of points mapped into an abstract Cartesian space.
https://en.wikipedia.org/wiki/Multidimensional_scaling
A large language model that is trained on text from multiple languages learning shared representations that enable zero-shot or few-shot transfer to new languages.
Multilingual Large Language Model
cross-lingual transfer
Multilingual LLM
A deep neural network that processes and links information using various modalities.
Multimodal Deep Learning
A deep neural network that processes and links information using various modalities.
https://arxiv.org/abs/2105.11087
A large language model that learns joint representations across different modalities like text vision and audio in an end-to-end fashion for better cross-modal understanding and generation.
Multimodal Fusion LLM
cross-modal grounding
Multimodal Fusion LLM
A multimodal large language model that learns joint representations across different modalities like text vision and audio in an end-to-end fashion for better cross-modal understanding and generation.
Multimodal Large Language Model
cross-modal grounding
Multimodal LLM
A multimodal large language model that learns joint representations across different modalities like text vision and audio in an end-to-end fashion for better cross-modal understanding and generation.
https://arxiv.org/abs/2303.17580
A language model that learns joint representations across different modalities like text vision and audio in an end-to-end fashion for better cross-modal understanding and generation.
Mulimodal LM
Multimodal Language Model
A language model that learns joint representations across different modalities like text vision and audio in an end-to-end fashion for better cross-modal understanding and generation.
https://arxiv.org/abs/2205.12630
A type of machine learning that uses multiple modalities of data such as text audio and images to improve learning outcomes.
Multimodal Learning
A type of machine learning that uses multiple modalities of data such as text audio and images to improve learning outcomes.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A multimodal LLM which processes prompts that include multiple modalities, such as both text and images, to generate relevant responses.
Multimodal Prompt-based Language Model
Multimodal Prompt-based Language Model
A multimodal LLM which processes prompts that include multiple modalities, such as both text and images, to generate relevant responses.
https://arxiv.org/abs/2210.03094
A transformer network that processes and relates information from different modalities such as text images and audio using a shared embedding space and attention mechanism to learn joint representations across modalities.
Multimodal Transformer
unified encoder
vision-language model
Multimodal Transformer
A merging layer that multiplies (element-wise) a list of inputs.
Multiply Layer
A merging layer that multiplies (element-wise) a list of inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Multiply
A subfield of machine learning focused on the interactions between computers and human language including programming computers to process and analyze large amounts of natural language data.
NLP
Natural Language Processing
A subfield of machine learning focused on the interactions between computers and human language including programming computers to process and analyze large amounts of natural language data.
https://en.wikipedia.org/wiki/Natural_language_processing
A system of interconnected nodes or entities for communication computation or data exchange.
Network
A deep feedforward network that combines neural network pattern matching with the algorithmic power of programmable computers.
NTM
Layers: Input, Hidden, Spiking Hidden, Output
Neural Turing Machine Network
A deep feedforward network that combines neural network pattern matching with the algorithmic power of programmable computers.
https://en.wikipedia.org/wiki/Neural_Turing_machine
A LLM which combines neural language modeling with symbolic reasoning components, leveraging structured knowledge representations and logical inferences to improve reasoning capabilities.
Neuro-Symbolic Large Language Model
knowledge reasoning
symbolic grounding
Neuro-Symbolic LLM
A layer that is a densely-connected neural network layer with added noise for regularization.
Noise Dense Layer
A layer that is a densely-connected neural network layer with added noise for regularization.
https://www.tensorflow.org/addons/api_docs/python/tfa/layers/NoisyDense
An input layer that adds noise to each value.
Noisy Input Layer
An input layer that adds noise to each value.
https://doi.org/10.1109/21.155944
The technique of transforming data into a standard format or scale typically to reduce redundancy and improve consistency often involving the adjustment of values measured on different scales to a common scale.
Normalization
A preprocessing layer that normalizes continuous features.
Normalization Layer
A preprocessing layer that normalizes continuous features.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Normalization
A layer that performs numerical data preprocessing operations.
Numerical Features Preprocessing Layer
A layer that performs numerical data preprocessing operations.
https://keras.io/guides/preprocessing_layers/
A deep neural network that classified objects from one or only a few examples.
OSL
One-shot Learning
A deep neural network that classified objects from one or only a few examples.
https://en.wikipedia.org/wiki/One-shot_learning
A large language model that is trained to model ordinal relationships and rank outputs rather than model probability distributions over text sequences directly.
Ordinal Large Language Model
preference modeling
ranking
Ordinal LLM
A layer containing the last neurons in the network that produces given outputs for the program.
Output Layer
A layer containing the last neurons in the network that produces given outputs for the program.
https://www.techopedia.com/definition/33263/output-layer-neural-networks
An activation layer that applies parametric rectified linear unit function element-wise.
PReLU Layer
An activation layer that applies parametric rectified linear unit function element-wise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/PReLU
An artificial neural network with a supervised learning algorithm for binary classification using a linear predictor function.
FFN
Feed-Forward Network
SLP
Single Layer Perceptron
Layers: Input, Output
Perceptron
A layer that permutes the dimensions of the input according to a given pattern.
Permute Layer
A layer that permutes the dimensions of the input according to a given pattern.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Permute
A large language model that adapts its language modeling and generation to the preferences style and persona of individual users or audiences.
Personalized Large Language Model
user adaptation LLM
Personalized LLM
A layer that, after taking a set of states or values as input, predicts a probability distribution of actions to take.
Policy Layer
A layer that serves to mitigate the sensitivity of convolutional layers to location and spatially downsample representations.
Pooling Layer
A layer that serves to mitigate the sensitivity of convolutional layers to location and spatially downsample representations.
https://d2l.ai/chapter_convolutional-neural-networks/pooling.html
A selection and sampling bias where more popular items are more exposed under-representing less popular items.
Popularity Bias
A selection and sampling bias where more popular items are more exposed under-representing less popular items.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A selection and sampling bias characterized by systematic distortions in demographics or other user characteristics between represented users and the target population.
Population Bias
A selection and sampling bias characterized by systematic distortions in demographics or other user characteristics between represented users and the target population.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
The series of steps applied to raw data before it is used in a machine learning model including tasks such as normalization scaling encoding and transformation to ensure the data is in an appropriate format and quality for analysis.
Preprocessing
The series of steps applied to raw data before it is used in a machine learning model including tasks such as normalization scaling encoding and transformation to ensure the data is in an appropriate format and quality for analysis.
https://doi.org/10.1109/ICDE.2019.00245
A layer that performs data preprocessing operations.
Preprocessing Layer
A layer that performs data preprocessing operations.
https://www.tensorflow.org/guide/keras/preprocessing_layers
An individual bias arising from how information is presented on the Web via a user interface due to rating or ranking of output or through users' self-selected biased interaction.
Presentation Bias
An individual bias arising from how information is presented on the Web via a user interface due to rating or ranking of output or through users' self-selected biased interaction.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A dimensionality reduction method for analyzing large datasets with high-dimensional features per observation increasing data interpretability while preserving maximum information and enabling visualization.
PCA
Principal Component Analysis
A dimensionality reduction method for analyzing large datasets with high-dimensional features per observation increasing data interpretability while preserving maximum information and enabling visualization.
https://en.wikipedia.org/wiki/Principal_component_analysis
A machine learning model in which a graph expresses the conditional dependence structure between random variables.
Graphical Model
PGM
Structure Probabilistic Model
Probabilistic Graphical Model
A machine learning model in which a graph expresses the conditional dependence structure between random variables.
https://en.wikipedia.org/wiki/Graphical_model
A hidden layer that estimates the probability of a sample being within a certain category.
Probabilistic Hidden Layer
A probabilistic graphical model that uses statistical techniques to analyze the words in each text to discover common themes their connections and their changes over time.
Probabilistic Topic Model
A probabilistic graphical model that uses statistical techniques to analyze the words in each text to discover common themes their connections and their changes over time.
https://pyro.ai/examples/prodlda.html
A computational bias resulting from judgment modulated by affect influenced by the level of efficacy and efficiency in information processing.
Validation Bias
Processing Bias
A computational bias resulting from judgment modulated by affect influenced by the level of efficacy and efficiency in information processing.
GTP-4o with Seppala et al. 2017
https://en.wikipedia.org/wiki/Bias_(statistics)
A LLM which is fine-tuned on a small number of examples or prompts, rather than full task datasets. This allows for rapid adaptation to new tasks with limited data, leveraging the model's few-shot learning capabilities.
Prompt-based Fine-Tuning Large Language Model
Prompt-tuned Large Language Model
few-shot learning
in-context learning
Prompt-based Fine-Tuning LLM
A regression analysis method for survival analysis where the unique effect of a unit increase in a covariate is multiplicative with respect to the hazard rate.
Proportional Hazards Model
A regression analysis method for survival analysis where the unique effect of a unit increase in a covariate is multiplicative with respect to the hazard rate.
https://en.wikipedia.org/wiki/Proportional_hazards_model
The base class for recurrent layers.
RNN Layer
The base class for recurrent layers.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RNN
A deep feedforward network that uses radial basis functions as activation functions for pattern recognition and interpolation.
RBFN
RBN
Radial Basis Function Network
Layers: Input, Hidden, Output
Radial Basis Network
A deep feedforward network that uses radial basis functions as activation functions for pattern recognition and interpolation.
https://en.wikipedia.org/wiki/Radial_basis_function_network
An image preprocessing layer that randomly adjusts brightness during training.
RandomBrightness Layer
An image preprocessing layer that randomly adjusts brightness during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomBrightness
An image preprocessing layer that randomly adjusts contrast during training.
RandomContrast Layer
An image preprocessing layer that randomly adjusts contrast during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomContrast
An image preprocessing layer that randomly crops images during training.
RandomCrop Layer
An image preprocessing layer that randomly crops images during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomCrop
A regression analysis model where the model parameters are random variables.
REM
Random Effects Model
A regression analysis model where the model parameters are random variables.
https://en.wikipedia.org/wiki/Random_effects_model
An image preprocessing layer that randomly flips images during training.
RandomFlip Layer
An image preprocessing layer that randomly flips images during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomFlip
An ensemble learning method for classification regression and other tasks that constructs a multitude of decision trees during training.
Random Forest
An ensemble learning method for classification regression and other tasks that constructs a multitude of decision trees during training.
https://en.wikipedia.org/wiki/Random_forest
An image preprocessing layer that randomly varies image height during training.
RandomHeight Layer
An image preprocessing layer that randomly varies image height during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomHeight
An image preprocessing layer that randomly rotates images during training.
RandomRotation Layer
An image preprocessing layer that randomly rotates images during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomRotation
An image preprocessing layer that randomly translates images during training.
RandomTranslation Layer
An image preprocessing layer that randomly translates images during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomTranslation
An image preprocessing layer that randomly varies image width during training.
RandomWidth Layer
An image preprocessing layer that randomly varies image width during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomWidth
An image preprocessing layer that randomly zooms in or out on images during training.
RandomZoom Layer
An image preprocessing layer that randomly zooms in or out on images during training.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RandomZoom
An anchoring bias characterized by the idea that top-ranked results are the most relevant and important leading to more clicks than other results.
Ranking Bias
An anchoring bias characterized by the idea that top-ranked results are the most relevant and important leading to more clicks than other results.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An individual bias characterized by differences in perspective memory recall interpretation and reporting of the same event by multiple persons or witnesses.
Rashomon Effect
Rashomon Principle
Rashomon Effect Bias
An individual bias characterized by differences in perspective memory recall interpretation and reporting of the same event by multiple persons or witnesses.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An activation function that returns max(x 0) the element-wise maximum of 0 and the input tensor.
ReLU
Rectified Linear Unit
ReLU Function
An activation function that returns max(x 0) the element-wise maximum of 0 and the input tensor.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/relu
An activation layer that applies the rectified linear unit function element-wise.
ReLU Layer
An activation layer that applies the rectified linear unit function element-wise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ReLU
A large language model that incorporates explicit reasoning capabilities leveraging logical rules axioms or external knowledge to make deductive inferences during language tasks.
Rational Large Language Model
Reasoning Large Language Model
logical inferences
reasoning
Reasoning LLM
A large language model that incorporates explicit reasoning capabilities leveraging logical rules axioms or external knowledge to make deductive inferences during language tasks.
https://doi.org/10.18653/v1/2023.acl-long.347
A layer composed of recurrent units with the number equal to the hidden size of the layer.
Recurrent Layer
A layer composed of recurrent units with the number equal to the hidden size of the layer.
https://docs.nvidia.com/deepLearning/performance/dl-performance-recurrent/index.html#recurrent-layer
A deep neural network with connections forming a directed graph along a temporal sequence enabling dynamic behavior.
RN
RecNN
Recurrent Network
Recurrent Neural Network
A large language model that uses recursive neural network architectures like TreeLSTMs to learn syntactic composition functions improving systematic generalization abilities.
Recursive Large Language Model
Self-Attending Large Language Model
iterative refinement
self-attention
Recursive LLM
A large language model that uses recursive neural network architectures like TreeLSTMs to learn syntactic composition functions improving systematic generalization abilities.
https://doi.org/10.1609/aaai.v33i01.33017450
A language model that uses recursive neural network architectures like TreeLSTMs to learn syntactic composition functions improving systematic generalization abilities.
RLM
Compositional generalization
Layers: Input, Memory Cell, Output
Recursive Language Model
A language model that uses recursive neural network architectures like TreeLSTMs to learn syntactic composition functions improving systematic generalization abilities.
https://en.wikipedia.org/wiki/Recurrent_neural_network
A deep neural network that recursively applies weights over structured input to generate structured or scalar predictions.
RecuNN
RvNN
Recursive Neural Network
A deep neural network that recursively applies weights over structured input to generate structured or scalar predictions.
https://en.wikipedia.org/wiki/Recursive_neural_network
A set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables.
Regression analysis
Regression model
Regression Analysis
A set of statistical processes for estimating the relationships between a dependent variable and one or more independent variables.
https://en.wikipedia.org/wiki/Regression_analysis
A layer that applies penalties on layer parameters or layer activity during optimization summed into the loss function that the network optimizes.
Regularization Layer
A layer that applies penalties on layer parameters or layer activity during optimization summed into the loss function that the network optimizes.
https://keras.io/api/layers/regularizers/
A type of machine learning focused on methods that do not require labeled input/output pairs or explicit correction of sub-optimal actions focusing instead on balancing exploration and exploitation to optimize performance over time.
Reinforcement Learning
A type of machine learning focused on methods that do not require labeled input/output pairs or explicit correction of sub-optimal actions focusing instead on balancing exploration and exploitation to optimize performance over time.
https://en.wikipedia.org/wiki/Reinforcement_learning
A large language model that is fine-tuned using reinforcement learning where the model receives rewards for generating text that satisfies certain desired properties or objectives improving the quality safety or alignment of generated text.
RL-Large Language Model
Reinforcement Learning Large Language Model
decision transformers
reward modeling
Reinforcement Learning LLM
A layer that repeats the input n times.
RepeatVector Layer
A layer that repeats the input n times.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/RepeatVector
A selection and sampling bias due to non-random sampling of subgroups making trends non-generalizable to new populations.
Representation Bias
A selection and sampling bias due to non-random sampling of subgroups making trends non-generalizable to new populations.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A deep neural network that discovers representations required for feature detection or classification from raw data.
Feature Learning
Representation Learning
A deep neural network that discovers representations required for feature detection or classification from raw data.
https://en.wikipedia.org/wiki/Feature_learning
A preprocessing layer that rescales input values to a new range.
Rescaling Layer
A preprocessing layer that rescales input values to a new range.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Rescaling
A layer that reshapes the inputs into the given shape.
Reshape Layer
A layer that reshapes the inputs into the given shape.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Reshape
A layer that is used to change the shape of the input.
Reshape Layer
Reshaping Layer
A layer that is used to change the shape of the input.
https://keras.io/api/layers/reshaping_layers/reshape/
A deep neural network that employs skip connections to bypass layers facilitating learning of residual functions.
DRN
Deep Residual Network
ResNN
ResNet
Layers: Input, Weight, BN, ReLU, Weight, BN, Addition, ReLU
Residual Neural Network
A deep neural network that employs skip connections to bypass layers facilitating learning of residual functions.
https://en.wikipedia.org/wiki/Residual_neural_network
A preprocessing layer that resizes images to a target size.
Resizing Layer
A preprocessing layer that resizes images to a target size.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Resizing
A Boltzmann machine network that learns the probability distribution of its input data.
RBM
Layers: Backfed Input, Probabilistic Hidden
Restricted Boltzmann Machine
A Boltzmann machine network that learns the probability distribution of its input data.
https://en.wikipedia.org/wiki/Restricted_Boltzmann_machine
A LLM which combines a pre-trained language model with a retrieval system that can access external knowledge sources. This allows the model to condition its generation on relevant retrieved knowledge, improving factual accuracy and knowledge grounding.
Retrieval-Augmented Large Language Model
knowledge grounding
open-book question answering
Retrieval-Augmented LLM
A regression analysis method that estimates the coefficients of multiple regression models in scenarios where the independent variables are highly correlated.
Ridge Regression
A regression analysis method that estimates the coefficients of multiple regression models in scenarios where the independent variables are highly correlated.
https://en.wikipedia.org/wiki/Ridge_regression
An activation function that multiplies scale (> 1) with the output of the ELU function to ensure a slope larger than one for positive inputs.
SELU
Scaled Exponential Linear Unit
SELU Function
An activation function that multiplies scale (> 1) with the output of the ELU function to ensure a slope larger than one for positive inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/selu
A model that extends ARIMA, explicitly supporting univariate time series data with a seasonal component, combining seasonal differencing with ARIMA modeling.
SARIMA
Seasonal Autoregressive Integrated Moving-Average
A computational bias introduced by non-random selection of individuals groups or data failing to ensure representativeness.
Sampling Bias
Selection Bias
Selection Effect
Selection And Sampling Bias
A computational bias introduced by non-random selection of individuals groups or data failing to ensure representativeness.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An individual bias characterized by the tendency to selectively adopt algorithmic advice that matches pre-existing beliefs and stereotypes.
Selective Adherence Bias
An individual bias characterized by the tendency to selectively adopt algorithmic advice that matches pre-existing beliefs and stereotypes.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A LLM which learns rich representations by solving pretext tasks that involve predicting parts of the input from other observed parts of the data, without relying on human-annotated labels.
Self-Supervised LLM
Pretext tasks
Self-Supervised LLM
A machine learning that is intermediate between supervised and unsupervised learning and predicts parts of the input data from other observed parts without relying on human-annotated labels.
Self-supervised Learning
A machine learning that is intermediate between supervised and unsupervised learning and predicts parts of the input data from other observed parts without relying on human-annotated labels.
https://en.wikipedia.org/wiki/Self-supervised_learning
A LLM which combines self-supervised pretraining on unlabeled data with supervised fine-tuning on labeled task data.
Semi-Supervised Large Language Model
self-training
Semi-Supervised LLM
A layer that performs depthwise separable 1D convolution.
SeparableConv1D Layer
SeparableConvolution1D Layer
A layer that performs depthwise separable 1D convolution.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SeparableConv1D
A layer that performs depthwise separable 2D convolution.
SeparableConv2D Layer
SeparableConvolution2D Layer
A layer that performs depthwise separable 2D convolution.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SeparableConv2D
An activation function that applies the sigmoid activation function sigmoid(x) = 1 / (1 + exp(-x)) always returning a value between 0 and 1.
Sigmoid Function
An activation function that applies the sigmoid activation function sigmoid(x) = 1 / (1 + exp(-x)) always returning a value between 0 and 1.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/sigmoid
A layer that processes one step within the whole time sequence input for a SimpleRNN layer.
SimpleRNNCell Layer
A layer that processes one step within the whole time sequence input for a SimpleRNN layer.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SimpleRNNCell
A recurrent layer that implements a fully-connected RNN where the output is to be fed back to input.
SimpleRNN Layer
A recurrent layer that implements a fully-connected RNN where the output is to be fed back to input.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SimpleRNN
#N/A
Simpson's Paradox
Simpon's Paradox Bias
#N/A
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A bias characterized by being for or against groups or individuals based on social identities demographic factors or immutable physical characteristics often manifesting as stereotypes.
Social Bias
Societal Bias
A bias characterized by being for or against groups or individuals based on social identities demographic factors or immutable physical characteristics often manifesting as stereotypes.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An activation function where the elements of the output vector are in range (0 1) and sum to 1 and each vector is handled independently.
Softmax Function
An activation function where the elements of the output vector are in range (0 1) and sum to 1 and each vector is handled independently.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/softmax
An activation layer that applies the softmax function to the inputs.
Softmax Layer
An activation layer that applies the softmax function to the inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Softmax
An activation function that is softplus(x) = log(exp(x) + 1).
Softplus Function
An activation function that is softplus(x) = log(exp(x) + 1).
https://www.tensorflow.org/api_docs/python/tf/keras/activations/softplus
An activation function that is softsign(x) = x / (abs(x) + 1).
Softsign Function
An activation function that is softsign(x) = x / (abs(x) + 1).
https://www.tensorflow.org/api_docs/python/tf/keras/activations/softsign
An autoencoder network with more hidden units than inputs that constrains only a few hidden units to be active at once.
SAE
Sparse AE
Sparse Autoencoder
Layers: Input, Hidden, Matched Output-Input
Sparse Auto Encoder
A large language model that uses techniques like pruning or quantization to reduce the number of non-zero parameters in the model making it more parameter-efficient and easier to deploy on resource-constrained devices.
Sparse Large Language Model
model compression
parameter efficiency
Sparse LLM
A representation learning network that finds sparse representations of input data as a linear combination of basic elements and identifies those elements.
Sparse coding
Sparse dictionary Learning
Sparse Learning
A representation learning network that finds sparse representations of input data as a linear combination of basic elements and identifies those elements.
https://en.wikipedia.org/wiki/Sparse_dictionary_learning
A regularization layer that performs the same function as Dropout but drops entire 1D feature maps instead of individual elements.
SpatialDropout1D Layer
A regularization layer that performs the same function as Dropout but drops entire 1D feature maps instead of individual elements.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SpatialDropout1D
A regularization layer that performs the same function as Dropout but drops entire 2D feature maps instead of individual elements.
SpatialDropout2D Layer
A regularization layer that performs the same function as Dropout but drops entire 2D feature maps instead of individual elements.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SpatialDropout2D
A regularization layer that performs the same function as Dropout but drops entire 3D feature maps instead of individual elements.
SpatialDropout3D Layer
A regularization layer that performs the same function as Dropout but drops entire 3D feature maps instead of individual elements.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/SpatialDropout3D
A regression analysis method used to model spatial relationships.
Spatial Regression
A regression analysis method used to model spatial relationships.
https://gisgeography.com/spatial-regression-models-arcgis/
A hidden layer that makes connections to an additional, heterogeneous hidden layer; modeled after biological neural networks.
Spiking Hidden Layer
A hidden layer that makes connections to an additional, heterogeneous hidden layer; modeled after biological neural networks.
https://doi.org/10.1016/S0893-6080(97)00011-7
A layer that allows a stack of RNN cells to behave as a single cell.
StackedRNNCells Layer
A layer that allows a stack of RNN cells to behave as a single cell.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/StackedRNNCells
An individual bias where people search only where it is easiest to look.
Streetlight Effect
Streetlight Effect Bias
An individual bias where people search only where it is easiest to look.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A categorical features preprocessing layer that maps string features to integer indices.
StringLookup Layer
A categorical features preprocessing layer that maps string features to integer indices.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/StringLookup
A merging layer that subtracts two inputs.
Subtract Layer
A merging layer that subtracts two inputs.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Subtract
The process of dividing text into subword units which are smaller than words but larger than individual characters to improve the efficiency and effectiveness of natural language processing models by capturing meaningful subunits of words.
Fragmentation
Part-word Division
Byte Pair Encoding
SentencePiece
Subword Segmentation
A bias characterized by the tendency to continue an endeavor due to previously invested resources despite costs outweighing benefits.
Sunk Cost Fallacy
Sunk Cost Fallacy Bias
A bias characterized by the tendency to continue an endeavor due to previously invested resources despite costs outweighing benefits.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A biclustering task focused on methods that simultaneously cluster the rows and columns of a labeled matrix considering data labels to enhance cluster coherence.
Supervised Block Clustering
Supervised Co-clustering
Supervised Joint Clustering
Supervised Two-mode Clustering
Supervised Two-way Clustering
Supervised Biclustering
A biclustering task focused on methods that simultaneously cluster the rows and columns of a labeled matrix considering data labels to enhance cluster coherence.
https://en.wikipedia.org/wiki/Biclustering
A clustering task focused on methods that group labeled objects such that objects in the same group have similar labels relative to those in other groups.
Cluster analysis
Supervised Clustering
A clustering task focused on methods that group labeled objects such that objects in the same group have similar labels relative to those in other groups.
https://en.wikipedia.org/wiki/Cluster_analysis
A type of machine learning focused on methods that learn a function mapping input to output based on example input-output pairs.
Supervised Learning
A type of machine learning focused on methods that learn a function mapping input to output based on example input-output pairs.
https://en.wikipedia.org/wiki/Supervised_learning
A network with supervised learning models for classification and regression that maps training examples to points in space maximizing the gap between categories.
SVM
SVN
Supper Vector Network
Layers: Input, Hidden, Output
Support Vector Machine
A network with supervised learning models for classification and regression that maps training examples to points in space maximizing the gap between categories.
https://en.wikipedia.org/wiki/Support-vector_machine
A machine learning task focused on methods for analyzing the expected duration of time until one or more events occur such as death in biological organisms or failure in mechanical systems.
Survival Analysis
A machine learning task focused on methods for analyzing the expected duration of time until one or more events occur such as death in biological organisms or failure in mechanical systems.
https://en.wikipedia.org/wiki/Survival_analysis
A processing bias characterized by the tendency to focus on items observations or people that "survive" a selection process overlooking those that did not.
Survivorship Bias
A processing bias characterized by the tendency to focus on items observations or people that "survive" a selection process overlooking those that did not.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An activation function that is x*sigmoid(x) a smooth non-monotonic function that consistently matches or outperforms ReLU on deep networks.
Swish Function
An activation function that is x*sigmoid(x) a smooth non-monotonic function that consistently matches or outperforms ReLU on deep networks.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/swish
A network that is a type of recurrent neural network where connections between units are symmetrical with equal weights in both directions.
SCN
Symmetrically Connected Network
A network that is a type of recurrent neural network where connections between units are symmetrical with equal weights in both directions.
https://ieeexplore.ieee.org/document/287176
A batch normalization layer that applies synchronous Batch Normalization across multiple devices.
SyncBatchNorm
SyncBatchNorm Layer
A batch normalization layer that applies synchronous Batch Normalization across multiple devices.
https://pytorch.org/docs/stable/nn.html#normalization-layers
A bias resulting from procedures and practices of institutions that operate in ways which result in certain social groups being advantaged or favored and others being disadvantaged or devalued.
Institutional Bias
Societal Bias
Systemic Bias
A bias resulting from procedures and practices of institutions that operate in ways which result in certain social groups being advantaged or favored and others being disadvantaged or devalued.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An activation function that is the hyperbolic tangent activation function.
hyperbolic tangent
Tanh Function
An activation function that is the hyperbolic tangent activation function.
https://www.tensorflow.org/api_docs/python/tf/keras/activations/tanh
A selection and sampling bias arising from differences in populations and behaviors over time.
Temporal Bias
A selection and sampling bias arising from differences in populations and behaviors over time.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A layer that performs text data preprocessing operations.
Text Preprocessing Layer
A layer that performs text data preprocessing operations.
https://keras.io/guides/preprocessing_layers/
A preprocessing layer that maps text features to integer sequences.
TextVectorization Layer
A preprocessing layer that maps text features to integer sequences.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/TextVectorization
A model that allows for different autoregressive processes depending on the regime or state of the time series, enabling the capture of nonlinear behaviors.
TAR
Threshold Autoregressive
An activation layer that applies the thresholded rectified linear unit function element-wise.
ThresholdedReLU Layer
An activation layer that applies the thresholded rectified linear unit function element-wise.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ThresholdedReLU
A wrapper layer that applies a layer to every temporal slice of an input.
TimeDistributed Layer
A wrapper layer that applies a layer to every temporal slice of an input.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/TimeDistributed
A machine learning task focused on methods for analyzing time series data to extract meaningful statistics and characteristics.
Time Series Analysis
A machine learning task focused on methods for analyzing time series data to extract meaningful statistics and characteristics.
https://en.wikipedia.org/wiki/Time_series
A machine learning task focused on methods that predict future values based on previously observed values.
Time Series Forecasting
A machine learning task focused on methods that predict future values based on previously observed values.
https://en.wikipedia.org/wiki/Time_series
The process of converting a sequence of text into smaller meaningful units called tokens typically words or subwords for the purpose of analysis or processing by language models.
Lexical Analysis
Text Segmentation
Tokenization
The methodologies and approaches used to train machine learning models including techniques such as supervised learning unsupervised learning reinforcement learning and transfer learning aimed at optimizing model performance.
Instructional Methods
Learning Techniques
Training Strategies
A type of machine learning focused on methods that reuse or transfer information from previously learned tasks to facilitate the learning of new tasks.
Transfer Learning
A type of machine learning focused on methods that reuse or transfer information from previously learned tasks to facilitate the learning of new tasks.
https://en.wikipedia.org/wiki/Transfer_learning
A large language model that leverages knowledge acquired during training on one task to improve performance on different but related tasks facilitating more efficient learning and adaptation.
Transfer LLM
transfer learning
Transfer Learning LLM
A transformer language model with large training corpuses and sets of parameters that uses the transformer architecture based on multi-head attention mechanisms allowing it to contextualize tokens within a context window for effective language understanding and generation.
Transformer Large Language Model
Transformer LLM
A transformer language model with large training corpuses and sets of parameters that uses the transformer architecture based on multi-head attention mechanisms allowing it to contextualize tokens within a context window for effective language understanding and generation.
https://en.wikipedia.org/wiki/Transformer_(deep_learning_architecture)
A language model that uses the transformer architecture based on multi-head attention mechanisms allowing it to contextualize tokens within a context window for effective language understanding and generation.
Transformer LM
Transformer Language Model
A language model that uses the transformer architecture based on multi-head attention mechanisms allowing it to contextualize tokens within a context window for effective language understanding and generation.
https://arxiv.org/abs/1706.03762
A deep neural network that utilizes attention mechanisms to weigh the significance of input data.
Transformer Network
A deep neural network that utilizes attention mechanisms to weigh the significance of input data.
https://en.wikipedia.org/wiki/Transformer_(machine_Learning_model)
A selection and sampling bias favoring groups better represented in training data due to less prediction uncertainty.
Uncertainty Bias
A selection and sampling bias favoring groups better represented in training data due to less prediction uncertainty.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
A normalization layer that normalizes a batch of inputs so that each input in the batch has a L2 norm equal to 1.
UnitNormalization Layer
A normalization layer that normalizes a batch of inputs so that each input in the batch has a L2 norm equal to 1.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/UnitNormalization
A biclustering task focused on methods that simultaneously cluster the rows and columns of an unlabeled input matrix to identify submatrices with coherent patterns.
Block Clustering
Co-clustering
Joint Clustering
Two-mode Clustering
Two-way Clustering
Unsupervised Biclustering
A biclustering task focused on methods that simultaneously cluster the rows and columns of an unlabeled input matrix to identify submatrices with coherent patterns.
https://en.wikipedia.org/wiki/Biclustering
A clustering task focused on methods that group a set of unlabeled objects such that objects in the same group are more similar to each other than to those in other groups.
Cluster analysis
Unsupervised Clustering
A clustering task focused on methods that group a set of unlabeled objects such that objects in the same group are more similar to each other than to those in other groups.
https://en.wikipedia.org/wiki/Cluster_analysis
A large language model that is trained solely on unlabeled data using self-supervised objectives like masked language modeling without any supervised fine-tuning.
Unsupervised Large Language Model
self-supervised
Unsupervised LLM
A type of machine learning focused on algorithms that learn patterns from unlabeled data.
Unsupervised Learning
A type of machine learning focused on algorithms that learn patterns from unlabeled data.
https://en.wikipedia.org/wiki/Unsupervised_learning
A network that initializes a discriminative neural net from one trained using an unsupervised criterion.
UPN
Unsupervised Pretrained Network
A network that initializes a discriminative neural net from one trained using an unsupervised criterion.
https://metacademy.org/graphs/concepts/unsupervised_pre_training#:~:text=Unsupervised%20pre%2Dtraining%20initializes%20a,optimization%20and%20the%20overfitting%20issues
A layer that upsamples the input by repeating each temporal step size times along the time axis.
UpSampling1D Layer
A layer that upsamples the input by repeating each temporal step size times along the time axis.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/UpSampling1D
A layer that upsamples the input by repeating each row and column size times.
UpSampling2D Layer
A layer that upsamples the input by repeating each row and column size times.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/UpSampling2D
A layer that upsamples the input by repeating each depth
UpSampling3D Layer
A layer that upsamples the input by repeating each depth
https://www.tensorflow.org/api_docs/python/tf/keras/layers/UpSampling3D
A computational bias characterized by inappropriately analyzing ambiguous stimuli scenarios and events.
Interpretive Bias
Use And Interpretation Bias
A computational bias characterized by inappropriately analyzing ambiguous stimuli scenarios and events.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An individual bias arising when a user imposes their own biases during interaction with data output results etc.
User Interaction Bias
An individual bias arising when a user imposes their own biases during interaction with data output results etc.
GTP-4o with Seppala et al. 2017
https://doi.org/10.6028/NIST.SP.1270
An autoencoder network that imposes a probabilistic structure on the latent space for unsupervised learning.
VAE
Layers: Input, Probabilistic Hidden, Matched Output-Input
Variational Auto Encoder
A model that captures the linear interdependencies among multiple time series, where each variable is modeled as a linear function of its own past values and the past values of all other variables in the system.
VAR
Vector Autoregression
The technique of limiting the number of unique tokens in a language model's vocabulary by merging or eliminating less frequent tokens thereby optimizing computational efficiency and resource usage.
Lexical Simplification
Lexicon Pruning
Vocabulary Condensation
Vocabulary Reduction
A layer of values to be applied to other cells or neurons in a network.
Weighted Layer
An abstract base class for wrappers that augment the functionality of another layer.
Wrapper Layer
An abstract base class for wrappers that augment the functionality of another layer.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/Wrapper
A layer that zero-pads the input along the time axis.
ZeroPadding1D Layer
A layer that zero-pads the input along the time axis.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ZeroPadding1D
A layer that zero-pads the input along the height and width dimensions.
ZeroPadding2D Layer
A layer that zero-pads the input along the height and width dimensions.
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ZeroPadding2D
A layer that zero-pads the input along the depth
ZeroPadding3D Layer
A layer that zero-pads the input along the depth
https://www.tensorflow.org/api_docs/python/tf/keras/layers/ZeroPadding3D
A LLM which performs tasks or understands concepts it has not explicitly been trained on, demonstrating a high degree of generalization and understanding.
Zero-Shot LLM
zero-shot learning
Zero-Shot Learning LLM
A deep neural network that predicts classes at test time from classes not observed during training.
ZSL
Zero-shot Learning
A deep neural network that predicts classes at test time from classes not observed during training.
https://en.wikipedia.org/wiki/Zero-shot_learning
A machine learning designed to learn continuous feature representations for nodes in a graph by optimizing a neighborhood-preserving objective.
N2V
node2vec
Layers: Input, Hidden, Output
node2vec
A machine learning designed to learn continuous feature representations for nodes in a graph by optimizing a neighborhood-preserving objective.
https://en.wikipedia.org/wiki/Node2vec
A node2vec that predicts the current node from a window of surrounding context nodes, with the order of context nodes not influencing prediction.
N2V-CBOW
CBOW
Layers: Input, Hidden, Output
node2vec-CBOW
A node2vec that predicts the current node from a window of surrounding context nodes, with the order of context nodes not influencing prediction.
https://en.wikipedia.org/wiki/Word2vec
A node2vec that uses the current node to predict the surrounding window of context nodes, weighing nearby context nodes more heavily than distant ones.
N2V-SkipGram
SkipGram
Layers: Input, Hidden, Output
node2vec-SkipGram
A node2vec that uses the current node to predict the surrounding window of context nodes, weighing nearby context nodes more heavily than distant ones.
https://en.wikipedia.org/wiki/Word2vec
A dimensionality reduction for visualizing high-dimensional data by giving each datapoint a location in a two or three-dimensional map.
t-SNE
tSNE
t-Distributed Stochastic Neighbor embedding
A dimensionality reduction for visualizing high-dimensional data by giving each datapoint a location in a two or three-dimensional map.
https://en.wikipedia.org/wiki/T-distributed_stochastic_neighbor_embedding
A machine learning that generates distributed representations of words by training a shallow neural network model, which aims to predict the context of each word within a corpus. This algorithm captures semantic meanings of words through their contextual usage in the text.
W2V
word2vec
Layers: Input, Hidden, Output
word2vec
A machine learning that generates distributed representations of words by training a shallow neural network model, which aims to predict the context of each word within a corpus. This algorithm captures semantic meanings of words through their contextual usage in the text.
https://en.wikipedia.org/wiki/Word2vec
A word2vec that predicts the current word from a window of surrounding context words, ignoring the order of context words.
W2V-CBOW
CBOW
Layers: Input, Hidden, Output
word2vec-CBOW
A word2vec that predicts the current word from a window of surrounding context words, ignoring the order of context words.
https://en.wikipedia.org/wiki/Word2vec
A word2vec that predicts surrounding context words from the current word, giving more weight to nearby context words than distant ones.
W2V-SkipGram
SkipGram
Layers: Input, Hidden, Output
word2vec-SkipGram
A word2vec that predicts surrounding context words from the current word, giving more weight to nearby context words than distant ones.
https://en.wikipedia.org/wiki/Word2vec
Bias Subset
Class Subset
Function Subset
Layer Subset
Machine Learning Subset
Model Subset
Network Subset
Preprocessing Subset