GeLU activation |
gaussian error linear activation function |
HSigmoid activation |
hard sigmoid activation function |
HSwish activation |
hard swish activation function |
Leaky ReLU activation |
leaky rectified linear unit activation function |
Log Softmax activation |
logarithmic softmax activation function |
ReLU activation |
rectified linear unit activation function |
Sigmoid activation |
sigmoid activation function |
Softmax activation |
softmax activation function |
Softplus activation |
softplus activation function |
Swish activation |
swish activation function |
Tanh activation |
hyperbolic tangent activation function |
ArgMax |
returns indices where values is the maximum value of each row in the given dimension |
ArgMin |
returns indices where values is the minimum value of each row in the given dimension |
Average pooling |
2D averaging over an input tensor |
Batch expander |
broadcast input tensor with shape [1, D, H, W] to [BatchSize, D, H, W] |
Batchnorm 2D |
batch normalization over 4D input tensor ([batch, channel, 2D inputs]) |
Clamp |
clamp all elements in input into the range [ min, max ] and return a resulting tensor |
Concatenation |
layer combine sub-tensors to one |
Convolution 1D |
1D convolution over input tensor |
Convolution 2D |
2D convolution over input tensor |
Convolution deptwise |
2D convolution over input tensor, each channel processed separately |
CumSum |
cumulative sum of elements |
Data |
entry point for data to a model |
Dropout |
dropout layer |
Dynamic depthwise conv 2D |
channel-wise dynamic convolution 2D layer |
Elementwise compare |
element-wise comparison layer |
Elementwise div |
element-wise division layer |
Elementwise max |
element-wise maximum layer |
Elementwise min |
element-wise minimum layer |
Elementwise mul |
element-wise multiplication |
Elementwise sub |
element-wise subtraction |
Elementwise sum |
element-wise addition layer |
Embedding |
word embeddings using lookup table |
Exp |
element-wise exponential layer |
Fake quant |
floating-point quantization layer simulating quantization and dequantization |
Fixed bias |
layer that adds a scalar to tensor |
Global average pool |
global average pooling layer |
Index fill |
fills the elements of the tensor with specified value |
L2 norm |
divides all elements in input tensor by L2 norm calculated across chosen dimension |
L2 squared norm |
L2 squared normalizing layer |
Label smoothing |
label smoothing layer |
LayerNorm |
layer normalization 1D |
LayerNorm2D |
layer normalization 2D |
Linear |
affine transformation layer |
Log |
natural logarithm layer |
Masked fill |
fills input tensor elements corresponding to ones in mask with fill value |
Matmul |
scalar multiplication of last two dimensions |
Maxpool |
2D max-pooling over input |
Non-zero mask |
element-wise non-zero mask |
Padding |
adds paddings to input tensors |
Positional encoding |
encodes symbol position in sequence into embedding vector |
Random choice |
randomly outputs one of it's input tensors |
Random select |
returns a tensor of elements selected from either x or y, depending on dropout rate |
Random tensor |
creates tensor filled with values from normal distribution |
Reduce batch mean |
computes mean of elements across dimensions of a tensor |
Reduce max |
returns maximum values of each row of the input tensor in the given dimension |
Reduce mean |
computes mean of elements across dimensions of a tensor |
Reduce min |
returns minimum values of each row of the input tensor in the given dimension |
Reduce non-zero |
computes the number of non-zero elements along dimensions of a tensor |
Reduce std |
computes the standard deviation of elements across dimensions of a tensor |
Reduce sum |
computes the sum of elements across dimensions of a tensor |
Repeate interleave |
creates a new tensor repeating elements along chosen dimension |
Reshape |
reshaping of a tensor |
Reverse |
reverse the order of a tensor |
Roll |
layer that rolls tensor along the given dimension |
Round |
returns a tensor with each of the elements of input rounded to the closest integer |
RSqrt |
returns a new tensor with the reciprocal of the square-root of each of the elements of the input |
Scale |
layer of multiplication by a scalar |
Select |
returns a tensor of elements selected from either x or y, depending on condition |
Slicer |
extracting sub-tensors |
Splitter |
duplication of a tensor |
Sqrt |
returns a new tensor with the square-root of each of the elements of input |
Square |
returns a new tensor with the square of each of the elements of input |
Tensor |
inserts a constant tensor into a topology |
Tile |
creates a new tensor by replicating input multiples times |
Transpose |
swap dimensions according to parameters |
Transposed convolution 1D |
1D transposed convolution operator over an input image |
Transposed convolution 2D |
2D transposed convolution operator over an input image |