Main Content

**Tip**

This topic explains how to define custom deep learning layers for your problems. For a list of built-in layers in Deep Learning Toolbox™, see List of Deep Learning Layers.

To learn how to define custom output layers, see Define Custom Deep Learning Output Layers.

If Deep Learning Toolbox does not provide the layer that you require for your task, then you can define your own custom layer using this topic as a guide. After defining the custom layer, you can automatically check that the layer is valid, GPU compatible, and outputs correctly defined gradients.

During training, the software iteratively performs forward and backward passes through the network.

When making a forward pass through the network, each layer takes the outputs of the previous layers, applies a function, and then outputs (forward propagates) the results to the next layers. Stateful layers, such as LSTM layers, also update the layer state.

Layers can have multiple inputs or outputs. For example, a layer can take
*X _{1}*, …,

At the end of a forward pass of the network, the output layer calculates the loss
*L* between the predictions *Y* and the targets
*T*.

During the backward pass of a network, each layer takes the derivatives of the loss
with respect to the outputs of the layer, computes the derivatives of the loss
*L* with respect to the inputs, and then backward propagates the
results. If the layer has learnable parameters, then the layer also computes the
derivatives of the layer weights (learnable parameters). The layer uses the derivatives
of the weights to update the learnable parameters.

The following figure describes the flow of data through a deep neural network and
highlights the data flow through a layer with a single input *X*, a
single output *Z*, and a learnable parameter
*W*.

To define a custom intermediate layer, use this class definition template. This template outlines the structure of an intermediate layer class definition. It outlines:

The optional

`properties`

blocks for the layer properties, learnable parameters, and state parameters. For more information, see Intermediate Layer Properties.The layer constructor function.

The

`predict`

function and the optional`forward`

function. For more information, see Forward Functions.The optional

`resetState`

function for layers with state properties. For more information, see Reset State Function.The optional

`backward`

function. For more information, see Backward Function.

classdef myLayer < nnet.layer.Layer % & nnet.layer.Formattable (Optional) properties % (Optional) Layer properties. % Declare layer properties here. end properties (Learnable) % (Optional) Layer learnable parameters. % Declare learnable parameters here. end properties (State) % (Optional) Layer state parameters. % Declare state parameters here. end properties (Learnable, State) % (Optional) Nested dlnetwork objects with both learnable % parameters and state. % Declare nested networks with learnable and state parameters here. end methods function layer = myLayer() % (Optional) Create a myLayer. % This function must have the same name as the class. % Define layer constructor function here. end function [Z,state] = predict(layer,X) % Forward input data through the layer at prediction time and % output the result and updated state. % % Inputs: % layer - Layer to forward propagate through % X - Input data % Outputs: % Z - Output of layer forward function % state - (Optional) Updated layer state. % % - For layers with multiple inputs, replace X with X1,...,XN, % where N is the number of inputs. % - For layers with multiple outputs, replace Z with % Z1,...,ZM, where M is the number of outputs. % - For layers with multiple state parameters, replace state % with state1,...,stateK, where K is the number of state % parameters. % Define layer predict function here. end function [Z,state,memory] = forward(layer,X) % (Optional) Forward input data through the layer at training % time and output the result, updated state, and a memory % value. % % Inputs: % layer - Layer to forward propagate through % X - Layer input data % Outputs: % Z - Output of layer forward function % state - (Optional) Updated layer state % memory - (Optional) Memory value for custom backward % function % % - For layers with multiple inputs, replace X with X1,...,XN, % where N is the number of inputs. % - For layers with multiple outputs, replace Z with % Z1,...,ZM, where M is the number of outputs. % - For layers with multiple state parameters, replace state % with state1,...,stateK, where K is the number of state % parameters. % Define layer forward function here. end function layer = resetState(layer) % (Optional) Reset layer state. % Define reset state function here. end function [dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory) % (Optional) Backward propagate the derivative of the loss % function through the layer. % % Inputs: % layer - Layer to backward propagate through % X - Layer input data % Z - Layer output data % dLdZ - Derivative of loss with respect to layer % output % dLdSout - (Optional) Derivative of loss with respect % to state output % memory - Memory value from forward function % Outputs: % dLdX - Derivative of loss with respect to layer input % dLdW - (Optional) Derivative of loss with respect to % learnable parameter % dLdSin - (Optional) Derivative of loss with respect to % state input % % - For layers with state parameters, the backward syntax must % include both dLdSout and dLdSin, or neither. % - For layers with multiple inputs, replace X and dLdX with % X1,...,XN and dLdX1,...,dLdXN, respectively, where N is % the number of inputs. % - For layers with multiple outputs, replace Z and dlZ with % Z1,...,ZM and dLdZ,...,dLdZM, respectively, where M is the % number of outputs. % - For layers with multiple learnable parameters, replace % dLdW with dLdW1,...,dLdWP, where P is the number of % learnable parameters. % - For layers with multiple state parameters, replace dLdSin % and dLdSout with dLdSin1,...,dLdSinK and % dLdSout1,...dldSoutK, respectively, where K is the number % of state parameters. % Define layer backward function here. end end end

`dlarray`

objects makes working with high
dimensional data easier by allowing you to label the dimensions. For example, you can label
which dimensions correspond to spatial, time, channel, and batch dimensions using the
`"S"`

, `"T"`

, `"C"`

, and
`"B"`

labels, respectively. For unspecified and other dimensions, use the
`"U"`

label. For `dlarray`

object functions that operate
over particular dimensions, you can specify the dimension labels by formatting the
`dlarray`

object directly, or by using the `DataFormat`

option.Using formatted `dlarray`

objects in custom layers also allows you to define
layers where the inputs and outputs have different formats, such as layers that permute,
add, or remove dimensions. For example, you can define a layer that takes as input a
mini-batch of images with format `"SSCB"`

(spatial, spatial, channel,
batch) and output a mini-batch of sequences with format `"CBT"`

(channel,
batch, time). Using formatted `dlarray`

objects also allows you to define
layers that can operate on data with different input formats, for example, layers that
support inputs with formats `"SSCB"`

(spatial, spatial, channel, batch) and
`"CBT"`

(channel, batch, time).

If you do not specify a backward function, then the layer functions, by default, receive
*unformatted*
`dlarray`

objects as input. To specify that the layer receives
*formatted*
`dlarray`

objects as input and also outputs formatted
`dlarray`

objects, also inherit from the
`nnet.layer.Formattable`

class when defining the custom layer.

For an example showing how to define a custom layer with formatted inputs, see Define Custom Deep Learning Layer with Formatted Inputs.

Declare the layer properties in the `properties`

section of the class
definition.

By default, custom intermediate layers have these properties. Do not declare these properties
in the `properties`

section.

Property | Description |
---|---|

`Name` | Layer name, specified as a character vector or a string scalar.
For `Layer` array input, the `trainNetwork` ,
`assembleNetwork` , `layerGraph` , and
`dlnetwork` functions automatically assign names to layers with
`Name` set to `''` . |

`Description` | One-line description of the layer, specified as a string scalar or a character vector. This
description appears when the layer is displayed in a If you do not specify a layer description, then the software displays the layer class name. |

`Type` | Type of the layer, specified as a character vector or a string scalar. The value of If you do not specify a layer type, then the software displays the layer class name. |

`NumInputs` | Number of inputs of the layer, specified as a positive integer. If
you do not specify this value, then the software automatically sets
`NumInputs` to the number of names in
`InputNames` . The default value is 1. |

`InputNames` | Input names of the layer, specified as a cell array of character
vectors. If you do not specify this value and
`NumInputs` is greater than 1, then the software
automatically sets `InputNames` to
`{'in1',...,'inN'}` , where `N` is
equal to `NumInputs` . The default value is
`{'in'}` . |

`NumOutputs` | Number of outputs of the layer, specified as a positive integer. If
you do not specify this value, then the software automatically sets
`NumOutputs` to the number of names in
`OutputNames` . The default value is 1. |

`OutputNames` | Output names of the layer, specified as a cell array of character
vectors. If you do not specify this value and
`NumOutputs` is greater than 1, then the software
automatically sets `OutputNames` to
`{'out1',...,'outM'}` , where `M`
is equal to `NumOutputs` . The default value is
`{'out'}` . |

If the layer has no other properties, then you can omit the `properties`

section.

**Tip**

If you are creating a layer with multiple inputs, then you must
set either the `NumInputs`

or `InputNames`

properties in the
layer constructor. If you are creating a layer with multiple outputs, then you must set either
the `NumOutputs`

or `OutputNames`

properties in the layer
constructor. For an example, see Define Custom Deep Learning Layer with Multiple Inputs.

Declare the layer learnable parameters in the ```
properties
(Learnable)
```

section of the class definition. You can specify numeric
arrays or `dlnetwork`

objects as learnable parameters. If the
`dlnetwork`

object has both learnable and state parameters (for
example, a `dlnetwork`

object that contains an LSTM layer), then you
must specify it in the `properties (Learnable, State)`

section. If
the layer has no learnable parameters, then you can omit the
`properties`

sections with the `Learnable`

attribute.

Optionally, you can specify the learning rate factor and the L2 factor of the
learnable parameters. By default, each learnable parameter has its learning rate
factor and L2 factor set to `1`

. For both built-in and custom
layers, you can set and get the learn rate factors and L2 regularization factors
using the following functions.

Function | Description |
---|---|

`setLearnRateFactor` | Set the learn rate factor of a learnable parameter. |

`setL2Factor` | Set the L2 regularization factor of a learnable parameter. |

`getLearnRateFactor` | Get the learn rate factor of a learnable parameter. |

`getL2Factor` | Get the L2 regularization factor of a learnable parameter. |

To specify the learning rate factor and the L2 factor of a learnable parameter,
use the syntaxes ```
layer =
setLearnRateFactor(layer,parameterName,value)
```

and ```
layer =
setL2Factor(layer,parameterName,value)
```

, respectively.

To get the value of the learning rate factor and the L2 factor of a learnable
parameter, use the syntaxes
`getLearnRateFactor(layer,parameterName)`

and
`getL2Factor(layer,parameterName)`

respectively.

For example, this syntax sets the learn rate factor of the learnable parameter
with the name `"Alpha"`

to `0.1`

.

`layer = setLearnRateFactor(layer,"Alpha",0.1);`

For stateful layers, such as recurrent layers, declare the layer state parameters
in the `properties (State)`

section of the class definition. For
`dlnetwork`

objects that have both learnable and state parameters
(for example, a `dlnetwork`

object that contains an LSTM layer), then
you must specify it in the `properties (Learnable, State)`

section.
If the layer has no state parameters, then you can omit the
`properties`

sections with the `State`

attribute.

If the layer has state parameters, then the forward functions must also return the updated layer state. For more information, see Forward Functions.

To specify a custom reset state function, include a function with syntax
`layer = resetState(layer)`

in the class definition. For more
information, see Reset State Function.

Some layers behave differently during training and during prediction. For example, a
dropout layer performs dropout only during training and has no effect during prediction. A
layer uses one of two functions to perform a forward pass: `predict`

or
`forward`

. If the forward pass is at prediction time, then the layer
uses the `predict`

function. If the forward pass is at training time, then
the layer uses the `forward`

function. If you do not require two different
functions for prediction time and training time, then you can omit the
`forward`

function. In this case, the layer uses
`predict`

at training time.

If the layer has state parameters, then the forward functions must also return the updated layer state parameters as numeric arrays.

If you define both a custom `forward`

function and a custom
`backward`

function, then the forward function must return a
`memory`

output.

The `predict`

function syntax depends on the type of layer.

`Z = predict(layer,X)`

forwards the input data`X`

through the layer and outputs the result`Z`

, where`layer`

has a single input, a single output.`[Z,state] = predict(layer,X)`

also outputs the updated state parameter`state`

, where`layer`

has a single state parameter.

You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters:

For layers with multiple inputs, replace

`X`

with`X1,...,XN`

, where`N`

is the number of inputs. The`NumInputs`

property must match`N`

.For layers with multiple outputs, replace

`Z`

with`Z1,...,ZM`

, where`M`

is the number of outputs. The`NumOutputs`

property must match`M`

.For layers with multiple state parameters, replace

`state`

with`state1,...,stateK`

, where`K`

is the number of state parameters.

**Tip**

If the number of inputs to the layer can vary, then use `varargin`

instead of `X1,…,XN`

. In this case, `varargin`

is a
cell array of the inputs, where `varargin{i}`

corresponds to
`Xi`

.

If the number of outputs can vary, then use `varargout`

instead of
`Z1,…,ZN`

. In this case, `varargout`

is a cell
array of the outputs, where `varargout{j}`

corresponds to
`Zj`

.

**Tip**

If the custom layer has a `dlnetwork`

object for a learnable parameter, then in
the `predict`

function of the custom layer, use the
`predict`

function for the `dlnetwork`

. Using the
`dlnetwork`

object `predict`

function ensures that the
software uses the correct layer operations for prediction.

The `forward`

function syntax depends on the type of layer:

`Z = forward(layer,X)`

forwards the input data`X`

through the layer and outputs the result`Z`

, where`layer`

has a single input, a single output.`[Z,state] = forward(layer,X)`

also outputs the updated state parameter`state`

, where`layer`

has a single state parameter.`[__,memory] = forward(layer,X)`

also returns a memory value for a custom`backward`

function using any of the previous syntaxes. If the layer has both a custom`forward`

function and a custom`backward`

function, then the forward function must return a memory value.

You can adjust the syntaxes for layers with multiple inputs, multiple outputs, or multiple state parameters:

For layers with multiple inputs, replace

`X`

with`X1,...,XN`

, where`N`

is the number of inputs. The`NumInputs`

property must match`N`

.For layers with multiple outputs, replace

`Z`

with`Z1,...,ZM`

, where`M`

is the number of outputs. The`NumOutputs`

property must match`M`

.For layers with multiple state parameters, replace

`state`

with`state1,...,stateK`

, where`K`

is the number of state parameters.

**Tip**

If the number of inputs to the layer can vary, then use `varargin`

instead of `X1,…,XN`

. In this case, `varargin`

is a
cell array of the inputs, where `varargin{i}`

corresponds to
`Xi`

.

If the number of outputs can vary, then use `varargout`

instead of
`Z1,…,ZN`

. In this case, `varargout`

is a cell
array of the outputs, where `varargout{j}`

corresponds to
`Zj`

.

**Tip**

If the custom layer has a `dlnetwork`

object for a learnable parameter, then in
the `forward`

function of the custom layer, use the
`forward`

function of the `dlnetwork`

object. Using the
`dlnetwork`

object `forward`

function ensures that the
software uses the correct layer operations for training.

The dimensions of the inputs depend on the type of data and the output of the connected layers:

Layer Input | Input Size | Observation Dimension |
---|---|---|

Feature vectors | c-by-N, where c
corresponds to the number of channels and N is the number of
observations. | 2 |

2-D images | h-by-w-by-c-by-N,
where h, w, and c
correspond to the height, width, and number of channels of the images,
respectively, and N is the number of observations. | 4 |

3-D images | h-by-w-by-d-by-c-by-N,
where h, w, d, and
c correspond to the height, width, depth, and number of
channels of the 3-D images, respectively, and N is the number
of observations. | 5 |

Vector sequences | c-by-N-by-S, where c is the number of features of the sequences, N is the number of observations, and S is the sequence length. | 2 |

2-D image sequences | h-by-w-by-c-by-N-by-S,
where h, w, and c
correspond to the height, width, and number of channels of the images,
respectively, N is the number of observations, and
S is the sequence length. | 4 |

3-D image sequences | h-by-w-by-d-by-c-by-N-by-S,
where h, w, d, and
c correspond to the height, width, depth, and number of
channels of the 3-D images, respectively, N is the number of
observations, and S is the sequence length. | 5 |

For layers that output sequences, the layers can output sequences of any length or output data
with no time dimension. Note that when training a network that outputs sequences using the
`trainNetwork`

function, the lengths of the input and output
sequences must match.

The outputs of the custom layer forward functions must not be
complex. If the `predict`

or `forward`

functions of your
custom layer involve complex numbers, convert all outputs to real values before returning them.
Using complex numbers in the `predict`

or `forward`

functions of your custom layer can lead to complex learnable parameters. If you are using
automatic differentiation (in other words, you are not writing a backward function for your
custom layer) then convert all learnable parameters to real values at the beginning of the
function computation. Doing so ensures that the outputs of automatically generated backward
functions are not complex.

When `DAGNetwork`

or `SeriesNetwork`

objects contain layers with state parameters, you can make predictions and update the layer states using the `predictAndUpdateState`

and `classifyAndUpdateState`

functions. You can reset the network state using the `resetState`

function.

The `resetState`

function for `DAGNetwork`

,
`SeriesNetwork`

, and `dlnetwork`

objects, by default, has
no effect on custom layers with state parameters. To define the layer behavior for the
`resetState`

function for network objects, define the optional layer
`resetState`

function in the layer definition that resets the state
parameters.

The `resetState`

function must have the syntax ```
layer =
resetState(layer)
```

, where the returned layer has the state properties
reset.

The layer backward function computes the derivatives of the loss with respect to the
input data and then outputs (backward propagates) results to the previous layer. If the
layer has learnable parameters (for example, layer weights), then
`backward`

also computes the derivatives of the learnable
parameters. When using the `trainNetwork`

function, the layer
automatically updates the learnable parameters using these derivatives during the
backward pass.

Defining the backward function is optional. If you do not specify a backward function,
and the layer forward functions support `dlarray`

objects, then the
software automatically determines the backward function using automatic differentiation.
For a list of functions that support `dlarray`

objects, see List of Functions with dlarray Support. Define a custom
backward function when you want to:

Use a specific algorithm to compute the derivatives.

Use operations in the forward functions that do not support

`dlarray`

objects.

Custom layers with learnable `dlnetwork`

objects do not support custom backward functions.

To define a custom backward function, create a function named
`backward`

.

The `backward`

function syntax depends on the type of layer.

`dLdX = backward(layer,X,Z,dLdZ,memory)`

returns the derivatives`dLdX`

of the loss with respect to the layer input, where`layer`

has a single input and a single output.`Z`

corresponds to the forward function output and`dLdZ`

corresponds to the derivative of the loss with respect to`Z`

. The function input`memory`

corresponds to the memory output of the forward function.`[dLdX,dLdW] = backward(layer,X,Z,dLdZ,memory)`

also returns the derivative`dLdW`

of the loss with respect to the learnable parameter, where`layer`

has a single learnable parameter.`[dLdX,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)`

also returns the derivative`dLdSin`

of the loss with respect to the state input using any of the previous syntaxes, where`layer`

has a single state parameter and`dLdSout`

corresponds to the derivative of the loss with respect to the layer state output.`[dLdX,dLdW,dLdSin] = backward(layer,X,Z,dLdZ,dLdSout,memory)`

also returns the derivative`dLdW`

of the loss with respect to the learnable parameter and returns the derivative`dLdSin`

of the loss with respect to the layer state input using any of the previous syntaxes, where`layer`

has a single state parameter and single learnable parameter.

You can adjust the syntaxes for layers with multiple inputs, multiple outputs, multiple learnable parameters, or multiple state parameters:

For layers with multiple inputs, replace

`X`

and`dLdX`

with`X1,...,XN`

and`dLdX1,...,dLdXN`

, respectively, where`N`

is the number of inputs.For layers with multiple outputs, replace

`Z`

and`dLdZ`

with`Z1,...,ZM`

and`dLdZ1,...,dLdZM`

, respectively, where`M`

is the number of outputs.For layers with multiple learnable parameters, replace

`dLdW`

with`dLdW1,...,dLdWP`

, where`P`

is the number of learnable parameters.For layers with multiple state parameters, replace

`dLdSin`

and`dLdSout`

with`dLdSin1,...,dLdSinK`

and`dLdSout1,...,dLdSoutK`

, respectively, where`K`

is the number of state parameters.

To reduce memory usage by preventing unused variables being saved between the forward and
backward pass, replace the corresponding input arguments with `~`

.

**Tip**

If the number of inputs to `backward`

can vary, then use
`varargin`

instead of the input arguments after
`layer`

. In this case, `varargin`

is a cell array
of the inputs, where the first `N`

elements correspond to the
`N`

layer inputs, the next `M`

elements correspond
to the `M`

layer outputs, the next `M`

elements
correspond to the derivatives of the loss with respect to the `M`

layer
outputs, the next `K`

elements correspond to the `K`

derivatives of the loss with respect to the `K`

states outputs, and the
last element corresponds to `memory`

.

If the number of outputs can vary, then use `varargout`

instead of the
output arguments. In this case, `varargout`

is a cell array of the
outputs, where the first `N`

elements correspond to the
`N`

the derivatives of the loss with respect to the
`N`

layer inputs, the next `P`

elements correspond
to the derivatives of the loss with respect to the `P`

learnable
parameters, and the next `K`

elements correspond to the derivatives of
the loss with respect to the `K`

state inputs.

The values of `X`

and `Z`

are the same as in the
forward functions. The dimensions of `dLdZ`

are the same as the
dimensions of `Z`

.

The dimensions and data type of `dLdX`

are the same as the dimensions
and data type of `X`

. The dimensions and data types of
`dLdW`

are the same as the dimensions and data types of
`W`

.

To calculate the derivatives of the loss, you can use the chain rule:

$$\frac{\partial L}{\partial {X}^{(i)}}={\displaystyle \sum _{j}\frac{\partial L}{\partial {Z}_{j}}\frac{\partial {Z}_{j}}{\partial {X}^{(i)}}}$$

$$\frac{\partial L}{\partial {W}_{i}}={\displaystyle \sum _{j}\frac{\partial L}{\partial {Z}_{j}}\frac{\partial {Z}_{j}}{\partial {W}_{i}}}$$

When using the `trainNetwork`

function, the layer automatically
updates the learnable parameters using the derivatives `dLdW`

during
the backward pass.

For an example showing how to define a custom backward function, see Specify Custom Layer Backward Function.

The outputs of the custom layer backward function must not be complex. If your backward function involves complex numbers, then convert all outputs of the backward function to real values before returning them.

If the layer forward functions fully support `dlarray`

objects, then the layer
is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs
and return outputs of type `gpuArray`

(Parallel Computing Toolbox).

Many MATLAB^{®} built-in functions support `gpuArray`

(Parallel Computing Toolbox) and `dlarray`

input arguments. For a list of
functions that support `dlarray`

objects, see List of Functions with dlarray Support. For a list of functions
that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
To use a GPU for deep
learning, you must also have a supported GPU device. For information on supported devices, see
GPU Support by Release (Parallel Computing Toolbox). For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).

To create a custom layer that supports code generation:

The layer must specify the pragma

`%#codegen`

in the layer definition.The inputs of

`predict`

must be:Consistent in dimension. Each input must have the same number of dimensions.

Consistent in batch size. Each input must have the same batch size.

The outputs of

`predict`

must be consistent in dimension and batch size with the layer inputs.Nonscalar properties must have type single, double, or character array.

Scalar properties must have type numeric, logical, or string.

Code generation supports intermediate layers with 2-D image or feature input only. Code generation does not support layers with state properties (properties with attribute `State`

).

For an example showing how to create a custom layer that supports code generation, see Define Custom Deep Learning Layer for Code Generation.

To create a custom layer that itself defines a layer graph, you can declare a
`dlnetwork`

object as a learnable parameter in the ```
properties
(Learnable)
```

section of the layer definition. This method is known as
*network composition*. You can use network composition to:

Create a single custom layer that represents a block of learnable layers, for example, a residual block.

Create a network with control flow, for example, a network with a section that can dynamically change depending on the input data.

Create a network with loops, for example, a network with sections that feed the output back into itself.

For nested networks that have both learnable and state parameters, for example, networks
with batch normalization or LSTM layers, declare the network in the ```
properties
(Learnable, State)
```

section of the layer definition.

If the layer forward functions fully support `dlarray`

objects, then the layer
is GPU compatible. Otherwise, to be GPU compatible, the layer functions must support inputs
and return outputs of type `gpuArray`

(Parallel Computing Toolbox).

Many MATLAB built-in functions support `gpuArray`

(Parallel Computing Toolbox) and `dlarray`

input arguments. For a list of
functions that support `dlarray`

objects, see List of Functions with dlarray Support. For a list of functions
that execute on a GPU, see Run MATLAB Functions on a GPU (Parallel Computing Toolbox).
To use a GPU for deep
learning, you must also have a supported GPU device. For information on supported devices, see
GPU Support by Release (Parallel Computing Toolbox). For more information on working with GPUs in MATLAB, see GPU Computing in MATLAB (Parallel Computing Toolbox).

If you create a custom deep learning layer, then you can use
the `checkLayer`

function
to check that the layer is valid. The function checks layers for validity, GPU compatibility,
correctly defined gradients, and code generation compatibility. To check that a layer is valid,
run the following command:

checkLayer(layer,validInputSize)

`layer`

is an instance of the layer, `validInputSize`

is a
vector or cell array specifying the valid input sizes to the layer. To check with multiple
observations, use the `ObservationDimension`

option. To check for code generation compatibility, set the
`CheckCodegenCompatibility`

option to `1`

(true). For large input sizes, the gradient checks take longer to run. To speed up the tests, specify a smaller valid input size.For more information, see Check Custom Layer Validity.

`checkLayer`

Check the layer validity of the custom layer `preluLayer`

.

The custom layer `preluLayer`

, attached to this is example as a supporting file, applies the PReLU operation to the input data. To access this layer, open this example as a live script.

Create an instance of the layer and check its validity using `checkLayer`

. Specify the valid input size to be the size of a single observation of typical input to the layer. The layer expects 4-D array inputs, where the first three dimensions correspond to the height, width, and number of channels of the previous layer output, and the fourth dimension corresponds to the observations.

Specify the typical size of the input of an observation and set the `ObservationDimension`

option to 4.

layer = preluLayer(20); validInputSize = [24 24 20]; checkLayer(layer,validInputSize,ObservationDimension=4)

Skipping GPU tests. No compatible GPU device found. Skipping code generation compatibility tests. To check validity of the layer for code generation, specify the 'CheckCodegenCompatibility' and 'ObservationDimension' options. Running nnet.checklayer.TestLayerWithoutBackward .......... ........ Done nnet.checklayer.TestLayerWithoutBackward __________ Test Summary: 18 Passed, 0 Failed, 0 Incomplete, 10 Skipped. Time elapsed: 1.9312 seconds.

Here, the function does not detect any issues with the layer.

`functionLayer`

| `checkLayer`

| `setLearnRateFactor`

| `setL2Factor`

| `getLearnRateFactor`

| `getL2Factor`

| `findPlaceholderLayers`

| `replaceLayer`

| `assembleNetwork`

| `PlaceholderLayer`

- Define Custom Deep Learning Layers
- Define Custom Deep Learning Output Layers
- Define Custom Deep Learning Layer with Learnable Parameters
- Define Custom Deep Learning Layer with Multiple Inputs
- Define Custom Deep Learning Layer with Formatted Inputs
- Define Custom Recurrent Deep Learning Layer
- Specify Custom Layer Backward Function
- Define Custom Deep Learning Layer for Code Generation
- Deep Learning Network Composition
- Define Nested Deep Learning Layer
- Check Custom Layer Validity