ai.djl.nn.recurrent

Class RecurrentBlock

java.lang.Object

ai.djl.nn.AbstractBlock

ai.djl.nn.ParameterBlock

ai.djl.nn.recurrent.RecurrentBlock

All Implemented Interfaces:

Block

Direct Known Subclasses:

GRU, LSTM, RNN

public abstract class RecurrentBlock
extends ParameterBlock

RecurrentBlock is an abstract implementation of recurrent neural networks.

Recurrent neural networks are neural networks with hidden states. They are very popular for natural language processing tasks, and other tasks which involve sequential data.

This [article](http://karpathy.github.io/2015/05/21/rnn-effectiveness/) written by Andrej Karpathy provides a detailed explanation of recurrent neural networks.

Currently, vanilla RNN, LSTM and GRU are implemented, with both multi-layer and bidirectional support.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	RecurrentBlock.BaseBuilder<T extends RecurrentBlock.BaseBuilder> The Builder to construct a RecurrentBlock type of Block.

Field Summary

Fields

Modifier and Type	Field and Description
protected NDArray	beginState
protected float	dropRate
protected int	gates
protected java.lang.String	mode
protected int	numDirections
protected int	numStackedLayers
protected java.util.List<Parameter>	parameters
protected boolean	stateOutputs
protected long	stateSize
protected boolean	useSequenceLength

Fields inherited from class ai.djl.nn.AbstractBlock

inputNames, inputShapes

Constructor Summary

Constructors

Constructor and Description
RecurrentBlock(RecurrentBlock.BaseBuilder<?> builder) Creates a RecurrentBlock object.

Method Summary

Modifier and Type	Method and Description
void	beforeInitialize(Shape[] inputs) Performs any action necessary before initialization.
NDList	forward(ParameterStore parameterStore, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params) Applies the operating function of the block once.
java.util.List<Parameter>	getDirectParameters() Returns a list of all the direct parameters of the block.
Shape[]	getOutputShapes(NDManager manager, Shape[] inputs) Returns the expected output shapes of the block for the specified input shapes.
Shape	getParameterShape(java.lang.String name, Shape[] inputShapes) Returns the shape of the specified direct parameter of this block given the shapes of the input to the block.
protected boolean	isBidirectional()
void	loadParameters(NDManager manager, java.io.DataInputStream is) Loads the parameters from the given input stream.
protected NDList	opInputs(ParameterStore parameterStore, NDList inputs)
protected void	resetBeginStates()
void	saveParameters(java.io.DataOutputStream os) Writes the parameters of the block to the given outputStream.
void	setBeginStates(NDList beginStates) Sets the initial NDArray value for the hidden states.
void	setStateOutputs(boolean stateOutputs) Sets the parameter that indicates whether the output must include the hidden states.
protected NDList	updateInputLayoutToTNC(NDList inputs)
protected void	validateInputSize(NDList inputs)

Methods inherited from class ai.djl.nn.ParameterBlock

getChildren, initialize, toString

Methods inherited from class ai.djl.nn.AbstractBlock

cast, clear, describeInput, getParameters, isInitialized, readInputShapes, saveInputShapes, setInitializer, setInitializer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ai.djl.nn.Block

forward, validateLayout

Field Detail

stateSize

protected long stateSize

dropRate

protected float dropRate

numStackedLayers

protected int numStackedLayers

mode

protected java.lang.String mode

useSequenceLength

protected boolean useSequenceLength

numDirections

protected int numDirections

gates

protected int gates

stateOutputs

protected boolean stateOutputs

beginState

protected NDArray beginState

parameters

protected java.util.List<Parameter> parameters

Constructor Detail

RecurrentBlock

public RecurrentBlock(RecurrentBlock.BaseBuilder<?> builder)

Creates a RecurrentBlock object.

Parameters:

builder - the Builder that has the necessary configurations

Method Detail

validateInputSize

protected void validateInputSize(NDList inputs)

setStateOutputs

public void setStateOutputs(boolean stateOutputs)

Sets the parameter that indicates whether the output must include the hidden states.

Parameters:

stateOutputs - whether the output must include the hidden states.

forward

public NDList forward(ParameterStore parameterStore,
                      NDList inputs,
                      boolean training,
                      ai.djl.util.PairList<java.lang.String,java.lang.Object> params)

Applies the operating function of the block once. This method should be called only on blocks that are initialized.

Parameters:

parameterStore - the parameter store

inputs - the input NDList

training - true for a training forward pass

params - optional parameters

Returns:

the output of the forward pass

setBeginStates

public void setBeginStates(NDList beginStates)

Sets the initial NDArray value for the hidden states.

Parameters:

beginStates - the NDArray value for the hidden states

resetBeginStates

protected void resetBeginStates()

getOutputShapes

public Shape[] getOutputShapes(NDManager manager,
                               Shape[] inputs)

Returns the expected output shapes of the block for the specified input shapes.

Parameters:

manager - an NDManager

inputs - the shapes of the inputs

Returns:

the expected output shapes of the block

getDirectParameters

public java.util.List<Parameter> getDirectParameters()

Returns a list of all the direct parameters of the block.

Returns:

the list of Parameter

beforeInitialize

public void beforeInitialize(Shape[] inputs)

Performs any action necessary before initialization.

Overrides:

beforeInitialize in class AbstractBlock

Parameters:

inputs - the expected shapes of the input

getParameterShape

public Shape getParameterShape(java.lang.String name,
                               Shape[] inputShapes)

Returns the shape of the specified direct parameter of this block given the shapes of the input to the block.

Parameters:

name - the name of the parameter

inputShapes - the shapes of the input to the block

Returns:

the shape of the parameter specified

saveParameters

public void saveParameters(java.io.DataOutputStream os)
                    throws java.io.IOException

Writes the parameters of the block to the given outputStream.

Parameters:

os - the outputstream to save the parameters to

Throws:

java.io.IOException - if an I/O error occurs

loadParameters

public void loadParameters(NDManager manager,
                           java.io.DataInputStream is)
                    throws java.io.IOException,
                           MalformedModelException

Loads the parameters from the given input stream.

Parameters:

manager - an NDManager to create the parameter arrays

is - the inputstream that stream the parameter values

Throws:

java.io.IOException - if an I/O error occurs

MalformedModelException - if the model file is corrupted or unsupported

isBidirectional

protected boolean isBidirectional()

opInputs

protected NDList opInputs(ParameterStore parameterStore,
                          NDList inputs)

updateInputLayoutToTNC

protected NDList updateInputLayoutToTNC(NDList inputs)