class GradientMachine

Gradient machine: machine which can be trained with a gradient descent.

Inheritance:

Public Fields

int n_inputs

Frame size of inputs sequences

int n_outputs

Frame size of outputs sequences

Parameters* params

Contains all parameters which will be updated with the gradient descent.

Parameters* der_params

Contains the derivatives for all parameters.

Sequence* beta

Contains the derivative with respect to the inputs

Public Methods

GradientMachine(int n_inputs_, int n_outputs_, int n_params_=0)

Initialize a gradient machine with n_inputs_ for the input frame size, n_outputs_ for the output frame size and n_params_ parameters.

virtual void iterInitialize()

This function is called before each training iteration.

virtual void forward(Sequence* inputs)

Given a sequence, update outputs.

virtual void backward(Sequence* inputs, Sequence* alpha)

Given a sequence, update the derivative with respect to the input (beta) and der_params.

virtual void setPartialBackprop(bool flag=true)

Set the partial backprop flag

virtual void frameForward(int t, real* f_inputs, real* f_outputs)

Given a frame f_inputs, updates f_outputs.

virtual void frameBackward(int t, real* f_inputs, real* beta_, real* f_outputs, real* alpha_)

Given the f_inputs and the derivatives alpha_ with respect to the outputs, updates the derivative with respect to the inputs (beta_) and der_params.

virtual void loadXFile(XFile* file)

By default, load the params field

virtual void saveXFile(XFile* file)

By default, save the params field

Inherited from Machine:

Public Fields

Sequence* outputs

Public Methods

virtual void reset()

virtual void setDataSet(DataSet* dataset_)

Inherited from Object:

Public Fields

Allocator* allocator

Public Methods

void addOption(const char* name, int size, void* ptr, const char* help="")

void addIOption(const char* name, int* ptr, int init_value, const char* help="")

void addROption(const char* name, real* ptr, real init_value, const char* help="")

void addBOption(const char* name, bool* ptr, bool init_value, const char* help="")

void addOOption(const char* name, Object** ptr, Object* init_value, const char* help="")

void setOption(const char* name, void* ptr)

void setIOption(const char* name, int option)

void setROption(const char* name, real option)

void setBOption(const char* name, bool option)

void setOOption(const char* name, Object* option)

void load(const char* filename)

void save(const char* filename)

void* operator new(size_t size, Allocator* allocator_=NULL)

void* operator new(size_t size, Allocator* allocator_, void* ptr_)

void operator delete(void* ptr)

Documentation

Gradient machine: machine which can be trained with a gradient descent.

Gradient machines take in inputs sequences which have always the same frame size, given by n_inputs, and outputs sequences which have always the same frame size too, given by n_outputs.

int n_inputs

Frame size of inputs sequences

int n_outputs

Frame size of outputs sequences

Parameters* params

Contains all parameters which will be updated with the gradient descent. Almost all machines will have only one node in params.

Parameters* der_params

Contains the derivatives for all parameters. Warning: params and der_params must have the same structure.

Sequence* beta

Contains the derivative with respect to the inputs

GradientMachine(int n_inputs_, int n_outputs_, int n_params_=0)

Initialize a gradient machine with n_inputs_ for the input frame size, n_outputs_ for the output frame size and n_params_ parameters. If n_inputs_ is 0, no beta sequence will be allocated. If n_outputs_ is 0, no outputs sequence will be allocated.

virtual void iterInitialize()

This function is called before each training iteration. By default, do nothing.

virtual void forward(Sequence* inputs)

Given a sequence, update outputs. By default, it uses frameForward(), to update each output frame given each input frame. It supposes by default the number of input and output frames is the same.

virtual void backward(Sequence* inputs, Sequence* alpha)

Given a sequence, update the derivative with respect to the input (beta) and der_params. If partial_backprop is false, don't update beta. By default, it uses frameBackward(), to update each beta frame given each input and alpha frame. It supposes by default the number of input and output frames is the same.

virtual void setPartialBackprop(bool flag=true)

Set the partial backprop flag

virtual void frameForward(int t, real* f_inputs, real* f_outputs)

Given a frame f_inputs, updates f_outputs. Used to easily create new classes. It is called by the default forward(), and it does nothing by default. If your machine needs to do special things on sequence (if input sequence do not have the same size as the output sequence), don't overload this function, but overload forward(). t is the current frame to be forwarded.

virtual void frameBackward(int t, real* f_inputs, real* beta_, real* f_outputs, real* alpha_)

Given the f_inputs and the derivatives alpha_ with respect to the outputs, updates the derivative with respect to the inputs (beta_) and der_params. It is called by the default backward(), and it does nothing by default. If your machine needs to do special things on sequence (if input sequence do not have the same size as the output sequence), don't overload this function, but overload backward(). t is the current frame to be back-propagated.

virtual void loadXFile(XFile* file)

By default, load the params field

virtual void saveXFile(XFile* file)

By default, save the params field

Direct child classes:

TemporalSubSampling
TemporalMean
TemporalConvolution
Tanh
SumMachine
SpatialSubSampling
SpatialConvolution
SoftPlus
SoftMax
Sigmoid
Mixer
LogSoftMax
LogSigmoid
LogRBF
LogMixer
Linear
InputsSelect
Exp
Distribution
Criterion
ConnectedMachine

Author:

Ronan Collobert (collober@idiap.ch)

See Also:

StochasticGradient

Alphabetic index HTML hierarchy of classes or Java