Inheritance diagram for classias::train::truncated_gradient_base< error_tmpl >:
The detail of the algorithm is described in: John Langford, Lihong Li, and Tong Zhang. Sparse Online Learning via Truncated Gradient. JMLR 10(Mar):777-801, 2009.
This class implements internal variables, operations, and interface that are common for training a binary/multi classification.
| error_tmpl | The type of the error (loss) function. |
Public Types | |
| typedef error_tmpl | error_type |
| The type implementing an error function. | |
| typedef error_type::model_type | model_type |
| The type implementing a model (weight vector for features). | |
| typedef model_type::value_type | value_type |
| The type representing a value. | |
|
typedef truncated_gradient_base< error_tmpl > | this_class |
| A synonym of this class. | |
Public Member Functions | |
| truncated_gradient_base () | |
| Constructs the object. | |
| virtual | ~truncated_gradient_base () |
| Destructs the object. | |
| void | clear () |
| Resets the internal states and parameters to default. | |
| void | set_num_features (size_t size) |
| Sets the number of features. | |
| void | start () |
| Starts a training process. | |
| void | finish () |
| Terminates a training process. | |
| void | discontinue () |
| void | copyright (std::ostream &os) |
| Shows the copyright information. | |
| void | report (std::ostream &os) |
| Reports the current state of the training process. | |
| parameter_exchange & | params () |
| Obtains the parameter interface. | |
| model_type & | model () |
| Obtains an access to the weight vector (model). | |
| const model_type & | model () const |
| Obtains a read-only access to the weight vector (model). | |
| value_type | loss () const |
Data Fields | |
| report_type | m_report |
Protected Member Functions | |
| value_type | learning_rate (int t) |
| Computes the learning rate for the update count. | |
| void | accumulate_penalty (int t, value_type eta) |
| Accumulates the L1 penalty for the current update. | |
| void | finalize_penalty (int t, value_type eta) |
| Finalizes the accmulation of L1 penalties. | |
| void | initialize_weights () |
| Initializes the weight vector. | |
| void | apply_penalty () |
| Applies the L1 penalties to the weight vector. | |
| void | apply_penalty (int i) |
| Applies the L1 penalty to the weight of a feature. | |
Protected Attributes | |
| model_type | m_w |
| The array of feature weights. | |
| model_type | m_penalty |
| The array of L1 penalties previously applied to weights. | |
| value_type | m_lambda |
| The lambda (coefficient for L1 regularization). | |
| value_type | m_eta |
| The current learning rate. | |
| value_type | m_t0 |
| The offset of the update count. | |
| int | m_t |
| The update count. | |
| value_type | m_loss |
| The loss. | |
| value_type | m_sum_penalty |
| The total amount of L1 penalty. | |
| parameter_exchange | m_params |
| Parameter interface. | |
| value_type | m_c |
| The coefficient for L2 regularization. | |
| value_type | m_n |
| The number of instances in the data set. | |
| value_type | m_eta0 |
| The initial learning rate. | |
| int | m_truncate_period |
| The period for truncations. | |
| bool | m_truncated |
| The boolean value indicating whether m_w is truncated. | |
Data Structures | |
| struct | report_type |
| The type of progress information. More... | |
| void classias::train::truncated_gradient_base< error_tmpl >::set_num_features | ( | size_t | size | ) | [inline] |
Sets the number of features.
This function resizes the weight vector.
| size | The number of features. |
| void classias::train::truncated_gradient_base< error_tmpl >::start | ( | ) | [inline] |
Starts a training process.
This function resets the internal states, and prepares for a training process.
| void classias::train::truncated_gradient_base< error_tmpl >::finish | ( | ) | [inline] |
Terminates a training process.
This function performs a post-processing after a training process.
| void classias::train::truncated_gradient_base< error_tmpl >::copyright | ( | std::ostream & | os | ) | [inline] |
Shows the copyright information.
| os | The output stream. |
| void classias::train::truncated_gradient_base< error_tmpl >::report | ( | std::ostream & | os | ) | [inline] |
Reports the current state of the training process.
| os | The output stream. |
| value_type classias::train::truncated_gradient_base< error_tmpl >::learning_rate | ( | int | t | ) | [inline, protected] |
Computes the learning rate for the update count.
| t | The update count. |
| void classias::train::truncated_gradient_base< error_tmpl >::accumulate_penalty | ( | int | t, | |
| value_type | eta | |||
| ) | [inline, protected] |
Accumulates the L1 penalty for the current update.
| t | The update count. | |
| eta | The learning rate for the update count. |
| void classias::train::truncated_gradient_base< error_tmpl >::finalize_penalty | ( | int | t, | |
| value_type | eta | |||
| ) | [inline, protected] |
Finalizes the accmulation of L1 penalties.
| t | The update count. | |
| eta | The learning rate for the update count. |
| void classias::train::truncated_gradient_base< error_tmpl >::initialize_weights | ( | ) | [inline, protected] |
Initializes the weight vector.
This function sets W = 0.
| void classias::train::truncated_gradient_base< error_tmpl >::apply_penalty | ( | int | i | ) | [inline, protected] |
Applies the L1 penalty to the weight of a feature.
| i | The feature index. |
| parameter_exchange& classias::train::truncated_gradient_base< error_tmpl >::params | ( | ) | [inline] |
Obtains the parameter interface.
| model_type& classias::train::truncated_gradient_base< error_tmpl >::model | ( | ) | [inline] |
Obtains an access to the weight vector (model).
| const model_type& classias::train::truncated_gradient_base< error_tmpl >::model | ( | ) | const [inline] |
Obtains a read-only access to the weight vector (model).