Section outline

  • Teach To Learn


    The perceptron needs to be trained on what to remember to be able to learn. Since the ANN can be trained using a data set to produce the desired output, this type of learning is also known as assisted learning. It is important to note that since this method is for a binary type classification, the input data must be linearly separable. Eq. 5 can be defined to update the synaptic weights W values.


    $$ \color{orange} { \mathbf{{W}_{new}} = \mathbf{{W}_{old}}+\mathbf{{Y}_{d}}.\mathbf{X} \quad (5)} $$
    \( \color{orange} {\ Donde: \\ \; \\ \mathbf{{W}_{new}} \text{ : Vector of synaptic weights updated} \\ \mathbf{{W}_{old}} \text{ : Vector of prior synaptic weights.}\\ \mathbf{{Y}_{d}} \text{ : Desired output vector.}\\ \mathbf{X} \text{ : Input vector.}} \)

    The update rule presented in Eq. 5 is valid if the labels are {-1, 1}. For example, if we have the following scenario:


    $$ \color{orange} { \begin{cases}{W}_{0}= 0 +(-1*-1) \\ {W}_{1}=0 +(-1*\;\; 1\;)\end{cases} \quad (6)} $$

    Equation (5) is applied for the input and output requirements, and it will allow the coefficients w0 and w1 values to change the direction of the update, Eq. 6. But, in the case that the labels are {0, 1} and there is an incorrect classification of input X and its true value is 0, then the coefficients will never be updated, Eq. 7.


    $$ \color{orange} { \begin{cases}{W}_{0}= 0 +(0*1) \\ {W}_{1}=0 +(0*1)\end{cases} \quad (7)} $$

    Another updating rule, Eq. 8, can be proposed to solve this inconvenience and to generalize both for -1, 1 and 0, 1 and make it learn.


    $$ \color{orange} { \mathbf{{W}_{new}}=\mathbf{{W}_{old}}+(\mathbf{{Y}_{d}-{Y}_{e}}).\mathbf{X} \quad (8)} $$
    \( \color{orange} {\ Donde: \\ \; \\ \mathbf{{W}_{new}} \text{ : Vector of synaptic weights updated}\\ \mathbf{{W}_{old}} \text{ : Vector of prior synaptic weights.}\\ \mathbf{{Y}_{d}} \text{ : Desired output vector.}\\ \mathbf{{Y}_{e}} \text{ : Vector of the estimated output}\\ \mathbf{X} \text{ : Input vector.}} \)

    The change in Eq. 8 lies in adopting an error that is made up of the difference between the desired output and the corresponding estimated output by combining Eq. 3 and Eq. 2 or Eq. 3 and Eq. 4. In this context, it can be said if the labels are {0, 1}, when the perceptron classifies correctly, the error will be 0. Therefore, it will not be necessary to change the coefficients W. In the scenario where the labels are {-1, 1 }, the error will be 0 if the classification is correct, and in the case of incorrect classification it will be 2 or -2 and the direction of updating the coefficients can be changed.