Section outline

  • Perceptron evolution


    McCulloch-Pitts model

    The model proposed by McCulloch-Pitts [3] has a simplified structure and behavior of a physiological neuron, Fig. 1.a, commonly called the bioinspired model. The Biological Neural Network (BRN) has dendrites, which are responsible for receiving stimuli from other neurons, and then transmitting this information to the cell body, regulating its intensity. The cell body is responsible for adding the above characteristics to generate an electrical signal, which, according to the threshold of the action potential given by the different concentrations of Na+, K+ and Cl− ions, generates a trigger towards other neurons. In the case of exceeding the firing potential threshold, the axon transmits the electrical signal generated in the cell body to other neurons through the synapses. The aforementioned mode of operation, by way of example, is artificially replicated by a perceptron, where in figure 1.b the same parts that form part of the biological model can be seen. The Artificial Neural Network (ANN) model consists of inputs (dendrites) that propagate through coefficients (synapses). The linear combination, Eq. 1 (cell body) between inputs and synaptic weights converges in a non-linear operation called activation function (axon).


    $$ \color{orange} {S=\sum_{i=1}^{n}{w_i\ast x_i} \quad (1)} $$

    A perceptron takes multiple binary inputs, such as x1, x2, and so on, and produces a single binary output. The W coefficients are real numbers that express the contribution of each input to the output. If the sum of the coefficients multiplied by the inputs is greater or less than a certain threshold, the neuron's output will be 1 or 0.



    (a) Biological neuron
    (b) McCulloch-Pitts model

    Figure 1: Comparison between a biological and artificial neuron


    Activation function proposed by McCulloch-Pitts

    Figure 1.b shows that between the exit of the cell body (S) and the exit (Y) there is an activation function (φ(S)). The McCulloch-Pitts mathematical model outputs binary type, labeled {0, 1}. To get the mentioned labels in your output, it is necessary that the activation function has only those possible values. In this sense, the Heaviside function [4], known in this way due to its creator, the mathematician Oliver Heaviside, generates a 0 or a 1 in its output according to the value of S, Eq. 2.

    $$ \color{orange} { \varphi\left(S\right) =\begin{cases}1 & S \geq 0\\0 & S <0\end{cases} \quad (2)} $$

    Rosenblatt model

    Frank Rosenblatt [1] inspired by the work proposed by his colleagues McCulloch-Pitts [3] developed the concept of the perceptron. The perceptron introduces some changes to the model explained in section 2.1. One of the changes he proposes is to introduce an additional input X0 that will have a fixed value of 1 with the coefficient W0. Adding these new parameters is the same as adding a bias to the linear combination of Eq. 1. For this reason, Eq.1 becomes Eq. 3, and as can be seen, the lower limit of the sum starts from i=0.

    $$ \color{orange} { S=\sum_{i=0}^{n}{w_i\ast x_i} \quad (3) }$$

    The other change made by [1] consists of changing the activation function called the sign function. In this case, the function generates a 0, -1 or 1 in its output according to the value of S, Eq. 4.

    $$ \color{orange} {\varphi\left(S\right) =\begin{cases}1 & S>0 \\0 & S=0 \\-1 & S <0\end{cases} \quad (4)} $$