Perceptron

What is perceptron in machine learning?

Perceptron is an algorithm that is used for supervised learning of binary classifiers. Binary classifiers are essentially functions that have the ability to determine whether an input, represented by a vector, belongs to a particular class. It’s a kind of linear classifier (a classification algorithm that uses a linear prediction function, combining a set of weights with the feature vector to make its predictions.

The perceptron algorithm was invented by Frank Rosenblatt at the Cornell Aeronautical Laboratory in 1958. It is an algorithm that is used to learn a binary classifier known as a threshold function. This is a function that maps its input x to an output value f(x):

f(x) = {0  otherwise1  if w . x + b > 0,

Here w is a vector of real-valued weights, w . x is the dot product i=1mwixi, where m denotes the number of inputs to the perceptron and b is the bias that shifts the decision boundary away from the origin and does not have any dependence on input values.

The idea of perceptron is rooted in two words: perception (the ability to sense something) and neuron ((nerve cells in the brain that transform sensory inputs into meaningful information).

A perceptron is made up of four parts:

• Input values or One input layer: The numerical input values correspond to features.
• Weights and Bias: Every feature has a weight assigned to it that determines its importance. This bias allows you to shift the activation function curve upwards and downwards.
• Net sum: This is the output that is calculated by using inputs and weights.
• Activation Function: This is used for the purpose of mapping the input between the required values like (0,1) or (-1,1).

The perceptron is a building block of artificial neural networks and is essentially a simplified model of biological models that exist in the brain. There has been research that demonstrated that a linear model like a perceptron can be capable of producing some behavior that is seen in real neurons.

What are the types of perceptrons?

There are two types of perceptrons:

• Single layer perceptrons - These can only learn linearly separable patterns.
• Multilayer perceptrons - These have the greatest processing power. They are a class of feedforward neural networks. They have a hidden layer and use sophisticated algorithms like backpropagation.
  

A single layer perceptron does not contain any hidden layers. Input nodes are fully connected to a node or multiple nodes in the succeeding layer. Nodes in the next layer take a weighted sum of their inputs.

A multilayer perceptron generates a set of outputs from a set of inputs. It is a neural network that connects multiple layers in a directed graph. This means that the signal path through the nodes only goes one way.

A multilayer perceptron consists of input, output, and hidden layers. Each hidden layer is made up of numerous perceptron’s which are known as hidden layers or hidden unit.

‍

‍

Why is perceptron used?

A perceptron is used for the purpose of classifying data into two sections. That is why it is known as a linear binary classifier. The perceptron algorithm allows neurons to learn and processes elements in the training set one at a time. 

How does perceptron work?

The perceptron works on these steps:

• It multiplies all the inputs x with their weights w.
• All the multiplied values are then added. This sum is known as the weighted sum.
• After that, the weighted sum is applied to the appropriate activation function. 

‍

Can perceptron learn XOR?

The classes in XOR are not linearly separable. It will not be possible for you to draw a straight line to separate the points (0,0),(1,1) from the points (0,1),(1,0). Single layer perceptrons can only learn linearly separable patterns. 

Therefore, a single layer perceptron does not have the ability to implement XOR. This gave rise to the need for and the invention of multilayer networks and perceptrons.

The XOR problem can be solved and overcome by making use of multilayer perceptrons. This essentially involves using multiple perceptrons arranged in feed-forward networks.

What is the difference between perceptron and neuron?

The perceptron is essentially a mathematical model of biological neuron. It is a unit that has weighted inputs and produces a binary output based on a threshold. It was pretty much the precursor to the backpropagation artificial neural network model. 

A node or a neuron in a backpropagation artificial neural network (ANN) is a generalization of the idea of the perceptron.

A neuron (node) in an artificial neural network also add weighted inputs, but rather than producing a binary output on the basis of a threshold, it produces a graded value between 0 and 1 based on the proximity of the input is to the desired category (the “1” value). Network nodes are have a tendency of being biased toward favoring the extreme values of 0 or 1 using a sigmoidal output function. You can interpret these graded values to mean the probability that the input is in the category, or the degree to which the category describes the input.

Perceptrons use a very brittle activation function. So if w . x is greater than a certain value, the prediction is positive. If it is less than that specific value, the prediction is negative.

‍

What are the limitations of perceptron?

Perceptrons have many limitations. They are:

• Because of the hard-limit transfer function, the output values of a perceptron are only able to take on only one of two values (0 or 1).
• Perceptrons are only capable of classifying linearly separable sets of vectors (vectors can be considered to be linearly separable if a straight line can be drawn to separate the input vectors into the appropriate categories.
• If the vectors aren’t linearly separable, learning will fail to reach a point where each and every vector is classified correctly. On the converse, if the vectors are linearly separable, perceptrons trained adaptively will never fail to find a solution in finite time.