250 Yonge Street, Suite 2200, Toronto, ON, M5G 1B1

©2019 by objectsoftconsultants.

Neural Networks

Neuron- Humans inspiration

The subject of artificial neural networks has matured to a great extent over the last few years. especially with the advent of very high-performance computing, the subject has assumed tremendous significance and has gained a very high application potential in the very recent years.

Neural Network:

The word derives its origin from the human brain or the human nervous system which consists of a massively large, parallel interconnection of a huge number of neurons. This network achieves different perceptual, recognition, etc. tasks in an amazingly small interval of time. Even when compared to today’s supercomputers. This inspired researchers to look for ways in which a computer can be made to mimic the brain.


  • Highly complex, non-linear, parallel computer.

  • Brain structural constituents’ neurons

  • Billions of nerve cells with trillions of interconnections exists within the human brain.

  • Individual neurons are complicated, they have a myriad of parts, subsystems and control mechanisms.

  • They convey information via a host of electrochemical pathways

  • Together these neurons and their interconnections from a process which is not binary, not stable and not synchronous.

Neural networks:

  • Neural networks with their remarkable ability to derive meaning from complicated and imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques.

  • A trained neural network can be thought of as an ‘expert’ in a particular category of information it has been given to analyse.

Usefulness, Capabilities and advantages:

  • Adaptive learning: An ANN is endowed with the ability to learn how to do tasks based on the data given for training or initial experience.

  • Self-organization: An ANN can create its own organization or representation of the information it receives during the learning phase.

  • Real time operation: ANN computations may be carried out parallelly in real-time.

  • Fault tolerance via redundant information code: Neural networks degrade gracefully partial destruction of a NN leads to the degradation of performance, but some of the network capabilities may be retained and recovered even after that.

  • NNs exploit non-linearity: Therefore it becomes very easy to realize complex mappings and functions with the help of NNs. Non-linearity is distributed.

  • NNs can realize functions that can map a set of inputs to the desired set of outputs. If the mapping is not accurate, we can accordingly modify the free-parameters (weights) of the system and ensure that convergence is achieved.

  • NNs provide evidential response i.e response associated with a measure of confidence.

  • NNs are VLSI implementable.

Biological neural network :-

Source : Wikipedia

Dendrites is tree like networks made of nerve fibre. Axon is single large connection extending from the cell body and carrying signals from the neuron.

The end of the axon splits into fine strands each strand terminates in to a small bulb-synapse. It is through the synapse that the neuron introduces its signals to other nearby neurons. There are approximately 10 to the power of 4 synapses per neuron. Electric impulses are passed between the synapses and dendrites.

Signal transmission involves a chemical process in which specific transmitter substances are released. This results in increase or decrease in the electric potential inside the body of the cell. If the potential reaches a threshold then the receiving cell fires and a pulse or action potential of fixed strength and duration is sent out through the axon to the synaptic junction of the other cells. The synapses are said to be inhibitory if they let passing impulses hinder the firing of the receiving cell or excitatory if they let the passing impulses cause the firing of the receiving cell. Cell body is where all the inputs are combined. The inputs are combined in accordance with the strength of the connection they are entering the cell from. If the strength of the connection is strong, then the strength of the signal passing through it will also be strong. Strengths of all connections are different. All the strengths have an impact on the net input and output. Strengths can be negative.

Terminology relationship between biological and artificial neurons:

  • Cell -> neuron, node, unit

  • Dendrites -> Weights, interconnections

  • Cell body -> Net input

  • Axon -> Output

Source : Wikipedia

Artificial neural networks perform various tasks such as pattern matching and classification, optimization, approximation, vector quantization and data clustering.

Comparison of biological neurons and Artificial network give better understanding to the Machine learning and Neural network practitioners.