1   Introduction

In our previous white papers, we have introduced how Algebra allows to convert the real world in computable data. We then explained how we use Differential Calculus to construct mathematical functions to predict the future based on the past. We now need to start linking these mathematical marvels to more complex problems. These complex problems will be solved using the famous Neural Networks. In this paper, I want to introduce the “origins” of neural Networks. How did the idea come to computer scientists? How close are they to the biological ones?

This paper is not going to teach biology. For a start, I am not a biologist, but an IT guy and would have no credential to do so. I will have to simplify the world just enough to make the principles understandable to everyone. The biologists who might read this will forgive me.

2  What’s in a brain?

The part of the brain we are interested in is where information is processed. This is done by the neurons. Their job is basically to take some information in, process some transformation on this information and transmit the result to another bunch of neurons. The transmission is done by the mean of the synapses.

Now this is all well on a simple picture, but the reality of a brain is at a scale hardly imaginable. Think about that: an adult has typically 86 billion of neurons, each neuron has between 1 000 and 10 000 synapses. So, it is said that we have between 100 and 1000 trillions of synapses. That’s plenty enough to make it capable of complex information processing.

We will construct a far simpler digital neural network to illustrate the general principle. In practice, digital neural networks have far less neurons than the brain but they can have a huge number nonetheless. We will talk about scale in a later paper. For this one, our aim is only to introduce how the digital mimics the biological.

2.1     What information does a neuron process?

So, we now have established that these things called neurons are taking information to treat, but what information?

The most basic kind of information is coming from our senses: what we see, we taste, we feel, we smell, and we hear. I suspect that there are many more stuff that can go in, but we should ask an expert. So, what happens when we want to eat something, for instance? Well, we probably look at it first and these lovely neurons will assess if it is edible or not, if yes, we touch it and smell it, then we get confirmation that indeed we can safely eat it, then we taste it and before swallowing we’ll get a last check of the safety of the food. We do that all the time without thinking. And we all know that if you give some chicken to your cat and he does not want to eat it, we’d better be cautious, right?

So how does that work? Say for the smell, we smell the food. The smell is in reality molecules of that food getting into our nose. These molecules get into the nasal cavity and touch olfactory receptors in a specialised tissue. In this tissue are …millions of olfactory neurons which are dedicated to working with smell. When a molecule binds to a receptor, it triggers a biomedical reaction that will transform that into an electrical signal. These olfactory neurons will process the information and send the result to more neurons in various areas of the brain for more processing. I spare you the details because I don’t want to make this paper tedious and then lose you. But you get the idea: from the smell molecule to the brain deciding if it is edible, we mostly have neurons to thank. …a lot of them. A serious big load of them.

2.2     This process in the digital world

This neuron processing is well and nice but how on earth are we going to reproduce the same thing with 0 and 1? Because after all, computers are only aware of 0 and 1s. I will not go down this rabbit hole as I still don’t want to lose you, so let’s go directly to the concept.

Let’s draw a digital neural network as we all know them.

So, let’s imagine this smelling process with this simple neural network.

The circles are neurons and the lines are synapses.

A “molecule of smell” arrives into the system. Our system possesses 2 types of very specialised olfactory neurons capable of processing: the chemical structure of the molecule and its concentration. This 2 information will enter the 2 neurons. These 2 initial olfactory neurons process the information and send the result to 4 different neurons that can be more or less specialised, it does not matter for our story. These 4 neurons process the information again and send their result to a last neuron. Based on its input, the last neuron will send a simple result: “Edible : Yes/No”.

And here we are, we have our first digital neural network. Champagne !

But of course, what are these digital neurons and synapses doing?, will you ask. How do they work with 0s and 1s? Excellent question! We’ll discover in the next papers that these digital neurons use numbers—not just 0s and 1s—to process smells, with some mathematical magic we’ll explore soon. You can guess from the previous papers that mathematics will have a serious role to play in the story.

3  Where is the Intelligence?

I usually look for “intelligence” in the covered topic. I must admit that in this case, the intelligence is probably for the scientific community to join different fields to achieve something new. Understanding how a brain is working and trying to mimic its behaviour in order to re-create intelligence is not small feat.

In the next papers, I will study in more details how these digital Neural Networks work, how we train them and how they can produce such incredible results.