1. Mimicking Biology - Intro
This is the first topic of a series on mimicking biology.
Like many people, I have been following the Artificial Intelligence craze for about 2 years, reading a few papers/books, and watching a few videos.
One thing that keeps coming to mind is how although the neural network attempts to imitate the brain’s neural interactions (to some extent), large language models are still highly inefficient specifically in the amount of data and computing required to be perceived as intelligent.
I am far from an expert in these topics, however in the spirit of aimlessly experimenting and wasting my time I have some interesting hypotheses on how to build better intelligence by shamelessly mimicking biology. So I started reading up on resources to better understand current AI and the human brain.
I would make a shit ton of incorrect assumptions and I am very open to more expert opinions. This is more or less an intellectual gymnastic for me.
Just for fun. No specific goal.
The over-arching hypothesis is that there just might be an emergent property of intelligence that arises from the different parts of the nervous system working together doing their own independent job.
Emergent properties are qualities or behaviors that arise when individual components of a system interact in ways that are not predictable merely by examining those components in isolation.
In my particular experiment, it would be that an emergent property of intelligence and consciousness might exist if we can simulate the individual parts of the brain (e.g. Thalamos, Cerebral cortex, etc…) into an application system.
One thing that would stand out is that for the human part, different sub-systems (I could call them “neural networks”) are responsible for different segments just for comprehension to happen unlike the LLM approach which is a single neural network e.g the Auditory cortex processes sound for volume and pattern recognition for language. In contrast, the prefrontal cortex helps in decision-making by evaluating the question, considering relevant memories, and planning a response.
But as well known by experts, the human brain which is the most complex organ in the body and arguably the most complex system on earth relies on a lot less energy and data to start making inferences. A plate of rice in the morning and I am performing inferences as I go about my day (an oversimplification because I understand the body also has other energy sources e.g sunlight, oxygen etc… but you get the gist)
All of my yapping to say that over the next few months, I will be attempting to build an AI by simply mimicking the human brain structure.
I don’t know what to expect and will be doing a lot of research on the brain and AI as I experiment.
It would also be iterative and I assume that for my first n number of iterations (where n is a large number 😅), it would all be garbage science but fun.
Side Note:
My friend Taiwo made the following point against shamelessly mimicking biology:
So this is the Mixture of Experts (MoE) approach to building LLMs at the moment as well, where different parameters (experts) are responsible for different reasoning in the LLM, the french Mistral guys are the most popular MoE LLMs.
On the Biological ones, it’s sometimes way too hard to mimic nature, such that we end up going around and arriving at something that becomes superior (Maybe not as efficient, but efficient enough), with flying for example, a lot of early prototypes were trying to flap their wings in a motion similar to birds before the wright brothers came along with the fixed wing aircraft system that worked.
Let’s see…..
Some additional boring context:
The diagrams below provide more context for the interestingness of the experiment.
One thing that would stand out is that for the human part, different sub-systems (I could call them “neural networks”) are responsible for different segments just for comprehension to happen unlike the LLM approach which is a single neural network e.g the Auditory cortex processes sound for volume and pattern recognition for language. In contrast, the prefrontal cortex helps in decision-making by evaluating the question, considering relevant memories, and planning a response.
For a typical foundation LLM model here is its flow diagram (inferencing)
For a biological system
Resources
Books I am reading:
- NEUROSCIENCE: SCIENCE OF THE BRAIN AN INTRODUCTION FOR YOUNG STUDENTS by British Neuroscience Association.
Courses I am taking: