- I am currently working on a Badger architecture that has no outer-loop, only one infinite inner-loop
- New building blocks are supposed to emerge from the interaction of existing building blocks, where the initial building blocks are predefined
- My first step was to experiment with sexual selection (the topic of this article)
- The second step will be to experiment with memetic evolution
 |
| Predators (red dots) replicate by eating prey. Prey (colored dots) replicate using sexual selection. Strong prey speciation not observed. |
Intro
A few months ago I decided to start doing my own experiments on our GoodAI Badger architecture.
My initial motivation was that I really miss programming, and being able to just build stuff with my own hands. A secondary motivation is that I have an intuition about what I think our end-goal would look like, and the best way to explain it is by building prototypes.
How does my approach differ from Badger architecture? In Badger, the goal is to learn the shared expert policy through outer-loop. In my approach (which I call “Manual Badger”), there’s no outer-loop. Instead, agents/experts come pre-designed with a policy that can later evolve into something complex and interesting. This approach has similarities to evolution, except this will be quite a specific version of evolution (in some ways more limited and narrower) but at the same time is supposed to enable open-endedness and the emergence of intelligence.
*Note: the boundaries between what is outer-loop and inner-loop, or what is expert’s policy and activations, and on what time-scale the majority of adaptation happens is maybe blurry and not so relevant at this moment.
What is sexual selection? It is a mode of natural selection in which members of one biological sex choose partners of the other sex to mate with, and compete with members of the same sex for access to members of the opposite sex. What makes this interesting from the perspective of AGI and Badger is that preferences arising within the population can generate fitness and reward effects that overtake external pressures, resulting in a system that in some sense chooses its own goals.
More info: https://en.wikipedia.org/wiki/Sexual_selection
What surprised me
- Speciation emerged much faster than I anticipated. The Simulation starts with a random distribution of genes and within 1000 time-steps we can observe clear boundaries between species.
- The domination of ‘most selected for’ genomes (colors) – because of the way I implemented the selection process (basically two genomes competing in the sexual selection process, some genomes get an advantage and because the process is stochastic, there’s not much that the disadvantaged genome can do)
- Geographic bottlenecks reinforce speciation - this means that if I disable the world map and the environment becomes a uniform grid with no bottlenecks between continents and isolated islands, the simulation will never evolve into a state where we can see clear boundaries between species - everything stays kind of uniform, or at least with very diffused species boundaries. If we enable the world map, clear speciation emerges.
How is this relevant to AGI and Badger architecture?
This is the 'Manual Badger' take on our Badger architecture.
The difference is that we don't learn the expert policy in the outer loop. Instead, we hard code it and let the system evolve into something interesting (diversity of behaviors, evolvability of adaptation, etc).
Next steps
Memetic evolution - agents being able to spread their memes. Memes are programs/behaviors.
My expectation is that it should lead to a higher diversity of sexual selection and behavior, and to the emergence of new behavioral niches for other memes.
Hopefully, we will observe a few more evolutionary transitions - interactions that are not predictable from the initial state.
Experiments
Basic experiment
- What you see is a grid world.
- Each cell represents an agent (two sexes), an obstacle (ocean/sea), or an empty location.
- Agents move randomly.
- Each agent has a genome, a 3-dimensional vector, visualized as an RGB color.
- The genome has no impact on an agent's fitness or behavior (except its sexual preferences)
- The genome impacts the sexual selection process: agent ranks the opposite sex agents in its proximity (2 to 3 cell radius) by how close their genomes are, and only above minimum threshold similarity is considered. The agent then chooses the best match and a new offspring is born to an unoccupied location near them (if there’s an empty lot).
- The offspring’s genome is a crossover of both parents - a mathematical average of those two genomes (RGB colors).
- There's no explicit survival fitness, no food.
- Agents mate every 10 timesteps, and they die after 100 timesteps (those black cells that frequently appear show the location where the agent died and their cell is unoccupied)
Turn off preferences entirely and see that color boundaries don’t persist.
The boundaries become very soft and dispersed.
Make agents prefer mates with stronger versions of their own traits.
Mating scan area increased to 5 cells radius (instead of default 2 cells).
Can we see spontaneous speciation from a completely grey initial state when mutation is disabled?
The answer is no. The initial population of grey agents will grow and saturate every available cell, but the color will never change.
Can we see spontaneous speciation from a completely grey initial state when mutation is enabled?
The answer is yes. The initial population of grey agents with mutation leads to speciation. Sometimes the borders between species are quite distinct, sometimes more dispersed. I think a big contribution here is the ‘mutation frequency’ and also the ‘selection threshold’. Maybe if I set the latter to a smaller number, the borders would become more distinct.
What helped to make the species grow larger and the borders more distinct was when I added the ‘mutation frequency’. I guess the population became slightly more stable.
Can we see spontaneous speciation from a completely grey initial state - with mutation enabled & mutation frequency is part of the genome (initial mutation frequency is close to zero, but can increase due to mutations)?
This is an idea I got while watching the previous experiment: Can we see spontaneous speciation from a completely grey initial state - with mutation?
The answer is yes. I think what happens is that higher mutation frequency emerges at some places and that area populates, but then slowly evolves to lower, or almost zero mutation frequency, in order to stabilize its population.
Can we see spontaneous speciation from a completely grey initial state - with mutation enabled & mutation frequency is part of the genome (initial mutation frequency is high, but can decrease due to mutations)?
Remove the world map, so we get a uniform grid. Can we now observe different evolution, because the bottlenecks caused by having geographically separated continents and only some of them connected through a narrow bridge, was removed?
The result was surprising and cool :) The system evolved towards something resembling the “heat death” of the universe.
The above three pictures show the progression - the first one shows the start, the middle one shows speciations with blurry borders, and the last one shows a state where all genes drifted towards 0, 0, 0, and the system is now very unlikely to evolve into anything else, although, there are locations where genes mutate and try to survive for some time, the drive towards 0, 0, 0 is still stronger.
Another one surprise is that I was unable to replicate this “heath death” result. I kept it running for much longer, but never got to the heat death state. It just keeps oscillating between different dominant genes.
What happens when replication speed is limited so that no matter how good at finding a mate you are, you can have at most ~2.5 offsprings per mating pair in your life? E.g. track offspring and every time a pair has one kid, they each have a 20% chance of going sterile/dying. Tune background death rate so that you don't have the entire space filled up with solid colors. My guess is that with runaway sexual selection you might get some kind of coexistence here, but with no selectivity, it'll still go to brown/gray.
Starts with an initial phase where we observe speciation with blurry borders. Then it evolves to a regime where either R color dominates, or G color dominates. It oscillates between them as long as I let it run.
An additional question is why B color doesn’t dominate (used also as a mutation frequency).
Conclusion: the result doesn’t differ from other experiments where the max number of offsprings is not limited.
Sterility set to 100% after reproduction. Didn’t reach criticality in reproduction, the population soon died out
Initialize with R color only
Conclusion: no difference, mutation will create speciations, and then dominant color overtakes, and then it fluctuates
Random color init
1024 grid size
Comment: same as for 256 grid size, speciation to diffusion
Aged colored = gene is multiplied by a normalized age.
This results in visualization when young and old are clearly distinguishable.
Offspring genome is not a result of averaging genes of both parents, but instead, we take some genes from each of them - kind of crossover point
Add predators (they replicate by eating prey)
Predators are visualized by red color (age coded)
After I added the predators, prey stopped to speciate (no strong boundaries between species, they all are kind of grey-ish)
Enabled world map
Now we can see that the speciation where predators are non-existent, however on islands and other similarly isolated areas without predators, speciation emerges.
Conclusion
It’s fun to be doing my own experiments again! :-)
Implementing my own agent-based simulation of evolutionary dynamics of sexual selection gave me better intuition for my next step - memetic evolution. Stay tuned!
If you want to let me know your feedback, please get in touch via my personal email address marek.rosa@keenswh.com, or use our Keen Software House support site. I welcome all of the feedback we receive and we will use it to learn and provide better services to our players.
Thank you for reading this blog!
Best,
Marek Rosa
CEO, Creative Director, Founder at Keen Software House
CEO, CTO, Founder at GoodAI
For more news:
Space Engineers: www.SpaceEngineersGame.com
Keen Software House: www.keenswh.com
Medieval Engineers: www.MedievalEngineers.com
GoodAI: www.GoodAI.com
General AI Challenge: www.General-AI-Challenge.org
Personal bio:
Marek
Rosa is the CEO and CTO of GoodAI, a general artificial intelligence
R&D company, and the CEO and founder of Keen Software House, an
independent game development studio best known for its best-seller Space
Engineers (4 million copies sold). Both companies are based in Prague,
Czech Republic.
Marek has been interested in
artificial intelligence since childhood. He started his career as a
programmer but later transitioned to a leadership role. After the
success of the Keen Software House titles, Marek was able to personally
fund GoodAI, his new general AI research company building human-level
artificial intelligence.
GoodAI started in January
2014 and has over 30 research scientists, engineers, and consultants
working across its Research and Applied teams.
At
this time, Marek is developing Space Engineers, as well as leading daily
research and development on recursive self-improvement based general AI
architecture - Badger.
No comments:
Post a Comment