Marek Rosa - dev blog: 01/01/2021

Thursday, January 14, 2021

Review of 2020 & Plans for 2021

SUMMARY:

Space Engineers had a great year!
New developments on GoodAI Badger architecture
Opened our new headquarters - Oranžérie
My sabbatical in Italy

Intro

It is my tradition at the beginning of every year to publish a review of what we have achieved in the previous year, and what our plans are for the new one.

Review from 2018/2019 is here.

Review from last year (2019/2020) is here.

This year's review starts here. Enjoy!

Keen Software House

This was another great year for Space Engineers.

We released Space Engineers on Xbox, and since then have released some major additions - dedicated server, modding support, and all new updates. We are supporting PC and Xbox equally.

March 2020 - New Planet Triton, Visual Scripting Tool Overhaul & Frostbite Pack

April 2020 - Space Engineers: Xbox Release

June 2020 - Sparks of the Future

August 2020 - Server Optimizations

November 2020 - Wasteland

Space Engineers’ user base is steadily growing (Steam Charts depicting “active users”).

Recent reviews and all reviews are at ~90% (Very Positive), which is a very good sign after all these years.

We are re-organizing our team so that there’s one group focusing on supporting the next major updates, and another group that is focusing on more long-term avenues.

Recently we celebrated 7 years of Space Engineers and for this occasion, we made this video:

GoodAI

Badger summary

Last year we continued working on the cornerstone of our research - Badger architecture.

We continued working on various prototypes of Badger architectures and we summarized our progress in a number of articles:

Benefits of modular approach – generalization - article
Learning in Badger experts improves with episodic memory - article
Trainability of Badger – Why is Badger so hard to train? - article
Distributed Evolutionary Computation on Deep Reinforcement Learning Tasks - article
Task Representation for Badger - article

The biggest challenges of Badger architecture still lie ahead of us:

Demonstrate learning inside the inner loop
Demonstrate open-ended learning capabilities (recursive self-improvement)
Demonstrate continuous life-long learning: not forgetting, reuse of skills, reuse of skills to learn new skills
Demonstrate parallel learning

We have noticed a number of papers published in 2020 that are similar or relevant to Badger architecture:

Meta-Learning Backpropagation And Improving It by Louis Kirsch and Jürgen Schmidhuber.
Network of Evolvable Neural Units: Evolving to Learn at a Synaptic Level by Paul Bertens and Seong-Whan Lee.
Finding online neural update rules by learning to remember by Karol Gregor.
MPLP: LEARNING A MESSAGE PASSING LEARNING PROTOCOL by Ettore Randazzo, Eyvind Niklasson, and Alexander Mordvintsev.
Continual Learning with Deep Artificial Neurons by Blake Camp, Jaya Krishna Mandivarapu, and Rolando Estrada.
Self-Organizing Intelligent Matter: A blueprint for an AI generating algorithm by Anonymous authors.

This development brings hope that other researchers find these principles interesting and that this sub-field of AI is going to grow.

Workshop & GoodAI Grants

During the summer we organized the first of our Meta-Learning & Multi-Agent Learning Workshops. The workshop saw over 60 participants from across the world take part including speakers from Google Brain, DeepMind, OpenAI, University of Oxford, Stanford University, and MIT. Before that, we had two closed workshops (1 and 2), both related to Badger architecture, multi-agent learning, modular meta-learning, open-endedness, and connected topics.

At the MLMA workshop, we launched a new initiative - GoodAI Grants. Researchers and research groups can submit their research proposals and if it’s relevant for what we are doing, especially Badger architecture, we provide a financial grant.

The first grant was awarded to Tomas Mikolov and his team, and many more will be announced soon. We had to double the initial budget because we received more than the expected number of interesting projects.

We have also been involved in a number of grant projects:

We are part of the VeriDream project, an international consortium of six organizations across Europe that has been awarded €2 million by the European Innovation Council to implement AI into robotics.
We are also working on iv4XR a Horizon 2020 project that aims to build a novel verification and validation technology for Extended Reality systems based on techniques from AI to provide learning and reasoning over a virtual world.

Papers and blog posts

Our team has been busy publishing papers and blog posts outlining our work and our grand vision! Below you can find a selection of this year’s publications.
One of our senior researchers, Nicholas Guttenberg, had co-authored a paper in Nature. Nicholas and a team of collaborators used AI to analyze fossils and find surprising patterns in Earth’s biological mass extinctions. You can read about that in Digital Trends here and you can find the original paper in Nature here.

GoodAI COO, Olga Afanasjeva, has written an opinion on What kind of AGI do we want? It is a great summary of what kind of research framework we believe we need if we want AI to perpetuate some of the values we hold dearest: curiosity, creativity, a need to be surprised, and to marvel.

AI Policy and Social Impact Director, Marek Havrda, PhD, has also been busy. His article on AI’s Role in Community Engagement within the Democratic Process was published in Springer Journal. He also took part in many international meetings on AI on behalf of GoodAI. You can read his report from the first OECD network of experts on AI here and his report from the expert workshop on International Governance of AI here. Marek and Olga also worked together on a forward-looking blog outlining 9 ways AGI could shape the world for the better.

The start of the year also saw the results of the Animal-AI Olympics. This project was organized by the University of Cambridge in collaboration with Leverhulme Centre for the Future of Intelligence, Imperial College London and GoodAI..

Finally, one of our older architectures Toy Architecture was published in PLOS ONE.

And Senior Research Scientist Petr Hlubuček wrote a blog on the Implementation of Generative Teaching Networks for PyTorch.

What did we learn during 2020? Vanilla Badger architecture is not enough, at least a few additional ideas need to be invented in order to push forward. That’s what we are working on!

Oranžérie

In the summer we held an opening party for our new headquarters - Oranžérie.

Because of COVID situation, we had only a few months to experience our new premises. Then everyone left for the home office. We are looking forward to next year, after the situation gets better and people will be able to meet again.

Photo credit: Ondřej Pýcha

COVID

COVID had no negative effect on KSH and GoodAI.

There were actually few positives (even though it sounds strange in this context)

Our revenue grew during the first and most intensive months of lockdowns
We proved that home office and remote work is a viable option and it quickly became our standard
Remote first and not requiring our team members to be present in Prague opened doors for hiring people internationally. I always wanted this and was planning it in the future, but now the situation forced us.

At present, 75% of our team is located in the Czech Republic and 25% is abroad.

Personal

For many years, while I was performing my CEO duties, I was missing programming. I missed being able to implement my ideas directly and to see the results instantly.

During 2020 I worked on redesigning my schedule and my management style, to have more “alone time”, to be able to start working on my own experiments.

In order to kick it off, I took a 2 months sabbatical in Italy, during which I started working on what I call “Manual Badger”. I published the first blog post detailing my early experiments with the evolutionary dynamic of sexual selection.

I plan to continue on what I started, delegating more and having time to do focused work.

The trip to Italy lasted almost 4 months and because I love Italy and its ancient history, I was enjoying every day. We went by car, from Prague to Florence, Sicily, by ferry to Naples, and then we stayed in Rome for a couple of months. I have hundreds of photos, it’s hard to pick, so here are at least few.

Plans for 2021

Keen Software House

Development of Space Engineers will continue with regular major updates.

We will continue building our team, to be able to keep the support and updates, and also to focus on new and revolutionary areas.

GoodAI

We will be announcing a new batch of GoodAI grants soon, supporting interesting research groups around the globe.

We will continue experimenting with various approaches to Badger architecture.

In the summer we plan to organize another round of the MLMA Workshop.

Thank you for reading this blog!

Best,
Marek Rosa
CEO, Creative Director, Founder at Keen Software House
CEO, CTO, Founder at GoodAI

For more news:
Space Engineers: www.SpaceEngineersGame.com
Keen Software House: www.keenswh.com
Medieval Engineers: www.MedievalEngineers.com
GoodAI: www.GoodAI.com
General AI Challenge: www.General-AI-Challenge.org

Personal bio:
Marek Rosa is the CEO and CTO of GoodAI, a general artificial intelligence R&D company, and the CEO and founder of Keen Software House, an independent game development studio best known for its best-seller Space Engineers (4 million copies sold). Both companies are based in Prague, Czech Republic.

Marek has been interested in artificial intelligence since childhood. He started his career as a programmer but later transitioned to a leadership role. After the success of the Keen Software House titles, Marek was able to personally fund GoodAI, his new general AI research company building human-level artificial intelligence.

GoodAI started in January 2014 and has over 30 research scientists, engineers, and consultants working across its Research and Applied teams.

At this time, Marek is developing Space Engineers, as well as leading daily research and development on recursive self-improvement based general AI architecture - Badger.

Friday, January 8, 2021

My experiments with the evolutionary dynamics of sexual selection

SUMMARY:

I am currently working on a Badger architecture that has no outer-loop, only one infinite inner-loop
New building blocks are supposed to emerge from the interaction of existing building blocks, where the initial building blocks are predefined
My first step was to experiment with sexual selection (the topic of this article)
The second step will be to experiment with memetic evolution

Predators (red dots) replicate by eating prey. Prey (colored dots) replicate using sexual selection. Strong prey speciation not observed.

Intro

A few months ago I decided to start doing my own experiments on our GoodAI Badger architecture.

My initial motivation was that I really miss programming, and being able to just build stuff with my own hands. A secondary motivation is that I have an intuition about what I think our end-goal would look like, and the best way to explain it is by building prototypes.

How does my approach differ from Badger architecture? In Badger, the goal is to learn the shared expert policy through outer-loop. In my approach (which I call “Manual Badger”), there’s no outer-loop. Instead, agents/experts come pre-designed with a policy that can later evolve into something complex and interesting. This approach has similarities to evolution, except this will be quite a specific version of evolution (in some ways more limited and narrower) but at the same time is supposed to enable open-endedness and the emergence of intelligence.

*Note: the boundaries between what is outer-loop and inner-loop, or what is expert’s policy and activations, and on what time-scale the majority of adaptation happens is maybe blurry and not so relevant at this moment.

What is sexual selection? It is a mode of natural selection in which members of one biological sex choose partners of the other sex to mate with, and compete with members of the same sex for access to members of the opposite sex. What makes this interesting from the perspective of AGI and Badger is that preferences arising within the population can generate fitness and reward effects that overtake external pressures, resulting in a system that in some sense chooses its own goals.

More info: https://en.wikipedia.org/wiki/Sexual_selection

What surprised me

Speciation emerged much faster than I anticipated. The Simulation starts with a random distribution of genes and within 1000 time-steps we can observe clear boundaries between species.
The domination of ‘most selected for’ genomes (colors) – because of the way I implemented the selection process (basically two genomes competing in the sexual selection process, some genomes get an advantage and because the process is stochastic, there’s not much that the disadvantaged genome can do)

Geographic bottlenecks reinforce speciation - this means that if I disable the world map and the environment becomes a uniform grid with no bottlenecks between continents and isolated islands, the simulation will never evolve into a state where we can see clear boundaries between species - everything stays kind of uniform, or at least with very diffused species boundaries. If we enable the world map, clear speciation emerges.

How is this relevant to AGI and Badger architecture?

This is the 'Manual Badger' take on our Badger architecture.

The difference is that we don't learn the expert policy in the outer loop. Instead, we hard code it and let the system evolve into something interesting (diversity of behaviors, evolvability of adaptation, etc).

Next steps

Memetic evolution - agents being able to spread their memes. Memes are programs/behaviors.

My expectation is that it should lead to a higher diversity of sexual selection and behavior, and to the emergence of new behavioral niches for other memes.

Hopefully, we will observe a few more evolutionary transitions - interactions that are not predictable from the initial state.

Experiments

Basic experiment

What you see is a grid world.
Each cell represents an agent (two sexes), an obstacle (ocean/sea), or an empty location.
Agents move randomly.
Each agent has a genome, a 3-dimensional vector, visualized as an RGB color.
The genome has no impact on an agent's fitness or behavior (except its sexual preferences)
The genome impacts the sexual selection process: agent ranks the opposite sex agents in its proximity (2 to 3 cell radius) by how close their genomes are, and only above minimum threshold similarity is considered. The agent then chooses the best match and a new offspring is born to an unoccupied location near them (if there’s an empty lot).
The offspring’s genome is a crossover of both parents - a mathematical average of those two genomes (RGB colors).
There's no explicit survival fitness, no food.
Agents mate every 10 timesteps, and they die after 100 timesteps (those black cells that frequently appear show the location where the agent died and their cell is unoccupied)

Turn off preferences entirely and see that color boundaries don’t persist.

The boundaries become very soft and dispersed.

Make agents prefer mates with stronger versions of their own traits.

Mating scan area increased to 5 cells radius (instead of default 2 cells).

Can we see spontaneous speciation from a completely grey initial state when mutation is disabled?

The answer is no. The initial population of grey agents will grow and saturate every available cell, but the color will never change.

Can we see spontaneous speciation from a completely grey initial state when mutation is enabled?

The answer is yes. The initial population of grey agents with mutation leads to speciation. Sometimes the borders between species are quite distinct, sometimes more dispersed. I think a big contribution here is the ‘mutation frequency’ and also the ‘selection threshold’. Maybe if I set the latter to a smaller number, the borders would become more distinct.

What helped to make the species grow larger and the borders more distinct was when I added the ‘mutation frequency’. I guess the population became slightly more stable.

Can we see spontaneous speciation from a completely grey initial state - with mutation enabled & mutation frequency is part of the genome (initial mutation frequency is close to zero, but can increase due to mutations)?

This is an idea I got while watching the previous experiment: Can we see spontaneous speciation from a completely grey initial state - with mutation?

The answer is yes. I think what happens is that higher mutation frequency emerges at some places and that area populates, but then slowly evolves to lower, or almost zero mutation frequency, in order to stabilize its population.

Can we see spontaneous speciation from a completely grey initial state - with mutation enabled & mutation frequency is part of the genome (initial mutation frequency is high, but can decrease due to mutations)?

Remove the world map, so we get a uniform grid. Can we now observe different evolution, because the bottlenecks caused by having geographically separated continents and only some of them connected through a narrow bridge, was removed?

The result was surprising and cool :) The system evolved towards something resembling the “heat death” of the universe.

The above three pictures show the progression - the first one shows the start, the middle one shows speciations with blurry borders, and the last one shows a state where all genes drifted towards 0, 0, 0, and the system is now very unlikely to evolve into anything else, although, there are locations where genes mutate and try to survive for some time, the drive towards 0, 0, 0 is still stronger.

Another one surprise is that I was unable to replicate this “heath death” result. I kept it running for much longer, but never got to the heat death state. It just keeps oscillating between different dominant genes.

What happens when replication speed is limited so that no matter how good at finding a mate you are, you can have at most ~2.5 offsprings per mating pair in your life? E.g. track offspring and every time a pair has one kid, they each have a 20% chance of going sterile/dying. Tune background death rate so that you don't have the entire space filled up with solid colors. My guess is that with runaway sexual selection you might get some kind of coexistence here, but with no selectivity, it'll still go to brown/gray.

Starts with an initial phase where we observe speciation with blurry borders. Then it evolves to a regime where either R color dominates, or G color dominates. It oscillates between them as long as I let it run.

An additional question is why B color doesn’t dominate (used also as a mutation frequency).

Conclusion: the result doesn’t differ from other experiments where the max number of offsprings is not limited.

Sterility set to 100% after reproduction. Didn’t reach criticality in reproduction, the population soon died out

Initialize with R color only

Conclusion: no difference, mutation will create speciations, and then dominant color overtakes, and then it fluctuates

Random color init

1024 grid size

Comment: same as for 256 grid size, speciation to diffusion

Aged colored = gene is multiplied by a normalized age.

This results in visualization when young and old are clearly distinguishable.

Offspring genome is not a result of averaging genes of both parents, but instead, we take some genes from each of them - kind of crossover point

Add predators (they replicate by eating prey)

Predators are visualized by red color (age coded)

After I added the predators, prey stopped to speciate (no strong boundaries between species, they all are kind of grey-ish)

Enabled world map

Now we can see that the speciation where predators are non-existent, however on islands and other similarly isolated areas without predators, speciation emerges.

Conclusion

It’s fun to be doing my own experiments again! :-)

Implementing my own agent-based simulation of evolutionary dynamics of sexual selection gave me better intuition for my next step - memetic evolution. Stay tuned!

If you want to let me know your feedback, please get in touch via my personal email address marek.rosa@keenswh.com, or use our Keen Software House support site. I welcome all of the feedback we receive and we will use it to learn and provide better services to our players.

Thank you for reading this blog!