Thursday, November 16, 2023

Games in 2033

This is a revised version of my article for Level magazine in May 2023.

Introduction

The next decade is set to witness a revolutionary transformation in the gaming industry, driven by advancements in AI and neural interface technology. 

This article unfolds a five-stage evolution of gaming, each building upon the previous innovations and technological breakthroughs, significantly reshaping the gaming experience and its societal impact. From intelligent NPCs to immersive neural interfaces, we will explore how each stage contributes to this radical shift.

Content:

  1. Intelligent NPCs
  2. AI-Generated games and assets
  3. AI-Simulated games (mid-frequency simulations)
  4. AI-Simulated games (high-frequency simulations)
  5. Neural Interfaces
  6. Observer Effect in AI-Simulated Games
  7. Impact on Society

Stage 1: Intelligent NPCs

The first stage will introduce AI-generated intelligent NPCs (non-player characters) capable of observing, understanding, thinking, planning, making informed decisions, and interacting with their environment.

These behavioral NPCs will offer more than just conversational abilities; they will exhibit goal-oriented actions and long-term memory, spatial awareness and reasoning, and an ability to mimic human thought and emotional processes

They will be powered by LLMs (Large Language Models) like GPT-4 or Mistral 7B. 

Initially, the NPCs will be cloud-based, allowing games with advanced AI characters to run on standard gaming systems like PCs and consoles. 

As technology advances, smaller LLMs will enable these complex NPCs to run directly on players' devices, lessening cloud dependency. The Mistral 7B model is already a step in this direction. 

Stage 2: AI-Generated Games and Assets

Next, we enter an era where game creation will become a collaborative process between the user and an LLM agent. This stage will revolutionize the user's role from mere player to creator. Users will engage in a back-and-forth dialogue with the AI, refining and articulating their game concept. 

The AI will generate code and assets within engines like Unreal and VRAGE3, utilizing its deep understanding of game dynamics. This stage will democratize game development, enabling the creation of games through simple descriptive text.

Just by talking to AI agents, you will be able to create the game of your dreams.

Stage 3: AI-Simulated Games (mid-frequency simulations)

In this stage, the AI model will play a more direct role in simulating game logic and maintaining the game state in its memory, while rendering and physics will still be managed by a traditional game engine.

The AI's advanced simulation capabilities will dynamically evolve game mechanics and storytelling alongside the game state, all in real time. This will ensure a continuously adaptive and flexible gaming experience.

This hybrid model will allow the AI to simulate mid-frequency game logic (5-10 hertz), complementing the high-frequency tasks like physics and rendering (60-120 hertz) handled by the game engine.

Stage 4: AI-Simulated Games (high-frequency simulations)

The next step in gaming technology will see AI take over all aspects of game simulation, including graphics rendering and physics, ensuring a consistent and smooth performance at 60-120 frames per second (FPS). 

While traditional game engines will still be needed for mid-frequency simulations, high-frequency simulations will rely entirely on AI for both physics and graphics rendering tasks.

This will enable the creation of a simulated reality that dynamically adapts to the player's desires in real-time

Achieving this level of sophistication requires significant improvements in the speed and efficiency of AI inference, likely spurring the development of specialized hardware designed specifically for AI model inference. 

In these simulations, the distinction from actual reality will be indiscernible, both in visual aspects and in terms of interactive fidelity and adaptability. Minute details will be intricately simulated, surpassing the limitations traditionally set by programmer-defined features, leading to a groundbreaking and immersive gaming experience.



Stage 5: Neural Interfaces

The culmination of our journey is the integration of neural interfaces like Neuralink, directly connecting the human brain to AI-powered gaming simulations

This stage represents a monumental leap, creating immersive and personalized gaming experiences that become indistinguishable from reality. We may already be in such a simulation 😀

The neural interface represents a new way to play games and a complete transformation of the human experience in virtual realms.

Observer Effect in AI-Simulated Games

A key performance advantage in AI-simulated games is derived from the observer effect, contrasting sharply with classical physics. Here, the existence of objects and their effects hinges on the player's observation.

This selective simulation strategy, akin to an advanced LOD optimization, ensures efficient resource use and a more engaging gaming experience. 

An element outside the player's perceptual field or comprehension remains unrendered and unsimulated, allowing the environment to dynamically adapt to the player's interaction and focus 

Impact on Society

AI simulations, less resource-intensive yet more captivating than real-world experiences, will shift our primary experiences to virtual simulations. 

The tangible world will transition to a colossal center for computing and manufacturing, supporting these virtual realms. 

These changes offer endless possibilities for personal growth, entertainment, and exploration, heralding a renaissance of synthetic universes beyond our current imagination.

 Conclusion

At GoodAI, we are already working on stage 1 with our AI People game (LLM-powered NPCs running on the player's computer).



The next stages will be 2 and 3, but I think we can skip 2 and go directly to stage 3 (mid-frequency AI simulation of game logic and state).

The forthcoming changes in the gaming industry, driven by AI and neural interfaces, will significantly impact society on multiple levels. From altering energy consumption patterns to redefining leisure time and our perception of reality, this evolution in gaming is set to transform not just how we play games but how we experience and interact with the world itself.


If you want to let me know your feedback, please get in touch via my personal email address marek.rosa@keenswh.com, or use our Keen Software House support site. I welcome all of the feedback we receive and we will use it to learn and provide better services to our players.


Thank you for reading this blog!

 

Best,
Marek Rosa
CEO, Creative Director, Founder at Keen Software House
CEO, CTO, Founder at GoodAI

 

For more news:
Space Engineers: www.SpaceEngineersGame.com
Keen Software House: www.keenswh.com
VRAGE Engine: www.keenswh.com/vrage/
GoodAI: www.GoodAI.com

 

Personal bio:

Marek Rosa is the founder and CEO of GoodAI, a general artificial intelligence R&D company, and Keen Software House, an independent game development studio, started in 2010, and best known for its best-seller Space Engineers (5 million copies sold). Space Engineers has the 4th largest Workshop on Steam with over 500K mods, ships, stations, worlds, and more!

Marek has been interested in game development and artificial intelligence since childhood. He started his career as a programmer and later transitioned to a leadership role. After the success of Keen Software House titles, Marek was able to fund GoodAI in 2014 with a $10 Million personal investment.

Both companies now have over 100 engineers, researchers, artists, and game developers.

Marek's primary focus includes Space Engineers, the VRAGE3 engine, the AI Game, and LLM agents that learn continually.

GoodAI's mission is to develop AGI - as fast as possible - to help humanity and understand the universe. One of the commercial stepping stones is the "AI People'' game which features LLM-driven NPCs grounded in the game world with developing personalities and long-term memory. GoodAI also works on autonomous agents that can self-improve and solve any task that a human can.

Thursday, October 26, 2023

Space Engineers: 10 Year Anniversary Update

  SUMMARY:

  • 10 Years of Space Engineers Celebration
  • New Blocks in the base game: Engineer statue, Fireworks, Tuxedo Suit Skin and more!
  • New Premium Content: 10 Year Anniversary Pack
  • Sounds of Space Engineers 2023
  • Steam Free Weekend
  • New 10 Year Anniversary Merchandise


Hello, Engineers!

How time flies when you are having fun.

Ten years ago we began this incredible journey through the stars, laying the foundation for everything that has come to pass. 10 years, 5 million copies, hundreds of updates, and a half million workshop mods and creations later, Space Engineers is still thriving!

To every Space Engineer out in the vastness of space, those of you that have been with us from the beginning, to those that became a Space Engineers only now, thank you for being a part of this journey. Your creativity and passion for Space Engineers inspires us to our very best.

In celebration of this incredible moment, we have prepared a surprise update for you to commemorate 10 years of engineering, planning, grinding, welding, battles, and exploration! 

Due to popular demand we are bringing back our two most requested (limited edition) character skins: Veteran and Ghost. These will only be available for a limited time! You can get them along with a new Party Tuxedo Suit in the new 10 Year Anniversary Pack.


Free Content

Engineer Statue (1 block, L grid)

A memento of the first Space Engineers. Their work and perseverance created this incredible future we now explore. 

Fireworks Block & Color Variant Ammo (1 block, S + L grid)

Introducing the Fireworks Block in Space Engineers - Light up the cosmos and celebrate your achievements with dazzling displays of fireworks!

Ammo Colors: Blue, Green, Red, Yellow, Pink and Rainbow

Flare Gun & Ammo

When your celebration needs to truly impress. Illuminate the void and signal your presence with “flare” and style.

Tuxedo Suit Character Skin

Elegance meets exploration. 


10 Year Anniversary Pack

Similar to our previous major releases, we hope you will take this opportunity to support the continued development of Space Engineers. This limited time package will only be available until January 5th 5PM UTC. This pack consists of cosmetic character skins that enrich your game visually. None of the features in this package provide any advantage to players who purchase this pack. 


The price of the 10 Year Anniversary Pack is $2.00 USD or your regional equivalent. Show your continued support for Space Engineers and check out the 10 Year Anniversary Pack.

This package includes:

Party Tuxedo Suit Character Skin

Dress to impress with the Party Tuxedo Suit for your astronaut and celebrate in style.


Ghost Character Skin

Get into the Halloween spirit with the Ghost Character Skin - haunt the cosmos as a spectral presence!


Veteran Character Skin

Show your experience and dedication with the returning Veteran Skin - wear your hard-earned stripes and command respect among the stars.


Steam Free Weekend & 75% Sale

Space Engineers is free to play on Steam this weekend!

You can install and enjoy Space Engineers for free from October 26, 5 pm UTC to October 30, 2023.

We are also running a Steam Sale from October 26, 2023 to November 2, 2023. Space Engineers base game will be 75% off!

There was no better time to join our community - fulfill your need to create! 


10 Year Anniversary Celebration

People from around the world gathered at our HQ to join in the celebration. It was heartwarming to see fans, colleagues, and supporters coming together to mark a decade of innovation, exploration, and creativity. 

For those who couldn't be here, we made sure you didn't miss out. Our live stream ensured Space Engineers across the globe could be a part of the festivities. The evening featured presentations from our CEO Marek Rosa as well as from our talented colleagues Jan Hloušek and Natiq Aghayev, offering insights into the development of Space Engineers over the years. 

We couldn't forget the live orchestral concert featuring Space Engineers' music from our amazing composer Karel Antonín. It added a special touch to the evening, taking us on a journey through the game's universe. Of course, we had great food, and there were plenty of opportunities to meet and chat with our fans, supporters, and the brilliant minds behind Space Engineers. 

We are immensely grateful for your support, which has made Space Engineers what it is today. As we reflect on this memorable September event, we're excited about the future. Here's to more years of exploration, creativity, and possibilities!


VRAGE3 Anniversary Presentations




New Anniversary Merchandise

To commemorate this special occasion, we are releasing new 10 year anniversary merchandise. 

You can check it out here.



And for new Displate metal posters continue here.


Sounds of Space Engineers 2023

After an overwhelming response featuring more than 100 submissions, the top 15 tracks of the 2023 Space Engineers music competition have been selected by multiple judges at KeenSWH!
It was no easy task selecting these winners as there were many high quality and unique entries.

The judges tried to select a mix of music styles so that there is variety and hopefully something for every player to enjoy! All of these tracks are making their way into the Jukebox in this update. 

Enjoy listening to these amazing tunes!


Hiring

If you’re interested in working on awesome games like Space Engineers or our in-house engine VRAGE3, we’d love to hear from you! 

We are currently in the Research and development phase, exploring new features of the VRAGE3 engine. Our goal is to support extremely large and fully dynamic and destructible environments, including planets, large spaceships, and any other creations built by the players.

If you’re interested in working on awesome games like Space Engineers or our in-house game engine , and love solving unique challenges, we’d love to hear from you! Check out the open positions at Keen Software House and don’t forget to send us your English CV/resume and cover letter. 

Remote collaboration is possible!

Our team is global.

Finding the best candidates to join Keen Software House means exploring every possible solution, including remote work. While we strive to provide team members with the best possible work-life balance here in Prague at our incredible Oranzerie offices, we understand that it is not always possible to transition, therefore we are very remote-friendly. Here’s a map of where our teammates live.


Follow our social media to get the latest news!


If you want to let me know your feedback, please get in touch via my personal email address marek.rosa@keenswh.com, or use our Keen Software House support site. I welcome all of the feedback we receive and we will use it to learn and provide better services to our players.


Thank you for reading this blog!

 

Best,
Marek Rosa
CEO, Creative Director, Founder at Keen Software House
CEO, CTO, Founder at GoodAI

 

For more news:
Space Engineers: www.SpaceEngineersGame.com
Keen Software House: www.keenswh.com
VRAGE Engine: www.keenswh.com/vrage/
GoodAI: www.GoodAI.com

 

Personal bio:

Marek Rosa is the founder and CEO of GoodAI, a general artificial intelligence R&D company, and Keen Software House, an independent game development studio, started in 2010, and best known for its best-seller Space Engineers (5 million copies sold). Space Engineers has the 4th largest Workshop on Steam with over 500K mods, ships, stations, worlds, and more!

Marek has been interested in game development and artificial intelligence since childhood. He started his career as a programmer and later transitioned to a leadership role. After the success of Keen Software House titles, Marek was able to fund GoodAI in 2014 with a $10 Million personal investment.

Both companies now have over 100 engineers, researchers, artists, and game developers.

Marek's primary focus includes Space Engineers, the VRAGE3 engine, the AI Game, and LLM agents that learn continually.

GoodAI's mission is to develop AGI - as fast as possible - to help humanity and understand the universe. One of the commercial stepping stones is the "AI People'' game which features LLM-driven NPCs grounded in the game world with developing personalities and long-term memory. GoodAI also works on autonomous agents that can self-improve and solve any task that a human can.

Friday, September 1, 2023

LLM Agent taught to control drones

 

In this video above, we demonstrate the learning process of one of our LLM Agents being taught how to use an API to control a drone quadcopter. The initial stages require us to provide the Agent with detailed and comprehensive instructions about how to send HTTP requests and what commands are available through the API. As the video progresses, the Agent quickly grasps these instructions and leverages the knowledge it already has to perform advanced and intricate tasks, like flying the drone following a square trajectory. This showcases the Agent's resilience and adaptive learning capabilities - how the Agent recovers from errors and false assumptions. 


This version of the continual-learning agents represents a significant advancement from our first prototype (agent embodied in a Python terminal). This enhanced Agent has access to distinct forms of working memory and long-term memory, enabling it to effectively manage several types of memory inconsistencies, such as contradictions or outdated information, and can learn from user feedback and environmental cues. You can think about it like a cognitive architecture built on top of LLM.


The agent's response is a result of a sequence of nested steps. This method augments the LLM's cognitive resources and attention span to extend beyond the limits of LLM (LLMs are stateless, have fixed size context, don't pay sufficient attention to all instructions in the prompt, etc.). Notably, the process employs iterative prompting, using multiple prompts to accomplish tasks that the LLM can't perform in a single inference—such as retrieving and summarizing memories or maintaining the agent's state for future steps. Every step in this sequence receives input data, processes it to produce relevant output data, and shares this output across the entire chain. Consequently, formulating a response becomes a joint effort. Each step in the chain has the autonomy to determine the information it requires, what new state needs to be stored in working memory and long-term memory, if it needs to consult the LLM, and what specific prompt should be fed to the LLM for each iteration.



Initially, the agent is in a “blank state,” knowing nothing about the drone API. Therefore, we commence by closely guiding its actions, telling the agent the exact URL that it has to communicate with and the exact form and value of the HTTP requests it has to send. However, as time progresses, the Agent gradually accumulates knowledge and expertise, enabling it to tackle increasingly complex tasks based on its past experience. 

We actively incentivize the agent to create functions or any kind of reusable code for representing learned skills, which the agent ends up carrying out without us having to tell it to do so. In addition, the agent interacts with a Python terminal as its means for performing actions, so it will instantly become aware of any malfunction or error in its code, being able to fix it on the fly and learning from its mistakes. Finally, thanks to its episodic memory, the agent can relate current situations with past ones, remembering errors made during its instruction or functions that it wrote long ago and might leverage just now.

The video demonstrates that the terminal prototype ended up being converted into one stage of the pipeline (the TerminalAction), which leverages the Python terminal environment plus the memories retrieved by a previous stage to carry out the agent’s actions. On top of this, other stages add functionality like terminal session persistence and generation of new memories.

Take a look at more videos below. 



This footage demonstrates how we teach the agent to comb an area inside four GPS points defined by the user. First, we let the agent memorize the four GPS points by manually sending the drone to each position and asking the agent to remember them with different names. Then, we explain to the agent what we expect it to do: thoroughly fly over that area by following zig-zagging corridors of 10 meters in width. This is extremely useful for planning exhaustive search operations

The video has been sped up 4x during the teaching and 16x during the combing phases.


In this recording, we teach the Agent to fly the drone in a circle. Because the Agent still ignores some aspects of the drone API, we must be specific about certain things, like using the blocking commands instead of the asynchronous ones. The Agent flies the drone following the user specification: the circle has a 50-meter radius, and its center is the current drone position. 

We have sped up the video 4x for visualization purposes.




Here's a video showcasing speech-to-text as input and text-to-speech as output. The agent has already been given a list of functions that it can use to interact with the drone API.

This addition enhances the user interaction and adds a social dimension to the agent. It makes the interaction more engaging.

 Note this is still an early prototype, and many improvements are in the works.


Link to the YouTube playlist containing all videos.


Thank you for reading this blog!

 Best,

Marek Rosa
CEO, Creative Director, Founder at Keen Software House
CEO, CTO, Founder at GoodAI

 

For more news:
Space Engineers: www.SpaceEngineersGame.com
Keen Software House: www.keenswh.com
VRAGE Engine: www.keenswh.com/vrage/
GoodAI: www.GoodAI.com

 

Personal bio:

Marek Rosa is the founder and CEO of GoodAI, a general artificial intelligence R&D company, and Keen Software House, an independent game development studio, started in 2010, and best known for its best-seller Space Engineers (5 million copies sold). Space Engineers has the 4th largest Workshop on Steam with over 500K mods, ships, stations, worlds, and more!

Marek has been interested in game development and artificial intelligence since childhood. He started his career as a programmer and later transitioned to a leadership role. After the success of Keen Software House titles, Marek was able to fund GoodAI in 2014 with a $10 Million personal investment.

Both companies now have over 100 engineers, researchers, artists, and game developers.

Marek's primary focus includes Space Engineers, the VRAGE3 engine, the AI Game, and LLM agents that learn continually.

GoodAI's mission is to develop AGI - as fast as possible - to help humanity and understand the universe. One of the commercial stepping stones is the "AI game," which features LLM-driven NPCs grounded in the game world with developing personalities and long-term memory. GoodAI also works on autonomous agents that can self-improve and solve any task that a human can.