Because it's hard...
We've dedicated the last four years to this endeavor.
Making a simple proof of concept (PoC) is easy (we've seen many), but polishing it to production level requires 1,000 times more effort where deeper complexities emerge:
1) Cost: Using LLM via API means someone has to pay for those thousands of tokens per hour per player. Yes, you can run a smaller LLM on a player's GPU (e.g., Mistral, Llama3 8b), but the quality and capabilities may not be sufficient for an engaging narrative and gameplay.
2) It's too early: GPT-3.5 is only 1.5 years old, and GPT-4 is only one year old.
3) Current LLMs: They may not be capable of running multi-agent narratives for long game sessions. Context sizes are not large enough to hold scene history consistently. LLMs only attend perfectly to some instructions in the context and often hallucinate.
4) Multi-modal LLMs (with Vision): You may need these to unlock spatial reasoning and better grounding to the game environment.
For LLM-driven NPCs to fundamentally transform gameplay—making them indispensable, like a painkiller rather than a mere vitamin—they must do more than enhance; they must be crucial, realizing games that, without LLM-driven AI NPCs, would be unachievable (Beat Saber wouldn't be what it is without VR).
- NPCs need to interact with one another and their environment.
- NPCs must consistently understand their surroundings, avoiding any contradictions related to their past or actions that the game mechanics do not support.
- NPCs' behavior and the plot must be consistent over long periods.
- NPCs must have intriguing personalities and narratives that unfold with the drama and tension of a well-crafted movie, rather than resembling dull everyday interactions.
- These narratives should continuously evolve, navigating from problem-solving to new challenges repeatedly.
The use of "ai" is disgusting. LLM's are just scraped data off the internet with no regard for where that data is coming from, or who it belongs to. The complete ignorance of data privacy and intellectual property rights is putrid. Used to be a big fan of Space Engineers for a long time but you've been letting us down consistently now.
ReplyDeleteI am sorry that you feel it this way.
DeleteBut I must point out: modern LLMs are not trained only on internet scrapped data - there’s so much data curation and synthetic data generation happening in creation of the training data, that it largely over weights the internet data, and this difference will be only increasing as we get to more agentic LLMs.
Same feeling to apply llm powered npcs in Games, especially in large gamning company. I'm currently leading a game ai team in bytedance, mostly of our daily work is to develop aigc tools for game developing(lower the cost), how can we make AI NPCs indispensable to game, maybe to make game deeply immersive is the key.
ReplyDeleteThanks
DeleteWasn't there a Witcher mod that uses a LLM for making up more NPC dialog lines? Sorry, I can't find the link anymore.
DeleteNow those NPCs were merely talking, not showing new AI-driven behavior. But it would be a first step. A snarky NPC companion.
Yes, I have seen NPC mods for Witcher or Skyrim... but those are just the first step - just allowing the NPC to have free-form dialogue.
DeleteI think those are not game-changer.
What I am talking about is when the AI NPC becomes a new game mechanic, allowing to play the game in completelly different ways, and also having new experiences.