Tuesday, October 18, 2016

Medieval Engineers Re-Launch

Planets Update

Medieval Engineers is back in action with a great new update! With the introduction of a highly detailed planet, a phenomenal world map that shows a symbolic depiction of the world, a fully updated rendering engine and many optimizations. Engineering in medieval times has never been this awesome. Not to mention all the other new additions to the game, like doors, new particles, a wardrobe, land ownership, and finally the ability to play as a female engineer! 

Medieval Engineers: www.MedievalEngineers.com

Now it is possible to enjoy claiming territory, working together with allies to defend it from enemies, or siege and destroy other player’s castles! Go hunting for deer, fight off barbarian attacks, design your own banners, or simply enjoy the view of the world from atop your own castle. Most features of the game support modding and the Steam Workshop, which already has thousands of items available to use with the game.

Having a planet in Medieval Engineers creates a play area that is many times bigger than the flat worlds that we had before. The planet can have many plants, animals and barbarians with plenty of room for players.
We’ve designed areas of the terrain so that players can build fortifications to defend their territory. The planet and all of its settings can be modded and shared through the Steam Workshop.

For a behind-the-scenes look at how we've developed our default planet, check out our developer diaries episode 4 and episode 13.

Area ownership
Area ownership is a group of features that allows players to claim ownership of territory. The planet has been divided into many areas. We’ve made a new block that we call the claim block. It is a marker that allows the player to claim and control area.

Using the claim block, players can choose who to allow access to the area, rename the area, pay taxes, and several other administrative functions. The claim blocks can also be used as respawn points so that players always have access to areas under their control.

Check out developer diaries episode 7 and episode 8 for more information about Area Ownership.

Contextual GUI
We’ve designed contextual menus for use in the game. These menus give the player a way to interact directly with the world, instead of a full-screen menu that takes players out of the game.

The contextual menus are small, simple to use and are uniform throughout the game. The icons quickly become familiar to players making the menus easy to navigate and use.

World Map and Fast travel
We’ve created a custom word map that is generated from the planet. This means the map will work for any planet and it gave us the opportunity to create some beautiful hand-drawn art. The world map allows you to see the planet by selecting one of 6 kingdoms and zooming in on a sector to see the areas. There are color coded icons for players, castles, owned areas, battles and other events.

By selecting the fast travel button, the player is shown the map with an overlay of accessible and inaccessible areas. Accessibility is based on terrain difficulty and area ownership. Clicking on an accessible area allows the player to travel there instantly. Fast travel cuts the tedium from the large distances that can be traveled on the planet. Fast travel has several options that can be configured in the advanced world settings such as cooldown time or disabling it completely.

For more information about the world map and fast travel see developer diary episode 15.

In Medieval Engineers we have customizable banners and flags. Using a banner workstation the player can customize a personal banner as well as a house banner. These banners can be placed as decoration in the world and they are also used throughout the game to identify players. The banner becomes the player’s identity in the world. We've made the banners modable so, if the built in patterns aren’t adequate, the player can download new ones from the Steam Workshop or create new ones via modding.

Check out developer diaries episode 14, episode 16 and episode 17 for more information about banners. For banner modding see the Banner Modding Guide.

Other new features in the coming update include:
  • A decay system that will remove items and destroyed castles from the world to maintain performance.
  • An improved building system that makes construction easier and more intuitive.
  • An advanced world settings screen where many additional settings can changed.
  • A compass that shows directions and navigational aids such as nearby castles.
  • Doors that open and close and keep barbarians out of your castle.
  • Many more small features and a long list of bug fixes.

See the change log for complete details at http://forums.keenswh.com/threads/7388367

Marek Rosa
CEO, Keen Software House

Medieval Engineers: www.MedievalEngineers.com

Space Engineers on Facebook: https://www.facebook.com/SpaceEngineers
Space Engineers on Twitter: https://twitter.com/SpaceEngineersG
Medieval Engineers on Facebook: https://www.facebook.com/MedievalEngineers
Medieval Engineers on Twitter: https://twitter.com/MedievalEng

Monday, August 29, 2016

Opening the Search for a CEO, Senior Producer / Game Director, Senior Multiplayer Programmer at Keen Software House

Today I'm excited to tell you about several new things in this post:

- Keen Software House is scaling up
- My role is shifting, and I will focus primarily on research in GoodAI
- We're searching for a CEO for Keen Software House
- We're searching for a Senior Producer for Space Engineers
- We've announced a 1 million czk / $40k USD internal multiplayer contest
- We're searching for a Senior Multiplayer Programmer

Several months ago, I wrote a blog post explaining that in the last 1-2 years we have transitioned from an indie studio of 5 people to one of 50+. We’re growing fast, and we’ve experienced some challenges (like any expanding business does). We’re solving this by constantly improving our organizational and management processes, and by finding strong leaders for each project.

At that time I wrote that post, we were working to find the right person to lead Medieval Engineers - to help us scale up and keep bringing you quality, awesome, fun, polished games with big features. Not long after that, we announced a new producer for Medieval Engineers: Tim Toxopeus, who took the reins for the game and brought it back to a state where, in just a few short weeks, we’ll be ready to re-release it - this time with Planets, area ownership, fixed aiming, improved character animations, stabilized first person camera, as well as several unannounced features that you’ll learn about in our upcoming weekly developer diaries. The future of Medieval Engineers is bright, and in time will include mechanical blocks, farming, and more.

Today I have more news for you. There is an incredible amount of potential in our teams, our VRAGE engine, and our games at Keen Software House, and it’s time to give them what they deserve: superstar leaders who can raise Keen Software House games to their full potential and go above and beyond for our community.

In order to do that, I have decided to open the search for:

CEO (Chief Executive Officer) for Keen Software House, who will:
  • Continue to fulfill my creative vision and the vision of our community
  • Be able to deliver Space Engineers and Medieval Engineers at the same or better quality than I was since we started it up until mid-2015, when I started to divide my attention between KSH and GoodAI. Hiring someone able to dedicate 100% of their time to these games will raise them to their full potential.
  • Define and sustain the Keen Software House company culture

Senior Producer / Game Director for Space Engineers, who will finish the game and continue developing the Engineers franchise
  • This will allow Petr Minarik, current producer for Space Engineers and our most senior programmer on the team, to focus on his strengths as programmer who can solve the greatest technical challenges in our games
  • We are very thankful to Petr, who agreed to lead the Space Engineers game until we found a dedicated producer. Now it’s time for him to focus his talents less on the business and team management side of things, and more on programming.

Senior Producers / Game Directors for new projects

Senior Multiplayer Programmer, who will bring the multiplayer experience in our games to a new level.
  • Our games have very dynamic, destructible and large environments, populated with a vast number of objects that can change position and shape very frequently and suddenly. Players can build and destroy ships, stations, planets, asteroids, and more. Each of these changes leads to long computations in game mechanics and physics sub-systems. 
  • The Senior Multiplayer Programmer will design and implement a multiplayer and parallelization system that is capable of handling this level of complexity. It’s essential that we provide a smooth game play experience at all times, not allowing players to observe inconsistencies, bugs, deaths due to incorrect multiplayer synchronization, or the game (or game world) being non-responsive, etc.. This is definitely not a simple job. No game has had to solve this before, meaning that we’re in uncharted territory. 
  • However, despite the difficulty of the problem, we definitely take it seriously. In addition to searching for experienced multiplayer programmers to join our team, on Monday previous week we launched an internal competition in at Keen Software house: the group or individual on our team delivering the best multiplayer in under 3 months will receive a 1 million CZK (= $40,000 USD) bonus from me. At the moment, we have two teams competing: 
    1. Petr Minarik + Jan Hlousek + Sandra Lenardova 
    2. Jan Nekvapil + Michal Zak + the Medieval Engineers team
Improving our management structure means that we can scale up in a serious way - the studio can now be fully run by the game teams, rather than by me directly. 

After we find the CEO and producers, I will transition to the role of Chairman. This means I will stand at the head of the company to preserve the original vision and spirit of Keen Software House. However, most of my time will be dedicated to research and development of general AI and my role as CEO/CTO of GoodAI. 

At GoodAI, we have reached very promising research milestones. We are going to launch the AI Roadmap Institute, and we are inventing neural architectures for growing topologies. All of these things mean that for me, now is the right time to focus 100% on GoodAI. I cannot postpone this. My dedication to general AI research will no longer allow me to directly manage our games, or to give Keen Software house the amount of attention it deserves. For these reasons, I am opening the search for more leaders for Keen Software House.

While I know that handing over the management of Keen Software House to a new CEO is a significant change, I am confident that with strong leadership, our Keen Software House team will bring you more than you ever expected or even thought possible. We will continue the Engineers franchise and expand our portfolio to keep delivering to our players.

Still not sure about what we can do with the right leaders? ;-) 

Just look at what we managed in the past year alone in Space Engineers! Since releasing Planets last November, we have focused on making constant improvements to the game. This includes performance optimizations and bugfixes, as well as smaller additions like the new character animation system. In the near future we will deliver realistic sounds, block redesign, a new render engine (better looking and optimized directly for our games), as well as more improvements that are always being worked on. 

What can we offer the potential CEO or Senior Producer / Game Director that joins us?

Our new colleague will be joining a strong studio with lots of potential to grow. Keen Software House invented the Engineering genre in games, and we continue to define the 3D space environment genre as a whole. Just look at the number of recently released games that are inspired by Space Engineers!

Space Engineers August 2016

Our new team leaders will receive serious compensation, bonuses, and stock options / a share in the company based on how Keen is performing, and have the potential to earn millions. We want to focus on having a very elite and senior team that is directly involved and contributing to the success of our games, and is thus compensated very well. This means that even programmers, artists, and game designers can make big money after delivering the results our community wants and deserves.

I am looking forward to finding and welcoming our new colleagues, who will share our passion and dedication.

Thanks for reading! And many thanks to the Space Engineers and Medieval Engineers players out there - we’re grateful for your patience and especially for your feedback. We definitely can’t make these games without you.

Marek Rosa
CEO and Founder of Keen Software House
CEO, CTO, and Founder of GoodAI


Space Engineers on Facebook: https://www.facebook.com/SpaceEngineers
Space Engineers on Twitter: https://twitter.com/SpaceEngineersG
Medieval Engineers on Facebook: https://www.facebook.com/MedievalEngineers
Medieval Engineers on Twitter: https://twitter.com/MedievalEng

Thursday, August 25, 2016

Guest Post by Jan Hlousek: VRAGE Render & Upcoming Performance Optimizations

Hello, Engineers! I’m Jan Hlousek, and I lead the VRAGE Render team at Keen Software House. Today I wanted to share some of our documentation with you, so you can take a closer look at precisely how we’re working to improve the render even further. Things are looking very promising so far (just take a peek at these screenshots!), but please keep in mind that this is a bold plan that is subject to change as we move forward and learn more.

Here is the primary structure of our documentation on render performance optimization:
  • Bottlenecks
    • How to combat draw calls count
    • Hot to combat drawing unnecessary objects
    • How to reduce vertex processing
  • Implementation Challenges
  • Expected Performance
  • Implementation
    • Overview
      • Frame processing flow
      • GPU Data topology
    • Texture arrays for voxels
    • Texture arrays / atlases for models
    • Texture arrays for billboards
    • Lodding #1
    • New render component
      • Instancing
      • GPU culling
      • Draw Composer
      • Lodding #2
    • Static transparent geometry
    • Improved culling
    • Voxel merge
    • Armor rendering
    • Occlusion culling
    • Voxel occluders
    • Shadows
    • Foliage
    • Planet setup
  • Future improvements
    • Occlusion culling #2
    • Point cloud
    • Shader optimizations
    • GPU bubbles removal
  • Appendix A - Optimization possibilities
    • Transparent pipeline
    • Models
    • Voxels
    • Culling
    • Foliage
    • Lights
  • Appendix B - Performance analysis
  • Appendix C - Articles

Read on or take a look at the update video to learn more! The documentation is fairly technical, but the basic idea is this: We want to reduce the draw calls by moving culling to GPU with merge instance rendering. Also, we want to reduce the number of meshes / vertices processed by better lodding and occlusion culling.


Main Bottlenecks in Space Engineers are in too many draw calls with too many vertices. Dispatching so many draw calls chokes the CPU on both the render and parallel thread, but also in the driver’s kernel. Dispatching unnecessary (occluded) draw calls with large vertex buffers has a negative impact on GPU performance.

See the Performance analysis section for more.

How to combat draw calls count
  • Using instancing per model will reduce draw calls per model
  • Using merge instancing (collating vertex buffers) reduces draw calls overall
    • Those are implemented to some degree, but with limited usage
  • Moving visibility detection to GPU, operating on a static list of objects without CPU involvement
How to combat drawing unnecessary objects
  • Better frustum culling
    • Currently, lots of stuff is dispatched to render although it is not in frustum
  • Detecting what is occluded using occlusion culling
How to reduce vertex processing
  • Proper lodding
    • Currently, lod thresholds are set up by artists, not taking into account current resolution or field of view

Implementation Challenges

It is quite clear how to combat all problems we currently have. Huge consideration was given to decision where to make the cut between CPU and GPU processing, so the communication between them is fluent (non-blocking) and efficient. Therefore, we decided to move culling to GPU. It will eliminate all per-frame updates of buffers in GPU. All updates of GPU data will be bound to changes in the world or camera spatial changes. We will make sure updates don’t choke the CPU or bus via dispatching them to low-priority thread. 

Note: Some tasks are still under research, therefore the final design of the architecture may be slightly changed.

Expected Performance

On GPU, the processing of culling, lodding and instancing will add some load. A reduced amount of triangles and pixels processed will reduce the load. It can be expected that after all optimizations, GPU performance gain will be based on the complexity of the world. The more complex the scene, the better the performance gain.

CPU performance will be enhanced massively: render thread will take a fraction of the current time, plus removing all per-frame parallel tasks. The simulation thread and all async tasks will have much more processing power at their disposal, reducing sim speed problems.

Bus will be freed from lots of per frame per draw call data currently being dispatched to GPU.
Overall, as we are mostly CPU bound, we are expecting those gains in the tested scenarios (see the Performance analysis section):


The roadmap is separated into multiple self-contained tasks. Tasks are designed with the iterative implementation approach in mind: each task can be finished and released separately, and each task brings performance improvements in itself. Tasks should be implemented in a specified order, though, because of dependencies.


Frame processing flow

GPU Data topology

Texture arrays for voxels
Removing conditions in shaders. Theoretically, it's possible to render all voxels in two passes (single and multi) - this will be implemented in the Draw composer phase.

Texture arrays / atlases for models
Two draw calls per model at maximum (base and decals).
Modify model’s texcoords for vertices to address correctly texture atlas. 
Research pending: mipmapping / filtering issues for atlases, performance for updates of huge arrays, performance for rendering from huge arrays.

Texture arrays for billboards
Apply texture atlasing from models to billboards as well. Replace texture atlasing in GPU particles with the same approach.

Lodding #1
Lod thresholds for models, imposters and voxels deduced in the algorithm based on these factors:
  • Render target resolution
  • Distance from viewport
  • Field of view
  • Density of triangles in model (will be deduced on import; for older models, on load)
  • Quality bias (will be used to generally shift to worse lods - exposed in game settings)
Lodding per viewport.
Far away grids should be discarded from rendering completely.
Making shadows work with new lodding.

New render component
Remove lodding, merging, culling and geometry recording.

  • Gather instances for all models
  • Per instance transformation in grid + index to parent’s (grid) transformation
  • Rendering all instances per model at once, without any culling
GPU culling
  • Add brute force frustum culling of instances
  • Add draw indirect based on instance list generated from culling
Draw Composer
  • Collate models into shared vertex / index buffers, each buffer containing objects in stride based on number of triangles (4, 16, 64, 256, 1024, ...), the rest of each stride will be filled with degenerated triangles. Mesh can be contained in multiple buckets to minimize the amount of degenerated triangles. Research pending: performance of indexing vertices from custom buffers; use indexing (with sorted vertices) or not? (less memory vs coherent cache); apply triangle strips?
  • Generate multiple instance lists based on the bucket model is in.
  • Render indirect for each bucket
Lodding #2
  • Bring the lod algorithm on GPU, in draw composer select correct lod mesh for model
  • Output to specific bucket
Static transparent geometry
Apply this approach to static transparent geometry as well.

Improved culling
Spatial tree for frustum culling per grid. Grids themselves will either be culled brute force or they will have their own tree as well.
Research: what is the common amount of grids in the game, and do we need to optimize for that?

Voxel merge
Use new render component and mesh buckets for voxel rendering as well.
Research pending: Bus considerations when adding new voxel patches to existing buckets. Multiple bucket types? (for short / long lived meshes)

Armor rendering
Armor blocks has to be merged, removing invisible edges. Custom tessellation of planes - removing unneeded vertices.
Research pending: Tesselation of lower lods, removing grid details. Basis for physics shape construction.

Occlusion culling
Occluders (essentially meshes with few triangles) are grouped into one huge occluder group per one grid. Occluder group is updated anytime grid is updated.
Armor blocks will have occluder mesh generated only for outer shell.
Models will be able to contain custom occluder lod, which will be added to the occluder group.
Hierarchical z-buffer constructed by rendering occluder groups for every camera view in the frame. HZB used for quick culling per instance.

Voxel occluders
Generate occluder group from the original grid of voxel terrain.
Research: one occluder group per planet / asteroid or multiple?

Add PCF postprocessing, tweak and switch to new shadows.

Optimize shaders (try removing per frame geometry shader). Lower density of grass with distance. Couple both density and distance for foliage in settings.

Planet setup
Tweak planet setup according to performance:
  • Density and distance of foliage
  • Density of trees / bushes
Add slider affecting densities to settings.

Future improvements

Occlusion culling #2
Occluders can occlude each other, removing whole grids from rendering. For this purpose, every occluder has to have an occludee as well. A bounding box (or multiple bounding boxes in case of large grid) containing all the grid’s objects AABBs will probably be enough.
Occluder groups for farway grids won’t be rendered at all.

Point cloud
Add very far objects to point cloud renderer, containing only position and color, determining pixel size based on the distance. Render whole buffer at once, no culling. Adding and removing from the buffer from time to time. The whole point cloud could be disabled based on the user’s settings (reducing the visibility distance)

Shader optimizations

GPU bubbles removal

Appendix A - Optimization possibilities

Transparent pipeline
  • GPU particles
    • Manage alive particles list for simulation (do not update all particles always)
    • Measure possible gains for multiple particle buckets (Lighting, Collisions, streaks)
  • Static Glass
    • Add support for instancing
  • Billboards
    • Shared texture arrays with gpu particles
      • Render all cpu particles in one pass
    • Automatic atlasing of other billboards
      • Render in one pass
  • Texture arrays
    • Loading to three big texture arrays (cm, add, ng)
    • Research pending
      • Possible performance hit with big texture array locking on load 
      • Possible performance hit with accessing texture array in shader (memory throughput bottleneck)
      • Use atlasing or just arrays?
    • Prepare vertex data with uvs and index to atlas
  • Instancing
    • Create new renderable component with simple interface and clean tracking of instances
    • Eligible for static models without bones
    • List of instance data in structured buffer
  • Merge Instancing
    • Consider whether to merge clusters of objects into one mesh
  • Texture arrays
  • Voxel merge
  • Compute shader for frustum culling
    • Passing list of indices to instances to drawInstancedIndirect
  • Occlusion culling
    • OccluderGroup
      • Contains simple occlusion per block
        • Standard armor handled separately
        • Blocks having a custom occlusion geometry in model
        • No deformations
      • Contains simple occlusion per sector of voxels
      • Essentially a triangle mesh
      • Managed per grid or per block of voxels
  • Optimize shaders
  • Number of lights in world
    • Find out their owner and their purpose
    • Check
      • Medieval planet
      • Space planet
      • Space empty scene

Appendix B - Performance analysis

Setup: CPU i5 3.2GHz, 16G RAM, nVidia GTX 750 Ti

Appendix C - Articles

Bottlenecks of constant buffer access

Texture update costs

Direct3D11 Deferred Contexts

Direct3D11 Optimization guide

Hierarchical Z-Buffer

Voxel occlusion


Thanks for reading!

Marek Rosa
CEO and Founder, Keen Software House
CEO, CTO and Founder, GoodAI


Space Engineers on Facebook: https://www.facebook.com/SpaceEngineers
Space Engineers on Twitter: https://twitter.com/SpaceEngineersG
Medieval Engineers on Facebook: https://www.facebook.com/MedievalEngineers
Medieval Engineers on Twitter: https://twitter.com/MedievalEng