Kohi Engine Roadmap

The items in this list are not in any particular order. This list will be updated occasionally as development progresses.

Future Releases:

0.11.0 Release

Remove rendergraph - have application drive render process directly.
Rework xform system:
- Rename xform->ktransform
- Change to use typedefed u32s instead of khandles.
Rework hierarchy graph to use typedefed u32s instead of khandles.
Rework audio system to use typedefed u16s instead of khandles.
Rework timeline system to use typedefed u16s instead of khandles.
Rework renderer backend to use u16 indices for internal texture ids and samplers instead of khandles.
Rework Material system to replace khandles with typedefed u16s.
- Renamed material_system -> kmaterial_system, along with all associated types.
Rework Shader system to replace khandles with typedefed u16s.
Move renderer code out of kmaterial_system and into a kmaterial_renderer (or maybe just move to the bespoke renderer?)
Move scene to game code, game-specific.
- Change scenes to use single level of 'inheritance', i.e. have a base_entity that has props, then a static_mesh_entity that has base_entity base member.
- Remove all references to scenes in engine core/runtime.
Refactor Testbed project for the above changes.
Remove viewports, enhance camera system to include this instead.
Change to use multiple vertex buffers instead of a single one to handle extra data (i.e. bone ids/weights, terrain material weights, etc. should be in a second buffer). Could then use the same pipeline for static/animated geometry by binding a 'dummy' vertex binding for binding 1 and not using those attributes in the shader if there are no bone weights.
Remove legacy systems (i.e. hierarchy system, etc.)
Hoist Overdrive project to it's own repo, refactor similar to Testbed.
Bugfix: #274 Fix validation issues regarding window resize.
Bugfix: #268 Add app config renderer option for "require_discrete_gpu". Default to false and pass down to Vulkan
Bugfix: #252 Fix/ensure macOS build script works as intended.
Animation system/animated meshes.
Performance: Split back out static/animated model shaders.
Bugfix: #237 Fix/ensure Windows build script works as intended. Perhaps remove Powershell requirement?
Build: Make a setup script that creates the clangd 'compile_flags.txt' files that is independent of the build process and can be run separately when the project is first cloned.

0.12.0 Release

Bugfix: #264 Address window closing issue on i3wm.

Future Releases

Editor gizmo needs work when it comes to local translation on top of current/parent rotation Editor gizmo needs work when it comes to local translation on top of current/parent rotations.
Rework freelist to take alignment into account.
- Rework renderbuffer to take alignment in during creation, and use said alignment for allocations.
Rework Vulkan shaders to only use compiled SPIR-V resources in the plugin runtime. This means that "shaderc" would not be required in the release build of the plugin. This also means that it would need to be linked somewhere. Adding it to the utils plugin is one option, but having Vulkan-specific stuff there is strange. Could also make another plugin that just contains the importer. This makes the most sense but has the drawback of being yet another plugin that has to be distributed (although, perhaps) it could be next to/a part of the vulkan renderer plugin somehow?
Remove deprecated geometry functions in renderer.
Remove calls to deprecated darray functions.
Asset System
- Asset type reworks:
- Folders (Future)
  - Create/Destroy/Rename, etc., All happens on-disk.
  - Manifest Updates as items are moved in/out.
- Images
  - Importers
  - Importer for KTX (future)
  - Importer for PSD (future)
- Static meshes
  - OBJ Imports - make material import/generation optional (so we don't overwrite the hand-rolled materials);
  - Create a "default static mesh" (named "StaticMesh_Default") which is a cube with perhaps a warning material.
- Audio
  - Importers
  - .wav
  - .qoa (Quite OK Audio format)
Asset Packaging
- Manifest file generator (utility that looks at directory structure and auto-creates manifest.kson file from that)
- Create binary blob format (.kpackage file) and read/write.
- Point kpackage to .kpackage file for "release" builds.
Resources
- Heightmap Terrain (formerly just Terrain)
- Refactor to encode a "material index" into an unused int/float within the vertex data, and eliminate the need to have "terrain vertex data" entirely.
Audio
- Audio velocity
- Effects
- Reverb
- Low- and Hi-pass filtering
- Ability to apply effects to categories or to channels directly.
- Sound Volumes (namely, echo for caves/tunnels, with properties to adjust in scene config)
Material system
- Add a default "warning" material that stands out, to use in place of a non-existent material.
- Blended materials
- Material hot-reloading
- Shading models:
- Phong
- Unlit
- Custom (requires shader asset name OR eventually one generated by shader graph)
- Two-sided support
- Additional texture channels:
- Specular (scale specularity on non-metallic surfaces (0-1, default 0.5)) (PBR, Phong)
Rework hashtable to better handle collisions.
Custom cursor
- Hardware cursor (i.e. from the windowing system)
- Software cursor (i.e. a texture drawn at cursor pos)
- Option to set cursor image (and change on the fly)
BUG: Hierarchy graph destroy does not release xforms. Should optionally do so.
BUG: water plane - Fix frustum culling for reflections.
BUG: Kson assignment does not require '=', and just assigns anyway. Will be an issue when other operators are added.
Flag for toggling debug items on/off in scene.
Flag for toggling grid on/off in scene.
Profiling and performance pass - a few quick things to consider:
- Optional ability to set texture filtering to aniso, and level, or disable.
- Reduce draw calls
- Occlusion queries/blocking volumes
Combine duplicated platform code (such as device_pixel_ratio and callback assignments) to a general platform.c file.
billboards
particles
[?] Move descriptor pools to be global to the backend instead of one per shader.
Remove deprecated darray functions.
Material redux:
- Additional texture channels:
- Clear coat (PBR)
- Clear coat roughness. (PBR)

Engine general:

platform layer (Windows, Linux, macOS)
- UTF-8/Wide character handling for Win32 windowing.
- UTF-8/Wide character handling for Linux windowing.
- Actual Wayland support (not via XWayland)
event system
clock
testing framework
math library (vector math, etc)
Memory system
Generic sorting function/library.
Allocators:
- linear allocator
- dynamic allocator (variable-size allocations)
- pool allocator
Systems manager
Resource system
Resource Loaders:
- binary
- text
- image
- material
- bitmap font
- system font
- scene
mobile runtime support (Android, iOS)
SIMD
Containers:
- stack
- hashtable
- freelist
- dynamic arrays
- static-sized array (with iterator)
- ring buffer
- queue
- pool
- u64_bst
- registry
quadtrees/octrees
Threads
Flag to force single-threaded mode.
Semaphores
Job system
- Job dependencies
- Job semaphores/signaling
ThreadPools
Multi-threaded logger
Textures
- binary file format (.kbi)
Renderable (writeable) textures
Static geometry
Materials
Input:
- desktop
- touch
- gamepad
- keymaps/keybindings
- I18n / keyboard layouts
Conosole
- Console consumer interface
- Logger as consumer
- Debug console as consumer
- kvars (console variables)
- Console commands
Application-level configuration
Strings
- kname string hashing
- compile-time hashing (Requires either custom preprocessor or using C++ compiler)
- high-level string structure library (not c-strings)
- I18n strings
resource hot reloading
prefabs
Custom storage format (KSON - Kohi Storage Object Notation)
- Tokenizer/parser
- ToString/FromString API
Scenes
- Base implementation
- Load from file
- Adjustable global scene properties
- Save to file
Renderer System (front-end/backend plugin architecture)
Audio System (front-end)
Physics System (front-end)
networking
profiling
timeline system
skeletal animation system
skybox
skysphere (i.e dynamic day/night cycles)
water plane
Raycasting
Object picking
- Pixel-perfect picking
- Raycast picking
Gizmo (in-world object manipulation)
Viewports
terrain
- binary format (.kbt extension, Kohi Binary Terrain) - .kht imports to this.
- heightmap-based (.kht extension, Kohi Heightmap Terrain)
- voxel-based (.kvt extension, Kohi Voxel Terrain)
- smooth voxels
- pixel picking
- raycast picking
- chunking/culling
- BUG: culling is currently passing all chunks always.
- LOD
- Blending between LOD levels (geometry skirts vs gap-filling, etc.)
- tessellation
- holes
- collision
volumes
- visibility/occlusion
- triggers
- physics volumes
- weather
Multi-window applications
Asset packaging system, including package build process. https://excalidraw.com/#json=5krkRPmGHvqoYkufVE_ED,ujzx6tqRDUn63DzjraQ_jw
- Assets specific to rutime or plugins should be provided at that level to the package build process.
- Would provide an interface to the engine, and the implementation could either load from disk or binary blob.
For release builds, compile shaders to bytecode/SPIR-V and place into package binary.
Custom types capability for asset system.

Renderer:

geometry generation (2d and 3d, e.g. cube, cylinder, etc.)
advanced materials (WIP)
PBR Lighting model
batch rendering (2d and 3d)
instanced rendering
Per-scene vertex/index buffers
Queue-up of data uploads during scene load:
- Notes/ steps involved:
- Setup a queue in the scene to hold pending mesh data.
- For each mesh:
  - Make sure mesh is invalidated so it doesn't attempt to render.
  - Assign a unique id for it and add it to the queue
  - Load it from disk (multithreaded, currently done but needs some changes). Save off id, size, data, offsets, etc.
  - Reserve space in buffer freelist but don't upload to GPU. Save the offset in the queue as well.
  - NOTE: that this could be used to figure out how large the buffer needs to be.
- Repeat this for all meshes.
- In one swoop, upload all vertex and index data to GPU at once, perhaps on a separate (transfer) queue to avoid frame drops.
  - Probably the easiest way is a single vkCmdCopyBuffer call with multiple regions (1 per object), to a STAGING buffer.
  - Once this completes, perform a copy from staging to the appropriate vertex/index buffer at the beginning of the next available frame.
- After the above completes (may need to setup a fence), validate meshes all at once, enabling rendering.
shadow maps
- PCF
- cascading shadow maps
- Adjustable Directional Light properties
- max shadow distance/fade (200/25)
- cascade split multiplier (0.91)
- shadow mode (soft/hard shadows/none)
- Percentage Closer Soft Shadows (PCSS)
- Point light shadows
texture mipmapping
Specular maps (NOTE: removed in favour of PBR)
Normal maps
Phong Lighting model (NOTE: removed in favour of PBR)
Multiple/configurable renderpass support.
Forward rendering
Deferred rendering
Forward+ rendering
Compute Shader support (frontend)

Plugins:

ECS (Entity Component System)
Audio (OpenAL plugin)
Vulkan Renderer Plugin (WIP)
- Decouple renderpass from shader/pipeline and framebuffer/rendertargets
- multithreading
- texture data upload
- mesh data upload
- pipeline statistic querying
- compute support
Direct3D Renderer Plugin
- multithreading
Metal Renderer Plugin
OpenGL Renderer Plugin
Headless Renderer Plugin

Standard UI: (separated section because number of controls)

Standard UI system
Layering
UI file format
Load/Save UIs
UI Editor (as a plugin to the editor)
control focus (tab order?)
docking
drag and drop support
UI Controls (one of the few engine-level areas that uses OOP):
- Base control - all controls come from this
- panel
- image box
- viewport control (world/scenes will be switched to use these as render targets)
- rich text control (system text w/ multicolour, bold/italic, etc. and bitmap text with multicolour only)
- button
- checkbox
- radio buttons
- tabs
- windows/modals (complete with resize, min/max/restore, open/close, etc.)
- resizable multi-panels
- scrollbar
- scroll container
- textbox/textarea
- In-game debug console

Editor

Editor application and 'runtime' executable
- World editor
- UI editor
- editor logic library (dll/.so) hot reload
Move .obj, .mtl import logic to editor (output to binary .ksm format).
Move texture import logic to editor (output to binary .kbt format).
DDS/KTX texture format imports
FBX model imports

Other items:

Split off "core" items (defines, memory, strings, containers, etc.) into a "core" or "foundation" library so they may be used without having to pull in all of the engine.
Split off platform layers into separate libraries outside the engine.
Auto-Generated API documentation
Documentation
Continuous Integration
Add git tags to mark version releases (https://github.com/travisvroman/kohi/issues/174)
Nix build compatability (https://github.com/travisvroman/kohi/issues/175)
Scripting language - Kohi Language (KLang) - Thanks for the name idea Dr. Elliot. :)

Previous Release TODO lists:

0.7.0 Release

0.7 Reorg
- Split engine into "core" and "runtime"
- Rename plugin libs to consistent format, update builds, etc.
- Remove systems manager, and move all system init back to engine.
- External systems registry
- Plugin System implementation
- kruntime_plugin structure
- Convert Vulkan Plugin to kruntime_plugin
- Convert OpenAL Plugin to kruntime_plugin
- Convert Standard UI Plugin to kruntime_plugin
- Implement windowing logic in platform layer.
- Implement windowing logic in renderer front/backend
- Ability to deprecate code (mark as deprecated/warn).
- Deprecate "geometry" interface points in renderer frontend/
- Move loading of application lib from app to engine (config-powered!)
- frame/swapchain image count refactor:
- Move anything that's aware of double/triple/5x buffering exclusively to the renderer backend. Nothing outside that should know or care about it.
- Refactor renderer texture API to pass a new TEXTURE_FLAG_RENDERER_BUFFERING flag used to indicate to the backend that resources should be created "per-frame" if this is set (i.e. 3 internal textures for triple-buffering)
- Adjust render target logic to defer to the backend when reaching for a texture handle to use for the above case.
- Renderpasses should no longer own render targets.
- Framebuffers (i.e. render targets) should no longer require a renderpass to create. (i.e. use a dummy pass)
- Shaders (graphics pipelines) should no longer require a renderpass to create. (i.e. use a dummy pass)
0.7 Scene refactor (see notes below):
Rename simple scene to just "scene" and move to engine core.
Create a unique-per-system handle for each system to identify a resource. These handles would be linked to a resource array of some sort and an index element within that array via a structure that holds both.
Create new "xform" structure and system that uses handles and can manage dependencies in updates internally. NOTE: This system should be laid out in a data-oriented way.
Create hierarchy graph that handles transform hierarchy and can provide a view of it. Also generating world matrices.
Remove transform from mesh.
Replace any and all transforms with xform handles.
Update systems (and create some) that use handles:
- Create xform system that uses handles
x Refactor scene loader to a version 2 that is more expressive and allows "{}" syntax to nest objects.
[x] Write "(de)serialization" routines for savable resources and use those in the above loader. Scene Refactor notes: Refactor into node-based system using handles for various types. A node should contain 3 (maybe 4) things: a unique identifier, a handle id (which is a link into a scene-wide handle table, which itself points to an index into an array of resources), a potential parent handle id (which can be INVALID*ID if unused), and potentially a name. There would then be lists of resource types (think mesh, terrain, lights, skybox, etc) which would each have lookup tables of handle ids to indices into these arrays. Additionally there would be a set of a lookup table and transforms that would be used. Separating these would allow updates on these objects in cache-coherent loops as well as any sorting/dependency lookup that would need to be done. The above will require that meshes have transforms removed from them. The transform would then be also referenced by the *node* instead of the mesh. This would also facilitate batching of like meshes for rendering in the future. Transforms would also have the parent pointer removed, and instead also use handles. This would eliminate issues with invalid pointers when an array of a resource (i.e. transform) is expanded and realloced. This should be done in a phased approach, and thus perhaps a new "xform" should be created, and the "transform" structure could be deprecated. Note that the resource lookup for this would likely be global, not just within a scene.
```
We could then have a few different "graphs" in the scene: one for transforms, one for visibility (i.e a flat
array of currently-visible objects would likely suffice here), and others.

We might also think about, at this point, reworking the scene parser to better handle object heirarchy in a more
expressive language fasion (perhaps using some sort of scoping syntax like "{}" to surround objects).
```
[x] Remove specialized rendergraphs. Replaced by app config.
[x] Separate debug shapes out to new debug_rendergraph_node.
[x] Separate editor gizmo out to new editor_gizmo_rendergraph_node.

0.8.0 Release

KName system (case-insensitve string hashing)
KString Id system (case-sensitive string hashing)
Virtual file system (VFS)
- Sits on top of and manages packages, doles out requests to loaded packages, etc.
[x] Remove geometry system
- Rename geometry->kgeometry
- Move creation of geometries to geometry.h/c
- Add geometry_type:
- 2D_STATIC - Used for 2d geometry that doesn't change
- 2D_DYNAMIC - Used for 2d geometry that changes often (NOTE: initially not supported)
- 3D_STATIC - Used for 3d geometry that doesn't change
- 3D_HEIGHTMAP_TERRAIN - Used for heightmap terrain-specific geometry that rarely (if ever) changes - includes material index/weight data
- 3D_SKINNED - Used for skinned 3d geometry that changes potentially every frame, and includes bone/weight data
- 3D_DYNAMIC - Used for 3d geometry that changes often (NOTE: initially not supported)
- CUSTOM - User-defined geometry type. Vertex/index size will only be looked at for this type.
[x] New Resource System
- Remove old resource system after it is decomissioned.
- New replacement Resource System will not only replace old system but also all resource types within the engine to standardize resource handling. New system will make requests to new Asset System asynchronously, and be responsible for all reference counting and auto-releasing.
- Provide kresource structure which contains basic high-level items such as type, name, generation, etc.
- Resource type reworks
- Text (Simple, generic loading of a text file)
  - Handler
- Binary (Simple loading of all bytes in a file)
  - Handler
- Textures
  - Handler - Does this also include GPU uploads? Thinking so, versus a different "gpu loader"
- Material
  - Handler
  - Conversion of material .kmt files to version 3.
  - Material instances
  - Standard materials
  - Water materials
  - Water plane: Add various properties to configuration.
- Shader
  - Handler
  - Conversion of scopes to update_frequency (global/instance/local -> per_frame/per_group/per_draw)
- Scene
  - Handler
- Static Mesh (formerly just Mesh)
  - Handler
  - kresource_static_mesh structure
  - holds (and owns) static_geometry structure
- Bitmap Font
  - Resource Handler
- System Font
  - Resource Handler
- Audio Resource
- Resource Handler
[x] Split asset loading logic from Resource System to new Asset System
- New asset system will work with asynchronous nature of VFS and automatically handle asset parsing/processing, hot-reloading and interaction with systems where necessary (i.e. texture system).
- Provide kasset structure which contains basic high-level metadata such as name, type, size and pointer to data, as well as a general "asset state" that can be looked at by any system (i.e. Uninitialized->Initialized->Loading->Loaded->Unloading)
- When an asset is requested, a name and callback are provided. The name and a asset-type-specific handler callback are provided to the VFS, which responds in kind when the asset is loaded. In the event more than one VFS call is required, the asset handler will be responsible for managing this and will ultimately invoke the original callback provided when the top-level asset was requested.
[x] Asset packaging (kpackage)
- Reorganize assets from testbed.assets folder to go along with the respective "module".
- Rename all assets and asset references to use the format "..". I.e. "Testbed.Texture.arch"
- Setup one manifest file including a list of all these files in each "module.". Exclude "source" files (i.e. .obj and .mtl).
- Asset type reworks:
- Static meshes
  - Regenerate all .ksm files.
  - Asset handler
  - Importer from Wavefront OBJ
  - OBJ Serializer
  - Serializer
  - Rename all references to "mesh" in the engine to "static_mesh" to separate it from later mesh types.
- Images
  - Asset handler
  - Importer
  - Common formats (.bmp, .tga, .jpg, .png)
  - Binary .kbi format
  - Serializer
- Shaders
  - Fix hot-reloading/change watches to be called from package/vfs
  - Convert .shadercfg file to KSON-based .ksc (Kohi Shader Config)
  - Asset handler
  - Serializer
- Bitmap fonts
  - Rename .fnt files to .kbf (Kohi Bitmap Font)
  - Asset handler
  - Serializer
- System fonts
  - Convert .fontcfg to KSON-based .ksf file (Kohi System Font)
  - Asset handler
  - Primary format Serializer
- Materials
  - Convert .kmt to KSON
  - MTL Serializer
  - Asset handler
  - Serializer
- Terrains -> HeightmapTerrains
  - Convert .kterrain to KSON-based .kht file (Kohi Heightmap Terrain)
  - Asset handler
  - Serializer
- Scenes
  - Asset handler
  - Serializer
- Music/Audio effects
  - Binary container format (.kaf - Kohi Audio File)
  - Asset handler
  - Importers
  - mp3
  - ogg vorbis
  - Serializer
- Create kpackage interface in kohi.core.
- Point kpackage to files on disk for "debug" builds.
- Asset hot reloading
- [x] BUG: Fix release build hang on startup (creating logical device).
- [x] BUG: Fix macOS window resizing.
- [x] Fix release build hang on startup (creating logical device).
- [x] Blitting - Separate colourbuffer to its own texture separate from the swapchain/blit to swapchain image just before present.
- [x] Material system refactor
  - Have material/texture system create the "combined" image at runtime instead of it being stored as a static asset (will make it easier to change in an editor without having to run a utility to do it).
  - Convert material configs to KSON
- [x] Material redux:
  - Shading models:
  - PBR (default now)
  - Transparency flag (turning off ignores alpha on albedo). Also used for material sorting during deferred/forward passes.
  - Additional texture channels:
  - Emissive (PBR, Phong)
- [x] Static-sized, dynamically-allocated, type-safe array with iterator.
  - Tests
- [x] Static-sized, stack-allocated, type-safe array with iterator.
  - Tests
- [x] Dynamic-sized, type-safe darray with iterator.
  - Tests
- [x] Vulkan backend:
  - Support for separate image descriptors and sampler descriptors. Remove support for combined image sampler descriptors.
  - Add generic samplers usable everywhere (linear repeat, nearest repeat, linear border, linear clamp etc.)
  - Add tracking to shader system as to whether these are used.
  - Default sampler uniforms to use these.
  - Add 16 sampler/texture limitation to shader system. (see required limits doc)
  - Change samplers to use khandles
  - Bubble up sampler functions to renderer frontend.
  - Remove concept of texture maps in favour of separate samplers and images
  - Change shader system to set "texture" instead of "sampler".
  - Change shader system to no longer hold string names for uniforms, but use knames instead.
- [x] Handle refactoring
  - Convert shader system to use handles
  - Convert material system to use handles
- [ ] Audio system
  - kaudio system refactor.
  - Audio "instances"
  - Audio radius checks (think emitter vs lister pos./falloff)
  - Audio instance play position + radius vs. listener + radius indicates if currently bound/playing on a channel.
  - Auto-channel selection based on availability (gracefully handle out-of-channels i.e. discard oldest or simply don't play?)
  - Channel reservation/sound types or "categories"
- [x] Debug shape
  - debug_sphere3d (similar to the one generated for the editor gizmo)

TODO.md 28 KB

Постоянная ссылка История Исходник

Kohi Engine Roadmap

Future Releases:

0.11.0 Release

0.12.0 Release

Future Releases

Engine general:

Renderer:

Plugins:

Standard UI: (separated section because number of controls)

Editor

Other items:

Previous Release TODO lists:

0.7.0 Release

0.8.0 Release

0.10.0 Release

TODO.md 28 KB Постоянная ссылка История Исходник

Kohi Engine Roadmap

Future Releases:

0.11.0 Release

0.12.0 Release

Future Releases

Engine general:

Renderer:

Plugins:

Standard UI: (separated section because number of controls)

Editor

Other items:

Previous Release TODO lists:

0.7.0 Release

0.8.0 Release

0.10.0 Release

TODO.md 28 KB

Постоянная ссылка История Исходник