BansheeEngine/Notes.txt

----------------------------------------------------------------------------------------------
General longterm systems:   
   - Camera controls + world grid
   - Frustum culling and octree (or some other) acceleration structure
   - Render queue and sorting

----------------------------------------------------------------------------------------------
Reminders:
  - Assets manifest
     - I need to be able to create a list of assets used in the game. Assets referenced from the editor
       should be easy to account for. But assets loaded from code are special. Maybe be like Unity and allow
	   special Resources folder?
  - When displaying inspector data for a component, take into consideration that it will need to be able
    to display that data for user created C# classes as well. AND I will most certainly have C# versions of all my
	components. Therefore is there any purpose of having C++ only inspector parsing code?
  - When I'll be doing SRGB write make sure GUI textures are handled properly. Right now they are read in gamma space, and displayed
    as normal, but if I switch to SRGB write then gamma would be applied twice to those textures.
  - Make editor background have a tileable pattern (see Mini paper app on my cell). Since tileable images wouldn't work with scale9grid, it should be an overlay over the solid background.
    - Maybe use the background type I used for website in WebDIP?
    - Use black with yellow-orange highlights (highlight = selected button, selected frame border, selected entry box, etc.)
  - Async callbacks. I'd like to be able to assign a callback to an async method, that will execute on the calling thread once the async operation is complete.
     - For example when setting PixelData for a cursor I need to get PixelData from a texture, which is an async operation, in which case I need to block the calling thread
	   until I get the result. But I'd rather apply the result once render thread is finished.
  - GUI currently doesn't batch elements belonging to different GUIWidgets because each of them has its own transform. Implement some form of instancing for DX11 and GL
    so this isn't required. GUIManager already has the ability to properly group meshes, all that is needed is a shader.

Potential optimizations:
 - bulkPixelConversion is EXTREMELY poorly unoptimized. Each pixel it calls a separate method that does redudant operations every pixel.
 - UI buffer updates
   - https://developer.nvidia.com/sites/default/files/akamai/gamedev/files/gdc12/Efficient_Buffer_Management_McDonald.pdf
   - See here how to avoid locks and stalls when updating just parts of a GUI buffer. In need to modify my mesh update code because it currently always updates 100% of the buffer.
  - Maybe get rid of CPU UI buffers completely.
 - UI shader resolution params for gui should be in a separate constant buffer

 Memory allocation critical areas:
 - TextUtility a lot of allocs
 - Binding gpu params. It gets copied in DeferredRenderContext
 - GameObjectHandle often allocates its internal data
 - ResourceHandle often allocates its internal data
 - AsyncOp allocates  AsyncOpData internally
 - Deserialization, a lot of temporary allocations going on - But how much impact on performance will allocations have considering this is probably limited by disk read?
 - Creating SceneObjects and Components - I might want to pool them, as I suspect user might alloc many per frame
 - Log logMsg

----------------------------------------------------------------------------------------------
More detailed thought out system descriptions:

<<<<SceneManager/SceneObject>>>>
Two major parts:
SceneObject update:
 - Just takes care of updating world/local transforms
  - Transforms are only updated when requested
 - Marking a transform dirty will also mark it dirty in the SceneManager
 - Only SceneObjects that have an Interactable type component (Renderer, Collider, etc.) will reside in SceneManager

SceneManager maintains a list of world matrices, bounds and primitives
 - We can query them for various collision (ray, frustum, etc.)
 - Internally they likely use BVH or oct-tree
 - Uses a binary-tree (unlike SceneObject hierarchy which is n-ary), which is laid out neatly in memory for quick traversal.
   - What happens when object is removed/added? Tree keeps on growing, empty nodes have their space always reserved. Possibly add a method that can reduce tree size when enough empty nodes exist. 
     Adding a node might increase tree size which will involve a memcpy while we increase the size.
 - (Updating an object is SceneManager should optionally transfer its world matrix to its owner object as well)

This separation should work fine, as scripts requesting transforms is unlikely to be something that done often. Or not nearly as much as frustum culling and raycasting will be.

How will Physics update objects (and when?)
 - It's unlikely there will be a massive amount of rigidbodies in the scene, so updating them should not be a huge matter of performance. 
   Calling setTransform after physics simulation update (and before SceneManager update) should work fine.

Updating children dirty whenever setPos/rot/tfrm is called is potentially slow. Can it be avoided?
 - Then again such updates will only be done from the simulation thread, usually from scripts so its unlikely they will be many of them

Do I separate SceneObject and Transform?
 - I don' think I need to

Make sure to let the user know SceneManager only gets updated after simulation, so changes to objects wont be applied right away. 
(See that he doesn't transform something and call Raycast just so it fails)
 - It is unlikely functionality when query results are needed right after transform will be used much. 
   So it is acceptable to implement it like this. We might add SceneManager::forceUpdate method in case it is not acceptable.

!!!BUG!!! - When I change parent I don't update individual local position/rotation/scale on scene object
 - Also I don't have a way of setting world pos/rot directly

<<<<Reducing render state changes>>>>
 - Transparent objects get sorted back to front, always
 - Opaque objects I can choose between front to back, no sort or back to front
 - Then sort based on material-pass combo, rendering all passes of the same material at once, then moving to next pass, then to next material, etc.
   - For transucent objects I need to render entire material at once, and not group by pass
   - Ignore individual state and textures changes, just sort based on material
 - Use key-based approach as described here: http://realtimecollisiondetection.net/blog/?p=86

Questions/Notes:
 1. Could I make use of multiple texture slots so I don't have to re-assign textures for every material when rendering translucent objects pass by pass?
   - When sorting back to front (or front to back) it's highly unlikely that there will be many objects sharing the same material next to the same depth level anyway. 
     So probably ignore this problem for now, and just change the states.

 2. Should sorting be done on main or render thread?
   - Main thread. It's highly unlikely main thread will be using as much CPU as render thread will, so free up render thread as much as possible.

 3. Should oct-tree queries be done on main or render thread?
   - Main thread as I would need to save and copy the state of the entire scene, in order to pass it to the render thread. Otherwise we risk race conditions.

 4. Since render state and shader changes are much more expensive than shader constant/buffer/mesh (and even texture) changes, it might be a good idea to sort based on these,
    instead of exact material? A lot of materials might share shaders and render states but not textures.

 5. This guy: http://home.comcast.net/~tom_forsyth/blog.wiki.html#%5B%5BRenderstate%20change%20costs%5D%5D (who's a driver programmer) sorts all opaque objects based on shader/state
    into buckets, and then orders elements in these bucks in front to back order. This gives him best of two worlds, early z rejection and low state changes.

<<<<Input System>>>>
 - Input is currently ignoring all axes except for mouse axes
 - Remove/Improve smoothing
 - Add ability to get raw or smoothed axis input for any axis (currently you can only get mouse X and Y axes)
 - Allow the user to map axes to custom keys. e.g. Left/right axis can have A and D keys where A returns -1, and D 1
 - Add a way to handle multiple devices (e.g. 2 or more joysticks)
 - Hook up OIS joystick callbacks and test joysticks

<<<<DirectDraw>>>>
 - Used for quickly drawing something, usually for debug and editor purposes.
 - It consists of methods like: DrawLine, DrawPolygon, DrawCube, DrawSphere, etc.
 - It may also contain other fancier methods like DrawWireframeMesh, DrawWorldGrid etc.
 - Commands get queued from various Component::update methods and get executed at the end of frame. After they're executed they are cleared and need to be re-queued next frame.
 - Internally DirectDraw manages dynamic meshes so it can merge multiple DrawLine class into one and such. This can help performance, but generally performance of this class should not be a major concern.
 - Example uses for it: 
    - Drawing GUI element bounds when debugging GUI
    - Drawing a wireframe selection effect when a mesh is selected in the scene


<<<<Multithreaded memory allocator>>>>
 - Singlethreaded implementation from Game Engine Gems 2 book
 - Small and medium allocators separate for each thread. Memory overhead should be minimal considering how small the pages are. But performance benefits are great.
 - Large allocator just uses some simple form of page allocation and reuse, using atomics?
 - Must ensure that memory allocated on one thread can only be freed from that thread
 - How do I easily tell which allocator to call based on current thread? Require a thread ID with each alloc/dealloc?
 - Need to think this through more
 
<<<<More on memory allocator>>>>
 - Regarding potentially often allocating large amounts of memory:
  - Ignore this for now. Allocating large amounts (16K+ of memory often probably won't be the case). This will only happen when modifying textures or meshes and I can assume there won't be many of such updates.
    - (But there will be multiple such updates per frame when it comes to GUI meshes for example)
  - However I should implement allocation counter in my allocator so I can know if I have a bottleneck.
  - For those allocations that do hit this limit I should implement a FrameAllocator. Memory is allocated during simulation step and the entire block is cleared when the frame ends.
   - Allocations like copying MeshData, PixelData, PassParams, etc. when queing commands for render thread should all be using this.
   - Problem with such allocator is safety
  - Allocations that are created and deleted in a single function should use a Stack allocator

<<<<Resource changes and reimport>>>>
Use case example:
 - User reimports a .gpuproginc file
   - Dependencies are held by CoreGpuObjectManager. Objects are required to register/unregister
      their dependencies in their methods manually.
 - Editor calls SomeClass::reimport(handle) after it detects a change 
   - this will load a new resource, release the old one and assign the new one to Handle without changing the GUID
   - In order to make it thread safe force all threads to finish what they're doing and pause them until the switch is done
     - Should be okay considering this should only happen in edit-mode so performance isn't imperative
   - reimport is recursively called on all dependant objects as well.

<<<<Handle multithreaded object management>>>:
 - Make everything that is possible immutable. Once created it cant be changed.
  - Example are shaders, state objects and similar
 - Things like Textures, Vertex, Index buffers, GpuParams may be changed
  - Make Vertex/Index buffers and similar only accesible from render thread. Higher level classes like meshes can have deferred methods
  - TODO - How to handle the remaining actually deferred methods? Like Textures?

DirectX11 supports concurrent drawing and resource creation so all my resource updates should be direct calls to DX methods (I'll need a deferred context?)
 - DX9 doesn't so creating/updating resources should wait for render thread?
  - Although these are sync points which kill the whole concept of separate render thread
  - Updating via copy then? (DX11 driver does it internally if resource is used anyway)
 - OpenGL? No idea, need to study GL contexts
 - Although it seems DX11 also copies data when mapping/unmapping or updating on a non-immediate context. So maybe copy is the solution?

So final solution:
 - Copy all data that will be updated on a deferred context
  - Make deferred context have a scratch buffer it can use for storing temporary copied data
 - Immediate context will execute all commands right away
  - This applies when rendering thread calls resource create/update internally
  - Or when other thread blocks and waits for rendering thread
 - Create a simple distinction so user knows when is something executed deferred and when immediate?
  - Move resource update/create methods to DeferredContext?
    - Not ALL methods need to be moved, only those that are resource heavy
    - Smaller methods may remain and always stay async, but keep internal state?
 - Resource creation on DX11 should be direct though, without a queue (especially if we manage to populate a resource in the same step)
 - Remove & replace internal data copying in GpuParamBlock (or just use a allocator instead of new())

A POSSIBLY BETTER SOLUTION THAN COPYING ALL THE DATA?
Classes derive from ISharedMemoryBuffer
 - For example PixelData, used when setting texture pixels
 - They have lock, unlock & clone methods
  - Users can choose whether they want to lock themselves out from modifying the class, or clone it, before passing it to a threaded method
 - Downside is that I need to do this for every class that will be used in threaded methods
 - Upside is that I think that is how DX handles its buffers at the moment


<<<<RenderSystem needed modifications>>>>
  - Texture resource views (Specifying just a subresource of a texture as a shader parameter)
  - UAV for textures
  - Stream out (write vertex buffers) (DX11 and GL)
  - Texture buffers 
   - Just add a special texture type? OpenGL doesn't support getting offset from within a texture buffer anyway
  - Tesselation (hull/domain) shader
  - Detachable and readable depthstencil buffer (Window buffers not required as they behave a bit differently in OpenGL)
  - OpenGL provides image load/store which seems to be GL UAV equivalent (http://www.opengl.org/wiki/Image_Load_Store)
  - Resolving MSAA textures (i.e. copying them to non-MSAA so they can be displayed on-screen). DX has ResolveSubresource, and OpenGL might have something similar.
  - Single and dual channel textures (especially render textures, which are very important for effects like SSAO)
  - Compute pipeline
  - Instancing (DrawInstanced) (DX11 and GL)
  - OpenGL append/consume buffers
  - Indirect drawing via indirect argument buffers
  - Texture arrays
  - Rendertargets that aren't just 2D (Volumetric (3D) render targets in particular)
  - Shader support for doubles
  - Dynamic shader linkage (Interfaces and similar)
  - Multisampled texture resources
  - Multiple adapters (multi gpu)
  - Passing initial data when creating a resource (DX11, but possibly GL too)
  - Sample mask when setting blend state (DX11, check if equivalent exists in GL)
  - RGBA blend factor when setting blend state(DX11, check if equivalent exists in GL)
  - HLSL9/HLSL11/GLSL/Cg shaders need preprocessor defines & includes
  - One camera -> one task (thread) approach for multithreading
   - Also make sure to run off a thread pool (WorkQueue class already exists that provides needed interface)
  - The way I handle rendering currently is to discard simulation results if gpu thread isn't finished.
     - This reduces input lag but at worst case scenario the effect of multithreading might be completely eliminated as
	   GPU ends up waiting for GPU, just because it was few milliseconds late. Maybe better to wait for GPU?