BansheeEngine/Notes.txt

Reminders:
  - When I'll be doing SRGB write make sure GUI textures are handled properly. Right now they are read in gamma space, and displayed
    as normal, but if I switch to SRGB write then gamma would be applied twice to those textures.
  - Async callbacks. I'd like to be able to assign a callback to an async method, that will execute on the calling thread once the async operation is complete.
     - For example when setting PixelData for a cursor I need to get PixelData from a texture, which is an async operation, in which case I need to block the calling thread
	   until I get the result. But I'd rather apply the result once render thread is finished.
  - GUI currently doesn't batch elements belonging to different GUIWidgets because each of them has its own transform. Implement some form of instancing for DX11 and GL
    so this isn't required. GUIManager already has the ability to properly group meshes, all that is needed is a shader.
  - When specifying GUIElement layout using (for example) GUILayoutOptions::expandableX then it would be useful if I didn't have to provide the Y height, and instead make it use 
    the default value for that GUIElement type. For most elements I will only be changing width, and height will remain default (labels, buttons, toggles, drop down lists, etc.) 
	so this would be helpful in a way I wouldn't need to look up actual element height in EngineGUI.
  - GUI currently ignores tooltips in GUIContent. 
  - A way to initialize BansheeEngine without RenderSystem or any kind of UI. So that it may be used for server building as well.
  - Calls to "initialize()" should be protected. For example GUIWidget and all its derived classes require the user to separately call initialize() after construction.
    However a much better option would be to wrap construction and initialization in a create() method and make initialize protected.
	  - GUIWidget needs to be added using addComponent which makes "create()" method not practical. However I have trouble seeing the need for initialize(),
	    it could be replaced with a variable parameter version of addComponent.
  - Profiling: Create an easy to browse list of all loaded Resources (similar to Hierarchy in Unity, just for loaded resources)
    - Possibly also for all core objects
  - GUI ignores image in GUIContent for most elements
  - Each view (i.e. camera) of the scene should be put into its own thread
  - How do I handle multiple mesh formats? Some files need animation, other don't. Some would mabye like to use QTangent, others the proper tangent frame.
    - Asset postprocessor? Imports a regular mesh using normal importers and then postprocesses it into a specialized format?
  - Load texture mips separately so we can unload HQ textures from far away objects (like UE3)
  - Add Unified shader so I can easily switch between HLSL and GLSL shaders (they need same parameters usually, just different code)
    - UE4 has GLSL/HLSL shader cross compiler, so something similar
  - Remove HardwarePixelBuffer (DX11 doesn't use it, and DX9 and OpenGL textures can be rewritten so they have its methods internally)
  - Make sure my Log system uses XML + HTML
  - There is an issue that custom-UIs won't have their mesh shared. For example most game UIs will be advanced and will 
   likely use on GUIWidget per element. However currently I only perform batching within a single widget which 
   doesn't help in the mentioned case.
  - Input: Allow combinations like A+B+X on joystick to be a virtual key
  - Add a field that tracks % of resource deserialization in BinarySerializer
  - Add GL Texture buffers (They're equivalent to DX11 buffers) - http://www.opengl.org/wiki/Buffer_Texture
  - I should consider creating two special Mesh types:
     StreamMesh - constantly updated by CPU and read by GPU
     ReadMesh - written by GPU and easily read by CPU
	  - OpenGL especially has no good way of reading or streaming data. It has special STREAM and COPY buffer types which I never use.
  - (EXTREMELY LOW PRIORITY) Scripting: It might be good to make Mono classes more generic and move them to BansheeEngine. 
      e.g. MonoClass -> ScriptClass, where ScriptClass is just an abstract interface. Then I don't expose any Mono stuff to actually script libraries like 
	  SBansheeEngine. User could then fairly easily port the system to another scripting language just by implementing another ScriptSystem. 
    - This would probably come with an overhead of at least one extra function call for each script call, which is currently unacceptable 
	  considering that most people will definitely won't be writing new script systems.
  - Perhaps add code generation functionality to the engine through Mono? I know Mono has support for it though its Embed interface
  - Add instancing to engine: All I really need to add are per-instance attributes in VertexData (and MeshData). And then RenderSystem::renderInstance method that also accepts an instance count.
  - Debug console
    - Add ability to add colors tags like <color=#123>
	- When showing a debug message, also provide a (clickable?) reference to Component it was triggered on (if applicable)
	  - It really helps when you get an error on a Component that hundreds of SceneObjects use
	- When displaying an error with a callstack, make each line of the callstack clickable where it opens the external editor
  - std::function allocates memory but I have no got way of using custom allocators as I'd have to wrap std::bind and that seems non-trivial
  - Add a TaskScheduler profiler that neatly shows time slices of each task and on which thread they are run on
  - Add support for BC6H and BC7 file format compression
  - D-Pad on non-XInput devices will not be handled properly by the Input system because OIS reports it separately from other buttons (Simple to fix)
  - Possible improvement: I can only update entire Mesh at once with writeSubresource
  - Possible improvement: I keep bounds for the entire mesh and not per-submesh
  - Possible improvement: I don't serialize Mesh bounds and they are recalculated whenever a mesh is loaded
  - Add better mip map and compression options to the texture importer. Right now I'm ignoring a lot of the options even though I support them.
  - Add separable pass sorting to RenderQueue
  - DDS file import
  - Make hierarchical documentation. Organize stuff based on type. Once I actually generate the documentation add Doxygen grouping tags (or whatever they're called)
  - Make a Getting Started guide, along with the example project. Or just finish up the manual.
  - I'm not too happy with HStrings using so many events. They use one internally which I think I can replace completely quite easily. And one externally for notifying GUI components, replacing
    which would require more thought.
  - GUIElementBase::_getElementAreas is currently only implemented for layouts. It will work of child elements of layouts and layouts themselves but will not 
    work for child elements of custom element types. This method is used for calculating size and position of an element in its parent.
  - I shouldn't use WeakRef with GameObjects. They need be deserialized in the order of their hierarchy and weak ref can break that.
  - Make sure to add fixedUpdate() to run your game logic in. This should have an adjustable update rate. See: http://gameprogrammingpatterns.com/game-loop.html
  - Will also need GameObject::destroy and GameObject::destroyImmediate. So I can remove GameObjects that might still be referenced that same frame (destroy() would just queue for destruction)
  - When unloading unused resources I should do it recursively so it unloads any new ones that might have been released during the first unload. 
    (e.g. unloading a sprite texture will probably release its atlas texture as well)
  - Make importer multithreaded. I can add a flag to SpecificImporter that tells me if a certain importer supports multiple threads or not, but even if it doesn't I can run 
    different importers on different threads this way. And once I hook up progress dialog box, perhaps make it have multiple progress bars in a single window (per thread).
  - Add FolderManager, extensible C# script that can be attached to a folder (can be in a folder .meta) file. Handles processing of assets in that folder. Can get notified 
    whenever anything in the folder changes. But is that necessary if I can have asset post- and pre- processor that can filter by folder programatically?


Potential optimizations:
 - bulkPixelConversion is EXTREMELY poorly unoptimized. Each pixel it calls a separate method that does redudant operations every pixel.
 - UI shader resolution params for gui should be in a separate constant buffer

----------------------------------------------------------------------------------------------
More detailed thought out system descriptions:

 <<<<Memory allocation critical areas>>>>
 - Binding gpu params. It gets copied in DeferredRenderContext
 - GameObjectHandle often allocates its internal data
 - ResourceHandle often allocates its internal data
 - AsyncOp allocates  AsyncOpData internally
 - Deserialization, a lot of temporary allocations going on - But how much impact on performance will allocations have considering this is probably limited by disk read?
 - Creating SceneObjects and Components - I might want to pool them, as I suspect user might alloc many per frame
 - Log logMsg

<<Multithreaded GUI rendering>>
 - Event handling and normal "update" will still be done on the main thread
 - At the beginning of each frame a GUI mesh update is queued on the GUI thread
 - Since we're queuing the update at the beggining of the frame we will be using last frames transform and gui element states.
   - When queing we need to make sure to store GUIWidget transform, and specific element states (e.g. "text" in GUILabel)
 - At the end of simulation frame wait until GUI update is complete. After both simulation and GUI updates are complete, proceed with submitting it to render system.

<<Figure out how to store texture references in a font>>
 - Currently I store a copy of the textures but how do I automatically update the font if they change?
 - Flesh out the dependencies system?
 - I can import texture as normal, and keep it as an actual TextureHandle, only keep it hidden
   if it was created automatically (by FontImporter) for example?
    - But then who deletes the texture?
	- Set up an "internalResource" system where resources hold references to each other and also release them?
	 - In inspector they can be expanded as children of the main resource, but cannot be directly modified?
	 - Deleting the main resource deletes the children too

<<<<Reducing render state changes>>>>
 - Transparent objects get sorted back to front, always
 - Opaque objects I can choose between front to back, no sort or back to front
 - Then sort based on material-pass combo, rendering all passes of the same material at once, then moving to next pass, then to next material, etc.
   - For transucent objects I need to render entire material at once, and not group by pass
   - Ignore individual state and textures changes, just sort based on material
 - Use key-based approach as described here: http://realtimecollisiondetection.net/blog/?p=86

Questions/Notes:
 1. Could I make use of multiple texture slots so I don't have to re-assign textures for every material when rendering translucent objects pass by pass?
   - When sorting back to front (or front to back) it's highly unlikely that there will be many objects sharing the same material next to the same depth level anyway. 
     So probably ignore this problem for now, and just change the states.

 2. Should sorting be done on main or render thread?
   - Main thread. It's highly unlikely main thread will be using as much CPU as render thread will, so free up render thread as much as possible.

 3. Should oct-tree queries be done on main or render thread?
   - Main thread as I would need to save and copy the state of the entire scene, in order to pass it to the render thread. Otherwise we risk race conditions.

 4. Since render state and shader changes are much more expensive than shader constant/buffer/mesh (and even texture) changes, it might be a good idea to sort based on these,
    instead of exact material? A lot of materials might share shaders and render states but not textures.

 5. This guy: http://home.comcast.net/~tom_forsyth/blog.wiki.html#%5B%5BRenderstate%20change%20costs%5D%5D (who's a driver programmer) sorts all opaque objects based on shader/state
    into buckets, and then orders elements in these bucks in front to back order. This gives him best of two worlds, early z rejection and low state changes.

<<<<DirectDraw>>>>
 - Used for quickly drawing something, usually for debug and editor purposes.
 - It consists of methods like: DrawLine, DrawPolygon, DrawCube, DrawSphere, etc.
 - It may also contain other fancier methods like DrawWireframeMesh, DrawWorldGrid etc.
 - Commands get queued from various Component::update methods and get executed at the end of frame. After they're executed they are cleared and need to be re-queued next frame.
 - Internally DirectDraw manages dynamic meshes so it can merge multiple DrawLine class into one and such. This can help performance, but generally performance of this class should not be a major concern.
 - Example uses for it: 
    - Drawing GUI element bounds when debugging GUI
    - Drawing a wireframe selection effect when a mesh is selected in the scene

<<<<RenderSystem needed modifications>>>>
  - Texture resource views (Specifying just a subresource of a texture as a shader parameter)
  - UAV for textures
  - Stream out (write vertex buffers) (DX11 and GL)
  - Texture buffers 
   - Just add a special texture type? OpenGL doesn't support getting offset from within a texture buffer anyway
  - Tesselation (hull/domain) shader
  - Detachable and readable depthstencil buffer (Window buffers not required as they behave a bit differently in OpenGL)
  - OpenGL provides image load/store which seems to be GL UAV equivalent (http://www.opengl.org/wiki/Image_Load_Store)
  - Resolving MSAA textures (i.e. copying them to non-MSAA so they can be displayed on-screen). DX has ResolveSubresource, and OpenGL might have something similar.
  - Single and dual channel textures (especially render textures, which are very important for effects like SSAO)
  - Compute pipeline
  - Instancing (DrawInstanced) (DX11 and GL)
  - OpenGL append/consume buffers
  - Indirect drawing via indirect argument buffers
  - Texture arrays
  - Rendertargets that aren't just 2D (Volumetric (3D) render targets in particular)
  - Shader support for doubles
  - Dynamic shader linkage (Interfaces and similar)
  - Multisampled texture resources
  - Multiple adapters (multi gpu)
  - Passing initial data when creating a resource (DX11, but possibly GL too)
  - Sample mask when setting blend state (DX11, check if equivalent exists in GL)
  - RGBA blend factor when setting blend state(DX11, check if equivalent exists in GL)
  - HLSL9/HLSL11/GLSL/Cg shaders need preprocessor defines & includes
  - DirectX11 supports concurrent drawing and resource creation so all my resource updates should be direct calls to DX methods (I'll need a deferred context?)
  - One camera -> one task (thread) approach for multithreading
   - Also make sure to run off a thread pool (WorkQueue class already exists that provides needed interface)
  - The way I handle rendering currently is to discard simulation results if gpu thread isn't finished.
     - This reduces input lag but at worst case scenario the effect of multithreading might be completely eliminated as
	   GPU ends up waiting for GPU, just because it was few milliseconds late. Maybe better to wait for GPU?


<<<Localization notes for MUCH LATER>>>
 - It would be nice if HString identifier hash was being generated at compile time
 - I still need an easy way to edit the string table (Editor, importer or similar)
 - I might need font localization for non-standard character sets (e.g. russian, greek, asian, etc.)
   - I probably don't want to use one huge set of textures containing both latin and asian characters but want to keep them separate
   - Also asian sets might be too large for textures, in which case generating them at runtime might be necessary (or parsing string table and
     generating textures from only used characters)