java
/
jMonkeyEngine-wiki
peilaus alkaen https://github.com/jMonkeyEngine/wiki.git


			
				
					
						
						
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104
							<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]--><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="Asciidoctor 1.5.4"><title>Physically Based Rendering – Part Three</title><link rel="stylesheet" href="./asciidoctor.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.5.0/css/font-awesome.min.css">
<link rel="stylesheet" href="./coderay-asciidoctor.css"><link rel="stylesheet" href="https://cdn.jsdelivr.net/docsearch.js/2/docsearch.min.css"><link rel="stylesheet" href="/home/travis/build/jMonkeyEngine/wiki/build/asciidoc/html5/jme3/advanced/twemoji-awesome.css"></head><body class="article toc2 toc-left"><div id="header"><div id="toolbar"><a href="https://github.com/jMonkeyEngine/wiki/edit/master/src/docs/asciidoc/jme3/advanced/pbr_part3.adoc"><i class="fa fa-pencil-square" aria-hidden="true"></i></a><a href="https://github.com/jMonkeyEngine/wiki/new/master/src/docs/asciidoc/jme3/advanced/"><i class="fa fa-plus-square" aria-hidden="true"></i></a><input dir="auto" style="position: relative; vertical-align: top;" spellcheck="false" autocomplete="off" class="searchbox__input aa-input" id="doc-search" name="search" placeholder="Search in the doc" required="required" type="search"></div><h1>Physically Based Rendering – Part Three</h1><div class="details"><span class="author" id="author"></span><br><span id="revnumber">version ,</span> <span id="revdate">2018/01/15 23:16</span></div><div id="toc" class="toc2"><div id="toctitle">Table of Contents</div><ul class="sectlevel1"><li><a href="#image-based-lighting-in-pbr">Image Based Lighting in PBR</a></li><li><a href="#indirect-lighting-for-pbr-with-image-based-lighting">Indirect Lighting for PBR with Image Based Lighting.</a><ul class="sectlevel2"><li><a href="#reminder-on-environment-maps">Reminder on environment maps :</a></li><li><a href="#ibl-diffuse">IBL Diffuse</a></li><li><a href="#ibl-specular">IBL Specular</a></li><li><a href="#thanks-epic-games">Thanks Epic games!</a></li></ul></li><li><a href="#lexical">Lexical :</a></li></ul></div></div><div id="content"><div id="preamble"><div class="sectionbody"><div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/irradianceMap.png" alt="irradianceMap" width="320" height="250"></div></div>
<div class="paragraph"><p><strong>Note</strong> : after several discussions in the team, I realized that some points were not clear in the  “PBR for artists” post. I’ve made an update with additional information on how to handle metalness and specular. I invite you to read it.</p></div>
<div class="paragraph"><p><a href="../../jme3\advanced\pbr_part1.html">Physically Based Rendering – Part one</a></p></div></div></div>
<div class="sect1"><h2 id="image-based-lighting-in-pbr">Image Based Lighting in PBR</h2><div class="sectionbody"><div class="paragraph"><p>In <a href="../../jme3/advanced/pbr_part2.html">Physically Based Rendering – Part Two</a>, I talked about the basics of PBR for developers, and explained the different steps of the lighting process with PBR.</p></div>
<div class="paragraph"><p>As said before, PBR does not necessarily imply to have indirect lighting, But that’s what makes it look so good.</p></div>
<div class="paragraph"><p>Today I’m gonna focus on a technique called Image Based Lighting (IBL), that will allow us to cheaply compute this indirect lighting.</p></div>
<div class="paragraph"><p>As before you can find at the end of the article a lexical with definitions of various unusual terms you’ll come across.</p></div></div></div>
<div class="sect2"><h3 id="indirect-lighting-for-pbr-with-image-based-lighting">Indirect Lighting for PBR with Image Based Lighting.</h3><div class="paragraph"><p>Direct lighting is usually pretty clear for everyone as it uses common light sources (point light, directional light,…).</p></div>
<div class="paragraph"><p>However indirect lighting is not that obvious. First you need to understand what we want to simulate with indirect light.</p></div>
<div class="paragraph"><p>It is often referred as <strong>Global Illumination (or GI)</strong>. This represents the light bouncing on surrounding objects that is lighting the shaded surface. There are several techniques to implement global illumination, but the most common is <strong>Image Based Lighting (IBL)</strong>. It is very often associated with PBR pipelines.</p></div>
<div class="paragraph"><p>So basically, a light source in game is a color, and optionally other parameters like direction, position, radius, etc… An image has color informations, and this color can be considered as a light source.</p></div>
<div class="paragraph"><p>For global Illumination light is coming from everywhere. So a good way to simulate GI with IBL is to consider an environment map as a light source.</p></div>
<div class="sect2"><h3 id="reminder-on-environment-maps">Reminder on environment maps :</h3><div class="paragraph"><p>Most often, in-game environment maps are cube maps.</p></div>
<div class="paragraph"><p>How do we fetch a pixel from an environment map? We need a vector. Often called the reflect vector (refVec), because thats the reflection vector of the view direction on the shaded surface.</p></div>
<div class="paragraph"><p>A picture worth thousand words</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/Cube_mapped_reflection_example.jpg" alt="Cube_mapped_reflection_example" width="320" height="250"></div></div>
<div class="paragraph"><p>from wikipedia : TopherTH at the English language Wikipedia [GFDL (<a class="bare" href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>), GFDL (<a class="bare" href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>) or CC-BY-SA-3.0 (<a class="bare" href="http://creativecommons.org/licenses/by-sa/3.0/">http://creativecommons.org/licenses/by-sa/3.0/</a>)], via Wikimedia Commons Here the reflected Ray is our reflection vector.</p></div>
<div class="paragraph"><p>Here the reflected Ray is our reflection vector.</p></div>
<div class="paragraph"><p>Unfortunately we can’t take each pixel of the env map and compute light as if it was a direct light source and hope for the best.</p></div>
<div class="paragraph"><p>There’s crazy math around that topic, and to be honest I didn’t get all of it myself. So instead of explaining difficult math equations that may be confusing, I’m gonna go straight to the point : How are we going to compute lighting from the environment map?</p></div></div>
<div class="sect2"><h3 id="ibl-diffuse">IBL Diffuse</h3><div class="paragraph"><p>First we need to compute Diffuse factor from the environment map. Remember our diffuse BRDF from last post? <strong>Lambert</strong>.</p></div>
<div class="paragraph"><p>To simplify, Lambert diffuse BRDF, as it’s used in game, is the light color multiplied by a visibility factor.</p></div>
<div class="paragraph"><p>This visibility factor is a function of the normal of the shaded geometry and the light direction.</p></div>
<div class="paragraph"><p>Let’s say you have a direct light source lighting the front side of a geometry. The back side of this geometry is in the dark. Meaning the front side visibility factor is 1 and the back side visibility factor is 0.</p></div>
<div class="paragraph"><p>For indirect lighting, we can ditch out this visibility factor because the light is coming from everywhere. So all of this simplifies in Diffuse factor = light color.</p></div>
<div class="paragraph"><p>But what’s the light color for a given point?</p></div>
<div class="paragraph"><p>Technically, every pixel in the environment map is a light source, so a shaded point is lighten by a vast amount of pixels.</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/irradiance.png" alt="irradiance" width="320" height="250"></div></div>
<div class="paragraph"><p>In this picture the orange area represent the light rays coming from the cube map to the shaded pixel, that we have to consider to properly compute the light color. So the idea would be, for each shaded pixel, to fetch all that texels and combine the colors.</p></div>
<div class="paragraph"><p>As you can image that’s not practical for a real time rendering pipeline. Even with a 128×128 env map that’s around 50k texture fetches per pixel.</p></div>
<div class="paragraph"><p>Fortunately, We can use something called an <strong>Irradiance map</strong>. An irradiance map is basically the afford mentioned computation…except that it’s precomputed and baked into a texture. In practice here is what it looks like.</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/irradianceMap.png" alt="irradianceMap" width="320" height="250"></div></div>
<div class="paragraph"><p>On the left the original cube map, on the right, the pre computed irradiance map.</p></div>
<div class="paragraph"><p>So at run time you just have to do one texture fetch in that map with the reflection vector. Pretty cool heh?</p></div>
<div class="paragraph"><p>Except that to pre-compute that map we still have to sample the cube map literally billions of times, and even if it’s at design time…it’s painfully long.</p></div>
<div class="paragraph"><p><strong>Spherical Harmonics (SH) to the rescue</strong></p></div>
<div class="paragraph"><p>What’s that again? I won’t go into explaining them in details (because I can’t actually ;-P ), but just know that it’s once again some math magic with a big name on it. Here is a post where it’s explained with simple words, in terms of what you can use them for.</p></div>
<div class="paragraph"><p>To put it simple, SH can help us to compute the irradiance map way faster. This article explains that it all boils down to compute only 9 spherical harmonics coefficients to efficiently approximate an irradiance factor.</p></div>
<div class="paragraph"><p>At this point you can even skip the pre computation of the irradiance map, and use those coefficients directly in your shader for each shaded pixels. That’s fast enough to be real time, and use less memory than a cube map.</p></div>
<div class="paragraph"><p>But still…it’s slower than one texture fetch, so I chose to compute the Irradiance map and use it in the shader.</p></div>
<div class="paragraph"><p>With this technique I can compute a 128X128 irradiance cube map on the CPU in Java in about 200ms. Too slow to be done on each frame, but at design time that’s the blink of an eye.</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/DiffuseIBL.png" alt="DiffuseIBL" width="320" height="250"></div></div>
<div class="paragraph"><p>Here is the diffuse effect of indirect lighting using an irradiance cube map</p></div></div>
<div class="sect2"><h3 id="ibl-specular">IBL Specular</h3><div class="paragraph"><p>Indirect diffuse is cool, but we want “shiny”!! Shiny implies specular lighting.</p></div>
<div class="paragraph"><p>It’s important to understand what we want as a specular reflection. We want it to be very neat when the roughness is low and very blurry when it’s high.</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/Roughness.png" alt="Roughness" width="320" height="250"></div></div>
<div class="paragraph"><p>As roughness increase the reflection gets more blurry.</p></div>
<div class="paragraph"><p>To do this, we have to resolve an integral called the <strong>radiance integral.</strong></p></div>
<div class="paragraph"><p>There is a method to do it quite fast that is called <strong>importance sampling</strong>. However it requires a lot of samples to get correct results, and therefore it’s pretty slow.</p></div>
<div class="paragraph"><p>As an example, for the previous shot, I was using this method, with 1024 samples. It was barely interactive, because it ran at 16 fps on a GTX 670.</p></div></div>
<div class="sect2"><h3 id="thanks-epic-games">Thanks Epic games!</h3><div class="paragraph"><p>Epic games came with a solution to this issue for Unreal Engine 4. Others did too, actually, but Epic games made it public in this paper, from Brian Karis. I can’t thank them enough for this.</p></div>
<div class="paragraph"><p>In this <a href="http://blog.selfshadow.com/publications/s2013-shading-course/karis/s2013_pbs_epic_notes_v2.pdf">paper</a>, they explain how they do it in UE4. They use a method they called the <strong>Split Sum Approximation</strong>. It doesn’t make the computation faster, but it transforms it so that it can be baked in two prefiltered textures.</p></div>
<div class="ulist"><ul><li><p>The prefiltered environment map</p></li></ul></div>
<div class="paragraph"><p>We are going to pre process an env map on the CPU.</p></div>
<div class="paragraph"><p>As explained before, we need the reflection to be more blurry as the roughness increase. The main idea here is to store different levels of roughness in the env map mip maps. The first mip map level will match roughness = 0 and the last will match roughness = 1.</p></div>
<div class="paragraph"><p>From mip levels to mip levels we’re going to convolve (blur) the images depending on the roughness. The more the roughness increase the more samples we’re going to use, and the more spread out they will be.</p></div>
<div class="paragraph"><p>But that’s not all, we also want to “bake” the specular BRDF in the map, so for each pixel we are going to compute the Cook-Torrentz microfacet BRDF (remember last post).</p></div>
<div class="paragraph"><p>But, as we are preprocessing the map, we don’t have any information about the shaded surface normal and view direction. So we are going to assume they are all the same, and equal to the envVector we’ll use to fetch pixels from the map. Also we assume that the shading point is exactly at the center of the cube map.</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/prefilteredEnvMapSampling.png" alt="prefilteredEnvMapSampling" width="320" height="250"></div></div>
<div class="paragraph"><p>This is an approximation again, and it has a cost in quality, but we’re all for approximation as long as it’s perform faster while still looking good, right?</p></div>
<div class="paragraph"><p>Here is what the result looks like</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/PrefilteredEnvMap.png" alt="PrefilteredEnvMap" width="320" height="250"></div></div>
<div class="paragraph"><p>The prefiltered environment map, with the different mip levels. notice how the blur increases through them.</p></div>
<div class="paragraph"><p>So now we can evaluate the first sum of the split sum approximation with a single texture fetch. We are going to compute the Lod level (the mip level where to fetch the texel) according to the roughness.</p></div>
<div class="paragraph"><p>Note that the image needs to be set up so that textureCube interpolates linearly between mip maps so that if the roughness value is not right on the mip level, it will interpolate between the two closest mip levels.</p></div>
<div class="ulist"><ul><li><p>The BRDF integration Map</p></li></ul></div>
<div class="paragraph"><p>Now we need the second sum of the split sum approximation.</p></div>
<div class="paragraph"><p>It’s an integration that has two inputs, the <strong>roughness</strong> that varies from 0 to 1, and the dot product between the normal and the light direction (<strong>N.L</strong>, read N dot L) that also varies from 0 to 1.</p></div>
<div class="paragraph"><p>The outputs are a <strong>scale</strong>, and a <strong>bias</strong>, also varying from 0 to 1.</p></div>
<div class="paragraph"><p>So basically we can bake all combinations into a 2D map. roughness and N.L will be the texture coordinate. the red channel of the map will be the scale, and the green channel will be the bias. (the blue channel is not used)</p></div>
<div class="paragraph"><p>Here is what it looks like :</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/integrateBrdf.png" alt="integrateBrdf" width="320" height="250"></div></div>
<div class="paragraph"><p>The nice part is that this map is constant for white light. It does not depends on the environment. So you can bake it once and for all then use it as an asset in your shaders.</p></div>
<div class="paragraph"><p>Now we have to combine values fetched from these maps to get the specular lighting.</p></div>
<div class="paragraph"><p>Here is what indirect specular alone, looks like, with a roughness of 0.1.</p></div>
<div style="text-align: center;" class="imageblock"><div class="content"><img src="../../jme3/advanced/IndirectSpeculra.png" alt="IndirectSpeculra" width="320" height="250"></div></div>
<div class="paragraph"><p><strong>So in the end :</strong></p></div>
<div class="paragraph"><p>Our indirect lighting pseudo code looks like this :</p></div>
<div class="listingblock"><div class="content"><pre class="CodeRay highlight"><code>//diffuse
indirectDiffuse = textureCube(IrradianceMap, refVec)  * diffuseColor

//specular
lod = getMipLevelFromRoughness(roughness)
prefilteredColor =  textureCube(PrefilteredEnvMap, refVec, lod)
envBRDF = texture2D(BRDFIntegrationMap,vec2(roughness, ndotv)).xy
indirectSpecular = prefilteredColor * (specularColor * envBRDF.x + envBRDF.y)

indirectLighting = indirectDiffuse + indirectSpecular</code></pre></div></div>
<div class="paragraph"><p>That concludes the post. Quite a lot of information to process. Now you should have an idea of the whole thing. Next time, we are going to go under the hood, and YOU GONNA HAZ CODE!!</p></div></div></div>
<div class="sect1"><h2 id="lexical">Lexical :</h2><div class="sectionbody"><div class="paragraph"><p><strong>Global Illumination (GI):</strong> A concept that represent all the lighting of a scene that is not coming from a direct light source.</p></div>
<div class="paragraph"><p><strong>Image Based Lighting (IBL):</strong> A technique that uses an image as a light source</p></div>
<div class="paragraph"><p><strong>Irradiance map :</strong> Precomputed environment map that contains diffuse lighting data of the environment.</p></div>
<div class="paragraph"><p><strong>Spherical Harmonics (SH):</strong> <a href="https://dickyjim.wordpress.com/2013/09/04/spherical-harmonics-for-beginners/">Read this</a></p></div>
<div class="paragraph"><p><strong>Importance Sampling :</strong> A math technique to approximate the result of an integral.</p></div>
<div class="paragraph"><p><strong>Split Sum Approximation :</strong> A way,used in Unreal Engine 4, to transform the specular radiance integral into 2 sums that can be easily baked into prefiltered textures.</p></div>
<hr>
<div class="ulist"><ul><li><p><a href="../../jme3\advanced\pbr_part1.html">Physically Based Rendering – Part one</a></p></li><li><p><a href="../../jme3/advanced/pbr_part2.html">Physically Based Rendering – Part Two</a></p></li></ul></div></div></div></div><div id="footer"><div id="footer-text">Version <br>Last updated 2019-12-20 23:30:51 +00:00</div></div><script type="text/javascript" src="https://cdn.jsdelivr.net/docsearch.js/2/docsearch.min.js"></script><script>docsearch({
  apiKey: 'a736b6d93de805e26ec2f49b55013fbd',
  indexName: 'jmonkeyengine',
  inputSelector: '#doc-search',
  debug: false // Set debug to true if you want to inspect the dropdown
});</script></body></html>