Browse Source

Add readme page in tests and fix typos

[email protected] 5 years ago
parent
commit
a666176d52
2 changed files with 34 additions and 3 deletions
  1. 31 0
      test/diff/README.md
  2. 3 3
      test/diff/puppeteer.js

+ 31 - 0
test/diff/README.md

@@ -0,0 +1,31 @@
+# Three.js automatic regression testing with CI
+
+You probably shouldn't run this tests on PC because right now it's not optimized for local usage and you can get different results on different GPUs. Goal is to make quick automated testing inside CI and keep screenshot pack updated for it.
+
+### Local usage
+```shell
+# generate new screenshots
+npm run make-screenshot <example1_name> ... <exampleN_name>
+
+# check examples
+npm run test-diff <example1_name> ... <exampleN_name>
+
+# check all examples in browser
+npx cross-env VISIBLE=ture npm run test-diff
+```
+
+### How it works
+- ci configs with parallelism
+- deterministic random/timer/rAF/video for screenshots
+- increased robustness with hided text, datgui, different flags and timeouts.
+- pipeline: turn off rAF -> 'networkidle0' -> networkTax -> turn on rAF -> render promise
+- added 3 progressive attempts for robustness
+
+### Status
+97% examples are covered with tests. Random robusness in CI ~85%. Robustness on different machines ~97%. For example in Windows webgl_effects_ascii example always fails or on integrated GPU you will have additional artifacts: webgl_materials_texture_anisotropy, webgl_postprocessing_procedural, webgl_shaders_tonemapping.
+
+### Probably wrong screenshots
+webgl2_multisampled_renderbuffers, webgl_simple_gi, webgl_postprocessing_dof2, webgl_loader_texture_pvrtc
+
+### Contribution
+You can help to simplify puppeteer script by suggesting example with [HeadlessExperimental.beginFrame](https://chromedevtools.github.io/devtools-protocol/tot/HeadlessExperimental) CDP API.

+ 3 - 3
test/diff/puppeteer.js

@@ -14,7 +14,7 @@ const port = 1234;
 const pixelThreshold = 0.2;
 const pixelThreshold = 0.2;
 const maxFailedPixels = 0.05;
 const maxFailedPixels = 0.05;
 const networkTimeout = 600;
 const networkTimeout = 600;
-const networkTax = 2000;                   // additional timout tax for resources size
+const networkTax = 2000;                   // additional timeout for resources size
 const pageSizeMinTax = 1.0;                // in mb, when networkTax = 0
 const pageSizeMinTax = 1.0;                // in mb, when networkTax = 0
 const pageSizeMaxTax = 5.0;                // in mb, when networkTax = networkTax
 const pageSizeMaxTax = 5.0;                // in mb, when networkTax = networkTax
 const renderTimeout = 1200;
 const renderTimeout = 1200;
@@ -23,14 +23,14 @@ const maxAttemptId = 3;                    // progresseve attempts
 const exceptionList = [
 const exceptionList = [
 
 
 	'index',
 	'index',
-	'webgl_loader_texture_pvrtc',            // not supported in CI, usless
+	'webgl_loader_texture_pvrtc',            // not supported in CI, useless
 	'webgl_materials_envmaps_parallax',
 	'webgl_materials_envmaps_parallax',
 	'webgl_test_memory2',                    // gives fatal error in puppeteer
 	'webgl_test_memory2',                    // gives fatal error in puppeteer
 	'webgl_worker_offscreencanvas',          // in a worker, not robust
 	'webgl_worker_offscreencanvas',          // in a worker, not robust
 
 
 ].concat( ( process.platform === "win32" ) ? [
 ].concat( ( process.platform === "win32" ) ? [
 
 
-	'webgl_effects_ascii'                    // windows fonts
+	'webgl_effects_ascii'                    // windows fonts not supported
 
 
 ] : [] );
 ] : [] );