Khronos-reference front end for GLSL/ESSL, partial front end for HLSL, and a SPIR-V generator.

#compiler #validator #glsl #shader #spir-v #hlsl #glslang #glslangvalidator #essl #shader #dev-tools

Greg Fischer adbf0d3106 Merge pull request #2961 from jeremy-lunarg/hayes-update-changes 3 vuotta sitten
.github 509a39212d Migrate travis to github actions 3 vuotta sitten
External cb261e3c21 License headers: s/Google/The Khronos Group 5 vuotta sitten
OGLCompilersDLL 2a44064885 Revert changes that migrate to `thread_local`. 5 vuotta sitten
SPIRV 9e2b914722 Restore legacy interface for remap() 3 vuotta sitten
StandAlone b5aae62731 Add whitelist filtering for debug comments in SPIRV-Remap. 3 vuotta sitten
Test 6cdae46314 Avoid duplicate BuiltIn variables for ray tracing matrices (fix #2921) 3 vuotta sitten
build_overrides dac38b8fce Add basic GN configurations 4 vuotta sitten
glslang 7dda6a6347 Merge pull request #2952 from stu-s/EXT_fragment_shader_barycentric 3 vuotta sitten
gtests 7dda6a6347 Merge pull request #2952 from stu-s/EXT_fragment_shader_barycentric 3 vuotta sitten
hlsl c8ef4f8a9f cmake: Remove "conditions" from endif 3 vuotta sitten
kokoro 0f52e7ef12 Fix GN build and presubmits 4 vuotta sitten
ndk_test cb261e3c21 License headers: s/Google/The Khronos Group 5 vuotta sitten
.appveyor.yml 3406857053 Add missing libs to release 4 vuotta sitten
.clang-format 0e5d1bb66a clang-format correction and typo (clang format likely not complete) 6 vuotta sitten
.gitattributes 69d0c1acc2 Remove executable bits from code/data files (#2420) 4 vuotta sitten
.gitignore 17c8387adc Add macOS hidden files to gitignore 3 vuotta sitten
.gn dac38b8fce Add basic GN configurations 4 vuotta sitten
Android.mk df01afe7b8 Android.mk: Add CInterface files 3 vuotta sitten
BUILD.bazel 035a3bbc4a GL_EXT_spirv_intrinsics - Port extensions 3 vuotta sitten
BUILD.gn 978338025a build: Make action targets hermetic 3 vuotta sitten
CHANGES.md c411e58d78 Release 11.10.0 3 vuotta sitten
CMakeLists.txt 25e97a5b06 Fix build for clang + mingw32-make 3 vuotta sitten
CODE_OF_CONDUCT.md 4685f3246e Create CODE_OF_CONDUCT.md 6 vuotta sitten
ChooseMSVCCRT.cmake cb261e3c21 License headers: s/Google/The Khronos Group 5 vuotta sitten
DEPS 0f52e7ef12 Fix GN build and presubmits 4 vuotta sitten
LICENSE.txt d465ac12dd Update LICENSE.txt 3 vuotta sitten
README-spirv-remap.txt a8456415b8 WIP: SPV Remapper: add remapper test framework 9 vuotta sitten
README.md 7801230069 Add a new C interface example to README.md 3 vuotta sitten
WORKSPACE 23f3bdfea1 Build/Test: Dropping 2013 allows using the latest googletests. 5 vuotta sitten
_config.yml c8274e941e Make file formatting comply with POSIX and Unix standards 5 vuotta sitten
build_info.h.tmpl 05798c17fb Fixed issues 2496. Used option 4: parse_version.cmake to avoid python3 usage. 4 vuotta sitten
build_info.py 25555a7f62 Explicitly use Python 3 for scripts 3 vuotta sitten
gen_extension_headers.py 25555a7f62 Explicitly use Python 3 for scripts 3 vuotta sitten
known_good.json 3f369d4a16 Update known_good.json 3 vuotta sitten
known_good_khr.json d03da06ac1 Remove execute permissions 7 vuotta sitten
license-checker.cfg 509a39212d Migrate travis to github actions 3 vuotta sitten
parse_version.cmake e3dab6f0d2 Code review. Thx ben-clayton for comments. 4 vuotta sitten
standalone.gclient dac38b8fce Add basic GN configurations 4 vuotta sitten
update_glslang_sources.py 25555a7f62 Explicitly use Python 3 for scripts 3 vuotta sitten

README-spirv-remap.txt


VERSION
--------------------------------------------------------------------------------
spirv-remap 0.97

INTRO:
--------------------------------------------------------------------------------
spirv-remap is a utility to improve compression of SPIR-V binary files via
entropy reduction, plus optional stripping of debug information and
load/store optimization. It transforms SPIR-V to SPIR-V, remapping IDs. The
resulting modules have an increased ID range (IDs are not as tightly packed
around zero), but will compress better when multiple modules are compressed
together, since compressor's dictionary can find better cross module
commonality.

Remapping is accomplished via canonicalization. Thus, modules can be
compressed one at a time with no loss of quality relative to operating on
many modules at once. The command line tool operates on multiple modules
only in the trivial repetition sense, for ease of use. The remapper API
only accepts a single module at a time.

There are two modes of use: command line, and a C++11 API. Both are
described below.

spirv-remap is currently in an alpha state. Although there are no known
remapping defects, it has only been exercised on one real world game shader
workload.


FEEDBACK
--------------------------------------------------------------------------------
Report defects, enhancements requests, code improvements, etc to:
[email protected]


COMMAND LINE USAGE:
--------------------------------------------------------------------------------
Examples are given with a verbosity of one (-v), but more verbosity can be
had via -vv, -vvv, etc, or an integer parameter to --verbose, such as
"--verbose 4". With no verbosity, the command is silent and returns 0 on
success, and a positive integer error on failure.

Pre-built binaries for several OSs are available. Examples presented are
for Linux. Command line arguments can be provided in any order.

1. Basic ID remapping

Perform ID remapping on all shaders in "*.spv", writing new files with
the same basenames to /tmp/out_dir.

spirv-remap -v --map all --input *.spv --output /tmp/out_dir

2. Perform all possible size reductions

spirv-remap-linux-64 -v --do-everything --input *.spv --output /tmp/out_dir

Note that --do-everything is a synonym for:

--map all --dce all --opt all --strip all

API USAGE:
--------------------------------------------------------------------------------

The public interface to the remapper is defined in SPIRV/SPVRemapper.h as follows:

namespace spv {

class spirvbin_t
{
public:
enum Options { ... };
spirvbin_t(int verbose = 0); // construct

// remap an existing binary in memory
void remap(std::vector& spv, std::uint32_t opts = DO_EVERYTHING);

// Type for error/log handler functions
typedef std::function errorfn_t;
typedef std::function logfn_t;

// Register error/log handling functions (can be c/c++ fn, lambda fn, or functor)
static void registerErrorHandler(errorfn_t handler) { errorHandler = handler; }
static void registerLogHandler(logfn_t handler) { logHandler = handler; }
};

} // namespace spv

The class definition is in SPVRemapper.cpp.

remap() accepts an std::vector of SPIR-V words, modifies them per the
request given in 'opts', and leaves the 'spv' container with the result.
It is safe to instantiate one spirvbin_t per thread and process a different
SPIR-V in each.

The "opts" parameter to remap() accepts a bit mask of desired remapping
options. See REMAPPING AND OPTIMIZATION OPTIONS.

On error, the function supplied to registerErrorHandler() will be invoked.
This can be a standard C/C++ function, a lambda function, or a functor.
The default handler simply calls exit(5); The error handler is a static
member, so need only be set up once, not once per spirvbin_t instance.

Log messages are supplied to registerLogHandler(). By default, log
messages are eaten silently. The log handler is also a static member.

BUILD DEPENDENCIES:
--------------------------------------------------------------------------------
1. C++11 compatible compiler
2. cmake
3. glslang


BUILDING
--------------------------------------------------------------------------------
The standalone remapper is built along side glslangValidator through its
normal build process.


REMAPPING AND OPTIMIZATION OPTIONS
--------------------------------------------------------------------------------
API:
These are bits defined under spv::spirvbin_t::, and can be
bitwise or-ed together as desired.

MAP_TYPES = canonicalize type IDs
MAP_NAMES = canonicalize named data
MAP_FUNCS = canonicalize function bodies
DCE_FUNCS = remove dead functions
DCE_VARS = remove dead variables
DCE_TYPES = remove dead types
OPT_LOADSTORE = optimize unneeded load/stores
MAP_ALL = (MAP_TYPES | MAP_NAMES | MAP_FUNCS)
DCE_ALL = (DCE_FUNCS | DCE_VARS | DCE_TYPES)
OPT_ALL = (OPT_LOADSTORE)
ALL_BUT_STRIP = (MAP_ALL | DCE_ALL | OPT_ALL)
DO_EVERYTHING = (STRIP | ALL_BUT_STRIP)