Переглянути джерело

Hxb (#11504)

* use identity for field params

* support partial generic expansion for fields

* take off @:generic

* try something for enum constructor type params

* gencommon says no

* communicate field type parameters by index again

* surely anons are just a list of anon field refs

* fix generic type param names

* let's get dangerous

* handle CBOs in HxbRestore too

* well

* chdir

* handle warnings as CBOs

* don't try to hxb import.hx because there's nothing in there

* don't roundtrip on Cross and Eval

way too many false positives in tests from printing

* add CFLR, don't resolve fields like a crazy person

* fix but it's still broken

* stop the roundtrip nonsense for now, enough other problems

* decode binary modules in check_display_file

* remove flush_fields, should not be needed with CFLR

* maybe fix generic type parameter naming

* add ENFR too

* pull stricter tanon changes and remove some debug

* unused open

* fix overloads a bit more

but it's still not 100% working

* fix generic type param naming a bit more

* check for cf_overloads instead of CfOverloads

The flag isn't set for ghetto overloads, so this should catch all relevant cases.

* fix overloads even more

* add enum when adding enum field ref

* fix overloads for real

* small optimization for the VERY common depth = 0 case

* properly initialize typedef monos

* map anons before identifying them

* write tpp paths instead of names

* remove some debuggery from the writer

* embrace ttp

* don't follow away Null<Void> in function returns

* encode basic types directly

I'm not sure if I want to keep this optimization, but the changes to com.basic seem like a good idea regardless. At the moment it might be possible to get a hold of the ominous `m` monomorph in Common.create, and binding that one to anything would lead to interesting results.

* add ANFR for anon fields

* Happy new year!

* [typer] fix functional interface type parameter leak

see #11390

* ANFR earlier

* only write cf_expr_unoptimized if it differs from cf_expr

* write texpr positions are bit smarter

* add field contexts, hashcons expression type instances

* [display] populate server/modules from hxb since there's only that atm

* remove cf_expr_unoptimized optimization again because the hashconsing has already broken it

We need a different solution for this silly field because it takes up a lot of space and time for very little gain. Maybe something like a binary diff to cf_expr would make sense here.

* forward declare locals in texpr

way less Hashtbling

* be less awkward around ttps

They're your friend!

* purge some debug from the reader

We never want to write to stderr because that hangs the display tests.

* don't generate TLazy and remove random bool

Also add comment about empty anons

* make api an argument of the read function

This is a prerequisite for making the reader reentrant, which should now be possible.

* demo how continuations could work

* reorder tables to something that makes sense to me

* separate chunk classes more

* add cl_type to avoid hundreds of ClassStatics anons

* don't set anon fields too early

* properly deal with ClassStatics in unification and field typing

also convert unify_anons to something human-readable

* bring back missing checks

* add small ring cache per type kind

* add more elaborate stats behind -D hxb.stats

* move expr stuff out of the way because I keep scrolling past it

* change mono byte to 1, avoid extra 1 byte for immediate type instance values

* encode simple type instances immediately

* potentially avoid some GC write barriers by splitting up pos

* use less ignorant fast_eq

* warnings

* basic reader stats

* minor cleanup

* fix class field scoping

* infer nested status from stack

Did you know there can be anon fields inside enum fields? Crazy...

* field type parameters 2.0

* put all texpr type instances in an array

* change -bcp to --hxb-lib

* create directory before creating file...

* format stats output nicer because that's very important

* rename hxml so I find it easier

* encode Void directly because it actually appears a lot

* reorganize type instance bytes, inline common list lengths

* warnings

* write blocks less awkwardly

also don't read arrays only to turn them into lists

* don't write i32 if it can be helped

* pass current chunk to pos writer

* add texpr stats

* optimize TCall

* optimize static calls

* refactor pos writer slightly

* optimize this.field

* fix generator printing

* enable disabled tests

* remove overly fancy position optimization

* write TInt as leb too

* Revert "write TInt as leb too"

This reverts commit ce6c8de751133b2e5898d9d9239e51128358198f.

* let's see where we're at

* check context before checking cache for display files

see #11480

* stop leaving debug prints in your commits ffs

* adjust test

Calling exclude before saving means that the class is extern on the next iteration, and thus not generated.

* dodge arcane sourcemap test problem

* dodge more

* change chunk to module in the writer

* use SimnBuffer

* remove roundtrip

* avoid some object overhead in the reader

* use correct v_id when reading from .hxb

* [hxb] use cache hxb for --hxb too

* don't generate aux output (including hxb) during diagnostics

* warnings

* only cache context when using compilation server

* write chunk name before length, and remove crc

The 4 bytes of the name don't contribute to the length, so this makes more sense.

* minor refactor to make interface a bit saner

* skip hxb generation on display too

* warnings

* move module_cache to HxbData

That might not be the ideal place for it either but it's better than tType.ml

* rework reader interface a bit as well

* read metadata in forward data

see https://github.com/go2hx/go2hx/issues/174

* read anon field meta too

* persist ordered list of chunks instead of one big bytes thing

* remove feature nonsense and revert meta change

* warnings

* fix server/module, get all data from hxb cache directly

* align chunk order

* [hxb] don't overwrite taint reason with check_display_file

* [server] we're sending cacheState, not 'dirty'

* [tests] update test

* [hxb] generate proper warnings for unbound type parameters

* Simplify unbound ttp positions; not like they're often useful anyway...

* ocaml syntax is hard

* anon fields can have @:overload too...

* use singular empty anon

* slightly improve field vs. overloads handling

* remove some debug because hxb is perfect

* [TYPF] skip module name, only write pack when different from current module (private types)

* no need to complicate things

* [tests] add test for #11480

* add CFEX, but don't generate anything into it yet

* move local type params to expressions

* don't write type parameter length twice

* write ttp host

* write nested status so the reader doesn't have to track this

* more CFEX work

* rename chunks

* move all field references in front of definitions

There's currently no data dependency which requires this, but it's more future-proof in case there's ever something along the lines of dependent types.

* rename chunk reader functions too

* sanity catch Error when resolving types

* more sanity

* forward declare module type parameters in MTF

We need to know their identity early so the instance builder can work. Constraints and defaults are set later.

* fix overload handling again

* remove manual cl_build call, add enable_field_access to api

see #11493

* add marker chunks, read hxb modules delayed

see #11493

* work around java.Init issue so we can run JVM roundtrip on CI

see #11493

* fail nicer if we can't write the archive

* write cl_flags in MTF

We want to know early if a class is an interface

* remove enable_field_access, rely on typing passes instead

see #11493

* only load a native lib once we actually need it

* generate extern modules too

* activate EXD, handle field type parameters awfully for now

* don't write empty chunks

* reduce diff against development

* bucket class fields by their name to avoid long linear lookups

* remove rings

Not worth it anymore because the type instance simple/not_simple distinction is fast enough.

* comment out stats writes because this might be an observable overhead

* time writer per-module so we can find pathological modules

* lose some more byte-level OOP overhead

* remove some debug noise

* merge IOChunk and Chunk, remove string_pool

* no comment

* simplify type instance writing again

Reuse a single chunk that we reset when writing a texpr type instance.

* add custom StringPool implementation

* optimize locals a bit more

* maybe deal with duplicate var declaration

* revert local changes

* try local optimization again

* invert texpr order: kind, type, pos

This will allow us to omit some type instances in case they can be inferred from the kind.

* introduce implicit texpr type instances

* walk back a little

* don't double match type params

* avoid some EXD reading

* small cleanup

* avoid some list reading for things that aren't lists

* reverse field order and create both list and PMap at the same time

* Generate dump even with --no-output

* avoid write_list for anon fields too

Folding a PMap to a list only to then get its length (which is a linear operation) and iterate over it isn't the power move that I thought it was.

* write pos pairs where applicable

* delay IO.input creation, add per-chunk timer

* avoid IO in reader too

* don't look if you want to sleep at night

* less nightmares

* remove abstract reader

* do less in display requests

CI exploding in 3... 2... 1...

* start working on delayed expression reading

see #11498

* Add hxb.stats to ignored defines for signature

* put the dodge in place, activate

see #11498

* adjust to cl_init changes

* used hashed identity pool for anon fields too

* less classes

* less classes

* no classes

* avoid some pointless DynArray to List operations

* more DynArray

* add MDR to deal with import to @:keep

* write CLR and friends after CFR

CFR might add more CLR. Also add sanity checks to make sure we don't modify pools after exporting them.

* actually install cf_type

* change field type parameter storing to something more awkward and efficient

* avoid some unnecessary byte blitting

* fix HxbId insertion on fields without expression

* Warnings

* [server-hxb] only read expressions eagerly when full typing

* [hxb] write type for TMeta

* [hxb] whitespace nazi

* tanon identification / tunification: stricterer EqStricter

* Why was this debug even pushed..

* hacktoberfest

* [display] server/type: only restore hxb headers

* check if 3725 is still a problem

* remove TODO comment

* Remove TODOs

* clean up unit hxmls

* don't null-terminate bytes

* [tests] add display test for static field completion

Also run again a couple tests after caching type

* die when writing out empty module or type name

The reader would die anyway

---------

Co-authored-by: Rudy Ges <[email protected]>
Simon Krajewski 1 рік тому
батько
коміт
fe395efcd8

+ 6 - 0
src-json/meta.json

@@ -518,6 +518,12 @@
 		"platforms": ["hl"],
 		"targets": ["TClass", "TClassField"]
 	},
+	{
+		"name": "HxbId",
+		"metadata": ":hxb.id",
+		"doc": "Internally used by hxb",
+		"internal": true
+	},
 	{
 		"name": "HxCompletion",
 		"metadata": ":hx.completion",

+ 10 - 2
src-json/warning.json

@@ -117,6 +117,14 @@
 		"name": "WConstructorInliningCancelled",
 		"doc": "Constructor call could not be inlined because a field is uninitialized",
 		"parent": "WTyper"
+	},
+	{
+		"name": "WHxb",
+		"doc": "Hxb (either --hxb output or haxe compiler cache) related warnings"
+	},
+	{
+		"name": "WUnboundTypeParameter",
+		"doc": "Hxb (either --hxb output or haxe compiler cache) failed to link a type parameter to an actual type",
+		"parent": "WHxb"
 	}
-
-]
+]

+ 1 - 1
src/codegen/genxml.ml

@@ -81,7 +81,7 @@ let rec follow_param t =
 		t
 
 let gen_meta meta =
-	let meta = List.filter (fun (m,_,_) -> match m with Meta.Used | Meta.RealPath | Meta.Pure -> false | _ -> true) meta in
+	let meta = List.filter (fun (m,_,_) -> match m with Meta.Used | Meta.RealPath | Meta.Pure | Meta.HxbId -> false | _ -> true) meta in
 	match meta with
 	| [] -> []
 	| _ ->

+ 9 - 0
src/compiler/args.ml

@@ -48,6 +48,7 @@ let parse_args com =
 	let actx = {
 		classes = [([],"Std")];
 		xml_out = None;
+		hxb_out = None;
 		json_out = None;
 		cmds = [];
 		config_macros = [];
@@ -58,6 +59,7 @@ let parse_args com =
 		interp = false;
 		jvm_flag = false;
 		swf_version = false;
+		hxb_libs = [];
 		native_libs = [];
 		raise_usage = (fun () -> ());
 		display_arg = None;
@@ -131,6 +133,10 @@ let parse_args com =
 		("Compilation",[],["-libcp"],Arg.String (fun path ->
 			com.class_paths#add (new ClassPath.directory_class_path (Path.add_trailing_slash path) Lib);
 		),"<path>","add a directory to find source files");
+		("Compilation",["--hxb-lib"],["-hxb-lib"],Arg.String (fun file ->
+			let lib = create_native_lib file false HxbLib in
+			actx.hxb_libs <- lib :: actx.hxb_libs
+		),"<path>","add a hxb library");
 		("Compilation",["-m";"--main"],["-main"],Arg.String (fun cl ->
 			if com.main_class <> None then raise (Arg.Bad "Multiple --main classes specified");
 			let cpath = Path.parse_type_path cl in
@@ -272,6 +278,9 @@ let parse_args com =
 		("Services",["--json"],[],Arg.String (fun file ->
 			actx.json_out <- Some file
 		),"<file>","generate JSON types description");
+		("Services",["--hxb"],[], Arg.String (fun dir ->
+			actx.hxb_out <- Some dir;
+		),"<directory>", "generate haxe binary representation in target directory");
 		("Optimization",["--no-output"],[], Arg.Unit (fun() -> actx.no_output <- true),"","compiles but does not generate any file");
 		("Debug",["--times"],[], Arg.Unit (fun() -> Timer.measure_times := true),"","measure compilation times");
 		("Optimization",["--no-inline"],[],Arg.Unit (fun () ->

+ 45 - 13
src/compiler/compilationCache.ml

@@ -23,9 +23,18 @@ type cached_native_lib = {
 	c_nl_files : (path,Ast.package) Hashtbl.t;
 }
 
-class context_cache (index : int) = object(self)
+let get_module_name_of_cfile file cfile = match cfile.c_module_name with
+	| None ->
+		let name = Path.module_name_of_file file in
+		cfile.c_module_name <- Some name;
+		name
+	| Some name ->
+		name
+
+class context_cache (index : int) (sign : Digest.t) = object(self)
 	val files : (Path.UniqueKey.t,cached_file) Hashtbl.t = Hashtbl.create 0
 	val modules : (path,module_def) Hashtbl.t = Hashtbl.create 0
+	val binary_cache : (path,HxbData.module_cache) Hashtbl.t = Hashtbl.create 0
 	val removed_files = Hashtbl.create 0
 	val mutable json = JNull
 	val mutable initialized = false
@@ -57,17 +66,41 @@ class context_cache (index : int) = object(self)
 	method find_module_opt path =
 		Hashtbl.find_opt modules path
 
-	method cache_module path value =
-		Hashtbl.replace modules path value
+	method find_module_extra path =
+		try (Hashtbl.find modules path).m_extra with Not_found -> (Hashtbl.find binary_cache path).mc_extra
+
+	method cache_module warn anon_identification hxb_writer_stats path m =
+		match m.m_extra.m_kind with
+		| MImport ->
+			Hashtbl.add modules m.m_path m
+		| _ ->
+			let writer = HxbWriter.create warn anon_identification hxb_writer_stats in
+			HxbWriter.write_module writer m;
+			let chunks = HxbWriter.get_chunks writer in
+			Hashtbl.replace binary_cache path {
+				mc_path = path;
+				mc_id = m.m_id;
+				mc_chunks = chunks;
+				mc_extra = { m.m_extra with m_cache_state = MSGood }
+			}
+
+	method clear_cache =
+		Hashtbl.clear modules
 
 	(* initialization *)
 
 	method is_initialized = initialized
 	method set_initialized value = initialized <- value
 
+	method get_sign = sign
 	method get_index = index
 	method get_files = files
 	method get_modules = modules
+
+	method get_hxb = binary_cache
+	method get_hxb_module path = Hashtbl.find binary_cache path
+
+	(* TODO handle hxb cache there too *)
 	method get_removed_files = removed_files
 
 	method get_json = json
@@ -115,7 +148,7 @@ class cache = object(self)
 		try
 			Hashtbl.find contexts sign
 		with Not_found ->
-			let cache = new context_cache (Hashtbl.length contexts) in
+			let cache = new context_cache (Hashtbl.length contexts) sign in
 			context_list <- cache :: context_list;
 			Hashtbl.add contexts sign cache;
 			cache
@@ -174,7 +207,14 @@ class cache = object(self)
 		Hashtbl.iter (fun _ cc ->
 			Hashtbl.iter (fun _ m ->
 				if Path.UniqueKey.lazy_key m.m_extra.m_file = file_key then m.m_extra.m_cache_state <- MSBad (Tainted reason)
-			) cc#get_modules
+			) cc#get_modules;
+			let open HxbData in
+			Hashtbl.iter (fun _ mc ->
+				if Path.UniqueKey.lazy_key mc.mc_extra.m_file = file_key then
+					mc.mc_extra.m_cache_state <- match reason, mc.mc_extra.m_cache_state with
+					| CheckDisplayFile, (MSBad _ as state) -> state
+					| _ -> MSBad (Tainted reason)
+			) cc#get_hxb
 		) contexts
 
 	(* haxelibs *)
@@ -267,11 +307,3 @@ type context_options =
 	| NormalContext
 	| MacroContext
 	| NormalAndMacroContext
-
-let get_module_name_of_cfile file cfile = match cfile.c_module_name with
-	| None ->
-		let name = Path.module_name_of_file file in
-		cfile.c_module_name <- Some name;
-		name
-	| Some name ->
-		name

+ 4 - 0
src/compiler/compilationContext.ml

@@ -11,6 +11,7 @@ type native_lib_kind =
 	| NetLib
 	| JavaLib
 	| SwfLib
+	| HxbLib
 
 type native_lib_arg = {
 	lib_file : string;
@@ -21,6 +22,7 @@ type native_lib_arg = {
 type arg_context = {
 	mutable classes : Globals.path list;
 	mutable xml_out : string option;
+	mutable hxb_out : string option;
 	mutable json_out : string option;
 	mutable cmds : string list;
 	mutable config_macros : string list;
@@ -31,6 +33,7 @@ type arg_context = {
 	mutable interp : bool;
 	mutable jvm_flag : bool;
 	mutable swf_version : bool;
+	mutable hxb_libs : native_lib_arg list;
 	mutable native_libs : native_lib_arg list;
 	mutable raise_usage : unit -> unit;
 	mutable display_arg : string option;
@@ -56,6 +59,7 @@ and compilation_context = {
 type compilation_callbacks = {
 	before_anything : compilation_context -> unit;
 	after_target_init : compilation_context -> unit;
+	after_save : compilation_context -> unit;
 	after_compilation : compilation_context -> unit;
 }
 

+ 8 - 1
src/compiler/compiler.ml

@@ -295,6 +295,7 @@ let do_type ctx mctx actx display_file_dot_path =
 	CommonCache.maybe_add_context_sign cs com "before_init_macros";
 	enter_stage com CInitMacrosStart;
 	ServerMessage.compiler_stage com;
+	Setup.init_native_libs com actx.hxb_libs;
 	let mctx = List.fold_left (fun mctx path ->
 		Some (MacroContext.call_init_macro ctx.com mctx path)
 	) mctx (List.rev actx.config_macros) in
@@ -378,6 +379,11 @@ let compile ctx actx callbacks =
 		let (tctx,display_file_dot_path) = do_type ctx mctx actx display_file_dot_path in
 		DisplayProcessing.handle_display_after_typing ctx tctx display_file_dot_path;
 		finalize_typing ctx tctx;
+		let is_compilation = is_compilation com in
+		com.callbacks#add_after_save (fun () ->
+			callbacks.after_save ctx;
+			if is_compilation then Generate.check_hxb_output ctx actx;
+		);
 		if is_diagnostics com then
 			filter ctx tctx (fun () -> DisplayProcessing.handle_display_after_finalization ctx tctx display_file_dot_path)
 		else begin
@@ -385,7 +391,7 @@ let compile ctx actx callbacks =
 			filter ctx tctx (fun () -> ());
 		end;
 		if ctx.has_error then raise Abort;
-		if is_compilation com then Generate.check_auxiliary_output com actx;
+		if is_compilation then Generate.check_auxiliary_output com actx;
 		enter_stage com CGenerationStart;
 		ServerMessage.compiler_stage com;
 		Generate.maybe_generate_dump ctx tctx;
@@ -446,6 +452,7 @@ with
 
 let finalize ctx =
 	ctx.comm.flush ctx;
+	List.iter (fun lib -> lib#close) ctx.com.hxb_libs;
 	(* In server mode any open libs are closed by the lib_build_task. In offline mode
 		we should do it here to be safe. *)
 	if not ctx.comm.is_server then begin

+ 63 - 0
src/compiler/generate.ml

@@ -1,5 +1,7 @@
 open Globals
 open CompilationContext
+open TType
+open Tanon_identification
 
 let check_auxiliary_output com actx =
 	begin match actx.xml_out with
@@ -19,6 +21,67 @@ let check_auxiliary_output com actx =
 			Genjson.generate com.types file
 	end
 
+let export_hxb com cc platform zip m =
+	let open HxbData in
+	match m.m_extra.m_kind with
+		| MCode | MMacro | MFake | MExtern -> begin
+			(* Printf.eprintf "Export module %s\n" (s_type_path m.m_path); *)
+			let l = platform :: (fst m.m_path @ [snd m.m_path]) in
+			let path = (String.concat "/" l) ^ ".hxb" in
+
+			try
+				let hxb_cache = cc#get_hxb_module m.m_path in
+				let out = IO.output_string () in
+				write_header out;
+				List.iter (fun (kind,data) ->
+					write_chunk_prefix kind (Bytes.length data) out;
+					IO.nwrite out data
+				) hxb_cache.mc_chunks;
+				let data = IO.close_out out in
+				zip#add_entry data path;
+			with Not_found ->
+				let anon_identification = new tanon_identification in
+				let warn w s p = com.Common.warning w com.warning_options s p in
+				let writer = HxbWriter.create warn anon_identification com.hxb_writer_stats in
+				HxbWriter.write_module writer m;
+				let out = IO.output_string () in
+				HxbWriter.export writer out;
+				zip#add_entry (IO.close_out out) path;
+		end
+	| _ ->
+		()
+
+let check_hxb_output ctx actx =
+	let com = ctx.com in
+	let try_write path =
+		let t = Timer.timer ["generate";"hxb"] in
+		Path.mkdir_from_path path;
+		let zip = new Zip_output.zip_output path 6 in
+		let export com =
+			let cc = CommonCache.get_cache com in
+			let target = Common.platform_name_macro com in
+			List.iter (fun m ->
+				let t = Timer.timer ["generate";"hxb";s_type_path m.m_path] in
+				Std.finally t (export_hxb com cc target zip) m
+			) com.modules;
+		in
+		Std.finally (fun () ->
+			zip#close;
+			t()
+		) (fun () ->
+			export com;
+			Option.may export (com.get_macros());
+		) ()
+	in
+	begin match actx.hxb_out with
+		| None ->
+			()
+		| Some path ->
+			try
+				try_write path
+			with Sys_error s ->
+				error ctx (Printf.sprintf "Could not write to %s: %s" path s) null_pos
+	end
 
 let parse_swf_header ctx h = match ExtString.String.nsplit h ":" with
 		| [width; height; fps] ->

+ 127 - 0
src/compiler/hxb/hxbData.ml

@@ -0,0 +1,127 @@
+open Globals
+open Type
+
+exception HxbFailure of string
+
+(*
+	MD = module
+	MT = module type
+	CL = class
+	EN = enum
+	AB = abstract
+	TD = typedef
+	AN = anon
+	CF = class field
+	EF = enum field
+	AF = anon field
+	EX = expression
+	EO = end of (Types | Fields | Module)
+	..F = forward definition
+	..R = reference
+	..D = definition
+*)
+
+type chunk_kind =
+	| STR (* string pool *)
+	| DOC (* doc pool *)
+	| MDF (* module foward *)
+	| MTF (* module types forward *)
+	(* Module type references *)
+	| MDR (* module references *)
+	| CLR (* class references *)
+	| ENR (* enum references *)
+	| ABR (* abstract references *)
+	| TDR (* typedef references *)
+	(* Field references *)
+	| AFR (* anon field references *)
+	(* Own module type definitions *)
+	| CLD (* class definition *)
+	| END (* enum definition *)
+	| ABD (* abstract definition *)
+	| TDD (* typedef definition *)
+	| EOT (* end of module types *)
+	(* Field references *)
+	| EFR (* enum field references *)
+	| CFR (* class field references *)
+	(* Own field definitions *)
+	| CFD (* class fields *)
+	| EFD (* enum fields *)
+	| AFD (* abstract fields *)
+	| EOF (* end of fields *)
+	| EXD (* class field expressions *)
+	| EOM (* end of module *)
+
+type cached_chunk = chunk_kind * bytes
+type cached_chunks = cached_chunk list
+
+type module_cache = {
+	mc_path : path;
+	mc_id : int;
+	mc_chunks : cached_chunks;
+	mc_extra : module_def_extra;
+}
+
+let string_of_chunk_kind = function
+	| STR -> "STR"
+	| DOC -> "DOC"
+	| MDF -> "MDF"
+	| MTF -> "MTF"
+	| MDR -> "MDR"
+	| CLR -> "CLR"
+	| ENR -> "ENR"
+	| ABR -> "ABR"
+	| TDR -> "TDR"
+	| AFR -> "AFR"
+	| EFR -> "EFR"
+	| CFR -> "CFR"
+	| CLD -> "CLD"
+	| END -> "END"
+	| ABD -> "ABD"
+	| TDD -> "TDD"
+	| EOT -> "EOT"
+	| CFD -> "CFD"
+	| EFD -> "EFD"
+	| AFD -> "AFD"
+	| EOF -> "EOF"
+	| EXD -> "EXD"
+	| EOM -> "EOM"
+
+let chunk_kind_of_string = function
+	| "STR" -> STR
+	| "DOC" -> DOC
+	| "MDF" -> MDF
+	| "MTF" -> MTF
+	| "MDR" -> MDR
+	| "CLR" -> CLR
+	| "ENR" -> ENR
+	| "ABR" -> ABR
+	| "TDR" -> TDR
+	| "AFR" -> AFR
+	| "EFR" -> EFR
+	| "CFR" -> CFR
+	| "CLD" -> CLD
+	| "END" -> END
+	| "ABD" -> ABD
+	| "TDD" -> TDD
+	| "EOT" -> EOT
+	| "CFD" -> CFD
+	| "EFD" -> EFD
+	| "AFD" -> AFD
+	| "EOF" -> EOF
+	| "EXD" -> EXD
+	| "EOM" -> EOM
+	| name -> raise (HxbFailure ("Invalid chunk name: " ^ name))
+
+let error (s : string) =
+	Printf.eprintf "[error] %s\n" s;
+	raise (HxbFailure s)
+
+let hxb_version = 1
+
+let write_header ch =
+	IO.nwrite_string ch "hxb";
+	IO.write_byte ch hxb_version
+
+let write_chunk_prefix kind length ch =
+	IO.nwrite ch (Bytes.unsafe_of_string (string_of_chunk_kind kind));
+	IO.write_real_i32 ch (Int32.of_int length)

+ 63 - 0
src/compiler/hxb/hxbLib.ml

@@ -0,0 +1,63 @@
+open Globals
+open Common
+open ExtString
+
+class hxb_library file_path = object(self)
+	inherit abstract_hxb_lib
+	val zip = lazy (Zip.open_in file_path)
+
+	val mutable cached_files = []
+	val modules = Hashtbl.create 0
+	val mutable closed = false
+	val mutable loaded = false
+
+	method load =
+		if not loaded then begin
+			loaded <- true;
+			let close = Timer.timer ["hxblib";"read"] in
+			List.iter (function
+				| ({ Zip.is_directory = false; Zip.filename = filename } as entry) when String.ends_with filename ".hxb" ->
+					let pack = String.nsplit filename "/" in
+					begin match List.rev pack with
+						| [] -> ()
+						| name :: pack ->
+							let name = String.sub name 0 (String.length name - 4) in
+							let pack = List.rev pack in
+							Hashtbl.add modules (pack,name) (filename,entry);
+						end
+				| _ -> ()
+			) (Zip.entries (Lazy.force zip));
+			close();
+		end
+
+	method get_bytes (target : string) (path : path) =
+		try
+			let path = (target :: fst path,snd path) in
+			let (filename,entry) = Hashtbl.find modules path in
+			let close = Timer.timer ["hxblib";"get bytes"] in
+			let zip = Lazy.force zip in
+			let data = Zip.read_entry zip entry in
+			close();
+			Some (Bytes.unsafe_of_string data)
+		with Not_found ->
+			None
+
+	method close =
+		if not closed then begin
+			closed <- true;
+			Zip.close_in (Lazy.force zip)
+		end
+
+	method get_file_path = file_path
+end
+
+
+let create_hxb_lib com file_path =
+	let file = if Sys.file_exists file_path then
+		file_path
+	else try
+		Common.find_file com file_path
+	with Not_found ->
+		failwith ("hxb lib " ^ file_path ^ " not found")
+	in
+	new hxb_library file

+ 2002 - 0
src/compiler/hxb/hxbReader.ml

@@ -0,0 +1,2002 @@
+open Globals
+open Ast
+open Type
+open HxbData
+open HxbReaderApi
+
+type field_reader_context = {
+	t_pool : Type.t Array.t;
+	pos : pos ref;
+	vars : tvar Array.t;
+	mutable tthis : Type.t option;
+}
+
+let create_field_reader_context p ts vars tthis = {
+	t_pool = ts;
+	pos = ref p;
+	vars = vars;
+	tthis = tthis;
+}
+
+type hxb_reader_stats = {
+	modules_fully_restored : int ref;
+	modules_partially_restored : int ref;
+}
+
+let create_hxb_reader_stats () = {
+	modules_fully_restored = ref 0;
+	modules_partially_restored = ref 0;
+}
+
+module ClassFieldInfo = struct
+	type t = {
+		type_parameters : typed_type_param array;
+	}
+
+	let create params = {
+		type_parameters = params;
+	}
+end
+
+module ClassFieldInfos = struct
+	type t = {
+		infos : ClassFieldInfo.t DynArray.t;
+	}
+
+	let meta = Meta.HxbId
+
+	let create () = {
+		infos = DynArray.create ()
+	}
+
+	let get infos cf =
+		let _,_,p = Meta.get meta cf.cf_meta in
+		DynArray.get infos.infos p.pmin
+
+	let unset infos cf =
+		cf.cf_meta <- Meta.remove meta cf.cf_meta
+
+	let set infos info cf =
+		let index = DynArray.length infos.infos in
+		DynArray.add infos.infos info;
+		cf.cf_meta <- (meta,[],{null_pos with pmin = index}) :: cf.cf_meta
+end
+
+module BytesWithPosition = struct
+	type t = {
+		bytes : bytes;
+		mutable pos : int;
+	}
+
+	let create bytes = {
+		bytes;
+		pos = 0;
+	}
+
+	let read_byte b =
+		let i = Bytes.unsafe_get b.bytes b.pos in
+		b.pos <- b.pos + 1;
+		int_of_char i
+
+	let read_bytes b length =
+		let out = Bytes.create length in
+		Bytes.blit b.bytes b.pos out 0 length;
+		b.pos <- b.pos + length;
+		out
+
+	let read_i16 i =
+		let ch2 = read_byte i in
+		let ch1 = read_byte i in
+		let n = ch1 lor (ch2 lsl 8) in
+		if ch2 land 128 <> 0 then
+			n - 65536
+		else
+			n
+
+	let read_real_i32 ch =
+		let ch1 = read_byte ch in
+		let ch2 = read_byte ch in
+		let ch3 = read_byte ch in
+		let base = Int32.of_int (ch1 lor (ch2 lsl 8) lor (ch3 lsl 16)) in
+		let big = Int32.shift_left (Int32.of_int (read_byte ch)) 24 in
+		Int32.logor base big
+
+		let read_i64 ch =
+			let big = Int64.of_int32 (read_real_i32 ch) in
+			let ch4 = read_byte ch in
+			let ch3 = read_byte ch in
+			let ch2 = read_byte ch in
+			let ch1 = read_byte ch in
+			let base = Int64.of_int (ch1 lor (ch2 lsl 8) lor (ch3 lsl 16)) in
+			let small = Int64.logor base (Int64.shift_left (Int64.of_int ch4) 24) in
+			Int64.logor (Int64.shift_left big 32) small
+
+	let read_double ch =
+		Int64.float_of_bits (read_i64 ch)
+end
+
+open BytesWithPosition
+
+let rec read_uleb128 ch =
+	let b = read_byte ch in
+	if b >= 0x80 then
+		(b land 0x7F) lor ((read_uleb128 ch) lsl 7)
+	else
+		b
+
+let read_leb128 ch =
+	let rec read acc shift =
+		let b = read_byte ch in
+		let acc = ((b land 0x7F) lsl shift) lor acc in
+		if b >= 0x80 then
+			read acc (shift + 7)
+		else
+			(b, acc, shift + 7)
+	in
+	let last, acc, shift = read 0 0 in
+	let res = (if (last land 0x40) <> 0 then
+		acc lor ((lnot 0) lsl shift)
+	else
+		acc) in
+	res
+
+let dump_stats name stats =
+	print_endline (Printf.sprintf "hxb_reader stats for %s" name);
+	print_endline (Printf.sprintf "  modules partially restored: %i" (!(stats.modules_fully_restored) - !(stats.modules_partially_restored)));
+	print_endline (Printf.sprintf "  modules fully restored: %i" !(stats.modules_fully_restored));
+
+class hxb_reader
+	(mpath : path)
+	(stats : hxb_reader_stats)
+= object(self)
+	val mutable api = Obj.magic ""
+	val mutable current_module = null_module
+
+	val mutable ch = BytesWithPosition.create (Bytes.create 0)
+	val mutable string_pool = Array.make 0 ""
+	val mutable doc_pool = Array.make 0 ""
+
+	val mutable classes = Array.make 0 null_class
+	val mutable abstracts = Array.make 0 null_abstract
+	val mutable enums = Array.make 0 null_enum
+	val mutable typedefs = Array.make 0 null_typedef
+	val mutable anons = Array.make 0 null_tanon
+	val mutable anon_fields = Array.make 0 null_field
+	val mutable tmonos = Array.make 0 (mk_mono())
+	val mutable class_fields = Array.make 0 null_field
+	val mutable enum_fields = Array.make 0 null_enum_field
+
+	val mutable type_type_parameters = Array.make 0 (mk_type_param null_class TPHType None None)
+	val mutable field_type_parameters = Array.make 0 (mk_type_param null_class TPHMethod None None)
+	val mutable local_type_parameters = Array.make 0 (mk_type_param null_class TPHLocal None None)
+
+	val mutable field_type_parameter_offset = 0
+	val empty_anon = mk_anon (ref Closed)
+
+	method resolve_type pack mname tname =
+		try
+			api#resolve_type pack mname tname
+		with Not_found ->
+			dump_backtrace();
+			error (Printf.sprintf "[HXB] [%s] Cannot resolve type %s" (s_type_path current_module.m_path) (s_type_path ((pack @ [mname]),tname)))
+
+	(* Primitives *)
+
+	method read_i32 =
+		read_real_i32 ch
+
+	method read_i16 =
+		read_i16 ch
+
+	method read_f64 =
+		read_double ch
+
+	method read_bool =
+		read_byte ch <> 0
+
+	method read_from_string_pool pool =
+		pool.(read_uleb128 ch)
+
+	method read_string =
+		self#read_from_string_pool string_pool
+
+	method read_raw_string =
+		let l = read_uleb128 ch in
+		Bytes.unsafe_to_string (read_bytes ch l)
+
+	(* Basic compounds *)
+
+	method read_list : 'a . (unit -> 'a) -> 'a list = fun f ->
+		let l = read_uleb128 ch in
+		List.init l (fun _ -> f ())
+
+	method read_option : 'a . (unit -> 'a) -> 'a option = fun f ->
+		match read_byte ch with
+		| 0 ->
+			None
+		| _ ->
+			Some (f())
+
+	method read_path =
+		let pack = self#read_list (fun () -> self#read_string) in
+		let name = self#read_string in
+		(pack,name)
+
+	method read_full_path =
+		let pack = self#read_list (fun () -> self#read_string) in
+		let mname = self#read_string in
+		let tname = self#read_string in
+		(pack,mname,tname)
+
+	method read_documentation =
+		let doc_own = self#read_option (fun () ->
+			self#read_from_string_pool doc_pool
+		) in
+		let doc_inherited = self#read_list (fun () ->
+			self#read_from_string_pool doc_pool
+		) in
+		{doc_own;doc_inherited}
+
+	method read_pos =
+		let file = self#read_string in
+		let min = read_leb128 ch in
+		let max = read_leb128 ch in
+		let pos = {
+			pfile = file;
+			pmin = min;
+			pmax = max;
+		} in
+		pos
+
+	method read_pos_pair =
+		let file = self#read_string in
+		let min1 = read_leb128 ch in
+		let max1 = read_leb128 ch in
+		let min2 = read_leb128 ch in
+		let max2 = read_leb128 ch in
+		let pos1 = {
+			pfile = file;
+			pmin = min1;
+			pmax = max1;
+		} in
+		let pos2 = {
+			pos1 with
+			pmin = pos1.pmin + min2;
+			pmax = pos1.pmin + max2;
+		} in
+		pos1,pos2
+
+	method read_metadata_entry : metadata_entry =
+		let name = self#read_string in
+		let p = self#read_pos in
+		let el = self#read_list (fun () -> self#read_expr) in
+		(Meta.from_string name,el,p)
+
+	method read_metadata =
+		self#read_list (fun () -> self#read_metadata_entry)
+
+	(* References *)
+
+	method read_class_ref =
+		classes.(read_uleb128 ch)
+
+	method read_abstract_ref =
+		abstracts.(read_uleb128 ch)
+
+	method read_enum_ref =
+		enums.(read_uleb128 ch)
+
+	method read_typedef_ref =
+		typedefs.(read_uleb128 ch)
+
+	method read_field_ref =
+		class_fields.(read_uleb128 ch)
+
+	method read_enum_field_ref =
+		enum_fields.(read_uleb128 ch)
+
+	method read_anon_ref =
+		match read_byte ch with
+		| 0 ->
+			anons.(read_uleb128 ch)
+		| 1 ->
+			let an = anons.(read_uleb128 ch) in
+			self#read_anon an
+		| _ ->
+			assert false
+
+	method read_anon_field_ref =
+		match read_byte ch with
+		| 0 ->
+			anon_fields.(read_uleb128 ch)
+		| 1 ->
+			let cf = anon_fields.(read_uleb128 ch) in
+			self#read_class_field_and_overloads_data cf;
+			cf
+		| _ ->
+			assert false
+
+	(* Expr *)
+
+	method get_binop i = match i with
+		| 0 -> OpAdd
+		| 1 -> OpMult
+		| 2 -> OpDiv
+		| 3 -> OpSub
+		| 4 -> OpAssign
+		| 5 -> OpEq
+		| 6 -> OpNotEq
+		| 7 -> OpGt
+		| 8 -> OpGte
+		| 9 -> OpLt
+		| 10 -> OpLte
+		| 11 -> OpAnd
+		| 12 -> OpOr
+		| 13 -> OpXor
+		| 14 -> OpBoolAnd
+		| 15 -> OpBoolOr
+		| 16 -> OpShl
+		| 17 -> OpShr
+		| 18 -> OpUShr
+		| 19 -> OpMod
+		| 20 -> OpInterval
+		| 21 -> OpArrow
+		| 22 -> OpIn
+		| 23 -> OpNullCoal
+		| _ -> OpAssignOp (self#get_binop (i - 30))
+
+	method get_unop i = match i with
+		| 0 -> Increment,Prefix
+		| 1 -> Decrement,Prefix
+		| 2 -> Not,Prefix
+		| 3 -> Neg,Prefix
+		| 4 -> NegBits,Prefix
+		| 5 -> Spread,Prefix
+		| 6 -> Increment,Postfix
+		| 7 -> Decrement,Postfix
+		| 8 -> Not,Postfix
+		| 9 -> Neg,Postfix
+		| 10 -> NegBits,Postfix
+		| 11 -> Spread,Postfix
+		| _ -> assert false
+
+	method read_placed_name =
+		let s = self#read_string in
+		let p = self#read_pos in
+		(s,p)
+
+	method read_type_path =
+		let pack = self#read_list (fun () -> self#read_string) in
+		let name = self#read_string in
+		let tparams = self#read_list (fun () -> self#read_type_param_or_const) in
+		let tsub = self#read_option (fun () -> self#read_string) in
+		{
+			tpackage = pack;
+			tname = name;
+			tparams = tparams;
+			tsub = tsub;
+		}
+
+	method read_placed_type_path =
+		let tp = self#read_type_path in
+		let pfull,ppath = self#read_pos_pair in
+		{
+			path = tp;
+			pos_full = pfull;
+			pos_path = ppath;
+		}
+
+	method read_type_param =
+		let pn = self#read_placed_name in
+		let ttp = self#read_list (fun () -> self#read_type_param) in
+		let tho = self#read_option (fun () -> self#read_type_hint) in
+		let def = self#read_option (fun () -> self#read_type_hint) in
+		let meta = self#read_metadata in
+		{
+			tp_name = pn;
+			tp_params = ttp;
+			tp_constraints = tho;
+			tp_meta = meta;
+			tp_default = def;
+		}
+
+	method read_type_param_or_const =
+		match read_byte ch with
+		| 0 -> TPType (self#read_type_hint)
+		| 1 -> TPExpr (self#read_expr)
+		| _ -> assert false
+
+	method read_func_arg =
+		let pn = self#read_placed_name in
+		let b = self#read_bool in
+		let meta = self#read_metadata in
+		let tho = self#read_option (fun () -> self#read_type_hint) in
+		let eo = self#read_option (fun () -> self#read_expr) in
+		(pn,b,meta,tho,eo)
+
+	method read_func =
+		let params = self#read_list (fun () -> self#read_type_param) in
+		let args = self#read_list (fun () -> self#read_func_arg) in
+		let tho = self#read_option (fun () -> self#read_type_hint) in
+		let eo = self#read_option (fun () -> self#read_expr) in
+		{
+			f_params = params;
+			f_args = args;
+			f_type = tho;
+			f_expr = eo;
+		}
+
+	method read_complex_type =
+		match read_byte ch with
+		| 0 -> CTPath (self#read_placed_type_path)
+		| 1 ->
+			let thl = self#read_list (fun () -> self#read_type_hint) in
+			let th = self#read_type_hint in
+			CTFunction(thl,th)
+		| 2 -> CTAnonymous (self#read_list (fun () -> self#read_cfield))
+		| 3 -> CTParent (self#read_type_hint)
+		| 4 ->
+			let ptp = self#read_list (fun () -> self#read_placed_type_path) in
+			let cffl = self#read_list (fun () -> self#read_cfield) in
+			CTExtend(ptp,cffl)
+		| 5 -> CTOptional (self#read_type_hint)
+		| 6 ->
+			let pn = self#read_placed_name in
+			let th = self#read_type_hint in
+			CTNamed(pn,th)
+		| 7 -> CTIntersection (self#read_list (fun () -> self#read_type_hint))
+		| _ -> assert false
+
+	method read_type_hint =
+		let ct = self#read_complex_type in
+		let p = self#read_pos in
+		(ct,p)
+
+	method read_access =
+		match read_byte ch with
+		| 0 -> APublic
+		| 1 -> APrivate
+		| 2 -> AStatic
+		| 3 -> AOverride
+		| 4 -> ADynamic
+		| 5 -> AInline
+		| 6 -> AMacro
+		| 7 -> AFinal
+		| 8 -> AExtern
+		| 9 -> AAbstract
+		| 10 -> AOverload
+		| 11 -> AEnum
+		| _ -> assert false
+
+	method read_placed_access =
+		let ac = self#read_access in
+		let p = self#read_pos in
+		(ac,p)
+
+	method read_cfield_kind =
+		match read_byte ch with
+		| 0 ->
+			let tho = self#read_option (fun () -> self#read_type_hint) in
+			let eo = self#read_option (fun () -> self#read_expr) in
+			FVar(tho,eo)
+		| 1 -> FFun (self#read_func)
+		| 2 ->
+			let pn1 = self#read_placed_name in
+			let pn2 = self#read_placed_name in
+			let tho = self#read_option (fun () -> self#read_type_hint) in
+			let eo = self#read_option (fun () -> self#read_expr) in
+			FProp(pn1,pn2,tho,eo)
+		| _ -> assert false
+
+	method read_cfield =
+		let pn = self#read_placed_name in
+		let doc = self#read_option (fun () -> self#read_documentation) in
+		let pos = self#read_pos in
+		let meta = self#read_metadata in
+		let access = self#read_list (fun () -> self#read_placed_access) in
+		let kind = self#read_cfield_kind in
+		{
+			cff_name = pn;
+			cff_doc = doc;
+			cff_pos = pos;
+			cff_meta = meta;
+			cff_access = access;
+			cff_kind = kind;
+		}
+
+	method read_expr =
+		let p = self#read_pos in
+		let e = match read_byte ch with
+		| 0 ->
+			let s = self#read_string in
+			let suffix = self#read_option (fun () -> self#read_string) in
+			EConst (Int (s, suffix))
+		| 1 ->
+			let s = self#read_string in
+			let suffix = self#read_option (fun () -> self#read_string) in
+			EConst (Float (s, suffix))
+		| 2 ->
+			let s = self#read_string in
+			let qs = begin match read_byte ch with
+			| 0 -> SDoubleQuotes
+			| 1 -> SSingleQuotes
+			| _ -> assert false
+			end in
+			EConst (String (s,qs))
+		| 3 ->
+			EConst (Ident (self#read_string))
+		| 4 ->
+			let s1 = self#read_string in
+			let s2 = self#read_string in
+			EConst (Regexp(s1,s2))
+		| 5 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			EArray(e1,e2)
+		| 6 ->
+			let op = self#get_binop (read_byte ch) in
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			EBinop(op,e1,e2)
+		| 7 ->
+			let e = self#read_expr in
+			let s = self#read_string in
+			let kind = begin match read_byte ch with
+			| 0 -> EFNormal
+			| 1 -> EFSafe
+			| _ -> assert false
+			end in
+			EField(e,s,kind)
+		| 8 ->
+			EParenthesis (self#read_expr)
+		| 9 ->
+			let fields = self#read_list (fun () ->
+				let n = self#read_string in
+				let p = self#read_pos in
+				let qs = begin match read_byte ch with
+				| 0 -> NoQuotes
+				| 1 -> DoubleQuotes
+				| _ -> assert false
+				end in
+				let e = self#read_expr in
+				((n,p,qs),e)
+			) in
+			EObjectDecl fields
+		| 10 ->
+			let el = self#read_list (fun () -> self#read_expr) in
+			EArrayDecl el
+		| 11 ->
+			let e = self#read_expr in
+			let el = self#read_list (fun () -> self#read_expr) in
+			ECall(e,el)
+		| 12 ->
+			let ptp = self#read_placed_type_path in
+			let el = self#read_list (fun () -> self#read_expr) in
+			ENew(ptp,el)
+		| 13 ->
+			let (op,flag) = self#get_unop (read_byte ch) in
+			let e = self#read_expr in
+			EUnop(op,flag,e)
+		| 14 ->
+			let vl = self#read_list (fun () ->
+				let name = self#read_placed_name in
+				let final = self#read_bool in
+				let static = self#read_bool in
+				let t = self#read_option (fun () -> self#read_type_hint) in
+				let expr = self#read_option (fun () -> self#read_expr) in
+				let meta = self#read_metadata in
+				{
+					ev_name = name;
+					ev_final = final;
+					ev_static = static;
+					ev_type = t;
+					ev_expr = expr;
+					ev_meta = meta;
+				}
+			) in
+			EVars vl
+		| 15 ->
+			let fk = begin match read_byte ch with
+			| 0 -> FKAnonymous
+			| 1 ->
+				let pn = self#read_placed_name in
+				let b = self#read_bool in
+				FKNamed(pn,b)
+			| 2 -> FKArrow
+			| _ -> assert false end in
+			let f = self#read_func in
+			EFunction(fk,f)
+		| 16 ->
+			EBlock (self#read_list (fun () -> self#read_expr))
+		| 17 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			EFor(e1,e2)
+		| 18 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			EIf(e1,e2,None)
+		| 19 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			let e3 = self#read_expr in
+			EIf(e1,e2,Some e3)
+		| 20 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			EWhile(e1,e2,NormalWhile)
+		| 21 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			EWhile(e1,e2,DoWhile)
+		| 22 ->
+			let e1 = self#read_expr in
+			let cases = self#read_list (fun () ->
+				let el = self#read_list (fun () -> self#read_expr) in
+				let eg = self#read_option (fun () -> self#read_expr) in
+				let eo = self#read_option (fun () -> self#read_expr) in
+				let p = self#read_pos in
+				(el,eg,eo,p)
+			) in
+			let def = self#read_option (fun () ->
+				let eo = self#read_option (fun () -> self#read_expr) in
+				let p = self#read_pos in
+				(eo,p)
+			) in
+			ESwitch(e1,cases,def)
+		| 23 ->
+			let e1 = self#read_expr in
+			let catches = self#read_list (fun () ->
+				let pn = self#read_placed_name in
+				let th = self#read_option (fun () -> self#read_type_hint) in
+				let e = self#read_expr in
+				let p = self#read_pos in
+				(pn,th,e,p)
+			) in
+			ETry(e1,catches)
+		| 24 -> EReturn None
+		| 25 -> EReturn (Some (self#read_expr))
+		| 26 -> EBreak
+		| 27 -> EContinue
+		| 28 -> EUntyped (self#read_expr)
+		| 29 -> EThrow (self#read_expr)
+		| 30 -> ECast ((self#read_expr),None)
+		| 31 ->
+			let e1 = self#read_expr in
+			let th = self#read_type_hint in
+			ECast(e1,Some th)
+		| 32 ->
+			let e1 = self#read_expr in
+			let th = self#read_type_hint in
+			EIs(e1,th)
+		| 33 ->
+			let e1 = self#read_expr in
+			let dk = begin match read_byte ch with
+			| 0 -> DKCall
+			| 1 -> DKDot
+			| 2 -> DKStructure
+			| 3 -> DKMarked
+			| 4 -> DKPattern (self#read_bool)
+			| _ -> assert false end in
+			EDisplay(e1,dk)
+		| 34 ->
+			let e1 = self#read_expr in
+			let e2 = self#read_expr in
+			let e3 = self#read_expr in
+			ETernary(e1,e2,e3)
+		| 35 ->
+			let e1 = self#read_expr in
+			let th = self#read_type_hint in
+			ECheckType(e1,th)
+		| 36 ->
+			let m = self#read_metadata_entry in
+			let e = self#read_expr in
+			EMeta(m,e)
+		| _ -> assert false
+		in
+		(e,p)
+
+	(* Type instances *)
+
+	method resolve_ttp_ref = function
+		| 1 ->
+			let i = read_uleb128 ch in
+			(type_type_parameters.(i))
+		| 2 ->
+			let i = read_uleb128 ch in
+			(field_type_parameters.(i))
+		| 3 ->
+			let k = read_uleb128 ch in
+			local_type_parameters.(k)
+		| _ ->
+			die "" __LOC__
+
+	method read_type_instance =
+		let read_fun_arg () =
+			let name = self#read_string in
+			let opt = self#read_bool in
+			let t = self#read_type_instance in
+			(name,opt,t)
+		in
+		match (read_byte ch) with
+		| 0 ->
+			let i = read_uleb128 ch in
+			tmonos.(i)
+		| 1 ->
+			let i = read_uleb128 ch in
+			(type_type_parameters.(i)).ttp_type
+		| 2 ->
+			let i = read_uleb128 ch in
+			(field_type_parameters.(i)).ttp_type
+		| 3 ->
+			let k = read_uleb128 ch in
+			local_type_parameters.(k).ttp_type
+		| 4 ->
+			t_dynamic
+		| 10 ->
+			let c = self#read_class_ref in
+			c.cl_type
+		| 11 ->
+			let en = self#read_enum_ref in
+			en.e_type
+		| 12 ->
+			let a = self#read_abstract_ref in
+			TType(abstract_module_type a [],[])
+		| 13 ->
+			let e = self#read_expr in
+			let c = {null_class with cl_kind = KExpr e; cl_module = current_module } in
+			TInst(c, [])
+		| 20 ->
+			TFun([],api#basic_types.tvoid)
+		| 21 ->
+			let arg1 = read_fun_arg () in
+			TFun([arg1],api#basic_types.tvoid)
+		| 22 ->
+			let arg1 = read_fun_arg () in
+			let arg2 = read_fun_arg () in
+			TFun([arg1;arg2],api#basic_types.tvoid)
+		| 23 ->
+			let arg1 = read_fun_arg () in
+			let arg2 = read_fun_arg () in
+			let arg3 = read_fun_arg () in
+			TFun([arg1;arg2;arg3],api#basic_types.tvoid)
+		| 24 ->
+			let arg1 = read_fun_arg () in
+			let arg2 = read_fun_arg () in
+			let arg3 = read_fun_arg () in
+			let arg4 = read_fun_arg () in
+			TFun([arg1;arg2;arg3;arg4],api#basic_types.tvoid)
+		| 29 ->
+			let args = self#read_list read_fun_arg in
+			TFun(args,api#basic_types.tvoid)
+		| 30 ->
+			let ret = self#read_type_instance in
+			TFun([],ret)
+		| 31 ->
+			let arg1 = read_fun_arg () in
+			let ret = self#read_type_instance in
+			TFun([arg1],ret)
+		| 32 ->
+			let arg1 = read_fun_arg () in
+			let arg2 = read_fun_arg () in
+			let ret = self#read_type_instance in
+			TFun([arg1;arg2],ret)
+		| 33 ->
+			let arg1 = read_fun_arg () in
+			let arg2 = read_fun_arg () in
+			let arg3 = read_fun_arg () in
+			let ret = self#read_type_instance in
+			TFun([arg1;arg2;arg3],ret)
+		| 34 ->
+			let arg1 = read_fun_arg () in
+			let arg2 = read_fun_arg () in
+			let arg3 = read_fun_arg () in
+			let arg4 = read_fun_arg () in
+			let ret = self#read_type_instance in
+			TFun([arg1;arg2;arg3;arg4],ret)
+		| 39 ->
+			let args = self#read_list read_fun_arg in
+			let ret = self#read_type_instance in
+			TFun(args,ret)
+		| 40 ->
+			let c = self#read_class_ref in
+			TInst(c,[])
+		| 41 ->
+			let c = self#read_class_ref in
+			let t1 = self#read_type_instance in
+			TInst(c,[t1])
+		| 42 ->
+			let c = self#read_class_ref in
+			let t1 = self#read_type_instance in
+			let t2 = self#read_type_instance in
+			TInst(c,[t1;t2])
+		| 49 ->
+			let c = self#read_class_ref in
+			let tl = self#read_types in
+			TInst(c,tl)
+		| 50 ->
+			let en = self#read_enum_ref in
+			TEnum(en,[])
+		| 51 ->
+			let en = self#read_enum_ref in
+			let t1 = self#read_type_instance in
+			TEnum(en,[t1])
+		| 52 ->
+			let en = self#read_enum_ref in
+			let t1 = self#read_type_instance in
+			let t2 = self#read_type_instance in
+			TEnum(en,[t1;t2])
+		| 59 ->
+			let e = self#read_enum_ref in
+			let tl = self#read_types in
+			TEnum(e,tl)
+		| 60 ->
+			let td = self#read_typedef_ref in
+			TType(td,[])
+		| 61 ->
+			let td = self#read_typedef_ref in
+			let t1 = self#read_type_instance in
+			TType(td,[t1])
+		| 62 ->
+			let td = self#read_typedef_ref in
+			let t1 = self#read_type_instance in
+			let t2 = self#read_type_instance in
+			TType(td,[t1;t2])
+		| 69 ->
+			let t = self#read_typedef_ref in
+			let tl = self#read_types in
+			TType(t,tl)
+		| 70 ->
+			let a = self#read_abstract_ref in
+			TAbstract(a,[])
+		| 71 ->
+			let a = self#read_abstract_ref in
+			let t1 = self#read_type_instance in
+			TAbstract(a,[t1])
+		| 72 ->
+			let a = self#read_abstract_ref in
+			let t1 = self#read_type_instance in
+			let t2 = self#read_type_instance in
+			TAbstract(a,[t1;t2])
+		| 79 ->
+			let a = self#read_abstract_ref in
+			let tl = self#read_types in
+			TAbstract(a,tl)
+		| 80 ->
+			empty_anon
+		| 81 ->
+			TAnon self#read_anon_ref
+		| 89 ->
+			TDynamic (Some self#read_type_instance)
+		| 100 ->
+			api#basic_types.tvoid
+		| 101 ->
+			api#basic_types.tint
+		| 102 ->
+			api#basic_types.tfloat
+		| 103 ->
+			api#basic_types.tbool
+		| 104 ->
+			api#basic_types.tstring
+		| i ->
+			error (Printf.sprintf "Bad type instance id: %i" i)
+
+	method read_types =
+		self#read_list (fun () -> self#read_type_instance)
+
+	method read_type_parameters_forward =
+		let length = read_uleb128 ch in
+		Array.init length (fun _ ->
+			let path = self#read_path in
+			let pos = self#read_pos in
+			let host = match read_byte ch with
+				| 0 -> TPHType
+				| 1 -> TPHConstructor
+				| 2 -> TPHMethod
+				| 3 -> TPHEnumConstructor
+				| 4 -> TPHAnonField
+				| 5 -> TPHLocal
+				| i -> die (Printf.sprintf "Invalid type paramter host: %i" i) __LOC__
+			in
+			let c = mk_class current_module path pos pos in
+			let ttp = mk_type_param c host None None in
+			c.cl_kind <- KTypeParameter ttp;
+			ttp
+		)
+
+	method read_type_parameters_data (a : typed_type_param array) =
+		Array.iter (fun ttp ->
+			let meta = self#read_metadata in
+			let constraints = self#read_types in
+			let def = self#read_option (fun () -> self#read_type_instance) in
+			let c = ttp.ttp_class in
+			ttp.ttp_default <- def;
+			ttp.ttp_constraints <- Some (Lazy.from_val constraints);
+			c.cl_meta <- meta;
+		) a
+
+	(* Fields *)
+
+	method read_field_kind = match read_byte ch with
+		| 0 -> Method MethNormal
+		| 1 -> Method MethInline
+		| 2 -> Method MethDynamic
+		| 3 -> Method MethMacro
+		| 10 -> Var {v_read = AccNormal;v_write = AccNormal}
+		| 11 -> Var {v_read = AccNormal;v_write = AccNo}
+		| 12 -> Var {v_read = AccNormal;v_write = AccNever}
+		| 13 -> Var {v_read = AccNormal;v_write = AccCtor}
+		| 14 -> Var {v_read = AccNormal;v_write = AccCall}
+		| 20 -> Var {v_read = AccInline;v_write = AccNever}
+		| 30 -> Var {v_read = AccCall;v_write = AccNormal}
+		| 31 -> Var {v_read = AccCall;v_write = AccNo}
+		| 32 -> Var {v_read = AccCall;v_write = AccNever}
+		| 33 -> Var {v_read = AccCall;v_write = AccCtor}
+		| 34 -> Var {v_read = AccCall;v_write = AccCall}
+		| 100 ->
+			let f = function
+				| 0 -> AccNormal
+				| 1 -> AccNo
+				| 2 -> AccNever
+				| 3 -> AccCtor
+				| 4 -> AccCall
+				| 5 -> AccInline
+				| 6 ->
+					let s = self#read_string in
+					let so = self#read_option (fun () -> self#read_string) in
+					AccRequire(s,so)
+				| i ->
+					error (Printf.sprintf "Bad accessor kind: %i" i)
+			in
+			let r = f (read_byte ch) in
+			let w = f (read_byte ch) in
+			Var {v_read = r;v_write = w}
+		| i ->
+			error (Printf.sprintf "Bad field kind: %i" i)
+
+	method read_var_kind =
+		match read_byte ch with
+			| 0 -> VUser TVOLocalVariable
+			| 1 -> VUser TVOArgument
+			| 2 -> VUser TVOForVariable
+			| 3 -> VUser TVOPatternVariable
+			| 4 -> VUser TVOCatchVariable
+			| 5 -> VUser TVOLocalFunction
+			| 6 -> VGenerated
+			| 7 -> VInlined
+			| 8 -> VInlinedConstructorVariable
+			| 9 -> VExtractorVariable
+			| 10 -> VAbstractThis
+			| _ -> assert false
+
+	method read_var =
+		let id = read_uleb128 ch in
+		let name = self#read_string in
+		let kind = self#read_var_kind in
+		let flags = read_uleb128 ch in
+		let meta = self#read_metadata in
+		let pos = self#read_pos in
+		let v = {
+			v_id = api#get_var_id id;
+			v_name = name;
+			v_type = t_dynamic;
+			v_kind = kind;
+			v_meta = meta;
+			v_pos = pos;
+			v_extra = None;
+			v_flags = flags;
+		} in
+		v
+
+	method read_texpr fctx =
+
+		let declare_local () =
+			let v = fctx.vars.(read_uleb128 ch) in
+			v.v_extra <- self#read_option (fun () ->
+				let params = self#read_list (fun () ->
+					let i = read_uleb128 ch in
+					local_type_parameters.(i)
+				) in
+				let vexpr = self#read_option (fun () -> self#read_texpr fctx) in
+				{
+					v_params = params;
+					v_expr = vexpr;
+				};
+			);
+			v.v_type <- self#read_type_instance;
+			v
+		in
+		let update_pmin () =
+			fctx.pos := {!(fctx.pos) with pmin = read_leb128 ch};
+		in
+		let update_pmax () =
+			fctx.pos := {!(fctx.pos) with pmax = read_leb128 ch};
+		in
+		let update_pminmax () =
+			let pmin = read_leb128 ch in
+			let pmax = read_leb128 ch in
+			fctx.pos := {!(fctx.pos) with pmin; pmax};
+		in
+		let update_p () =
+			fctx.pos := self#read_pos;
+		in
+		let read_relpos () =
+			begin match read_byte ch with
+				| 0 ->
+					()
+				| 1 ->
+					update_pmin ()
+				| 2 ->
+					update_pmax ()
+				| 3 ->
+					update_pminmax ()
+				| 4 ->
+					update_p ()
+				| _ ->
+					assert false
+			end;
+			!(fctx.pos)
+		in
+		let rec loop () =
+			let loop2 () =
+				match read_byte ch with
+					(* values 0-19 *)
+					| 0 -> TConst TNull,None
+					| 1 -> TConst TThis,fctx.tthis
+					| 2 -> TConst TSuper,None
+					| 3 -> TConst (TBool false),(Some api#basic_types.tbool)
+					| 4 -> TConst (TBool true),(Some api#basic_types.tbool)
+					| 5 -> TConst (TInt self#read_i32),(Some api#basic_types.tint)
+					| 6 -> TConst (TFloat self#read_string),(Some api#basic_types.tfloat)
+					| 7 -> TConst (TString self#read_string),(Some api#basic_types.tstring)
+					| 13 -> TConst (TBool false),None
+					| 14 -> TConst (TBool true),None
+					| 15 -> TConst (TInt self#read_i32),None
+					| 16 -> TConst (TFloat self#read_string),None
+					| 17 -> TConst (TString self#read_string),None
+
+					(* vars 20-29 *)
+					| 20 ->
+						TLocal (fctx.vars.(read_uleb128 ch)),None
+					| 21 ->
+						let v = declare_local () in
+						TVar (v,None),(Some api#basic_types.tvoid)
+					| 22 ->
+						let v = declare_local () in
+						let e = loop () in
+						TVar (v, Some e),(Some api#basic_types.tvoid)
+
+					(* blocks 30-49 *)
+					| 30 ->
+						TBlock [],None
+					| 31 | 32 | 33 | 34 | 35 as i ->
+						let l = i - 30 in
+						let el = List.init l (fun _ -> loop ()) in
+						TBlock el,None
+					| 36 ->
+						let l = read_byte ch in
+						let el = List.init l (fun _ -> loop ()) in
+						TBlock el,None
+					| 39 ->
+						let el = self#read_list loop in
+						TBlock el,None
+
+					(* function 50-59 *)
+					| 50 ->
+						let read_tfunction_arg () =
+							let v = declare_local () in
+							let cto = self#read_option loop in
+							(v,cto)
+						in
+						let args = self#read_list read_tfunction_arg in
+						let r = self#read_type_instance in
+						let e = loop () in
+						TFunction {
+							tf_args = args;
+							tf_type = r;
+							tf_expr = e;
+						},None
+					(* texpr compounds 60-79 *)
+					| 60 ->
+						let e1 = loop () in
+						let e2 = loop () in
+						TArray (e1,e2),None
+					| 61 ->
+						let e = loop () in
+						TParenthesis e,Some e.etype
+					| 62 ->
+						TArrayDecl (loop_el()),None
+					| 63 ->
+						let fl = self#read_list (fun () ->
+							let name = self#read_string in
+							let p = self#read_pos in
+							let qs = match read_byte ch with
+								| 0 -> NoQuotes
+								| 1 -> DoubleQuotes
+								| _ -> assert false
+							in
+							let e = loop () in
+							((name,p,qs),e)
+						) in
+						TObjectDecl fl,None
+					| 65 ->
+						let m = self#read_metadata_entry in
+						let e1 = loop () in
+						TMeta (m,e1),None
+
+					(* calls 70 - 79 *)
+					| 70 ->
+						let e1 = loop () in
+						TCall(e1,[]),None
+					| 71 | 72 | 73 | 74 as i ->
+						let e1 = loop () in
+						let el = List.init (i - 70) (fun _ -> loop ()) in
+						TCall(e1,el),None
+					| 79 ->
+						let e1 = loop () in
+						let el = self#read_list loop in
+						TCall(e1,el),None
+
+					(* branching 80-89 *)
+					| 80 ->
+						let e1 = loop () in
+						let e2 = loop () in
+						TIf(e1,e2,None),(Some api#basic_types.tvoid)
+					| 81 ->
+						let e1 = loop () in
+						let e2 = loop () in
+						let e3 = loop () in
+						TIf(e1,e2,Some e3),None
+					| 82 ->
+						let subject = loop () in
+						let cases = self#read_list (fun () ->
+							let patterns = loop_el() in
+							let ec = loop () in
+							{ case_patterns = patterns; case_expr = ec}
+						) in
+						let def = self#read_option (fun () -> loop ()) in
+						TSwitch {
+							switch_subject = subject;
+							switch_cases = cases;
+							switch_default = def;
+							switch_exhaustive = true;
+						},None
+					| 83 ->
+						let e1 = loop () in
+						let catches = self#read_list (fun () ->
+							let v = declare_local () in
+							let e = loop () in
+							(v,e)
+						) in
+						TTry(e1,catches),None
+					| 84 ->
+						let e1 = loop () in
+						let e2 = loop () in
+						TWhile(e1,e2,NormalWhile),(Some api#basic_types.tvoid)
+					| 85 ->
+						let e1 = loop () in
+						let e2 = loop () in
+						TWhile(e1,e2,DoWhile),(Some api#basic_types.tvoid)
+					| 86 ->
+						let v  = declare_local () in
+						let e1 = loop () in
+						let e2 = loop () in
+						TFor(v,e1,e2),(Some api#basic_types.tvoid)
+
+					(* control flow 90-99 *)
+					| 90 ->
+						TReturn None,None
+					| 91 ->
+						TReturn (Some (loop ())),None
+					| 92 ->
+						TContinue,None
+					| 93 ->
+						TBreak,None
+					| 94 ->
+						TThrow (loop ()),None
+
+					(* access 100-119 *)
+					| 100 ->
+						TEnumIndex (loop ()),(Some api#basic_types.tint)
+					| 101 ->
+						let e1 = loop () in
+						let ef = self#read_enum_field_ref in
+						let i = read_uleb128 ch in
+						TEnumParameter(e1,ef,i),None
+					| 102 ->
+						let e1 = loop () in
+						let c = self#read_class_ref in
+						let tl = self#read_types in
+						let cf = self#read_field_ref in
+						TField(e1,FInstance(c,tl,cf)),None
+					| 103 ->
+						let e1 = loop () in
+						let c = self#read_class_ref in
+						let cf = self#read_field_ref in
+						TField(e1,FStatic(c,cf)),None
+					| 104 ->
+						let e1 = loop () in
+						let cf = self#read_anon_field_ref in
+						TField(e1,FAnon(cf)),None
+					| 105 ->
+						let e1 = loop () in
+						let c = self#read_class_ref in
+						let tl = self#read_types in
+						let cf = self#read_field_ref in
+						TField(e1,FClosure(Some(c,tl),cf)),None
+					| 106 ->
+						let e1 = loop () in
+						let cf = self#read_anon_field_ref in
+						TField(e1,FClosure(None,cf)),None
+					| 107 ->
+						let e1 = loop () in
+						let en = self#read_enum_ref in
+						let ef = self#read_enum_field_ref in
+						TField(e1,FEnum(en,ef)),None
+					| 108 ->
+						let e1 = loop () in
+						let s = self#read_string in
+						TField(e1,FDynamic s),None
+
+					| 110 ->
+						let p = read_relpos () in
+						let c = self#read_class_ref in
+						let cf = self#read_field_ref in
+						let e1 = Texpr.Builder.make_static_this c p in
+						TField(e1,FStatic(c,cf)),None
+					| 111 ->
+						let p = read_relpos () in
+						let c = self#read_class_ref in
+						let tl = self#read_types in
+						let cf = self#read_field_ref in
+						let ethis = mk (TConst TThis) (Option.get fctx.tthis) p in
+						TField(ethis,FInstance(c,tl,cf)),None
+
+					(* module types 120-139 *)
+					| 120 ->
+						let c = self#read_class_ref in
+						TTypeExpr (TClassDecl c),(Some c.cl_type)
+					| 121 ->
+						let en = self#read_enum_ref in
+						TTypeExpr (TEnumDecl en),(Some en.e_type)
+					| 122 ->
+						TTypeExpr (TAbstractDecl self#read_abstract_ref),None
+					| 123 ->
+						TTypeExpr (TTypeDecl self#read_typedef_ref),None
+					| 124 ->
+						TCast(loop (),None),None
+					| 125 ->
+						let e1 = loop () in
+						let (pack,mname,tname) = self#read_full_path in
+						let mt = self#resolve_type pack mname tname in
+						TCast(e1,Some mt),None
+					| 126 ->
+						let c = self#read_class_ref in
+						let tl = self#read_types in
+						let el = loop_el() in
+						TNew(c,tl,el),None
+					| 127 ->
+						let ttp = self#resolve_ttp_ref (read_uleb128 ch) in
+						let tl = self#read_types in
+						let el = loop_el() in
+						TNew(ttp.ttp_class,tl,el),None
+					| 128 ->
+						let ttp = self#resolve_ttp_ref (read_uleb128 ch) in
+						TTypeExpr (TClassDecl ttp.ttp_class),None
+
+					(* unops 140-159 *)
+					| i when i >= 140 && i < 160 ->
+						let (op,flag) = self#get_unop (i - 140) in
+						let e = loop () in
+						TUnop(op,flag,e),None
+
+					(* binops 160-219 *)
+					| i when i >= 160 && i < 220 ->
+						let op = self#get_binop (i - 160) in
+						let e1 = loop () in
+						let e2 = loop () in
+						TBinop(op,e1,e2),None
+					(* rest 250-254 *)
+					| 250 ->
+						TIdent (self#read_string),None
+
+					| i ->
+						die (Printf.sprintf "  [ERROR] Unhandled texpr %d at:" i) __LOC__
+				in
+				let e,t = loop2 () in
+				let t = match t with
+					| None -> fctx.t_pool.(read_uleb128 ch)
+					| Some t -> t
+				in
+				let p = read_relpos () in
+				let e = {
+					eexpr = e;
+					etype = t;
+					epos = p;
+				} in
+				e
+		and loop_el () =
+			self#read_list loop
+		in
+		loop()
+
+	method read_class_field_forward =
+		let name = self#read_string in
+		let pos,name_pos = self#read_pos_pair in
+		let overloads = self#read_list (fun () -> self#read_class_field_forward) in
+		{ null_field with cf_name = name; cf_pos = pos; cf_name_pos = name_pos; cf_overloads = overloads }
+
+	method start_texpr =
+		begin match read_byte ch with
+			| 0 ->
+				()
+			| 1 ->
+				let a = self#read_type_parameters_forward in
+				local_type_parameters <- a;
+				self#read_type_parameters_data a;
+			| i ->
+				die "" __LOC__
+		end;
+		let tthis = self#read_option (fun () -> self#read_type_instance) in
+		let l = read_uleb128 ch in
+		let ts = Array.init l (fun _ ->
+			self#read_type_instance
+		) in
+		let l = read_uleb128 ch in
+		let vars = Array.init l (fun _ ->
+			self#read_var
+		) in
+		create_field_reader_context self#read_pos ts vars tthis
+
+	method read_field_type_parameters =
+		let num_params = read_uleb128 ch in
+		begin match read_byte ch with
+			| 0 ->
+				()
+			| 1 ->
+				let a = self#read_type_parameters_forward in
+				field_type_parameters <- a;
+				self#read_type_parameters_data a;
+				field_type_parameter_offset <- 0; (* num_params is added below *)
+			| i ->
+				die "" __LOC__
+		end;
+		let params = List.init num_params (fun offset ->
+			field_type_parameters.(field_type_parameter_offset + offset)
+		) in
+		field_type_parameter_offset <- field_type_parameter_offset + num_params;
+		params
+
+	method read_expression (fctx : field_reader_context) =
+		let e = self#read_texpr fctx in
+		let e_unopt = self#read_option (fun () -> self#read_texpr fctx) in
+		e,e_unopt
+
+	val class_field_infos = ClassFieldInfos.create ()
+
+	method read_class_field_data (cf : tclass_field) : unit =
+		let params = self#read_field_type_parameters in
+
+		let t = self#read_type_instance in
+
+		let flags = read_uleb128 ch in
+
+		let doc = self#read_option (fun () -> self#read_documentation) in
+		cf.cf_meta <- self#read_metadata;
+		let kind = self#read_field_kind in
+
+		let expr,expr_unoptimized = match read_byte ch with
+			| 0 ->
+				None,None
+			| 1 ->
+				let fctx = self#start_texpr in
+				let e,e_unopt = self#read_expression fctx in
+				(Some e,e_unopt)
+			| 2 ->
+				(* store type parameter info for EXD *)
+				let info = ClassFieldInfo.create field_type_parameters in
+				ClassFieldInfos.set class_field_infos info cf;
+				None,None
+			| _ ->
+				die "" __LOC__
+		in
+
+		cf.cf_type <- t;
+		cf.cf_doc <- doc;
+		cf.cf_kind <- kind;
+		cf.cf_expr <- expr;
+		cf.cf_expr_unoptimized <- expr_unoptimized;
+		cf.cf_params <- params;
+		cf.cf_flags <- flags
+
+	method read_class_field_and_overloads_data (cf : tclass_field) =
+		let rec loop depth cfl = match cfl with
+			| cf :: cfl ->
+				assert (depth > 0);
+				self#read_class_field_data cf;
+				loop (depth - 1) cfl
+			| [] ->
+				assert (depth = 0)
+		in
+		loop (read_uleb128 ch) (cf :: cf.cf_overloads);
+
+	method select_class_type_parameters (c: tclass) =
+		match c.cl_kind with
+		| KAbstractImpl a ->
+			type_type_parameters <- Array.of_list a.a_params
+		| _ ->
+			type_type_parameters <- Array.of_list c.cl_params
+
+	method read_class_fields (c : tclass) =
+		self#select_class_type_parameters c;
+		let _ = self#read_option (fun f ->
+			let cf = Option.get c.cl_constructor in
+			self#read_class_field_and_overloads_data cf
+		) in
+		let _ = self#read_option (fun f ->
+			let cf = Option.get c.cl_init in
+			self#read_class_field_and_overloads_data cf
+		) in
+		let rec loop ref_kind num cfl = match cfl with
+			| cf :: cfl ->
+				assert (num > 0);
+				self#read_class_field_and_overloads_data cf;
+				loop ref_kind (num - 1) cfl
+			| [] ->
+				assert (num = 0)
+		in
+		loop CfrMember (read_uleb128 ch) c.cl_ordered_fields;
+		loop CfrStatic (read_uleb128 ch) c.cl_ordered_statics;
+		(match c.cl_kind with KModuleFields md -> md.m_statics <- Some c; | _ -> ());
+
+	method read_enum_fields (e : tenum) =
+		type_type_parameters <- Array.of_list e.e_params;
+		ignore(self#read_list (fun () ->
+			let name = self#read_string in
+			let ef = PMap.find name e.e_constrs in
+			ef.ef_params <- self#read_field_type_parameters;
+			ef.ef_type <- self#read_type_instance;
+			ef.ef_doc <- self#read_option (fun () -> self#read_documentation);
+			ef.ef_meta <- self#read_metadata;
+		))
+
+	(* Module types *)
+
+	method read_common_module_type (infos : tinfos) =
+		infos.mt_private <- self#read_bool;
+		infos.mt_doc <- self#read_option (fun () -> self#read_documentation);
+		infos.mt_meta <- self#read_metadata;
+		let params = Array.of_list infos.mt_params in
+		type_type_parameters <- params;
+		self#read_type_parameters_data params;
+		infos.mt_params <- Array.to_list type_type_parameters;
+		infos.mt_using <- self#read_list (fun () ->
+			let c = self#read_class_ref in
+			let p = self#read_pos in
+			(c,p)
+		)
+
+	method read_class_kind = match read_byte ch with
+		| 0 -> KNormal
+		| 1 -> die "" __LOC__
+		| 2 -> KExpr self#read_expr
+		| 3 -> KGeneric
+		| 4 ->
+			let c = self#read_class_ref in
+			let tl = self#read_types in
+			KGenericInstance(c,tl)
+		| 5 -> KMacroType
+		| 6 -> KGenericBuild (self#read_list (fun () -> self#read_cfield))
+		| 7 -> KAbstractImpl self#read_abstract_ref
+		| 8 -> KModuleFields current_module
+		| i ->
+			error (Printf.sprintf "Invalid class kind id: %i" i)
+
+	method read_class (c : tclass) =
+		self#read_common_module_type (Obj.magic c);
+		c.cl_kind <- self#read_class_kind;
+		let read_relation () =
+			let c = self#read_class_ref in
+			let tl = self#read_types in
+			(c,tl)
+		in
+		c.cl_super <- self#read_option read_relation;
+		c.cl_implements <- self#read_list read_relation;
+		c.cl_dynamic <- self#read_option (fun () -> self#read_type_instance);
+		c.cl_array_access <- self#read_option (fun () -> self#read_type_instance);
+
+	method read_abstract (a : tabstract) =
+		self#read_common_module_type (Obj.magic a);
+		a.a_impl <- self#read_option (fun () -> self#read_class_ref);
+		begin match read_byte ch with
+			| 0 ->
+				a.a_this <- TAbstract(a,extract_param_types a.a_params)
+			| _ ->
+				a.a_this <- self#read_type_instance;
+		end;
+		a.a_from <- self#read_list (fun () -> self#read_type_instance);
+		a.a_to <- self#read_list (fun () -> self#read_type_instance);
+		a.a_enum <- self#read_bool;
+
+	method read_abstract_fields (a : tabstract) =
+		a.a_array <- self#read_list (fun () -> self#read_field_ref);
+		a.a_read <- self#read_option (fun () -> self#read_field_ref);
+		a.a_write <- self#read_option (fun () -> self#read_field_ref);
+		a.a_call <- self#read_option (fun () -> self#read_field_ref);
+
+		a.a_ops <- self#read_list (fun () ->
+			let i = read_byte ch in
+			let op = self#get_binop i in
+			let cf = self#read_field_ref in
+			(op, cf)
+		);
+
+		a.a_unops <- self#read_list (fun () ->
+			let i = read_byte ch in
+			let (op, flag) = self#get_unop i in
+			let cf = self#read_field_ref in
+			(op, flag, cf)
+		);
+
+		a.a_from_field <- self#read_list (fun () ->
+			let cf = self#read_field_ref in
+			let t = match cf.cf_type with
+				| TFun((_,_,t) :: _, _) -> t
+				| _ -> die "" __LOC__
+			in
+			(t,cf)
+		);
+
+		a.a_to_field <- self#read_list (fun () ->
+			let cf = self#read_field_ref in
+			let t = match cf.cf_type with
+				| TFun(_, t) -> t
+				| _ -> die "" __LOC__
+			in
+			(t,cf)
+		);
+
+	method read_enum (e : tenum) =
+		self#read_common_module_type (Obj.magic e);
+		e.e_extern <- self#read_bool;
+		e.e_names <- self#read_list (fun () -> self#read_string);
+
+	method read_typedef (td : tdef) =
+		self#read_common_module_type (Obj.magic td);
+		let t = self#read_type_instance in
+		match td.t_type with
+		| TMono r ->
+			(match r.tm_type with
+			| None -> Monomorph.bind r t;
+			| Some t' -> die (Printf.sprintf "typedef %s is already initialized to %s, but new init to %s was attempted" (s_type_path td.t_path) (s_type_kind t') (s_type_kind t)) __LOC__)
+		| _ ->
+			die "" __LOC__
+
+	(* Chunks *)
+
+	method read_string_pool =
+		let l = read_uleb128 ch in
+		Array.init l (fun i ->
+			self#read_raw_string;
+		);
+
+	method read_efr =
+		let l = read_uleb128 ch in
+		let a = Array.init l (fun i ->
+			let en = self#read_enum_ref in
+			let name = self#read_string in
+			PMap.find name en.e_constrs
+		) in
+		enum_fields <- a
+
+	method read_afr =
+		let l = read_uleb128 ch in
+		let a = Array.init l (fun _ -> self#read_class_field_forward) in
+		anon_fields <- a
+
+	method read_cfr =
+		let l = read_uleb128 ch in
+		let a = Array.init l (fun i ->
+			let c = self#read_class_ref in
+			let kind = match read_byte ch with
+				| 0 -> CfrStatic
+				| 1 -> CfrMember
+				| 2 -> CfrConstructor
+				| 3 -> CfrInit
+				| _ -> die "" __LOC__
+			in
+			let cf =  match kind with
+				| CfrStatic ->
+					let name = self#read_string in
+					begin try
+						PMap.find name c.cl_statics
+					with Not_found ->
+						raise (HxbFailure (Printf.sprintf "Could not read static field %s on %s while hxbing %s" name (s_type_path c.cl_path) (s_type_path current_module.m_path)))
+					end;
+				| CfrMember ->
+					let name = self#read_string in
+					begin try
+						PMap.find name c.cl_fields
+					with Not_found ->
+						raise (HxbFailure (Printf.sprintf "Could not read instance field %s on %s while hxbing %s" name (s_type_path c.cl_path) (s_type_path current_module.m_path)))
+					end
+				| CfrConstructor ->
+					Option.get c.cl_constructor
+				| CfrInit ->
+					Option.get c.cl_init
+			in
+			let pick_overload cf depth =
+				let rec loop depth cfl = match cfl with
+					| cf :: cfl ->
+						if depth = 0 then
+							cf
+						else
+							loop (depth - 1) cfl
+					| [] ->
+						raise (HxbFailure (Printf.sprintf "Bad overload depth for %s on %s: %i" cf.cf_name (s_type_path c.cl_path) depth))
+				in
+				let cfl = cf :: cf.cf_overloads in
+				loop depth cfl
+			in
+			let depth = read_uleb128 ch in
+			if depth = 0 then
+				cf
+			else
+				pick_overload cf depth;
+		) in
+		class_fields <- a
+
+	method read_cfd =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let c = classes.(i) in
+			self#read_class_fields c;
+		done
+
+	method read_exd =
+		ignore(self#read_list (fun () ->
+			let c = self#read_class_ref in
+			self#read_list (fun () ->
+				let cf = self#read_field_ref in
+				let length = read_uleb128 ch in
+				let bytes = read_bytes ch length in
+				let ch_cf = BytesWithPosition.create bytes in
+				let read_expressions () =
+					self#select_class_type_parameters c;
+					field_type_parameters <- (ClassFieldInfos.get class_field_infos cf).type_parameters;
+					ClassFieldInfos.unset class_field_infos cf;
+					field_type_parameter_offset <- 0;
+					let old = ch in
+					ch <- ch_cf;
+					let fctx = self#start_texpr in
+					let e,e_unopt = self#read_expression fctx in
+					ch <- old;
+					cf.cf_expr <- Some e;
+					cf.cf_expr_unoptimized <- e_unopt;
+				in
+				if api#read_expression_eagerly cf then
+					read_expressions ()
+				else begin
+					let t = cf.cf_type in
+					let r = ref (lazy_available t) in
+					r := lazy_wait (fun() ->
+						cf.cf_type <- t;
+						r := lazy_available t;
+						read_expressions ();
+						t
+					);
+					cf.cf_type <- TLazy r
+				end
+			)
+		))
+
+	method read_afd =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let a = abstracts.(i) in
+			self#read_abstract_fields a;
+		done
+
+	method read_cld =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let c = classes.(i) in
+			self#read_class c;
+		done
+
+	method read_abd =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let a = abstracts.(i) in
+			self#read_abstract a;
+		done
+
+	method read_end =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let en = enums.(i) in
+			self#read_enum en;
+		done
+
+	method read_efd =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let e = enums.(i) in
+			self#read_enum_fields e;
+			Type.unify (TType(enum_module_type e,[])) e.e_type
+		done
+
+	method read_anon an =
+		let read_fields () =
+			let rec loop acc i =
+				if i = 0 then
+					acc
+				else begin
+					let cf = self#read_anon_field_ref in
+					loop (PMap.add cf.cf_name cf acc) (i - 1)
+				end
+			in
+			an.a_fields <- loop PMap.empty (read_uleb128 ch)
+		in
+
+		begin match read_byte ch with
+		| 0 ->
+			an.a_status := Closed;
+			read_fields ()
+		| 1 ->
+			an.a_status := Const;
+			read_fields ()
+		| 2 ->
+			an.a_status := Extend self#read_types;
+			read_fields ()
+		| _ -> assert false
+		end;
+
+		an
+
+	method read_tdd =
+		let l = read_uleb128 ch in
+		for i = 0 to l - 1 do
+			let t = typedefs.(i) in
+			self#read_typedef t;
+		done
+
+	method read_clr =
+		let l = read_uleb128 ch in
+		classes <- (Array.init l (fun i ->
+				let (pack,mname,tname) = self#read_full_path in
+				match self#resolve_type pack mname tname with
+				| TClassDecl c ->
+					c
+				| _ ->
+					error ("Unexpected type where class was expected: " ^ (s_type_path (pack,tname)))
+		))
+
+	method read_abr =
+		let l = read_uleb128 ch in
+		abstracts <- (Array.init l (fun i ->
+			let (pack,mname,tname) = self#read_full_path in
+			match self#resolve_type pack mname tname with
+			| TAbstractDecl a ->
+				a
+			| _ ->
+				error ("Unexpected type where abstract was expected: " ^ (s_type_path (pack,tname)))
+		))
+
+	method read_enr =
+		let l = read_uleb128 ch in
+		enums <- (Array.init l (fun i ->
+			let (pack,mname,tname) = self#read_full_path in
+			match self#resolve_type pack mname tname with
+			| TEnumDecl en ->
+				en
+			| _ ->
+				error ("Unexpected type where enum was expected: " ^ (s_type_path (pack,tname)))
+		))
+
+	method read_tdr =
+		let l = read_uleb128 ch in
+		typedefs <- (Array.init l (fun i ->
+			let (pack,mname,tname) = self#read_full_path in
+			match self#resolve_type pack mname tname with
+			| TTypeDecl tpd ->
+				tpd
+			| _ ->
+				error ("Unexpected type where typedef was expected: " ^ (s_type_path (pack,tname)))
+		))
+
+	method read_mdr =
+		let length = read_uleb128 ch in
+		for _ = 0 to length - 1 do
+			let path = self#read_path in
+			ignore(api#resolve_module path)
+		done
+
+	method read_mtf =
+		self#read_list (fun () ->
+			let kind = read_byte ch in
+			let path = self#read_path in
+			let pos,name_pos = self#read_pos_pair in
+			let params = self#read_type_parameters_forward in
+			let mt = match kind with
+			| 0 ->
+				let c = mk_class current_module path pos name_pos in
+				c.cl_params <- Array.to_list params;
+				c.cl_flags <- read_uleb128 ch;
+
+				let read_field () =
+					self#read_class_field_forward;
+				in
+
+				c.cl_constructor <- self#read_option read_field;
+				c.cl_init <- self#read_option read_field;
+				let read_fields i =
+					let rec loop acc_l acc_pm i =
+						if i = 0 then
+							acc_l,acc_pm
+						else begin
+							let cf = self#read_class_field_forward in
+							loop (cf :: acc_l) (PMap.add cf.cf_name cf acc_pm) (i - 1)
+						end
+					in
+					loop [] PMap.empty i
+				in
+				let num_fields = read_uleb128 ch in
+				let num_statics = read_uleb128 ch in
+				let l,pm = read_fields num_fields in
+				c.cl_ordered_fields <- l;
+				c.cl_fields <- pm;
+				let l,pm = read_fields num_statics in
+				c.cl_ordered_statics <- l;
+				c.cl_statics <- pm;
+
+				TClassDecl c
+			| 1 ->
+				let en = mk_enum current_module path pos name_pos in
+				en.e_params <- Array.to_list params;
+
+				let read_field () =
+					let name = self#read_string in
+					let pos,name_pos = self#read_pos_pair in
+					let index = read_byte ch in
+
+					{ null_enum_field with
+						ef_name = name;
+						ef_pos = pos;
+						ef_name_pos = name_pos;
+						ef_index = index;
+					}
+				in
+				let rec loop acc i =
+					if i = 0 then
+						acc
+					else begin
+						let ef = read_field () in
+						loop (PMap.add ef.ef_name ef acc) (i - 1)
+					end
+				in
+				en.e_constrs <- loop PMap.empty (read_uleb128 ch);
+				TEnumDecl en
+			| 2 ->
+				let td = mk_typedef current_module path pos name_pos (mk_mono()) in
+				td.t_params <- Array.to_list params;
+				typedefs <- Array.append typedefs (Array.make 1 td);
+				TTypeDecl td
+			| 3 ->
+				let a = mk_abstract current_module path pos name_pos in
+				a.a_params <- Array.to_list params;
+				abstracts <- Array.append abstracts (Array.make 1 a);
+				TAbstractDecl a
+			| _ ->
+				error ("Invalid type kind: " ^ (string_of_int kind));
+			in
+			mt
+		)
+
+	method read_mdf =
+		let path = self#read_path in
+		let file = self#read_string in
+
+		let l = read_uleb128 ch in
+		anons <- Array.init l (fun _ -> { a_fields = PMap.empty; a_status = ref Closed });
+		tmonos <- Array.init (read_uleb128 ch) (fun _ -> mk_mono());
+		api#make_module path file
+
+	method private read_chunk_prefix =
+		let name = Bytes.unsafe_to_string (read_bytes ch 3) in
+		let size = Int32.to_int self#read_i32 in
+		(name,size)
+
+	method private read_chunk_data' (kind : chunk_kind) =
+		match kind with
+		| STR ->
+			string_pool <- self#read_string_pool;
+		| DOC ->
+			doc_pool <- self#read_string_pool;
+		| MDF ->
+			current_module <- self#read_mdf;
+		| MTF ->
+			current_module.m_types <- self#read_mtf;
+			api#add_module current_module;
+		| MDR ->
+			self#read_mdr;
+		| CLR ->
+			self#read_clr;
+		| ENR ->
+			self#read_enr;
+		| ABR ->
+			self#read_abr;
+		| TDR ->
+			self#read_tdr;
+		| AFR ->
+			self#read_afr;
+		| CLD ->
+			self#read_cld;
+		| END ->
+			self#read_end;
+		| ABD ->
+			self#read_abd;
+		| TDD ->
+			self#read_tdd;
+		| EOT ->
+			()
+		| EFR ->
+			self#read_efr;
+		| CFR ->
+			self#read_cfr;
+		| CFD ->
+			self#read_cfd;
+		| EFD ->
+			self#read_efd;
+		| AFD ->
+			self#read_afd;
+		| EOF ->
+			()
+		| EXD ->
+			self#read_exd;
+		| EOM ->
+			incr stats.modules_fully_restored;
+
+	method private read_chunk_data kind =
+		let path = String.concat "_" (ExtLib.String.nsplit (s_type_path mpath) ".") in
+		let id = ["hxb";"read";string_of_chunk_kind kind;path] in
+		let close = Timer.timer id in
+		self#read_chunk_data' kind;
+		close()
+
+	method read_chunks (new_api : hxb_reader_api) (chunks : cached_chunks) =
+		fst (self#read_chunks_until new_api chunks EOM)
+
+	method read_chunks_until (new_api : hxb_reader_api) (chunks : cached_chunks) end_chunk =
+		api <- new_api;
+		let rec loop = function
+			| (kind,data) :: chunks ->
+				ch <- BytesWithPosition.create data;
+				self#read_chunk_data kind;
+				if kind = end_chunk then chunks else loop chunks
+			| [] -> die "" __LOC__
+		in
+		let remaining = loop chunks in
+		(current_module, remaining)
+
+	method read (new_api : hxb_reader_api) (bytes : bytes) =
+		api <- new_api;
+		ch <- BytesWithPosition.create bytes;
+		if (Bytes.to_string (read_bytes ch 3)) <> "hxb" then
+			raise (HxbFailure "magic");
+		let version = read_byte ch in
+		if version <> hxb_version then
+			raise (HxbFailure (Printf.sprintf "version mismatch: hxb version %i, reader version %i" version hxb_version));
+		(fun end_chunk ->
+			let rec loop () =
+				let (name,size) = self#read_chunk_prefix in
+				let kind = chunk_kind_of_string name in
+				self#read_chunk_data kind;
+				if kind <> end_chunk then begin
+					loop()
+				end
+			in
+			loop();
+			current_module
+		)
+end

+ 12 - 0
src/compiler/hxb/hxbReaderApi.ml

@@ -0,0 +1,12 @@
+open Globals
+open Type
+
+class virtual hxb_reader_api = object(self)
+	method virtual make_module : path -> string -> module_def
+	method virtual add_module : module_def -> unit
+	method virtual resolve_type : string list -> string -> string -> module_type
+	method virtual resolve_module : path -> module_def
+	method virtual basic_types : basic_types
+	method virtual get_var_id : int -> int
+	method virtual read_expression_eagerly : tclass_field -> bool
+end

+ 2330 - 0
src/compiler/hxb/hxbWriter.ml

@@ -0,0 +1,2330 @@
+open Globals
+open Ast
+open Type
+open HxbData
+open Tanon_identification
+
+let rec binop_index op = match op with
+	| OpAdd -> 0
+	| OpMult -> 1
+	| OpDiv -> 2
+	| OpSub -> 3
+	| OpAssign -> 4
+	| OpEq -> 5
+	| OpNotEq -> 6
+	| OpGt -> 7
+	| OpGte -> 8
+	| OpLt -> 9
+	| OpLte -> 10
+	| OpAnd -> 11
+	| OpOr -> 12
+	| OpXor -> 13
+	| OpBoolAnd -> 14
+	| OpBoolOr -> 15
+	| OpShl -> 16
+	| OpShr -> 17
+	| OpUShr -> 18
+	| OpMod -> 19
+	| OpInterval -> 20
+	| OpArrow -> 21
+	| OpIn -> 22
+	| OpNullCoal -> 23
+	| OpAssignOp op -> 30 + binop_index op
+
+let unop_index op flag = match op,flag with
+	| Increment,Prefix -> 0
+	| Decrement,Prefix -> 1
+	| Not,Prefix -> 2
+	| Neg,Prefix -> 3
+	| NegBits,Prefix -> 4
+	| Spread,Prefix -> 5
+	| Increment,Postfix -> 6
+	| Decrement,Postfix -> 7
+	| Not,Postfix -> 8
+	| Neg,Postfix -> 9
+	| NegBits,Postfix -> 10
+	| Spread,Postfix -> 11
+
+type hxb_writer_stats = {
+	type_instance_kind_writes : int array;
+	texpr_writes : int array;
+	type_instance_immediate : int ref;
+	type_instance_cache_hits : int ref;
+	type_instance_cache_misses : int ref;
+	pos_writes_full : int ref;
+	pos_writes_min : int ref;
+	pos_writes_max : int ref;
+	pos_writes_minmax : int ref;
+	pos_writes_eq : int ref;
+	chunk_sizes : (string,int ref * int ref) Hashtbl.t;
+}
+
+let create_hxb_writer_stats () = {
+	type_instance_kind_writes = Array.make 255 0;
+	texpr_writes = Array.make 255 0;
+	type_instance_immediate = ref 0;
+	type_instance_cache_hits = ref 0;
+	type_instance_cache_misses = ref 0;
+	pos_writes_full = ref 0;
+	pos_writes_min = ref 0;
+	pos_writes_max = ref 0;
+	pos_writes_minmax = ref 0;
+	pos_writes_eq = ref 0;
+	chunk_sizes = Hashtbl.create 0;
+}
+
+let dump_stats name stats =
+	let sort_and_filter_array a =
+		let _,kind_writes = Array.fold_left (fun (index,acc) writes ->
+			(index + 1,if writes = 0 then acc else (index,writes) :: acc)
+		) (0,[]) a in
+		let kind_writes = List.sort (fun (_,writes1) (_,writes2) -> compare writes2 writes1) kind_writes in
+		List.map (fun (index,writes) -> Printf.sprintf "    %3i: %9i" index writes) kind_writes
+	in
+	let t_kind_writes = sort_and_filter_array stats.type_instance_kind_writes in
+	print_endline (Printf.sprintf "hxb_writer stats for %s" name);
+	print_endline "  type instance kind writes:";
+	List.iter print_endline t_kind_writes;
+	let texpr_writes = sort_and_filter_array stats.texpr_writes in
+	print_endline "  texpr writes:";
+	List.iter print_endline texpr_writes;
+
+	print_endline "  type instance writes:";
+	print_endline (Printf.sprintf "     immediate: %9i" !(stats.type_instance_immediate));
+	print_endline (Printf.sprintf "    cache hits: %9i" !(stats.type_instance_cache_hits));
+	print_endline (Printf.sprintf "    cache miss: %9i" !(stats.type_instance_cache_misses));
+	print_endline "  pos writes:";
+	print_endline (Printf.sprintf "      full: %9i\n       min: %9i\n       max: %9i\n    minmax: %9i\n     equal: %9i" !(stats.pos_writes_full) !(stats.pos_writes_min) !(stats.pos_writes_max) !(stats.pos_writes_minmax) !(stats.pos_writes_eq));
+	(* let chunk_sizes = Hashtbl.fold (fun name (imin,imax) acc -> (name,!imin,!imax) :: acc) stats.chunk_sizes [] in
+	let chunk_sizes = List.sort (fun (_,imin1,imax1) (_,imin2,imax2) -> compare imax1 imax2) chunk_sizes in
+	print_endline "chunk sizes:";
+	List.iter (fun (name,imin,imax) ->
+		print_endline (Printf.sprintf "    %s: %i - %i" name imin imax)
+	) chunk_sizes *)
+
+module StringHashtbl = Hashtbl.Make(struct
+	type t = string
+
+	let equal =
+		String.equal
+
+	let hash s =
+		(* What's the best here? *)
+		Hashtbl.hash s
+end)
+
+module StringPool = struct
+	type t = {
+		lut : int StringHashtbl.t;
+		items : string DynArray.t;
+		mutable closed : bool;
+	}
+
+	let create () = {
+		lut = StringHashtbl.create 16;
+		items = DynArray.create ();
+		closed = false;
+	}
+
+	let add sp s =
+		assert (not sp.closed);
+		let index = DynArray.length sp.items in
+		StringHashtbl.add sp.lut s index;
+		DynArray.add sp.items s;
+		index
+
+	let get sp s =
+		StringHashtbl.find sp.lut s
+
+	let get_or_add sp s =
+		try
+			get sp s
+		with Not_found ->
+			add sp s
+
+	let finalize sp =
+		assert (not sp.closed);
+		sp.closed <- true;
+		DynArray.to_list sp.items,DynArray.length sp.items
+end
+
+module Pool = struct
+	type ('key,'value) t = {
+		lut : ('key,int) Hashtbl.t;
+		items : 'value DynArray.t;
+		mutable closed : bool;
+	}
+
+	let create () = {
+		lut = Hashtbl.create 0;
+		items = DynArray.create ();
+		closed = false;
+	}
+
+	let add pool (key : 'key) (value : 'value) =
+		assert (not pool.closed);
+		let index = DynArray.length pool.items in
+		DynArray.add pool.items value;
+		Hashtbl.add pool.lut key index;
+		index
+
+	let get pool (key : 'key) =
+		Hashtbl.find pool.lut key
+
+	let extract pool (key : 'key) =
+		DynArray.get pool.items (get pool key)
+
+	let has pool (key : 'key) =
+		Hashtbl.mem pool.lut key
+
+	let get_or_add pool (key : 'key) (value : 'value) =
+		try
+			get pool key
+		with Not_found ->
+			add pool key value
+
+	let is_empty pool =
+		DynArray.length pool.items = 0
+
+	let advance pool dummy =
+		DynArray.add pool.items dummy
+
+	let finalize pool =
+		assert (not pool.closed);
+		pool.closed <- true;
+		pool.items
+end
+
+module IdentityPool = struct
+	type ('key,'value) t = {
+		items : ('key * 'value) DynArray.t;
+		mutable closed : bool;
+	}
+
+	let create () = {
+		items = DynArray.create ();
+		closed = false;
+	}
+
+	let add pool (key : 'key) (value : 'value) =
+		assert (not pool.closed);
+		let index = DynArray.length pool.items in
+		DynArray.add pool.items (key,value);
+		index
+
+	let get pool (key : 'key) =
+		DynArray.index_of (fun (key',_) -> key == key') pool.items
+
+	let get_or_add pool (key : 'key) (value : 'value) =
+		try
+			get pool key
+		with Not_found ->
+			add pool key value
+
+	let to_list pool =
+		DynArray.to_list pool.items
+
+	let finalize pool =
+		assert (not pool.closed);
+		pool.closed <- true;
+		pool.items
+
+	let length pool = DynArray.length pool.items
+end
+
+module HashedIdentityPool = struct
+	type ('hkey,'key,'value) t = {
+		lut : ('hkey,('key * int)) Hashtbl.t;
+		items : ('key * 'value) DynArray.t;
+		mutable closed : bool;
+	}
+
+	let create () = {
+		lut = Hashtbl.create 16;
+		items = DynArray.create ();
+		closed = false;
+	}
+
+	let add pool (hkey : 'hkey) (key : 'key) (value : 'value) =
+		assert (not pool.closed);
+		let index = DynArray.length pool.items in
+		DynArray.add pool.items (key,value);
+		Hashtbl.add pool.lut hkey (key,index);
+		index
+
+	let get pool (hkey : 'hkey) (key : 'key) =
+		let l = Hashtbl.find_all pool.lut hkey in
+		List.assq key l
+
+	let finalize pool =
+		assert (not pool.closed);
+		pool.closed <- true;
+		pool.items
+end
+
+module SimnBuffer = struct
+	type t = {
+		buffer_size : int;
+		mutable buffer : bytes;
+		mutable buffers : bytes Queue.t;
+		mutable offset : int;
+	}
+
+	let create buffer_size = {
+		buffer = Bytes.create buffer_size;
+		buffers = Queue.create ();
+		offset = 0;
+		buffer_size = buffer_size;
+	}
+
+	let reset sb =
+		sb.buffer <- Bytes.create sb.buffer_size;
+		sb.buffers <- Queue.create ();
+		sb.offset <- 0
+
+	let promote_buffer sb =
+		Queue.add sb.buffer sb.buffers;
+		sb.buffer <- Bytes.create sb.buffer_size;
+		sb.offset <- 0
+
+	let add_u8 sb i =
+		if sb.offset = sb.buffer_size then begin
+			(* Current buffer is full, promote it. *)
+			promote_buffer sb;
+			Bytes.unsafe_set sb.buffer 0 i;
+			sb.offset <- 1;
+		end else begin
+			(* There's room, put it in. *)
+			Bytes.unsafe_set sb.buffer sb.offset i;
+			sb.offset <- sb.offset + 1
+		end
+
+	let add_bytes sb bytes =
+		let rec loop offset left =
+			let space = sb.buffer_size - sb.offset in
+			if left > space then begin
+				(* We need more than we have. Blit as much as we can, promote buffer, recurse. *)
+				Bytes.unsafe_blit bytes offset sb.buffer sb.offset space;
+				promote_buffer sb;
+				loop (offset + space) (left - space)
+			end else begin
+				(* It fits, blit it. *)
+				Bytes.unsafe_blit bytes offset sb.buffer sb.offset left;
+				sb.offset <- sb.offset + left;
+			end
+		in
+		loop 0 (Bytes.length bytes)
+
+	let contents sb =
+		let size = sb.offset + sb.buffer_size * Queue.length sb.buffers in
+		let out = Bytes.create size in
+		let offset = ref 0 in
+		(* We know that all sb.buffers are of sb.buffer_size length, so blit them together. *)
+		Queue.iter (fun bytes ->
+			Bytes.unsafe_blit bytes 0 out !offset sb.buffer_size;
+			offset := !offset + sb.buffer_size;
+		) sb.buffers;
+		(* Append our current buffer until sb.offset *)
+		Bytes.unsafe_blit sb.buffer 0 out !offset sb.offset;
+		out
+end
+
+module Chunk = struct
+	type t = {
+		kind : chunk_kind;
+		cp : StringPool.t;
+		ch : SimnBuffer.t;
+	}
+
+	let create kind cp initial_size = {
+		kind;
+		cp;
+		ch = SimnBuffer.create initial_size;
+	}
+
+	let reset chunk =
+		SimnBuffer.reset chunk.ch
+
+	let write_u8 io v =
+		SimnBuffer.add_u8 io.ch (Char.unsafe_chr v)
+
+	let write_i32 io v =
+		let base = Int32.to_int v in
+		let big = Int32.to_int (Int32.shift_right_logical v 24) in
+		write_u8 io base;
+		write_u8 io (base lsr 8);
+		write_u8 io (base lsr 16);
+		write_u8 io big
+
+	let write_i64 io v =
+		write_i32 io (Int64.to_int32 v);
+		write_i32 io (Int64.to_int32 (Int64.shift_right_logical v 32))
+
+	let write_f64 io v =
+		write_i64 io (Int64.bits_of_float v)
+
+	let write_bytes io b =
+		SimnBuffer.add_bytes io.ch b
+
+	let write_ui16 io i =
+		write_u8 io i;
+		write_u8 io (i lsr 8)
+
+	let get_bytes io =
+		SimnBuffer.contents io.ch
+
+	let rec write_uleb128 io v =
+		let b = v land 0x7F in
+		let rest = v lsr 7 in
+		if rest = 0 then
+			write_u8 io b
+		else begin
+			write_u8 io (b lor 0x80);
+			write_uleb128 io rest
+		end
+
+	let rec write_leb128 io v =
+		let b = v land 0x7F in
+		let rest = v asr 7 in
+		if (rest = 0 && (b land 0x40 = 0)) || (rest = -1 && (b land 0x40 = 0x40)) then
+			write_u8 io b
+		else begin
+			write_u8 io (b lor 0x80);
+			write_leb128 io rest
+		end
+
+	let write_bytes_length_prefixed io b =
+		write_uleb128 io (Bytes.length b);
+		write_bytes io b
+
+	let write_bool io b =
+		write_u8 io (if b then 1 else 0)
+
+	let export : 'a . hxb_writer_stats -> t -> 'a IO.output -> unit = fun stats io chex ->
+		let bytes = get_bytes io in
+		let length = Bytes.length bytes in
+		write_chunk_prefix io.kind length chex;
+		(* begin try
+			let (imin,imax) = Hashtbl.find stats.chunk_sizes io.name in
+			if length < !imin then imin := length;
+			if length > !imax then imax := length
+		with Not_found ->
+			Hashtbl.add stats.chunk_sizes io.name (ref length,ref length);
+		end; *)
+		IO.nwrite chex bytes
+
+	let write_string chunk s =
+		write_uleb128 chunk (StringPool.get_or_add chunk.cp s)
+
+	let write_list : 'b . t -> 'b list -> ('b -> unit) -> unit = fun chunk l f ->
+		write_uleb128 chunk (List.length l);
+		List.iter f l
+
+	let write_dynarray chunk d f =
+		write_uleb128 chunk (DynArray.length d);
+		DynArray.iter f d
+
+	let write_option : 'b . t -> 'b option -> ('b -> unit) -> unit = fun chunk v f -> match v with
+	| None ->
+		write_u8 chunk 0
+	| Some v ->
+		write_u8 chunk 1;
+		f v
+
+	let export_data chunk_from chunk_to =
+		let bytes = get_bytes chunk_from in
+		write_bytes chunk_to bytes
+end
+
+module PosWriter = struct
+	type t = {
+		stats : hxb_writer_stats;
+		mutable p_file : string;
+		mutable p_min : int;
+		mutable p_max : int;
+	}
+
+	let do_write_pos (chunk : Chunk.t) (p : pos) =
+		(* incr stats.pos_writes_full; *)
+		Chunk.write_string chunk p.pfile;
+		Chunk.write_leb128 chunk p.pmin;
+		Chunk.write_leb128 chunk p.pmax
+
+	let create stats chunk p =
+		do_write_pos chunk p;
+	{
+		stats;
+		p_file = p.pfile;
+		p_min = p.pmin;
+		p_max = p.pmax;
+	}
+
+	let write_pos pw (chunk : Chunk.t) (write_equal : bool) (offset : int) (p : pos) =
+		if p.pfile != pw.p_file then begin
+			(* File changed, write full pos *)
+			Chunk.write_u8 chunk (4 + offset);
+			do_write_pos chunk p;
+			pw.p_file <- p.pfile;
+			pw.p_min <- p.pmin;
+			pw.p_max <- p.pmax;
+		end else if p.pmin <> pw.p_min then begin
+			if p.pmax <> pw.p_max then begin
+				(* pmin and pmax changed *)
+				(* incr stats.pos_writes_minmax; *)
+				Chunk.write_u8 chunk (3 + offset);
+				Chunk.write_leb128 chunk p.pmin;
+				Chunk.write_leb128 chunk p.pmax;
+				pw.p_min <- p.pmin;
+				pw.p_max <- p.pmax;
+			end else begin
+				(* pmin changed *)
+				(* incr stats.pos_writes_min; *)
+				Chunk.write_u8 chunk (1 + offset);
+				Chunk.write_leb128 chunk p.pmin;
+				pw.p_min <- p.pmin;
+			end
+		end else if p.pmax <> pw.p_max then begin
+			(* pmax changed *)
+			(* incr stats.pos_writes_max; *)
+			Chunk.write_u8 chunk (2 + offset);
+			Chunk.write_leb128 chunk p.pmax;
+			pw.p_max <- p.pmax;
+		end else begin
+			(* incr stats.pos_writes_eq; *)
+			if write_equal then
+				Chunk.write_u8 chunk offset;
+		end
+end
+
+type field_writer_context = {
+	t_pool : StringPool.t;
+	pos_writer : PosWriter.t;
+	mutable texpr_this : texpr option;
+	vars : (tvar * int) DynArray.t;
+}
+
+let create_field_writer_context pos_writer = {
+	t_pool = StringPool.create ();
+	pos_writer = pos_writer;
+	texpr_this = None;
+	vars = DynArray.create ();
+}
+
+type hxb_writer = {
+	warn : Warning.warning -> string -> Globals.pos -> unit;
+	anon_id : Type.t Tanon_identification.tanon_identification;
+	stats : hxb_writer_stats;
+	mutable current_module : module_def;
+	chunks : Chunk.t DynArray.t;
+	cp : StringPool.t;
+	docs : StringPool.t;
+	mutable chunk : Chunk.t;
+
+	classes : (path,tclass) Pool.t;
+	enums : (path,tenum) Pool.t;
+	typedefs : (path,tdef) Pool.t;
+	abstracts : (path,tabstract) Pool.t;
+	anons : (path,tanon) Pool.t;
+	anon_fields : (string,tclass_field,unit) HashedIdentityPool.t;
+	tmonos : (tmono,unit) IdentityPool.t;
+
+	own_classes : (path,tclass) Pool.t;
+	own_enums : (path,tenum) Pool.t;
+	own_typedefs : (path,tdef) Pool.t;
+	own_abstracts : (path,tabstract) Pool.t;
+	type_param_lut : (path,(string,typed_type_param) Pool.t) Pool.t;
+	class_fields : (string,tclass_field,(tclass * class_field_ref_kind * int)) HashedIdentityPool.t;
+	enum_fields : ((path * string),(tenum * tenum_field)) Pool.t;
+	mutable type_type_parameters : (string,typed_type_param) Pool.t;
+	mutable field_type_parameters : (typed_type_param,unit) IdentityPool.t;
+	mutable local_type_parameters : (typed_type_param,unit) IdentityPool.t;
+	mutable field_stack : unit list;
+	unbound_ttp : (typed_type_param,unit) IdentityPool.t;
+	t_instance_chunk : Chunk.t;
+}
+
+module HxbWriter = struct
+	let in_nested_scope writer = match writer.field_stack with
+		| [] -> false (* can happen for cl_init and in EXD *)
+		| [_] -> false
+		| _ -> true
+
+	(* Chunks *)
+
+	let start_chunk writer (kind : chunk_kind) =
+		let initial_size = match kind with
+			| EOT | EOF | EOM -> 0
+			| MDF -> 16
+			| MTF | MDR | CLR | END | ABD | ENR | ABR | TDR | EFR | CFR | AFD -> 64
+			| AFR | CLD | TDD | EFD -> 128
+			| STR | DOC -> 256
+			| CFD | EXD -> 512
+		in
+		let new_chunk = Chunk.create kind writer.cp initial_size in
+		DynArray.add writer.chunks new_chunk;
+		writer.chunk <- new_chunk
+
+	let start_temporary_chunk : 'a . hxb_writer -> int -> (Chunk.t -> 'a) -> 'a = fun writer initial_size ->
+		let new_chunk = Chunk.create EOM (* TODO: something else? *) writer.cp initial_size in
+		let old_chunk = writer.chunk in
+		writer.chunk <- new_chunk;
+		(fun f ->
+			writer.chunk <- old_chunk;
+			f new_chunk
+		)
+
+	let write_inlined_list : 'a . hxb_writer -> int -> int -> (int -> unit) -> (unit -> unit) -> ('a -> unit) -> 'a list -> unit
+		= fun writer offset max f_byte f_first f_elt l ->
+		let length = List.length l in
+		if length > max then begin
+			f_byte (offset + 9);
+			f_first ();
+			Chunk.write_list writer.chunk l f_elt
+		end else begin
+			f_byte (offset + length);
+			f_first();
+			List.iter (fun elt ->
+				f_elt elt
+			) l
+		end
+
+	(* Basic compounds *)
+
+	let write_path writer (path : path) =
+		Chunk.write_list writer.chunk (fst path) (Chunk.write_string writer.chunk);
+		Chunk.write_string writer.chunk (snd path)
+
+	let write_full_path writer (pack : string list) (mname : string) (tname : string) =
+		Chunk.write_list writer.chunk pack (Chunk.write_string writer.chunk);
+		if mname = "" || tname = "" then
+			die (Printf.sprintf "write_full_path: pack = %s, mname = %s, tname = %s" (String.concat "." pack) mname tname) __LOC__;
+		Chunk.write_string writer.chunk mname;
+		Chunk.write_string writer.chunk tname
+
+	let write_documentation writer (doc : doc_block) =
+		Chunk.write_option writer.chunk doc.doc_own (fun s ->
+			Chunk.write_uleb128 writer.chunk (StringPool.get_or_add writer.docs s)
+		);
+		Chunk.write_list writer.chunk doc.doc_inherited (fun s ->
+			Chunk.write_uleb128 writer.chunk (StringPool.get_or_add writer.docs s)
+		)
+
+	let write_pos writer (p : pos) =
+		Chunk.write_string writer.chunk p.pfile;
+		Chunk.write_leb128 writer.chunk p.pmin;
+		Chunk.write_leb128 writer.chunk p.pmax
+
+	let write_pos_pair writer (p1 : pos) (p2 : pos) =
+		(* Write second position offset relative to first position's pmin, which is often within 1 byte range. *)
+		Chunk.write_string writer.chunk p1.pfile;
+		Chunk.write_leb128 writer.chunk p1.pmin;
+		Chunk.write_leb128 writer.chunk p1.pmax;
+		Chunk.write_leb128 writer.chunk (p2.pmin - p1.pmin);
+		Chunk.write_leb128 writer.chunk (p2.pmax - p1.pmin)
+
+	let rec write_metadata_entry writer ((meta,el,p) : metadata_entry) =
+		Chunk.write_string writer.chunk (Meta.to_string meta);
+		write_pos writer p;
+		Chunk.write_list writer.chunk el (write_expr writer)
+
+	and write_metadata writer ml =
+		Chunk.write_list writer.chunk ml (write_metadata_entry writer)
+
+	(* expr *)
+
+	and write_object_field_key writer (n,p,qs) =
+		Chunk.write_string writer.chunk n;
+		write_pos writer p;
+		begin match qs with
+			| NoQuotes -> Chunk.write_u8 writer.chunk 0
+			| DoubleQuotes -> Chunk.write_u8 writer.chunk 1
+		end
+
+	and write_type_path writer tp =
+		Chunk.write_list writer.chunk tp.tpackage (Chunk.write_string writer.chunk);
+		Chunk.write_string writer.chunk tp.tname;
+		Chunk.write_list writer.chunk tp.tparams (write_type_param_or_const writer);
+		Chunk.write_option writer.chunk tp.tsub (Chunk.write_string writer.chunk)
+
+	and write_placed_type_path writer ptp =
+		write_type_path writer ptp.path;
+		write_pos_pair writer ptp.pos_full ptp.pos_path
+
+	and write_type_param_or_const writer = function
+		| TPType th ->
+			Chunk.write_u8 writer.chunk 0;
+			write_type_hint writer th
+		| TPExpr e ->
+			Chunk.write_u8 writer.chunk 1;
+			write_expr writer e
+
+	and write_complex_type writer = function
+		| CTPath tp ->
+			Chunk.write_u8 writer.chunk 0;
+			write_placed_type_path writer tp
+		| CTFunction(thl,th) ->
+			Chunk.write_u8 writer.chunk 1;
+			Chunk.write_list writer.chunk thl (write_type_hint writer);
+			write_type_hint writer th
+		| CTAnonymous cffl ->
+			Chunk.write_u8 writer.chunk 2;
+			Chunk.write_list writer.chunk cffl (write_cfield writer);
+		| CTParent th ->
+			Chunk.write_u8 writer.chunk 3;
+			write_type_hint writer th
+		| CTExtend(ptp,cffl) ->
+			Chunk.write_u8 writer.chunk 4;
+			Chunk.write_list writer.chunk ptp (write_placed_type_path writer);
+			Chunk.write_list writer.chunk cffl (write_cfield writer);
+		| CTOptional th ->
+			Chunk.write_u8 writer.chunk 5;
+			write_type_hint writer th
+		| CTNamed(pn,th) ->
+			Chunk.write_u8 writer.chunk 6;
+			write_placed_name writer pn;
+			write_type_hint writer th
+		| CTIntersection(thl) ->
+			Chunk.write_u8 writer.chunk 7;
+			Chunk.write_list writer.chunk thl (write_type_hint writer)
+
+	and write_type_hint writer (ct,p) =
+		write_complex_type writer ct;
+		write_pos writer p
+
+	and write_type_param writer tp =
+		write_placed_name writer tp.tp_name;
+		Chunk.write_list writer.chunk tp.tp_params (write_type_param writer);
+		Chunk.write_option writer.chunk tp.tp_constraints (write_type_hint writer);
+		Chunk.write_option writer.chunk tp.tp_default (write_type_hint writer);
+		Chunk.write_list writer.chunk tp.tp_meta (write_metadata_entry writer)
+
+	and write_func_arg writer (pn,b,meta,tho,eo) =
+		write_placed_name writer pn;
+		Chunk.write_bool writer.chunk b;
+		write_metadata writer meta;
+		Chunk.write_option writer.chunk tho (write_type_hint writer);
+		Chunk.write_option writer.chunk eo (write_expr writer);
+
+	and write_func writer f =
+		Chunk.write_list writer.chunk f.f_params (write_type_param writer);
+		Chunk.write_list writer.chunk f.f_args (write_func_arg writer);
+		Chunk.write_option writer.chunk f.f_type (write_type_hint writer);
+		Chunk.write_option writer.chunk f.f_expr (write_expr writer)
+
+	and write_placed_name writer (s,p) =
+		Chunk.write_string writer.chunk s;
+		write_pos writer p
+
+	and write_access writer ac =
+		let i = match ac with
+		| APublic -> 0
+		| APrivate -> 1
+		| AStatic -> 2
+		| AOverride -> 3
+		| ADynamic -> 4
+		| AInline -> 5
+		| AMacro -> 6
+		| AFinal -> 7
+		| AExtern -> 8
+		| AAbstract -> 9
+		| AOverload -> 10
+		| AEnum -> 11
+		in
+		Chunk.write_u8 writer.chunk i
+
+	and write_placed_access writer (ac,p) =
+		write_access writer ac;
+		write_pos writer p
+
+	and write_cfield_kind writer = function
+		| FVar(tho,eo) ->
+			Chunk.write_u8 writer.chunk 0;
+			Chunk.write_option writer.chunk tho (write_type_hint writer);
+			Chunk.write_option writer.chunk eo (write_expr writer);
+		| FFun f ->
+			Chunk.write_u8 writer.chunk 1;
+			write_func writer f;
+		| FProp(pn1,pn2,tho,eo) ->
+			Chunk.write_u8 writer.chunk 2;
+			write_placed_name writer pn1;
+			write_placed_name writer pn2;
+			Chunk.write_option writer.chunk tho (write_type_hint writer);
+			Chunk.write_option writer.chunk eo (write_expr writer)
+
+	and write_cfield writer cff =
+		write_placed_name writer cff.cff_name;
+		Chunk.write_option writer.chunk cff.cff_doc (write_documentation writer);
+		write_pos writer cff.cff_pos;
+		write_metadata writer cff.cff_meta;
+		Chunk.write_list writer.chunk cff.cff_access (write_placed_access writer);
+		write_cfield_kind writer cff.cff_kind
+
+	and write_expr writer (e,p) =
+		write_pos writer p;
+		match e with
+		| EConst (Int (s, suffix)) ->
+			Chunk.write_u8 writer.chunk 0;
+			Chunk.write_string writer.chunk s;
+			Chunk.write_option writer.chunk suffix (Chunk.write_string writer.chunk);
+		| EConst (Float (s, suffix)) ->
+			Chunk.write_u8 writer.chunk 1;
+			Chunk.write_string writer.chunk s;
+			Chunk.write_option writer.chunk suffix (Chunk.write_string writer.chunk);
+		| EConst (String (s,qs)) ->
+			Chunk.write_u8 writer.chunk 2;
+			Chunk.write_string writer.chunk s;
+			begin match qs with
+			| SDoubleQuotes -> Chunk.write_u8 writer.chunk 0;
+			| SSingleQuotes -> Chunk.write_u8 writer.chunk 1;
+			end
+		| EConst (Ident s) ->
+			Chunk.write_u8 writer.chunk 3;
+			Chunk.write_string writer.chunk s;
+		| EConst (Regexp(s1,s2)) ->
+			Chunk.write_u8 writer.chunk 4;
+			Chunk.write_string writer.chunk s1;
+			Chunk.write_string writer.chunk s2;
+		| EArray(e1,e2) ->
+			Chunk.write_u8 writer.chunk 5;
+			write_expr writer e1;
+			write_expr writer e2;
+		| EBinop(op,e1,e2) ->
+			Chunk.write_u8 writer.chunk 6;
+			Chunk.write_u8 writer.chunk (binop_index op);
+			write_expr writer e1;
+			write_expr writer e2;
+		| EField(e1,s,kind) ->
+			Chunk.write_u8 writer.chunk 7;
+			write_expr writer e1;
+			Chunk.write_string writer.chunk s;
+			begin match kind with
+			| EFNormal -> Chunk.write_u8 writer.chunk 0;
+			| EFSafe -> Chunk.write_u8 writer.chunk 1;
+			end
+		| EParenthesis e1 ->
+			Chunk.write_u8 writer.chunk 8;
+			write_expr writer e1;
+		| EObjectDecl fl ->
+			Chunk.write_u8 writer.chunk 9;
+			let write_field (k,e1) =
+				write_object_field_key writer k;
+				write_expr writer e1
+			in
+			Chunk.write_list writer.chunk fl write_field;
+		| EArrayDecl el ->
+			Chunk.write_u8 writer.chunk 10;
+			Chunk.write_list writer.chunk el (write_expr writer);
+		| ECall(e1,el) ->
+			Chunk.write_u8 writer.chunk 11;
+			write_expr writer e1;
+			Chunk.write_list writer.chunk el (write_expr writer)
+		| ENew(ptp,el) ->
+			Chunk.write_u8 writer.chunk 12;
+			write_placed_type_path writer ptp;
+			Chunk.write_list writer.chunk el (write_expr writer);
+		| EUnop(op,flag,e1) ->
+			Chunk.write_u8 writer.chunk 13;
+			Chunk.write_u8 writer.chunk (unop_index op flag);
+			write_expr writer e1;
+		| EVars vl ->
+			Chunk.write_u8 writer.chunk 14;
+			let write_var v =
+				write_placed_name writer v.ev_name;
+				Chunk.write_bool writer.chunk v.ev_final;
+				Chunk.write_bool writer.chunk v.ev_static;
+				Chunk.write_option writer.chunk v.ev_type (write_type_hint writer);
+				Chunk.write_option writer.chunk v.ev_expr (write_expr writer);
+				write_metadata writer v.ev_meta;
+			in
+			Chunk.write_list writer.chunk vl write_var
+		| EFunction(fk,f) ->
+			Chunk.write_u8 writer.chunk 15;
+			begin match fk with
+			| FKAnonymous -> Chunk.write_u8 writer.chunk 0;
+			| FKNamed (pn,inline) ->
+				Chunk.write_u8 writer.chunk 1;
+				write_placed_name writer pn;
+				Chunk.write_bool writer.chunk inline;
+			| FKArrow -> Chunk.write_u8 writer.chunk 2;
+			end;
+			write_func writer f;
+		| EBlock el ->
+			Chunk.write_u8 writer.chunk 16;
+			Chunk.write_list writer.chunk el (write_expr writer)
+		| EFor(e1,e2) ->
+			Chunk.write_u8 writer.chunk 17;
+			write_expr writer e1;
+			write_expr writer e2;
+		| EIf(e1,e2,None) ->
+			Chunk.write_u8 writer.chunk 18;
+			write_expr writer e1;
+			write_expr writer e2;
+		| EIf(e1,e2,Some e3) ->
+			Chunk.write_u8 writer.chunk 19;
+			write_expr writer e1;
+			write_expr writer e2;
+			write_expr writer e3;
+		| EWhile(e1,e2,NormalWhile) ->
+			Chunk.write_u8 writer.chunk 20;
+			write_expr writer e1;
+			write_expr writer e2;
+		| EWhile(e1,e2,DoWhile) ->
+			Chunk.write_u8 writer.chunk 21;
+			write_expr writer e1;
+			write_expr writer e2;
+		| ESwitch(e1,cases,def) ->
+			Chunk.write_u8 writer.chunk 22;
+			write_expr writer e1;
+			let write_case (el,eg,eo,p) =
+				Chunk.write_list writer.chunk el (write_expr writer);
+				Chunk.write_option writer.chunk eg (write_expr writer);
+				Chunk.write_option writer.chunk eo (write_expr writer);
+				write_pos writer p;
+			in
+			Chunk.write_list writer.chunk cases write_case;
+			let write_default (eo,p) =
+				Chunk.write_option writer.chunk eo (write_expr writer);
+				write_pos writer p
+			in
+			Chunk.write_option writer.chunk def write_default;
+		| ETry(e1,catches) ->
+			Chunk.write_u8 writer.chunk 23;
+			write_expr writer e1;
+			let write_catch (pn,th,e,p) =
+				write_placed_name writer pn;
+				Chunk.write_option writer.chunk th (write_type_hint writer);
+				write_expr writer e;
+				write_pos writer p;
+			in
+			Chunk.write_list writer.chunk catches write_catch;
+		| EReturn None ->
+			Chunk.write_u8 writer.chunk 24;
+		| EReturn (Some e1) ->
+			Chunk.write_u8 writer.chunk 25;
+			write_expr writer e1;
+		| EBreak ->
+			Chunk.write_u8 writer.chunk 26;
+		| EContinue ->
+			Chunk.write_u8 writer.chunk 27;
+		| EUntyped e1 ->
+			Chunk.write_u8 writer.chunk 28;
+			write_expr writer e1;
+		| EThrow e1 ->
+			Chunk.write_u8 writer.chunk 29;
+			write_expr writer e1;
+		| ECast(e1,None) ->
+			Chunk.write_u8 writer.chunk 30;
+			write_expr writer e1;
+		| ECast(e1,Some th) ->
+			Chunk.write_u8 writer.chunk 31;
+			write_expr writer e1;
+			write_type_hint writer th;
+		| EIs(e1,th) ->
+			Chunk.write_u8 writer.chunk 32;
+			write_expr writer e1;
+			write_type_hint writer th;
+		| EDisplay(e1,dk) ->
+			Chunk.write_u8 writer.chunk 33;
+			write_expr writer e1;
+			begin match dk with
+			| DKCall -> Chunk.write_u8 writer.chunk 0;
+			| DKDot -> Chunk.write_u8 writer.chunk 1;
+			| DKStructure -> Chunk.write_u8 writer.chunk 2;
+			| DKMarked -> Chunk.write_u8 writer.chunk 3;
+			| DKPattern b ->
+				Chunk.write_u8 writer.chunk 4;
+				Chunk.write_bool writer.chunk b;
+			end
+		| ETernary(e1,e2,e3) ->
+			Chunk.write_u8 writer.chunk 34;
+			write_expr writer e1;
+			write_expr writer e2;
+			write_expr writer e3;
+		| ECheckType(e1,th) ->
+			Chunk.write_u8 writer.chunk 35;
+			write_expr writer e1;
+			write_type_hint writer th;
+		| EMeta(m,e1) ->
+			Chunk.write_u8 writer.chunk 36;
+			write_metadata_entry writer m;
+			write_expr writer e1
+
+	(* References *)
+
+	let write_class_ref writer (c : tclass) =
+		let i = Pool.get_or_add writer.classes c.cl_path c in
+		Chunk.write_uleb128 writer.chunk i
+
+	let write_enum_ref writer (en : tenum) =
+		let i = Pool.get_or_add writer.enums en.e_path en in
+		Chunk.write_uleb128 writer.chunk i
+
+	let write_typedef_ref writer (td : tdef) =
+		let i = Pool.get_or_add writer.typedefs td.t_path td in
+		Chunk.write_uleb128 writer.chunk i
+
+	let write_abstract_ref writer (a : tabstract) =
+		let i = Pool.get_or_add writer.abstracts a.a_path a in
+		Chunk.write_uleb128 writer.chunk i
+
+	let write_tmono_ref writer (mono : tmono) =
+		let index = IdentityPool.get_or_add writer.tmonos mono () in
+		Chunk.write_uleb128 writer.chunk index
+
+	let write_field_ref writer (c : tclass) (kind : class_field_ref_kind)  (cf : tclass_field) =
+		let index = try
+			HashedIdentityPool.get writer.class_fields cf.cf_name cf
+		with Not_found ->
+			let find_overload c cf_base =
+				let rec loop depth cfl = match cfl with
+					| cf' :: cfl ->
+						if cf' == cf then
+							Some(c,depth)
+						else
+							loop (depth + 1) cfl
+					| [] ->
+						None
+				in
+				let cfl = cf_base :: cf_base.cf_overloads in
+				loop 0 cfl
+			in
+			let find_overload c =
+				try
+					find_overload c (find_field c cf.cf_name kind)
+				with Not_found ->
+					None
+			in
+			let r = match kind with
+				| CfrStatic | CfrConstructor ->
+					find_overload c;
+				| CfrInit ->
+					Some(c,0)
+				| CfrMember ->
+					(* For member overloads we need to find the correct class, which is a mess. *)
+					let rec loop c = match find_overload c with
+						| Some _ as r ->
+							r
+						| None ->
+							if has_class_flag c CInterface then
+								let rec loopi l = match l with
+									| [] ->
+										None
+									| (c,_) :: l ->
+										match loop c with
+										| Some _ as r ->
+											r
+										| None ->
+											loopi l
+								in
+								loopi c.cl_implements
+							else match c.cl_super with
+								| Some(c,_) ->
+									loop c
+								| None ->
+									None
+					in
+					loop c;
+			in
+			let c,depth = match r with
+				| None ->
+					print_endline (Printf.sprintf "Could not resolve %s overload for %s on %s" (s_class_field_ref_kind kind) cf.cf_name (s_type_path c.cl_path));
+					c,0
+				| Some(c,depth) ->
+					c,depth
+			in
+			HashedIdentityPool.add writer.class_fields cf.cf_name cf (c,kind,depth)
+		in
+		Chunk.write_uleb128 writer.chunk index
+
+	let write_enum_field_ref writer (en : tenum) (ef : tenum_field) =
+		let key = (en.e_path,ef.ef_name) in
+		try
+			Chunk.write_uleb128 writer.chunk (Pool.get writer.enum_fields key)
+		with Not_found ->
+			ignore(Pool.get_or_add writer.enums en.e_path en);
+			Chunk.write_uleb128 writer.chunk (Pool.add writer.enum_fields key (en,ef))
+
+	let write_var_kind writer vk =
+		let b = match vk with
+			| VUser TVOLocalVariable -> 0
+			| VUser TVOArgument -> 1
+			| VUser TVOForVariable -> 2
+			| VUser TVOPatternVariable -> 3
+			| VUser TVOCatchVariable -> 4
+			| VUser TVOLocalFunction -> 5
+			| VGenerated -> 6
+			| VInlined -> 7
+			| VInlinedConstructorVariable -> 8
+			| VExtractorVariable -> 9
+			| VAbstractThis -> 10
+		in
+		Chunk.write_u8 writer.chunk b
+
+	let write_var writer fctx v =
+		Chunk.write_uleb128 writer.chunk v.v_id;
+		Chunk.write_string writer.chunk v.v_name;
+		write_var_kind writer v.v_kind;
+		Chunk.write_uleb128 writer.chunk v.v_flags;
+		write_metadata writer v.v_meta;
+		write_pos writer v.v_pos
+
+	let rec write_anon writer (an : tanon) (ttp : type_params) =
+		let write_fields () =
+			let restore = start_temporary_chunk writer 256 in
+			let i = ref 0 in
+			PMap.iter (fun _ cf ->
+				write_anon_field_ref writer cf;
+				incr i;
+			) an.a_fields;
+			let bytes = restore (fun new_chunk -> Chunk.get_bytes new_chunk) in
+			Chunk.write_uleb128 writer.chunk !i;
+			Chunk.write_bytes writer.chunk bytes;
+		in
+		begin match !(an.a_status) with
+		| Closed ->
+			Chunk.write_u8 writer.chunk 0;
+			write_fields ()
+		| Const ->
+			Chunk.write_u8 writer.chunk 1;
+			write_fields ()
+		| Extend tl ->
+			Chunk.write_u8 writer.chunk 2;
+			write_types writer tl;
+			write_fields ()
+		| ClassStatics _ ->
+			assert false
+		| EnumStatics _ ->
+			assert false
+		| AbstractStatics _ ->
+			assert false
+		end
+
+	and write_anon_ref writer (an : tanon) (ttp : type_params) =
+		let pfm = Option.get (writer.anon_id#identify_anon ~strict:true an) in
+		try
+			let index = Pool.get writer.anons pfm.pfm_path in
+			Chunk.write_u8 writer.chunk 0;
+			Chunk.write_uleb128 writer.chunk index
+		with Not_found ->
+			let index = Pool.add writer.anons pfm.pfm_path an in
+			Chunk.write_u8 writer.chunk 1;
+			Chunk.write_uleb128 writer.chunk index;
+			write_anon writer an ttp
+
+	and write_anon_field_ref writer cf =
+		try
+			let index = HashedIdentityPool.get writer.anon_fields cf.cf_name cf in
+			Chunk.write_u8 writer.chunk 0;
+			Chunk.write_uleb128 writer.chunk index
+		with Not_found ->
+			let index = HashedIdentityPool.add writer.anon_fields cf.cf_name cf () in
+			Chunk.write_u8 writer.chunk 1;
+			Chunk.write_uleb128 writer.chunk index;
+			ignore(write_class_field_and_overloads_data writer true cf)
+
+	(* Type instances *)
+
+	and write_type_parameter_ref writer (ttp : typed_type_param) =
+		begin try
+			begin match ttp.ttp_host with
+			| TPHType ->
+				let i = Pool.get writer.type_type_parameters ttp.ttp_name in
+				Chunk.write_u8 writer.chunk 1;
+				Chunk.write_uleb128 writer.chunk i
+			| TPHMethod | TPHEnumConstructor | TPHAnonField | TPHConstructor ->
+				let i = IdentityPool.get writer.field_type_parameters ttp in
+				Chunk.write_u8 writer.chunk 2;
+				Chunk.write_uleb128 writer.chunk i;
+			| TPHLocal ->
+				let index = IdentityPool.get writer.local_type_parameters ttp in
+				Chunk.write_u8 writer.chunk 3;
+				Chunk.write_uleb128 writer.chunk index;
+		end with Not_found ->
+			(try ignore(IdentityPool.get writer.unbound_ttp ttp) with Not_found -> begin
+				ignore(IdentityPool.add writer.unbound_ttp ttp ());
+				let p = { null_pos with pfile = (Path.UniqueKey.lazy_path writer.current_module.m_extra.m_file) } in
+				let msg = Printf.sprintf "Unbound type parameter %s" (s_type_path ttp.ttp_class.cl_path) in
+				writer.warn WUnboundTypeParameter msg p
+			end);
+			Chunk.write_u8 writer.chunk 4; (* TDynamic None *)
+		end
+
+	(*
+		simple references:
+			0 - mono
+			1 -> type ttp
+			2 -> field ttp
+			3 -> local ttp
+			4 -> Dynamic
+
+		special references:
+			10 - class statics
+			11 - enum statics
+			12 - abstract statics
+			13 - KExpr
+
+		void functions:
+			20: () -> Void
+			21: (A) -> Void
+			22: (A, B) -> Void
+			23: (A, B, C) -> Void
+			24: (A, B, C) -> Void
+			29: (?) -> Void
+
+		non-void functions:
+			30: () -> T
+			31: (A) -> T
+			32: (A, B) -> T
+			33: (A, B, C) -> T
+			34: (A, B, C, D) -> T
+			39: (?) -> T
+
+		class:
+			40: C
+			41: C<A>
+			42: C<A, B>
+			49: C<?>
+
+		enum:
+			50: E
+			51: E<A>
+			52: E<A, B>
+			59: E<?>
+
+		typedef:
+			60: T
+			61: T<A>
+			62: T<A, B>
+			69: T<?>
+
+		abstract:
+			70: A
+			71: A<A>
+			72: A<A, B>
+			79: A<?>
+
+		anons:
+			80: {}
+			81: any anon
+			89: Dynamic<T>
+
+		concrete types:
+			100: Void
+			101: Int
+			102: Float
+			103: Bool
+			104: String
+	*)
+	and write_type_instance writer t =
+		let write_function_arg (n,o,t) =
+			Chunk.write_string writer.chunk n;
+			Chunk.write_bool writer.chunk o;
+			write_type_instance writer t;
+		in
+		let write_inlined_list offset max f_first f_elt l =
+			write_inlined_list writer offset max (Chunk.write_u8 writer.chunk) f_first f_elt l
+		in
+		match t with
+			| TAbstract ({a_path = ([],"Void")},[]) ->
+				Chunk.write_u8 writer.chunk 100;
+			| TAbstract ({a_path = ([],"Int")},[]) ->
+				Chunk.write_u8 writer.chunk 101;
+			| TAbstract ({a_path = ([],"Float")},[]) ->
+				Chunk.write_u8 writer.chunk 102;
+			| TAbstract ({a_path = ([],"Bool")},[]) ->
+				Chunk.write_u8 writer.chunk 103;
+			| TInst ({cl_path = ([],"String")},[]) ->
+				Chunk.write_u8 writer.chunk 104;
+			| TMono r ->
+				Monomorph.close r;
+				begin match r.tm_type with
+				| None ->
+					Chunk.write_u8 writer.chunk 0;
+					write_tmono_ref writer r;
+				| Some t ->
+					(* Don't write bound monomorphs, write underlying type directly *)
+					write_type_instance writer t
+				end
+			| TLazy f ->
+				write_type_instance writer (lazy_type f)
+			| TInst({cl_kind = KTypeParameter ttp},[]) ->
+				write_type_parameter_ref writer ttp;
+			| TInst({cl_kind = KExpr e},[]) ->
+				Chunk.write_u8 writer.chunk 13;
+				write_expr writer e;
+			| TInst(c,[]) ->
+				Chunk.write_u8 writer.chunk 40;
+				write_class_ref writer c;
+			| TEnum(en,[]) ->
+				Chunk.write_u8 writer.chunk 50;
+				write_enum_ref writer en;
+			| TType(td,[]) ->
+				let default () =
+					Chunk.write_u8 writer.chunk 60;
+					write_typedef_ref writer td;
+				in
+				begin match td.t_type with
+				| TAnon an ->
+					begin match !(an.a_status) with
+						| ClassStatics c ->
+							Chunk.write_u8 writer.chunk 10;
+							write_class_ref writer c
+						| EnumStatics en ->
+							Chunk.write_u8 writer.chunk 11;
+							write_enum_ref writer en;
+						| AbstractStatics a ->
+							Chunk.write_u8 writer.chunk 12;
+							write_abstract_ref writer a
+						| _ ->
+							default()
+					end
+				| _ ->
+					default()
+				end;
+			| TAbstract(a,[]) ->
+				Chunk.write_u8 writer.chunk 70;
+				write_abstract_ref writer a;
+			| TDynamic None ->
+				Chunk.write_u8 writer.chunk 4;
+			| TFun([],t) when ExtType.is_void (follow_lazy_and_mono t) ->
+				Chunk.write_u8 writer.chunk 20;
+			| TFun(args,t) when ExtType.is_void (follow_lazy_and_mono t) ->
+				write_inlined_list 20 4 (fun () -> ()) write_function_arg args;
+			| TFun(args,t) ->
+				write_inlined_list 30 4 (fun () -> ()) write_function_arg args;
+				write_type_instance writer t;
+			| TInst(c,tl) ->
+				write_inlined_list 40 2 (fun () -> write_class_ref writer c) (write_type_instance writer) tl;
+			| TEnum(en,tl) ->
+				write_inlined_list 50 2 (fun () -> write_enum_ref writer en) (write_type_instance writer) tl;
+			| TType(td,tl) ->
+				write_inlined_list 60 2 (fun () -> write_typedef_ref writer td) (write_type_instance writer) tl;
+			| TAbstract(a,tl) ->
+				write_inlined_list 70 2 (fun () -> write_abstract_ref writer a) (write_type_instance writer) tl;
+			| TAnon an when PMap.is_empty an.a_fields ->
+				Chunk.write_u8 writer.chunk 80;
+			| TAnon an ->
+				Chunk.write_u8 writer.chunk 81;
+				write_anon_ref writer an []
+			| TDynamic (Some t) ->
+				Chunk.write_u8 writer.chunk 89;
+				write_type_instance writer t
+
+	and write_types writer tl =
+		Chunk.write_list writer.chunk tl (write_type_instance writer)
+
+	(* texpr *)
+
+	and write_texpr_type_instance writer (fctx : field_writer_context) (t: Type.t) =
+		let old_chunk = writer.chunk in
+		writer.chunk <- writer.t_instance_chunk;
+		Chunk.reset writer.chunk;
+		write_type_instance writer t;
+		let t_bytes = Chunk.get_bytes writer.chunk in
+		writer.chunk <- old_chunk;
+		let index = StringPool.get_or_add fctx.t_pool (Bytes.unsafe_to_string t_bytes) in
+		Chunk.write_uleb128 writer.chunk index
+
+	and write_texpr writer (fctx : field_writer_context) (e : texpr) =
+		let declare_var v =
+			let index = if has_var_flag v VHxb then begin
+				(* Duplicate var declaration! Can happen when writing both cf_expr and cf_expr_unoptimized,
+				   although it arguably shouldn't. In this case we don't add the var again and instead write
+				   out the existing ID.*)
+				   v.v_id
+			end else begin
+				let index = DynArray.length fctx.vars in
+				DynArray.add fctx.vars (v,v.v_id);
+				(* Store local index in v_id so we find it easily for all the TLocal expressions.
+				   This is set back by the var writer in start_texpr. *)
+				v.v_id <- index;
+				add_var_flag v VHxb;
+				index;
+			end in
+			Chunk.write_uleb128 writer.chunk index;
+			Chunk.write_option writer.chunk v.v_extra (fun ve ->
+				Chunk.write_list writer.chunk ve.v_params (fun ttp ->
+					let index = IdentityPool.add writer.local_type_parameters ttp () in
+					Chunk.write_uleb128 writer.chunk index
+				);
+				Chunk.write_option writer.chunk ve.v_expr (write_texpr writer fctx);
+			);
+			write_type_instance writer v.v_type;
+		in
+		let rec loop e =
+			let write_type = match e.eexpr with
+			(* values 0-19 *)
+			| TConst ct ->
+				begin match ct with
+				| TNull ->
+					Chunk.write_u8 writer.chunk 0;
+					true
+				| TThis ->
+					fctx.texpr_this <- Some e;
+					Chunk.write_u8 writer.chunk 1;
+					false;
+				| TSuper ->
+					Chunk.write_u8 writer.chunk 2;
+					true;
+				| TBool false when (ExtType.is_bool (follow_lazy_and_mono e.etype)) ->
+					Chunk.write_u8 writer.chunk 3;
+					false;
+				| TBool true when (ExtType.is_bool (follow_lazy_and_mono e.etype)) ->
+					Chunk.write_u8 writer.chunk 4;
+					false;
+				| TInt i32 when (ExtType.is_int (follow_lazy_and_mono e.etype)) ->
+					Chunk.write_u8 writer.chunk 5;
+					Chunk.write_i32 writer.chunk i32;
+					false;
+				| TFloat f when (ExtType.is_float (follow_lazy_and_mono e.etype)) ->
+					Chunk.write_u8 writer.chunk 6;
+					Chunk.write_string writer.chunk f;
+					false;
+				| TString s when (ExtType.is_string (follow_lazy_and_mono e.etype)) ->
+					Chunk.write_u8 writer.chunk 7;
+					Chunk.write_string writer.chunk s;
+					false
+				| TBool false ->
+					Chunk.write_u8 writer.chunk 13;
+					true;
+				| TBool true ->
+					Chunk.write_u8 writer.chunk 14;
+					true;
+				| TInt i32 ->
+					Chunk.write_u8 writer.chunk 15;
+					Chunk.write_i32 writer.chunk i32;
+					true;
+				| TFloat f ->
+					Chunk.write_u8 writer.chunk 16;
+					Chunk.write_string writer.chunk f;
+					true;
+				| TString s ->
+					Chunk.write_u8 writer.chunk 17;
+					Chunk.write_string writer.chunk s;
+					true;
+				end
+			(* vars 20-29 *)
+			| TLocal v ->
+				Chunk.write_u8 writer.chunk 20;
+				Chunk.write_uleb128 writer.chunk v.v_id;
+				true; (* I think there are cases where v_type != etype *)
+			| TVar(v,None) ->
+				Chunk.write_u8 writer.chunk 21;
+				declare_var v;
+				false;
+			| TVar(v,Some e1) ->
+				Chunk.write_u8 writer.chunk 22;
+				declare_var v;
+				loop e1;
+				false;
+			(* blocks 30-49 *)
+			| TBlock [] ->
+				Chunk.write_u8 writer.chunk 30;
+				true;
+			| TBlock el ->
+				let restore = start_temporary_chunk writer 256 in
+				let i = ref 0 in
+				List.iter (fun e ->
+					incr i;
+					loop e;
+				) el;
+				let bytes = restore (fun new_chunk -> Chunk.get_bytes new_chunk) in
+				let l = !i in
+				begin match l with
+				| 1 -> Chunk.write_u8 writer.chunk 31;
+				| 2 -> Chunk.write_u8 writer.chunk 32;
+				| 3 -> Chunk.write_u8 writer.chunk 33;
+				| 4 -> Chunk.write_u8 writer.chunk 34;
+				| 5 -> Chunk.write_u8 writer.chunk 35;
+				| _ ->
+					if l <= 0xFF then begin
+						Chunk.write_u8 writer.chunk 36;
+						Chunk.write_u8 writer.chunk l;
+					end else begin
+						Chunk.write_u8 writer.chunk 39;
+						Chunk.write_uleb128 writer.chunk l;
+					end;
+				end;
+				Chunk.write_bytes writer.chunk bytes;
+				true;
+			(* function 50-59 *)
+			| TFunction tf ->
+				Chunk.write_u8 writer.chunk 50;
+				Chunk.write_list writer.chunk tf.tf_args (fun (v,eo) ->
+					declare_var v;
+					Chunk.write_option writer.chunk eo loop;
+				);
+				write_type_instance writer tf.tf_type;
+				loop tf.tf_expr;
+				true;
+			(* texpr compounds 60-79 *)
+			| TArray(e1,e2) ->
+				Chunk.write_u8 writer.chunk 60;
+				loop e1;
+				loop e2;
+				true;
+			| TParenthesis e1 ->
+				Chunk.write_u8 writer.chunk 61;
+				loop e1;
+				false; (* surely this is always the nested type *)
+			| TArrayDecl el ->
+				Chunk.write_u8 writer.chunk 62;
+				loop_el el;
+				true;
+			| TObjectDecl fl ->
+				Chunk.write_u8 writer.chunk 63;
+				Chunk.write_list writer.chunk fl (fun ((name,p,qs),e) ->
+					Chunk.write_string writer.chunk name;
+					write_pos writer p;
+					begin match qs with
+					| NoQuotes -> Chunk.write_u8 writer.chunk 0;
+					| DoubleQuotes -> Chunk.write_u8 writer.chunk 1;
+					end;
+					loop e
+				);
+				true;
+			| TCall(e1,el) ->
+				write_inlined_list writer 70 4 (Chunk.write_u8 writer.chunk) (fun () -> loop e1) loop el;
+				true;
+			| TMeta(m,e1) ->
+				Chunk.write_u8 writer.chunk 65;
+				write_metadata_entry writer m;
+				loop e1;
+				true;
+			(* branching 80-89 *)
+			| TIf(e1,e2,None) ->
+				Chunk.write_u8 writer.chunk 80;
+				loop e1;
+				loop e2;
+				false;
+			| TIf(e1,e2,Some e3) ->
+				Chunk.write_u8 writer.chunk 81;
+				loop e1;
+				loop e2;
+				loop e3;
+				true;
+			| TSwitch s ->
+				Chunk.write_u8 writer.chunk 82;
+				loop s.switch_subject;
+				Chunk.write_list writer.chunk s.switch_cases (fun c ->
+					loop_el c.case_patterns;
+					loop c.case_expr;
+				);
+				Chunk.write_option writer.chunk s.switch_default loop;
+				true;
+			| TTry(e1,catches) ->
+				Chunk.write_u8 writer.chunk 83;
+				loop e1;
+				Chunk.write_list writer.chunk catches  (fun (v,e) ->
+					declare_var v;
+					loop e
+				);
+				true;
+			| TWhile(e1,e2,flag) ->
+				Chunk.write_u8 writer.chunk (if flag = NormalWhile then 84 else 85);
+				loop e1;
+				loop e2;
+				false;
+			| TFor(v,e1,e2) ->
+				Chunk.write_u8 writer.chunk 86;
+				declare_var v;
+				loop e1;
+				loop e2;
+				false;
+			(* control flow 90-99 *)
+			| TReturn None ->
+				Chunk.write_u8 writer.chunk 90;
+				true;
+			| TReturn (Some e1) ->
+				Chunk.write_u8 writer.chunk 91;
+				loop e1;
+				true;
+			| TContinue ->
+				Chunk.write_u8 writer.chunk 92;
+				true;
+			| TBreak ->
+				Chunk.write_u8 writer.chunk 93;
+				true;
+			| TThrow e1 ->
+				Chunk.write_u8 writer.chunk 94;
+				loop e1;
+				true;
+			(* access 100-119 *)
+			| TEnumIndex e1 ->
+				Chunk.write_u8 writer.chunk 100;
+				loop e1;
+				false;
+			| TEnumParameter(e1,ef,i) ->
+				Chunk.write_u8 writer.chunk 101;
+				loop e1;
+				let en = match follow ef.ef_type with
+					| TFun(_,tr) ->
+						begin match follow tr with
+							| TEnum(en,_) -> en
+							| _ -> die "" __LOC__
+						end
+					| _ ->
+						die "" __LOC__
+				in
+				write_enum_field_ref writer en ef;
+				Chunk.write_uleb128 writer.chunk i;
+				true;
+			| TField({eexpr = TConst TThis; epos = p1},FInstance(c,tl,cf)) when fctx.texpr_this <> None ->
+				Chunk.write_u8 writer.chunk 111;
+				PosWriter.write_pos fctx.pos_writer writer.chunk true 0 p1;
+				write_class_ref writer c;
+				write_types writer tl;
+				write_field_ref writer c CfrMember cf;
+				true;
+			| TField(e1,FInstance(c,tl,cf)) ->
+				Chunk.write_u8 writer.chunk 102;
+				loop e1;
+				write_class_ref writer c;
+				write_types writer tl;
+				write_field_ref writer c CfrMember cf;
+				true;
+			| TField({eexpr = TTypeExpr (TClassDecl c'); epos = p1},FStatic(c,cf)) when c == c' ->
+				Chunk.write_u8 writer.chunk 110;
+				PosWriter.write_pos fctx.pos_writer writer.chunk true 0 p1;
+				write_class_ref writer c;
+				write_field_ref writer c CfrStatic cf;
+				true;
+			| TField(e1,FStatic(c,cf)) ->
+				Chunk.write_u8 writer.chunk 103;
+				loop e1;
+				write_class_ref writer c;
+				write_field_ref writer c CfrStatic cf;
+				true;
+			| TField(e1,FAnon cf) ->
+				Chunk.write_u8 writer.chunk 104;
+				loop e1;
+				write_anon_field_ref writer cf;
+				true;
+			| TField(e1,FClosure(Some(c,tl),cf)) ->
+				Chunk.write_u8 writer.chunk 105;
+				loop e1;
+				write_class_ref writer c;
+				write_types writer tl;
+				write_field_ref writer c CfrMember cf;
+				true;
+			| TField(e1,FClosure(None,cf)) ->
+				Chunk.write_u8 writer.chunk 106;
+				loop e1;
+				write_anon_field_ref writer cf;
+				true;
+			| TField(e1,FEnum(en,ef)) ->
+				Chunk.write_u8 writer.chunk 107;
+				loop e1;
+				write_enum_ref writer en;
+				write_enum_field_ref writer en ef;
+				true;
+			| TField(e1,FDynamic s) ->
+				Chunk.write_u8 writer.chunk 108;
+				loop e1;
+				Chunk.write_string writer.chunk s;
+				true;
+			(* module types 120-139 *)
+			| TTypeExpr (TClassDecl ({cl_kind = KTypeParameter ttp})) ->
+				Chunk.write_u8 writer.chunk 128;
+				write_type_parameter_ref writer ttp;
+				true;
+			| TTypeExpr (TClassDecl c) ->
+				Chunk.write_u8 writer.chunk 120;
+				write_class_ref writer c;
+				false;
+			| TTypeExpr (TEnumDecl en) ->
+				Chunk.write_u8 writer.chunk 121;
+				write_enum_ref writer en;
+				false;
+			| TTypeExpr (TAbstractDecl a) ->
+				Chunk.write_u8 writer.chunk 122;
+				write_abstract_ref writer a;
+				true;
+			| TTypeExpr (TTypeDecl td) ->
+				Chunk.write_u8 writer.chunk 123;
+				write_typedef_ref writer td;
+				true;
+			| TCast(e1,None) ->
+				Chunk.write_u8 writer.chunk 124;
+				loop e1;
+				true;
+			| TCast(e1,Some md) ->
+				Chunk.write_u8 writer.chunk 125;
+				loop e1;
+				let infos = t_infos md in
+				let m = infos.mt_module in
+				write_full_path writer (fst m.m_path) (snd m.m_path) (snd infos.mt_path);
+				true;
+			| TNew(({cl_kind = KTypeParameter ttp}),tl,el) ->
+				Chunk.write_u8 writer.chunk 127;
+				write_type_parameter_ref writer ttp;
+				write_types writer tl;
+				loop_el el;
+				true;
+			| TNew(c,tl,el) ->
+				Chunk.write_u8 writer.chunk 126;
+				write_class_ref writer c;
+				write_types writer tl;
+				loop_el el;
+				true;
+			(* unops 140-159 *)
+			| TUnop(op,flag,e1) ->
+				Chunk.write_u8 writer.chunk (140 + unop_index op flag);
+				loop e1;
+				true;
+			(* binops 160-219 *)
+			| TBinop(op,e1,e2) ->
+				Chunk.write_u8 writer.chunk (160 + binop_index op);
+				loop e1;
+				loop e2;
+				true;
+			(* rest 250-254 *)
+			| TIdent s ->
+				Chunk.write_u8 writer.chunk 250;
+				Chunk.write_string writer.chunk s;
+				true;
+			in
+			if write_type then
+				write_texpr_type_instance writer fctx e.etype;
+			PosWriter.write_pos fctx.pos_writer writer.chunk true 0 e.epos;
+
+		and loop_el el =
+			Chunk.write_list writer.chunk el loop
+		in
+		loop e
+
+	and write_type_parameters_forward writer (ttps : typed_type_param list) =
+		let write_type_parameter_forward ttp =
+			write_path writer ttp.ttp_class.cl_path;
+			write_pos writer ttp.ttp_class.cl_name_pos;
+			let i = match ttp.ttp_host with
+				| TPHType -> 0
+				| TPHConstructor -> 1
+				| TPHMethod -> 2
+				| TPHEnumConstructor -> 3
+				| TPHAnonField -> 4
+				| TPHLocal -> 5
+			in
+			Chunk.write_u8 writer.chunk i
+		in
+		Chunk.write_list writer.chunk ttps write_type_parameter_forward
+
+	and write_type_parameters_data writer (ttps : typed_type_param list) =
+		let write_type_parameter_data ttp =
+			let c = ttp.ttp_class in
+			write_metadata writer c.cl_meta;
+			write_types writer (get_constraints ttp);
+			Chunk.write_option writer.chunk ttp.ttp_default (write_type_instance writer)
+		in
+		List.iter write_type_parameter_data ttps
+
+	and write_type_parameters writer (ttps : typed_type_param list) =
+		write_type_parameters_forward writer ttps;
+		write_type_parameters_data writer ttps;
+
+	(* Fields *)
+
+	and write_field_kind writer = function
+		| Method MethNormal -> Chunk.write_u8 writer.chunk 0;
+		| Method MethInline -> Chunk.write_u8 writer.chunk 1;
+		| Method MethDynamic -> Chunk.write_u8 writer.chunk 2;
+		| Method MethMacro -> Chunk.write_u8 writer.chunk 3;
+		(* normal read *)
+		| Var {v_read = AccNormal; v_write = AccNormal } -> Chunk.write_u8 writer.chunk 10
+		| Var {v_read = AccNormal; v_write = AccNo } -> Chunk.write_u8 writer.chunk 11
+		| Var {v_read = AccNormal; v_write = AccNever } -> Chunk.write_u8 writer.chunk 12
+		| Var {v_read = AccNormal; v_write = AccCtor } -> Chunk.write_u8 writer.chunk 13
+		| Var {v_read = AccNormal; v_write = AccCall } -> Chunk.write_u8 writer.chunk 14
+		(* inline read *)
+		| Var {v_read = AccInline; v_write = AccNever } -> Chunk.write_u8 writer.chunk 20
+		(* getter read *)
+		| Var {v_read = AccCall; v_write = AccNormal } -> Chunk.write_u8 writer.chunk 30
+		| Var {v_read = AccCall; v_write = AccNo } -> Chunk.write_u8 writer.chunk 31
+		| Var {v_read = AccCall; v_write = AccNever } -> Chunk.write_u8 writer.chunk 32
+		| Var {v_read = AccCall; v_write = AccCtor } -> Chunk.write_u8 writer.chunk 33
+		| Var {v_read = AccCall; v_write = AccCall } -> Chunk.write_u8 writer.chunk 34
+		(* weird/overlooked combinations *)
+		| Var {v_read = r;v_write = w } ->
+			Chunk.write_u8 writer.chunk 100;
+			let f = function
+				| AccNormal -> Chunk.write_u8 writer.chunk 0
+				| AccNo -> Chunk.write_u8 writer.chunk 1
+				| AccNever -> Chunk.write_u8 writer.chunk 2
+				| AccCtor -> Chunk.write_u8 writer.chunk 3
+				| AccCall -> Chunk.write_u8 writer.chunk 4
+				| AccInline -> Chunk.write_u8 writer.chunk 5
+				| AccRequire(s,so) ->
+					Chunk.write_u8 writer.chunk 6;
+					Chunk.write_string writer.chunk s;
+					Chunk.write_option writer.chunk so (Chunk.write_string writer.chunk)
+			in
+			f r;
+			f w
+
+	and open_field_scope writer (params : type_params) =
+		writer.field_stack <- () :: writer.field_stack;
+		let nested = in_nested_scope writer in
+		let old_field_params = writer.field_type_parameters in
+		let old_local_params = writer.local_type_parameters in
+		if not nested then begin
+			writer.local_type_parameters <- IdentityPool.create ();
+			writer.field_type_parameters <- IdentityPool.create ();
+		end;
+		List.iter (fun ttp ->
+			ignore(IdentityPool.add writer.field_type_parameters ttp ());
+		) params;
+		(fun () ->
+			writer.field_type_parameters <- old_field_params;
+			writer.local_type_parameters <- old_local_params;
+			writer.field_stack <- List.tl writer.field_stack
+		)
+
+	and write_class_field_forward writer cf =
+		Chunk.write_string writer.chunk cf.cf_name;
+		write_pos_pair writer cf.cf_pos cf.cf_name_pos;
+		Chunk.write_list writer.chunk cf.cf_overloads (fun cf ->
+			write_class_field_forward writer cf;
+		);
+
+	and start_texpr writer (p: pos) =
+		let restore = start_temporary_chunk writer 512 in
+		let fctx = create_field_writer_context (PosWriter.create writer.stats writer.chunk p) in
+		fctx,(fun () ->
+			restore(fun new_chunk ->
+				let restore = start_temporary_chunk writer 512 in
+				if in_nested_scope writer then
+					Chunk.write_u8 writer.chunk 0
+				else begin
+					Chunk.write_u8 writer.chunk 1;
+					let ltp = List.map fst (IdentityPool.to_list writer.local_type_parameters) in
+					write_type_parameters writer ltp
+				end;
+				Chunk.write_option writer.chunk fctx.texpr_this (fun e -> write_type_instance writer e.etype);
+				let items,length = StringPool.finalize fctx.t_pool in
+				Chunk.write_uleb128 writer.chunk length;
+				List.iter (fun bytes ->
+					Chunk.write_bytes writer.chunk (Bytes.unsafe_of_string bytes)
+				) items;
+				Chunk.write_uleb128 writer.chunk (DynArray.length fctx.vars);
+				DynArray.iter (fun (v,v_id) ->
+					v.v_id <- v_id;
+					remove_var_flag v VHxb;
+					write_var writer fctx v;
+				) fctx.vars;
+				restore(fun newer_chunk -> newer_chunk,new_chunk)
+			)
+		)
+
+	and commit_field_type_parameters writer (params : type_params) =
+		Chunk.write_uleb128 writer.chunk (List.length params);
+		if in_nested_scope writer then
+			Chunk.write_u8 writer.chunk 0
+		else begin
+			Chunk.write_u8 writer.chunk 1;
+			let ftp = List.map fst (IdentityPool.to_list writer.field_type_parameters) in
+			write_type_parameters writer ftp
+		end
+
+	and write_class_field_data writer (write_expr_immediately : bool) (cf : tclass_field) =
+		let restore = start_temporary_chunk writer 512 in
+		write_type_instance writer cf.cf_type;
+		Chunk.write_uleb128 writer.chunk cf.cf_flags;
+		Chunk.write_option writer.chunk cf.cf_doc (write_documentation writer);
+		write_metadata writer cf.cf_meta;
+		write_field_kind writer cf.cf_kind;
+		let expr_chunk = match cf.cf_expr with
+			| None ->
+				Chunk.write_u8 writer.chunk 0;
+				None
+			| Some e when not write_expr_immediately ->
+				Chunk.write_u8 writer.chunk 2;
+				let fctx,close = start_texpr writer e.epos in
+				write_texpr writer fctx e;
+				Chunk.write_option writer.chunk cf.cf_expr_unoptimized (write_texpr writer fctx);
+				let expr_chunk = close() in
+				Some expr_chunk
+			| Some e ->
+				Chunk.write_u8 writer.chunk 1;
+				let fctx,close = start_texpr writer e.epos in
+				write_texpr writer fctx e;
+				Chunk.write_option writer.chunk cf.cf_expr_unoptimized (write_texpr writer fctx);
+				let expr_pre_chunk,expr_chunk = close() in
+				Chunk.export_data expr_pre_chunk writer.chunk;
+				Chunk.export_data expr_chunk writer.chunk;
+				None
+		in
+		restore (fun new_chunk ->
+			commit_field_type_parameters writer cf.cf_params;
+			Chunk.export_data new_chunk writer.chunk
+		);
+		expr_chunk
+
+	and write_class_field_and_overloads_data writer (write_expr_immediately : bool) (cf : tclass_field) =
+		let cfl = cf :: cf.cf_overloads in
+		Chunk.write_uleb128 writer.chunk (List.length cfl);
+		ExtList.List.filter_map (fun cf ->
+			let close = open_field_scope writer cf.cf_params in
+			let expr_chunk = write_class_field_data writer write_expr_immediately cf in
+			close();
+			Option.map (fun expr_chunk -> (cf,expr_chunk)) expr_chunk
+		) cfl
+
+	(* Module types *)
+
+	let select_type writer (path : path) =
+		writer.type_type_parameters <- Pool.extract writer.type_param_lut path
+
+	let write_common_module_type writer (infos : tinfos) : unit =
+		Chunk.write_bool writer.chunk infos.mt_private;
+		Chunk.write_option writer.chunk infos.mt_doc (write_documentation writer);
+		write_metadata writer infos.mt_meta;
+		write_type_parameters_data writer infos.mt_params;
+		Chunk.write_list writer.chunk infos.mt_using (fun (c,p) ->
+			write_class_ref writer c;
+			write_pos writer p;
+		)
+
+	let write_class_kind writer = function
+		| KNormal ->
+			Chunk.write_u8 writer.chunk 0
+		| KTypeParameter ttp ->
+			die "" __LOC__
+		| KExpr e ->
+			Chunk.write_u8 writer.chunk 2;
+			write_expr writer e;
+		| KGeneric ->
+			Chunk.write_u8 writer.chunk 3;
+		| KGenericInstance(c,tl) ->
+			Chunk.write_u8 writer.chunk 4;
+			write_class_ref writer c;
+			write_types writer tl
+		| KMacroType ->
+			Chunk.write_u8 writer.chunk 5;
+		| KGenericBuild l ->
+			Chunk.write_u8 writer.chunk 6;
+			Chunk.write_list writer.chunk l (write_cfield writer);
+		| KAbstractImpl a ->
+			Chunk.write_u8 writer.chunk 7;
+			write_abstract_ref writer a;
+		| KModuleFields md ->
+			Chunk.write_u8 writer.chunk 8
+
+	let write_class writer (c : tclass) =
+		begin match c.cl_kind with
+		| KAbstractImpl a ->
+			select_type writer a.a_path
+		| _ ->
+			select_type writer c.cl_path;
+		end;
+		write_common_module_type writer (Obj.magic c);
+		write_class_kind writer c.cl_kind;
+		Chunk.write_option writer.chunk c.cl_super (fun (c,tl) ->
+			write_class_ref writer c;
+			write_types writer tl
+		);
+		Chunk.write_list writer.chunk c.cl_implements (fun (c,tl) ->
+			write_class_ref writer c;
+			write_types writer tl
+		);
+		Chunk.write_option writer.chunk c.cl_dynamic (write_type_instance writer);
+		Chunk.write_option writer.chunk c.cl_array_access (write_type_instance writer)
+
+	let write_abstract writer (a : tabstract) =
+		begin try
+			select_type writer a.a_path
+		with Not_found ->
+			prerr_endline ("Could not select abstract " ^ (s_type_path a.a_path));
+		end;
+		write_common_module_type writer (Obj.magic a);
+		Chunk.write_option writer.chunk a.a_impl (write_class_ref writer);
+		if Meta.has Meta.CoreType a.a_meta then
+			Chunk.write_u8 writer.chunk 0
+		else begin
+			Chunk.write_u8 writer.chunk 1;
+			write_type_instance writer a.a_this;
+		end;
+		Chunk.write_list writer.chunk a.a_from (write_type_instance writer);
+		Chunk.write_list writer.chunk a.a_to (write_type_instance writer);
+		Chunk.write_bool writer.chunk a.a_enum
+
+	let write_abstract_fields writer (a : tabstract) =
+		let c = match a.a_impl with
+			| None ->
+				null_class
+			| Some c ->
+				c
+		in
+
+		Chunk.write_list writer.chunk a.a_array (write_field_ref writer c CfrStatic);
+		Chunk.write_option writer.chunk a.a_read (write_field_ref writer c CfrStatic );
+		Chunk.write_option writer.chunk a.a_write (write_field_ref writer c CfrStatic);
+		Chunk.write_option writer.chunk a.a_call (write_field_ref writer c CfrStatic);
+
+		Chunk.write_list writer.chunk a.a_ops (fun (op, cf) ->
+			Chunk.write_u8 writer.chunk (binop_index op);
+			write_field_ref writer c CfrStatic cf
+		);
+
+		Chunk.write_list writer.chunk a.a_unops (fun (op, flag, cf) ->
+			Chunk.write_u8 writer.chunk (unop_index op flag);
+			write_field_ref writer c CfrStatic cf
+		);
+
+		Chunk.write_list writer.chunk a.a_from_field (fun (t,cf) ->
+			write_field_ref writer c CfrStatic cf;
+		);
+
+		Chunk.write_list writer.chunk a.a_to_field (fun (t,cf) ->
+			write_field_ref writer c CfrStatic cf;
+		)
+
+	let write_enum writer (e : tenum) =
+		select_type writer e.e_path;
+		write_common_module_type writer (Obj.magic e);
+		Chunk.write_bool writer.chunk e.e_extern;
+		Chunk.write_list writer.chunk e.e_names (Chunk.write_string writer.chunk)
+
+	let write_typedef writer (td : tdef) =
+		select_type writer td.t_path;
+		write_common_module_type writer (Obj.magic td);
+		write_type_instance writer td.t_type
+
+	(* Module *)
+
+	let forward_declare_type writer (mt : module_type) =
+		let name = ref "" in
+		let i = match mt with
+		| TClassDecl c ->
+			ignore(Pool.add writer.classes c.cl_path c);
+			ignore(Pool.add writer.own_classes c.cl_path c);
+			name := snd c.cl_path;
+			0
+		| TEnumDecl e ->
+			ignore(Pool.add writer.enums e.e_path e);
+			ignore(Pool.add writer.own_enums e.e_path e);
+			name := snd e.e_path;
+			1
+		| TTypeDecl t ->
+			ignore(Pool.add writer.typedefs t.t_path t);
+			ignore(Pool.add writer.own_typedefs t.t_path t);
+			name := snd t.t_path;
+			2
+		| TAbstractDecl a ->
+			ignore(Pool.add writer.abstracts a.a_path a);
+			ignore(Pool.add writer.own_abstracts a.a_path a);
+			name := snd a.a_path;
+			3
+		in
+
+		let infos = t_infos mt in
+		Chunk.write_u8 writer.chunk i;
+		write_path writer (fst infos.mt_path, !name);
+		write_pos_pair writer infos.mt_pos infos.mt_name_pos;
+		write_type_parameters_forward writer infos.mt_params;
+		let params = Pool.create () in
+		writer.type_type_parameters <- params;
+		ignore(Pool.add writer.type_param_lut infos.mt_path params);
+		List.iter (fun ttp ->
+			ignore(Pool.add writer.type_type_parameters ttp.ttp_name ttp)
+		) infos.mt_params;
+
+		(* Forward declare fields *)
+		match mt with
+		| TClassDecl c ->
+			Chunk.write_uleb128 writer.chunk c.cl_flags;
+			Chunk.write_option writer.chunk c.cl_constructor (write_class_field_forward writer);
+			Chunk.write_option writer.chunk c.cl_init (write_class_field_forward writer);
+
+			(* Write in reverse order so reader can read tail-recursively without List.rev *)
+			let write_fields cfl =
+				let i = ref 0 in
+				let rec loop cfl = match cfl with
+					| [] ->
+						()
+					| cf :: cfl ->
+						loop cfl;
+						write_class_field_forward writer cf;
+						incr i;
+				in
+				let restore = start_temporary_chunk writer 256 in
+				loop cfl;
+				let bytes = restore (fun new_chunk -> Chunk.get_bytes new_chunk) in
+				!i,bytes
+			in
+			let num_fields,field_bytes = write_fields c.cl_ordered_fields in
+			let num_statics,static_bytes = write_fields c.cl_ordered_statics in
+			Chunk.write_uleb128 writer.chunk num_fields;
+			Chunk.write_uleb128 writer.chunk num_statics;
+			Chunk.write_bytes writer.chunk field_bytes;
+			Chunk.write_bytes writer.chunk static_bytes;
+
+		| TEnumDecl e ->
+			Chunk.write_list writer.chunk (PMap.foldi (fun s f acc -> (s,f) :: acc) e.e_constrs []) (fun (s,ef) ->
+				Chunk.write_string writer.chunk s;
+				write_pos_pair writer ef.ef_pos ef.ef_name_pos;
+				Chunk.write_u8 writer.chunk ef.ef_index
+			);
+		| TAbstractDecl a ->
+			()
+		| TTypeDecl t ->
+			()
+
+	let write_module writer (m : module_def) =
+		writer.current_module <- m;
+
+		start_chunk writer MTF;
+		Chunk.write_list writer.chunk m.m_types (forward_declare_type writer);
+
+		let items = Pool.finalize writer.own_abstracts in
+		if DynArray.length items > 0 then begin
+			start_chunk writer ABD;
+			Chunk.write_dynarray writer.chunk items (write_abstract writer);
+			start_chunk writer AFD;
+			Chunk.write_dynarray writer.chunk items (write_abstract_fields writer);
+		end;
+		let items = Pool.finalize writer.own_classes in
+		if DynArray.length items > 0 then begin
+			start_chunk writer CLD;
+			Chunk.write_dynarray writer.chunk items (write_class writer);
+			start_chunk writer CFD;
+			let expr_chunks = DynArray.create () in
+			Chunk.write_dynarray writer.chunk items (fun c ->
+				begin match c.cl_kind with
+				| KAbstractImpl a ->
+					select_type writer a.a_path
+				| _ ->
+					select_type writer c.cl_path;
+				end;
+
+				let c_expr_chunks = DynArray.create () in
+				let write_field ref_kind cf =
+					let l = write_class_field_and_overloads_data writer false cf in
+					List.iter (fun (cf,e) ->
+						DynArray.add c_expr_chunks (cf,ref_kind,e);
+					) l
+				in
+
+				Chunk.write_option writer.chunk c.cl_constructor (write_field CfrConstructor);
+				Chunk.write_option writer.chunk c.cl_init (write_field CfrInit);
+				Chunk.write_list writer.chunk c.cl_ordered_fields (write_field CfrMember);
+				Chunk.write_list writer.chunk c.cl_ordered_statics (write_field CfrStatic);
+				if DynArray.length c_expr_chunks > 0 then
+					DynArray.add expr_chunks (c,c_expr_chunks)
+			);
+			if DynArray.length expr_chunks > 0 then begin
+				start_chunk writer EXD;
+				Chunk.write_dynarray writer.chunk expr_chunks (fun (c,l) ->
+					write_class_ref writer c;
+					Chunk.write_dynarray writer.chunk l (fun (cf,ref_kind,(e_pre,e)) ->
+						write_field_ref writer c ref_kind cf;
+						let bytes_pre = Chunk.get_bytes e_pre in
+						let bytes_e = Chunk.get_bytes e in
+						Chunk.write_uleb128 writer.chunk (Bytes.length bytes_pre + Bytes.length bytes_e);
+						Chunk.write_bytes writer.chunk bytes_pre;
+						Chunk.write_bytes writer.chunk bytes_e;
+					)
+				)
+			end
+		end;
+		let items = Pool.finalize writer.own_enums in
+		if DynArray.length items > 0 then begin
+			start_chunk writer END;
+			Chunk.write_dynarray writer.chunk items (write_enum writer);
+			start_chunk writer EFD;
+			Chunk.write_dynarray writer.chunk items (fun e ->
+				Chunk.write_list writer.chunk (PMap.foldi (fun s f acc -> (s,f) :: acc) e.e_constrs []) (fun (s,ef) ->
+					select_type writer e.e_path;
+					let close = open_field_scope writer ef.ef_params in
+					Chunk.write_string writer.chunk s;
+					let restore = start_temporary_chunk writer 32 in
+					write_type_instance writer ef.ef_type;
+					let t_bytes = restore (fun new_chunk -> Chunk.get_bytes new_chunk) in
+					commit_field_type_parameters writer ef.ef_params;
+					Chunk.write_bytes writer.chunk t_bytes;
+					Chunk.write_option writer.chunk ef.ef_doc (write_documentation writer);
+					write_metadata writer ef.ef_meta;
+					close();
+				);
+			)
+		end;
+		let items = Pool.finalize writer.own_typedefs in
+		if DynArray.length items > 0 then begin
+			start_chunk writer TDD;
+			Chunk.write_dynarray writer.chunk items (write_typedef writer);
+		end;
+
+		let items = HashedIdentityPool.finalize writer.class_fields in
+		if DynArray.length items > 0 then begin
+			start_chunk writer CFR;
+			Chunk.write_uleb128 writer.chunk (DynArray.length items);
+			DynArray.iter (fun (cf,(c,kind,depth)) ->
+				write_class_ref writer c;
+				begin match kind with
+				| CfrStatic ->
+					Chunk.write_u8 writer.chunk 0;
+					Chunk.write_string writer.chunk cf.cf_name
+				| CfrMember ->
+					Chunk.write_u8 writer.chunk 1;
+					Chunk.write_string writer.chunk cf.cf_name
+				| CfrConstructor ->
+					Chunk.write_u8 writer.chunk 2;
+				| CfrInit ->
+					Chunk.write_u8 writer.chunk 3;
+				end;
+				Chunk.write_uleb128 writer.chunk depth
+			) items;
+		end;
+
+		let items = Pool.finalize writer.enum_fields in
+		if DynArray.length items > 0 then begin
+			start_chunk writer EFR;
+			Chunk.write_uleb128 writer.chunk (DynArray.length items);
+			DynArray.iter (fun (en,ef) ->
+				write_enum_ref writer en;
+				Chunk.write_string writer.chunk ef.ef_name;
+			) items;
+		end;
+
+		let items = HashedIdentityPool.finalize writer.anon_fields in
+		if DynArray.length items > 0 then begin
+			start_chunk writer AFR;
+			Chunk.write_uleb128 writer.chunk (DynArray.length items);
+			DynArray.iter (fun (cf,_) ->
+				write_class_field_forward writer cf
+			) items;
+		end;
+
+		let items = Pool.finalize writer.classes in
+		if DynArray.length items > 0 then begin
+			start_chunk writer CLR;
+			Chunk.write_dynarray writer.chunk items (fun c ->
+				let m = c.cl_module in
+				write_full_path writer (fst m.m_path) (snd m.m_path) (snd c.cl_path);
+			)
+		end;
+		let items = Pool.finalize writer.abstracts in
+		if DynArray.length items > 0 then begin
+			start_chunk writer ABR;
+			Chunk.write_dynarray writer.chunk items (fun a ->
+				let m = a.a_module in
+				write_full_path writer (fst m.m_path) (snd m.m_path) (snd a.a_path);
+			)
+		end;
+		let items = Pool.finalize writer.enums in
+		if DynArray.length items > 0 then begin
+			start_chunk writer ENR;
+			Chunk.write_dynarray writer.chunk items (fun en ->
+				let m = en.e_module in
+				write_full_path writer (fst m.m_path) (snd m.m_path) (snd en.e_path);
+			)
+		end;
+		let items = Pool.finalize writer.typedefs in
+		if DynArray.length items > 0 then begin
+			start_chunk writer TDR;
+			Chunk.write_dynarray writer.chunk items (fun td ->
+				let m = td.t_module in
+				write_full_path writer (fst m.m_path) (snd m.m_path) (snd td.t_path);
+			)
+		end;
+
+		start_chunk writer MDF;
+		write_path writer m.m_path;
+		Chunk.write_string writer.chunk (Path.UniqueKey.lazy_path m.m_extra.m_file);
+		Chunk.write_uleb128 writer.chunk (DynArray.length (Pool.finalize writer.anons));
+		Chunk.write_uleb128 writer.chunk (DynArray.length (IdentityPool.finalize writer.tmonos));
+
+		begin
+			let deps = DynArray.create () in
+			PMap.iter (fun _ mdep ->
+				match mdep.md_kind with
+				| MCode | MExtern | MFake when mdep.md_sign = m.m_extra.m_sign ->
+						DynArray.add deps mdep.md_path;
+				| _ ->
+					()
+			) m.m_extra.m_deps;
+			if DynArray.length deps > 0 then begin
+				start_chunk writer MDR;
+				Chunk.write_uleb128 writer.chunk (DynArray.length deps);
+				DynArray.iter (fun path ->
+					write_path writer path
+				) deps
+			end
+		end;
+
+		start_chunk writer EOT;
+		start_chunk writer EOF;
+		start_chunk writer EOM;
+
+		let finalize_string_pool kind items length =
+			start_chunk writer kind;
+			Chunk.write_uleb128 writer.chunk length;
+			List.iter (fun s ->
+				let b = Bytes.unsafe_of_string s in
+				Chunk.write_bytes_length_prefixed writer.chunk b;
+			) items
+		in
+		begin
+			let items,length = StringPool.finalize writer.cp in
+			finalize_string_pool STR items length
+		end;
+		begin
+			let items,length = StringPool.finalize writer.docs in
+			if length > 0 then
+				finalize_string_pool DOC items length
+		end
+
+	let get_sorted_chunks writer =
+		let l = DynArray.to_list writer.chunks in
+		let l = List.sort (fun chunk1 chunk2 ->
+			(Obj.magic chunk1.Chunk.kind - (Obj.magic chunk2.kind))
+		) l in
+		l
+end
+
+let create warn anon_id stats =
+	let cp = StringPool.create () in
+	{
+		warn;
+		anon_id;
+		stats;
+		current_module = null_module;
+		chunks = DynArray.create ();
+		cp = cp;
+		docs = StringPool.create ();
+		chunk = Obj.magic ();
+		classes = Pool.create ();
+		enums = Pool.create ();
+		typedefs = Pool.create ();
+		abstracts = Pool.create ();
+		anons = Pool.create ();
+		anon_fields = HashedIdentityPool.create ();
+		tmonos = IdentityPool.create ();
+		own_classes = Pool.create ();
+		own_abstracts = Pool.create ();
+		own_enums = Pool.create ();
+		own_typedefs = Pool.create ();
+		type_param_lut = Pool.create ();
+		class_fields = HashedIdentityPool.create ();
+		enum_fields = Pool.create ();
+		type_type_parameters = Pool.create ();
+		field_type_parameters = IdentityPool.create ();
+		local_type_parameters = IdentityPool.create ();
+		field_stack = [];
+		unbound_ttp = IdentityPool.create ();
+		t_instance_chunk = Chunk.create EOM cp 32;
+	}
+
+let write_module writer m =
+	HxbWriter.write_module writer m
+
+let get_chunks writer =
+	List.map (fun chunk ->
+		(chunk.Chunk.kind,Chunk.get_bytes chunk)
+	) (HxbWriter.get_sorted_chunks writer)
+
+let export : 'a . hxb_writer -> 'a IO.output -> unit = fun writer ch ->
+	write_header ch;
+	let l = HxbWriter.get_sorted_chunks writer in
+	List.iter (fun io ->
+		Chunk.export writer.stats io ch
+	) l

+ 174 - 26
src/compiler/server.ml

@@ -3,11 +3,13 @@ open Common
 open CompilationCache
 open Timer
 open Type
+open Typecore
 open DisplayProcessingGlobals
 open Ipaddr
 open Json
 open CompilationContext
 open MessageReporting
+open HxbData
 
 exception Dirty of module_skip_reason
 exception ServerError of string
@@ -313,7 +315,7 @@ let check_module sctx ctx m_path m_extra p =
 			end
 		in
 		let find_module_extra sign mpath =
-			((com.cs#get_context sign)#find_module mpath).m_extra
+			(com.cs#get_context sign)#find_module_extra mpath
 		in
 		let check_dependencies () =
 			PMap.iter (fun _ mdep ->
@@ -392,6 +394,75 @@ let check_module sctx ctx m_path m_extra p =
 	end;
 	state
 
+class hxb_reader_api_server
+	(ctx : Typecore.typer)
+	(cc : context_cache)
+= object(self)
+
+	method make_module (path : path) (file : string) =
+		let mc = cc#get_hxb_module path in
+		{
+			m_id = mc.mc_id;
+			m_path = path;
+			m_types = [];
+			m_statics = None;
+			m_extra = mc.mc_extra
+		}
+
+	method add_module (m : module_def) =
+		ctx.com.module_lut#add m.m_path m
+
+	method resolve_type (pack : string list) (mname : string) (tname : string) =
+		let path = (pack,mname) in
+		let m = self#resolve_module path in
+		List.find (fun t -> snd (t_path t) = tname) m.m_types
+
+	method resolve_module (path : path) =
+		match self#find_module path with
+		| GoodModule m ->
+			m
+		| BinaryModule mc ->
+			let reader = new HxbReader.hxb_reader path ctx.com.hxb_reader_stats in
+			let f_next chunks until =
+				let t_hxb = Timer.timer ["server";"module cache";"hxb read"] in
+				let r = reader#read_chunks_until (self :> HxbReaderApi.hxb_reader_api) chunks until in
+				t_hxb();
+				r
+			in
+			let m,chunks = f_next mc.mc_chunks EOF in
+
+			(* We try to avoid reading expressions as much as possible, so we only do this for
+				 our current display file if we're in display mode. *)
+			let is_display_file = DisplayPosition.display_position#is_in_file (Path.UniqueKey.lazy_key m.m_extra.m_file) in
+			if is_display_file || ctx.com.display.dms_full_typing then ignore(f_next chunks EOM);
+			m
+		| BadModule reason ->
+			die (Printf.sprintf "Unexpected BadModule %s" (s_type_path path)) __LOC__
+		| NoModule ->
+			die (Printf.sprintf "Unexpected NoModule %s" (s_type_path path)) __LOC__
+
+	method find_module (m_path : path) =
+		try
+			GoodModule (ctx.com.module_lut#find m_path)
+		with Not_found -> try
+			let mc = cc#get_hxb_module m_path in
+			begin match mc.mc_extra.m_cache_state with
+				| MSBad reason -> BadModule reason
+				| _ -> BinaryModule mc
+			end
+		with Not_found ->
+			NoModule
+
+	method basic_types =
+		ctx.com.basic
+
+	method get_var_id (i : int) =
+		i
+
+	method read_expression_eagerly (cf : tclass_field) =
+		ctx.com.display.dms_full_typing
+end
+
 let handle_cache_bound_objects com cbol =
 	DynArray.iter (function
 		| Resource(name,data) ->
@@ -404,25 +475,44 @@ let handle_cache_bound_objects com cbol =
 
 (* Adds module [m] and all its dependencies (recursively) from the cache to the current compilation
    context. *)
-let add_modules sctx ctx m p =
+let rec add_modules sctx ctx (m : module_def) (from_binary : bool) (p : pos) =
 	let com = ctx.Typecore.com in
+	let own_sign = CommonCache.get_cache_sign com in
 	let rec add_modules tabs m0 m =
 		if m.m_extra.m_added < ctx.com.compilation_step then begin
+			m.m_extra.m_added <- ctx.com.compilation_step;
 			(match m0.m_extra.m_kind, m.m_extra.m_kind with
 			| MCode, MMacro | MMacro, MCode ->
 				(* this was just a dependency to check : do not add to the context *)
 				handle_cache_bound_objects com m.m_extra.m_cache_bound_objects;
 			| _ ->
-				m.m_extra.m_added <- ctx.com.compilation_step;
 				ServerMessage.reusing com tabs m;
 				List.iter (fun t ->
 					(t_infos t).mt_restore()
 				) m.m_types;
-				TypeloadModule.ModuleLevel.add_module ctx m p;
+				(* The main module gets added when reading hxb already, so let's not add it again. Note that we
+				   can't set its m_added ahead of time because we want the rest of the logic here to run. *)
+				if not from_binary || m != m then
+					com.module_lut#add m.m_path m;
 				handle_cache_bound_objects com m.m_extra.m_cache_bound_objects;
 				PMap.iter (fun _ mdep ->
-					let m2 = (com.cs#get_context mdep.md_sign)#find_module mdep.md_path in
-					add_modules (tabs ^ "  ") m0 m2
+					let mpath = mdep.md_path in
+					if mdep.md_sign = own_sign then begin
+						let m2 = try
+							com.module_lut#find mpath
+						with Not_found ->
+							match type_module sctx ctx mpath p with
+							| GoodModule m ->
+								m
+							| BinaryModule mc ->
+								failwith (Printf.sprintf "Unexpectedly found unresolved binary module %s as a dependency of %s" (s_type_path mpath) (s_type_path m0.m_path))
+							| NoModule ->
+								failwith (Printf.sprintf "Unexpectedly could not find module %s as a dependency of %s" (s_type_path mpath) (s_type_path m0.m_path))
+							| BadModule reason ->
+								failwith (Printf.sprintf "Unexpected bad module %s (%s) as a dependency of %s" (s_type_path mpath) (Printer.s_module_skip_reason reason) (s_type_path m0.m_path))
+						in
+						add_modules (tabs ^ "  ") m0 m2
+					end
 				) m.m_extra.m_deps
 			)
 		end
@@ -431,29 +521,83 @@ let add_modules sctx ctx m p =
 
 (* Looks up the module referred to by [mpath] in the cache. If it exists, a check is made to
    determine if it's still valid. If this function returns None, the module is re-typed. *)
-let type_module sctx (ctx:Typecore.typer) mpath p =
+and type_module sctx (ctx:Typecore.typer) mpath p =
 	let t = Timer.timer ["server";"module cache"] in
 	let com = ctx.Typecore.com in
 	let cc = CommonCache.get_cache com in
-	try
-		let m = cc#find_module mpath in
-		let tcheck = Timer.timer ["server";"module cache";"check"] in
-		begin match check_module sctx ctx m.m_path m.m_extra p with
-		| None -> ()
-		| Some reason ->
-			ServerMessage.skipping_dep com "" (m.m_path,(Printer.s_module_skip_reason reason));
-			tcheck();
-			raise Not_found;
-		end;
-		tcheck();
+	let skip m_path reason =
+		ServerMessage.skipping_dep com "" (m_path,(Printer.s_module_skip_reason reason));
+		BadModule reason
+	in
+	let add_modules from_binary m =
 		let tadd = Timer.timer ["server";"module cache";"add modules"] in
-		add_modules sctx ctx m p;
+		add_modules sctx ctx m from_binary p;
 		tadd();
-		t();
-		Some m
-	with Not_found ->
-		t();
-		None
+		GoodModule m
+	in
+	let check_module sctx ctx m_path m_extra p =
+		let tcheck = Timer.timer ["server";"module cache";"check"] in
+		let r = check_module sctx ctx mpath m_extra p in
+		tcheck();
+		r
+	in
+	let find_module_in_cache ctx cc m_path p =
+		try
+			let m = cc#find_module m_path in
+			begin match m.m_extra.m_cache_state with
+				| MSBad reason -> BadModule reason
+				| _ -> GoodModule m
+			end;
+		with Not_found -> try
+			let mc = cc#get_hxb_module m_path in
+			begin match mc.mc_extra.m_cache_state with
+				| MSBad reason -> BadModule reason
+				| _ -> BinaryModule mc
+			end
+		with Not_found ->
+			NoModule
+	in
+	(* Should not raise anything! *)
+	let m = match find_module_in_cache ctx cc mpath p with
+		| GoodModule m ->
+			(* "Good" here is an assumption, it only means that the module wasn't explicitly invalidated
+			   in the cache. The true cache state will be known after check_module. *)
+			begin match check_module sctx ctx mpath m.m_extra p with
+				| None ->
+					add_modules false m;
+				| Some reason ->
+					skip m.m_path reason
+			end
+		| BinaryModule mc ->
+			(* Similarly, we only know that a binary module wasn't explicitly tainted. Decode it only after
+			   checking dependencies. This means that the actual decoding never has any reason to fail. *)
+			begin match check_module sctx ctx mpath mc.mc_extra p with
+				| None ->
+					let reader = new HxbReader.hxb_reader mpath com.hxb_reader_stats in
+					let api = (new hxb_reader_api_server ctx cc :> HxbReaderApi.hxb_reader_api) in
+					let f_next chunks until =
+						let t_hxb = Timer.timer ["server";"module cache";"hxb read"] in
+						let r = reader#read_chunks_until api chunks until in
+						t_hxb();
+						r
+					in
+					let m,chunks = f_next mc.mc_chunks EOF in
+					(* We try to avoid reading expressions as much as possible, so we only do this for
+					   our current display file if we're in display mode. *)
+					let is_display_file = DisplayPosition.display_position#is_in_file (Path.UniqueKey.lazy_key m.m_extra.m_file) in
+					if is_display_file || ctx.com.display.dms_full_typing then ignore(f_next chunks EOM);
+					add_modules true m;
+				| Some reason ->
+					skip mpath reason
+			end
+		| BadModule reason ->
+			(* A BadModule state here means that the module is already invalidated in the cache, e.g. from server/invalidate. *)
+			skip mpath reason
+		| NoModule as mr ->
+			mr
+	in
+	t();
+	m
 
 let before_anything sctx ctx =
 	ensure_macro_setup sctx
@@ -477,10 +621,13 @@ let after_target_init sctx ctx =
 		Hashtbl.add sctx.class_paths sign class_path_strings;
 		()
 
-let after_compilation sctx ctx =
-	if not (has_error ctx) then
+let after_save sctx ctx =
+	if ctx.comm.is_server && not (has_error ctx) then
 		maybe_cache_context sctx ctx.com
 
+let after_compilation sctx ctx =
+	()
+
 let mk_length_prefixed_communication allow_nonblock chin chout =
 	let sin = Unix.descr_of_in_channel chin in
 	let chin = IO.input_channel chin in
@@ -629,6 +776,7 @@ let rec process sctx comm args =
 		callbacks = {
 			before_anything = before_anything sctx;
 			after_target_init = after_target_init sctx;
+			after_save = after_save sctx;
 			after_compilation = after_compilation sctx;
 		};
 		init_wait_socket = init_wait_socket;

+ 15 - 0
src/context/common.ml

@@ -333,6 +333,13 @@ class module_lut = object(self)
 	method get_type_lut = type_lut
 end
 
+class virtual abstract_hxb_lib = object(self)
+	method virtual load : unit
+	method virtual get_bytes : string -> path -> bytes option
+	method virtual close : unit
+	method virtual get_file_path : string
+end
+
 type context = {
 	compilation_step : int;
 	mutable stage : compiler_stage;
@@ -401,6 +408,7 @@ type context = {
 	mutable neko_lib_paths : string list;
 	mutable include_files : (string * string) list;
 	mutable native_libs : native_libraries;
+	mutable hxb_libs : abstract_hxb_lib list;
 	mutable net_std : string list;
 	net_path_map : (path,string list * string list * string) Hashtbl.t;
 	mutable c_args : string list;
@@ -408,6 +416,8 @@ type context = {
 	(* misc *)
 	mutable basic : basic_types;
 	memory_marker : float array;
+	hxb_reader_stats : HxbReader.hxb_reader_stats;
+	hxb_writer_stats : HxbWriter.hxb_writer_stats;
 }
 
 let enter_stage com stage =
@@ -823,6 +833,7 @@ let create compilation_step cs version args display_mode =
 		resources = Hashtbl.create 0;
 		net_std = [];
 		native_libs = create_native_libs();
+		hxb_libs = [];
 		net_path_map = Hashtbl.create 0;
 		c_args = [];
 		neko_lib_paths = [];
@@ -866,6 +877,8 @@ let create compilation_step cs version args display_mode =
 		has_error = false;
 		report_mode = RMNone;
 		is_macro_context = false;
+		hxb_reader_stats = HxbReader.create_hxb_reader_stats ();
+		hxb_writer_stats = HxbWriter.create_hxb_writer_stats ();
 	} in
 	com
 
@@ -912,6 +925,8 @@ let clone com is_macro_context =
 		module_to_file = new hashtbl_lookup;
 		overload_cache = new hashtbl_lookup;
 		module_lut = new module_lut;
+		hxb_reader_stats = HxbReader.create_hxb_reader_stats ();
+		hxb_writer_stats = HxbWriter.create_hxb_writer_stats ();
 		std = null_class;
 		empty_class_path = new ClassPath.directory_class_path "" User;
 		class_paths = new ClassPaths.class_paths;

+ 17 - 6
src/context/commonCache.ml

@@ -77,18 +77,29 @@ let get_cache com = match com.Common.cache with
 	| Some cache ->
 		cache
 
+let get_cache_sign com = match com.Common.cache with
+	| None -> Define.get_signature com.defines
+	| Some cache -> cache#get_sign
+
 let rec cache_context cs com =
 	let cc = get_cache com in
 	let sign = Define.get_signature com.defines in
+	let anon_identification = new Tanon_identification.tanon_identification in
 	let cache_module m =
 		(* If we have a signature mismatch, look-up cache for module. Physical equality check is fine as a heueristic. *)
-		let cc = if m.m_extra.m_sign == sign then cc else cs#get_context m.m_extra.m_sign in
-		cc#cache_module m.m_path m;
+		let cc = if m.m_extra.m_sign = sign then cc else cs#get_context m.m_extra.m_sign in
+		let warn w s p = com.warning w com.warning_options s p in
+		cc#cache_module warn anon_identification com.hxb_writer_stats m.m_path m;
 	in
 	List.iter cache_module com.modules;
-	match com.get_macros() with
-	| None -> ()
-	| Some com -> cache_context cs com
+	begin match com.get_macros() with
+		| None -> ()
+		| Some com -> cache_context cs com
+	end;
+	if Define.raw_defined com.defines "hxb.stats" then begin
+		HxbReader.dump_stats (platform_name com.platform) com.hxb_reader_stats;
+		HxbWriter.dump_stats (platform_name com.platform) com.hxb_writer_stats
+	end
 
 let maybe_add_context_sign cs com desc =
 	let sign = Define.get_signature com.defines in
@@ -97,4 +108,4 @@ let maybe_add_context_sign cs com desc =
 let lock_signature com name =
 	let cs = com.cs in
 	maybe_add_context_sign cs com name;
-	com.cache <- Some (get_cache com)
+	com.cache <- Some (get_cache com)

+ 55 - 5
src/context/display/displayJson.ml

@@ -104,6 +104,55 @@ class display_handler (jsonrpc : jsonrpc_handler) com (cs : CompilationCache.t)
 		end
 end
 
+class hxb_reader_api_com
+	~(headers_only : bool)
+	(com : Common.context)
+	(cc : CompilationCache.context_cache)
+= object(self)
+	method make_module (path : path) (file : string) =
+		let mc = cc#get_hxb_module path in
+		{
+			m_id = mc.mc_id;
+			m_path = path;
+			m_types = [];
+			m_statics = None;
+			m_extra = mc.mc_extra
+		}
+
+	method add_module (m : module_def) =
+		com.module_lut#add m.m_path m;
+
+	method resolve_type (pack : string list) (mname : string) (tname : string) =
+		let path = (pack,mname) in
+		let m = self#find_module path in
+		List.find (fun t -> snd (t_path t) = tname) m.m_types
+
+	method resolve_module (path : path) =
+		self#find_module path
+
+	method find_module (m_path : path) =
+		try
+			com.module_lut#find m_path
+		with Not_found -> try
+			cc#find_module m_path
+		with Not_found ->
+			let mc = cc#get_hxb_module m_path in
+			let reader = new HxbReader.hxb_reader mc.mc_path com.hxb_reader_stats in
+			fst (reader#read_chunks_until (self :> HxbReaderApi.hxb_reader_api) mc.mc_chunks (if headers_only then MTF else EOM))
+
+	method basic_types =
+		com.basic
+
+	method get_var_id (i : int) =
+		i
+
+	method read_expression_eagerly (cf : tclass_field) =
+		false
+end
+
+let find_module ~(headers_only : bool) com cc path =
+	(new hxb_reader_api_com ~headers_only com cc)#find_module path
+
 type handler_context = {
 	com : Common.context;
 	jsonrpc : jsonrpc_handler;
@@ -280,9 +329,10 @@ let handler =
 		"server/modules", (fun hctx ->
 			let sign = Digest.from_hex (hctx.jsonrpc#get_string_param "signature") in
 			let cc = hctx.display#get_cs#get_context sign in
+			let open HxbData in
 			let l = Hashtbl.fold (fun _ m acc ->
-				if m.m_extra.m_kind <> MFake then jstring (s_type_path m.m_path) :: acc else acc
-			) cc#get_modules [] in
+				if m.mc_extra.m_kind <> MFake then jstring (s_type_path m.mc_path) :: acc else acc
+			) cc#get_hxb [] in
 			hctx.send_result (jarray l)
 		);
 		"server/module", (fun hctx ->
@@ -291,11 +341,11 @@ let handler =
 			let cs = hctx.display#get_cs in
 			let cc = cs#get_context sign in
 			let m = try
-				cc#find_module path
+				find_module ~headers_only:true hctx.com cc path
 			with Not_found ->
 				hctx.send_error [jstring "No such module"]
 			in
-			hctx.send_result (generate_module cs cc m)
+			hctx.send_result (generate_module (cc#get_hxb) (find_module ~headers_only:true hctx.com cc) m)
 		);
 		"server/type", (fun hctx ->
 			let sign = Digest.from_hex (hctx.jsonrpc#get_string_param "signature") in
@@ -303,7 +353,7 @@ let handler =
 			let typeName = hctx.jsonrpc#get_string_param "typeName" in
 			let cc = hctx.display#get_cs#get_context sign in
 			let m = try
-				cc#find_module path
+				find_module ~headers_only:true hctx.com cc path
 			with Not_found ->
 				hctx.send_error [jstring "No such module"]
 			in

+ 15 - 4
src/context/display/displayTexpr.ml

@@ -170,10 +170,21 @@ let check_display_file ctx cs =
 			TypeloadParse.PdiHandler.handle_pdi ctx.com cfile.c_pdi;
 			(* We have to go through type_module_hook because one of the module's dependencies could be
 			   invalid (issue #8991). *)
-			begin match !TypeloadModule.type_module_hook ctx path null_pos with
-			| None -> raise Not_found
-			| Some m -> check_display_module ctx cfile.c_decls m
-			end
+			let m = try
+				ctx.com.module_lut#find path
+			with Not_found ->
+				begin match !TypeloadModule.type_module_hook ctx path null_pos with
+				| NoModule | BadModule _ -> raise Not_found
+				| BinaryModule mc ->
+					let api = (new TypeloadModule.hxb_reader_api_typeload ctx TypeloadModule.load_module' p :> HxbReaderApi.hxb_reader_api) in
+					let reader = new HxbReader.hxb_reader path ctx.com.hxb_reader_stats in
+					let m = reader#read_chunks api mc.mc_chunks in
+					m
+				| GoodModule m ->
+					m
+				end
+			in
+			check_display_module ctx cfile.c_decls m
 		with Not_found ->
 			let fkey = DisplayPosition.display_position#get_file_key in
 			(* force parsing again : if the completion point have been changed *)

+ 6 - 0
src/context/nativeLibraryHandler.ml

@@ -48,3 +48,9 @@ let add_native_lib com lib =
 			| _ -> failwith ("unsupported file@`std` format: " ^ file)
 		in
 		Dotnet.add_net_lib com file is_std is_extern
+	| HxbLib ->
+		let hxb_lib = HxbLib.create_hxb_lib com file in
+		com.hxb_libs <- hxb_lib :: com.hxb_libs;
+		(fun () ->
+			hxb_lib#load
+		)

+ 6 - 0
src/context/typecore.ml

@@ -211,6 +211,12 @@ type dot_path_part = {
 	pos : pos
 }
 
+type find_module_result =
+	| GoodModule of module_def
+	| BadModule of module_skip_reason
+	| BinaryModule of HxbData.module_cache
+	| NoModule
+
 let make_build_info kind path params extern apply = {
 	build_kind = kind;
 	build_path = path;

+ 1 - 1
src/core/define.ml

@@ -152,7 +152,7 @@ let get_signature def =
 			   Parser.parse_macro_ident as well (issue #5682).
 			   Note that we should removed flags like use_rtti_doc here.
 			*)
-			| "display" | "use_rtti_doc" | "macro_times" | "display_details" | "no_copt" | "display_stdin"
+			| "display" | "use_rtti_doc" | "macro_times" | "display_details" | "no_copt" | "display_stdin" | "hxb.stats"
 			| "message.reporting" | "message.log_file" | "message.log_format" | "message.no_color"
 			| "dump" | "dump_dependencies" | "dump_ignore_var_ids" -> acc
 			| _ -> (k ^ "=" ^ v) :: acc

+ 8 - 8
src/core/json/genjson.ml

@@ -701,7 +701,7 @@ let generate_module_type ctx mt =
 
 (* module *)
 
-let generate_module cs cc m =
+let generate_module modules find_module m =
 	jobject [
 		"id",jint m.m_id;
 		"path",generate_module_path m.m_path;
@@ -715,16 +715,16 @@ let generate_module cs cc m =
 		"dependencies",jarray (PMap.fold (fun mdep acc ->
 			(jobject [
 				"path",jstring (s_type_path mdep.md_path);
-				"sign",jstring (Digest.to_hex ((cs#get_context mdep.md_sign)#find_module mdep.md_path).m_extra.m_sign);
+				"sign",jstring (Digest.to_hex (find_module mdep.md_path).m_extra.m_sign);
 			]) :: acc
 		) m.m_extra.m_deps []);
-		"dependents",jarray (List.map (fun m -> (jobject [
-			"path",jstring (s_type_path m.m_path);
-			"sign",jstring (Digest.to_hex m.m_extra.m_sign);
-		])) (Hashtbl.fold (fun _ m' acc ->
-			if PMap.mem m.m_id m'.m_extra.m_deps then m' :: acc
+		"dependents",jarray (List.map (fun (path, sign) -> (jobject [
+			"path",jstring (s_type_path path);
+			"sign",jstring (Digest.to_hex sign);
+		])) (Hashtbl.fold (fun _ (m':HxbData.module_cache) acc ->
+			if PMap.mem m.m_id m'.mc_extra.m_deps then (m'.mc_path, m'.mc_extra.m_sign) :: acc
 			else acc
-		) cc#get_modules []));
+		) modules []));
 	]
 
 let create_context ?jsonrpc gm = {

+ 1 - 0
src/core/tType.ml

@@ -502,6 +502,7 @@ type flag_tvar =
 	| VCaught
 	| VStatic
 	| VUsedByTyper (* Set if the typer looked up this variable *)
+	| VHxb (* Flag used by hxb *)
 
 let flag_tvar_names = [
 	"VCaptured";"VFinal";"VAnalyzed";"VAssigned";"VCaught";"VStatic";"VUsedByTyper"

+ 1 - 1
src/core/timer.ml

@@ -212,4 +212,4 @@ class timer (id : string list) = object(self)
 
 	method nest (name : string) =
 		new timer (id @ [name])
-end
+end

+ 80 - 4
src/typing/typeloadModule.ml

@@ -772,18 +772,94 @@ let type_module ctx mpath file ?(dont_check_path=false) ?(is_extern=false) tdecl
 	let timer = Timer.timer ["typing";"type_module"] in
 	Std.finally timer (type_module ctx mpath file ~is_extern tdecls) p *)
 
-let type_module_hook = ref (fun _ _ _ -> None)
+let type_module_hook = ref (fun _ _ _ -> NoModule)
 
-let load_module' ctx m p =
+class hxb_reader_api_typeload
+	(ctx : typer)
+	(load_module : typer -> path -> pos -> module_def)
+	(p : pos)
+= object(self)
+	method make_module (path : path) (file : string) =
+		let m = ModuleLevel.make_module ctx path file p in
+		m.m_extra.m_processed <- 1;
+		m
+
+	method add_module (m : module_def) =
+		ctx.com.module_lut#add m.m_path m
+
+	method resolve_type (pack : string list) (mname : string) (tname : string) =
+		let m = load_module ctx (pack,mname) p in
+		List.find (fun t -> snd (t_path t) = tname) m.m_types
+
+	method resolve_module (path : path) =
+		load_module ctx path p
+
+	method basic_types =
+		ctx.com.basic
+
+	method get_var_id (i : int) =
+		(* The v_id in .hxb has no relation to this context, make a new one. *)
+		let uid = fst alloc_var' in
+		incr uid;
+		!uid
+
+	method read_expression_eagerly (cf : tclass_field) =
+		ctx.com.is_macro_context || match cf.cf_kind with
+			| Var _ ->
+				true
+			| Method _ ->
+				delay ctx PTypeField (fun () -> ignore(follow cf.cf_type));
+				false
+end
+
+let rec load_hxb_module ctx path p =
+	let read file bytes =
+		try
+			let api = (new hxb_reader_api_typeload ctx load_module' p :> HxbReaderApi.hxb_reader_api) in
+			let reader = new HxbReader.hxb_reader path ctx.com.hxb_reader_stats in
+			let read = reader#read api bytes in
+			let m = read MTF in
+			delay ctx PBuildClass (fun () ->
+				ignore(read EOT);
+				delay ctx PConnectField (fun () ->
+					ignore(read EOM);
+				);
+			);
+			m
+		with e ->
+			Printf.eprintf "\x1b[30;41mError loading %s from %s\x1b[0m\n" (snd path) file;
+			let msg = Printexc.to_string e and stack = Printexc.get_backtrace () in
+			Printf.eprintf " => %s\n%s\n" msg stack;
+			raise e
+	in
+	let target = Common.platform_name_macro ctx.com in
+	let rec loop l = match l with
+		| hxb_lib :: l ->
+			begin match hxb_lib#get_bytes target path with
+				| Some bytes ->
+					read hxb_lib#get_file_path bytes
+				| None ->
+					loop l
+			end
+		| [] ->
+			raise Not_found
+	in
+	loop ctx.com.hxb_libs
+
+and load_module' ctx m p =
 	try
 		(* Check current context *)
 		ctx.com.module_lut#find m
 	with Not_found ->
 		(* Check cache *)
 		match !type_module_hook ctx m p with
-		| Some m ->
+		| GoodModule m ->
 			m
-		| None ->
+		| BinaryModule _ ->
+			die "" __LOC__ (* The server builds those *)
+		| NoModule | BadModule _ -> try
+			load_hxb_module ctx m p
+		with Not_found ->
 			let raise_not_found () = raise_error_msg (Module_not_found m) p in
 			if ctx.com.module_nonexistent_lut#mem m then raise_not_found();
 			if ctx.g.load_only_cached_modules then raise_not_found();

+ 0 - 2
std/jvm/Jvm.hx

@@ -40,8 +40,6 @@ import jvm.annotation.EnumValueReflectionInformation;
 @:keep
 @:native('haxe.jvm.Jvm')
 class Jvm {
-	extern static final init:java.Init;
-
 	extern static public function instanceof<S, T>(obj:S, type:T):Bool;
 
 	extern static public function referenceEquals<T>(v1:T, v2:T):Bool;

+ 7 - 1
tests/runci/targets/Jvm.hx

@@ -14,11 +14,17 @@ class Jvm {
 
 		for (level in 0...3) {
 			final args = args.concat(["-D", "jvm.dynamic-level=" + level]);
-			runCommand("haxe", ["compile-jvm-only.hxml", "--times"].concat(args));
+			runCommand("haxe", ["compile-jvm-only.hxml", "--hxb", "bin/hxb/jvm.zip"].concat(args));
+			runCommand("java", ["-jar", "bin/unit.jar"]);
+
+			runCommand("haxe", ["compile-jvm-only.hxml", "--hxb-lib", "bin/hxb/jvm.zip"].concat(args));
 			runCommand("java", ["-jar", "bin/unit.jar"]);
 
 			runCommand("haxe", ["compile-jvm-only.hxml","-dce","no"].concat(args));
 			runCommand("java", ["-jar", "bin/unit.jar"]);
+
+			runCommand("haxe", ["compile-jvm-only.hxml", "--hxb-lib", "bin/hxb/jvm.zip"].concat(args));
+			runCommand("java", ["-jar", "bin/unit.jar"]);
 		}
 
 		changeDirectory(miscJavaDir);

+ 2 - 1
tests/runci/targets/Macro.hx

@@ -5,7 +5,8 @@ import runci.Config.*;
 
 class Macro {
 	static public function run(args:Array<String>) {
-		runCommand("haxe", ["compile-macro.hxml"].concat(args));
+		runCommand("haxe", ["compile-macro.hxml", "--hxb", "bin/hxb/eval.zip"].concat(args));
+		runCommand("haxe", ["compile-macro.hxml", "--hxb-lib", "bin/hxb/eval.zip"].concat(args));
 
 		changeDirectory(displayDir);
 		haxelibInstallGit("Simn", "haxeserver");

+ 1 - 1
tests/server/src/cases/ServerTests.hx

@@ -130,7 +130,7 @@ class ServerTests extends TestCase {
 		runHaxe(args2);
 
 		runHaxe(args);
-		assertSkipping("HelloWorld", Tainted("check_display_file"));
+		assertSkipping("HelloWorld", Tainted("server/invalidate"));
 	}
 
 	function testMutuallyDependent() {

+ 7 - 0
tests/unit/compile-hxb-interp-roundtrip.hxml

@@ -0,0 +1,7 @@
+compile-macro.hxml
+--hxb bin/hxb/eval.zip
+
+--next
+
+compile-macro.hxml
+--hxb-lib bin/hxb/eval.zip

+ 7 - 0
tests/unit/compile-hxb-jvm-roundtrip.hxml

@@ -0,0 +1,7 @@
+compile-jvm-only.hxml
+--hxb bin/hxb/jvm.zip
+
+--next
+
+compile-jvm-only.hxml
+--hxb-lib bin/hxb/jvm.zip