Fix tests, fix identity issue

doc/SECURITY.md

@@ -6,38 +6,51 @@ Author(s): Adam Ierymenko <[email protected]>
 
 # Introduction
 
-This document briefly describes the core components of ZeroTier's cryptographic and security architecture. It focuses primarily on version 2.0, ignoring deprecated v1.x constructions that are in the process of being phased out.
+This document describes the core components of ZeroTier's cryptographic and security architecture. It focuses primarily on version 2.0 and only briefly touches on v1.x constructions that are being phased out.
 
 The intended audience for this document is developers, auditors, and security professionals wishing to understand ZeroTier's design from a security posture point of view. It's also written to serve as the basis for professional security audits of the ZeroTier protocol and code base.
 
 ## High-Level Protocol Design
 
-ZeroTier's protocol is split into two conceptual layers that we term **VL1** and **VL2**. VL1 stands for *virtual layer 1* and is a cryptographically addressed secure global peer-to-peer network responsible for moving packets between ZeroTier nodes. VL2 stands for *virtual layer 2* and is a full Ethernet emulation layer incorporating cryptographic certificate and token based access control. It's conceptually separate from VL1 but for the sake of simplicity and user experience leverages VL1's cryptographic keys and identifiers to implement this access control scheme.
+ZeroTier's protocol is split into two conceptual layers that we term **VL1** and **VL2**.
 
-## Asymmetric Cryptography, Identities, and Addressing
+VL1 stands for *virtual layer 1* and is a cryptographically addressed secure global peer-to-peer network responsible for moving packets between ZeroTier nodes. It's a virtual analogue of the physical wire or radio transciever in an Ethernet or WiFi network respectively. Think of it as a gigantic wire closet for planet Earth.
 
-VL1 peers are cryptographically addressed. This means that their public keys constitute their addresses. Cryptographic addressing is extremely convenient in peer-to-peer networks as it makes the authentication of addresses implicit via authenticated encryption and/or cryptographic signatures.
+VL2 stands for *virtual layer 2* and is a full Ethernet emulation layer incorporating cryptographic certificate and token based access control. It is similar (but not identical) to other Ethernet virtualization protocols like VXLAN. VL2 is conceptually separate from VL1 but for the sake of simplicity and ease of use leverages VL1's cryptographic infrastructure for its own authentication needs.
 
-A ZeroTier identity consists of one or more cryptographic public keys and a short address derived from a hash of those keys. A fingerprint is simply the SHA-384 hash of these keys and is longer than an address but also shorter than the keys themselves and size-invariant across different identity types.
+## VL1 Asymmetric Cryptography: Identities, and Addressing
+
+VL1 peers are cryptographically addressed, meaning addresses are strongly bound to public keys. Cryptographic addressing is extremely convenient in peer-to-peer networks as it leverages authenticated (AEAD) encryption to implicity authenticate endpoint addresses.
+
+A ZeroTier identity is comprised of one or more cryptographic public keys and a short **ZeroTier address** derived from a hash of those keys. In addition to this short address there also exists a longer fingerprint in the form of a SHA-384 hash of identity public key(s).
 
 #### Identity Types and Corresponding Algorithms
 
-* Type 0: one Curve25519 key for elliptic curve Diffie-Hellman and one Ed25519 key for Ed25519 signatures, with the address computed from a hash of both.
-* Type 1: Curve25519, Ed25519, and NIST P-384 public keys, with the latter being used for signatures (the Ed25519 key is still there but is presently unused) and with *both* keys being used for elliptic curve Diffie-Hellman key agreement. In key agreement the resulting raw secret keys are hashed together using SHA-384 to combine them and yield a single session key.
+* **Type 0** (v1.x and v2.x): one Curve25519 key for elliptic curve Diffie-Hellman and one Ed25519 key for Ed25519 signatures, with the address and fingerprint computed from a hash of both.
+* **Type 1** (v2.x only): Curve25519, Ed25519, and NIST P-384 public keys, with the latter being used for signatures (the Ed25519 key is still there but is presently unused) and with *both* keys being used for elliptic curve Diffie-Hellman key agreement. In key agreement the resulting raw secret keys are hashed together using SHA-384 to combine them and yield a single session key.
+
+Session keys resulting from identity key exchange and agreement are *long-lived keys* that remain static for the lifetime of a particular pair of identities. A different mechanism is used for ephemeral key negotiation.
 
-(Session keys resulting from identity key exchange and agreement are long-lived session keys. A different mechanism is used for ephemeral key negotiation.)
+#### ZeroTier Addresses
 
-#### Short Addresses
+In the simplest form of cryptographic addressing, keys are used directly as addresses throughout the system. Unfortunately even public key cryptosystems with short keys like Curve25519 still result in string representations that are prohibitively long for human beings to type. ZeroTier mitigates this usability problem by using a short hash of the public key termed a **ZeroTier address** to refer to a peer's full identity. This short address is also used at the wire level to reduce the size of the packet header. Peers may request full identities based on addresses from from root servers.
 
-In simple cryptographic addressing, keys are used directly as addresses throughout the system. Unfortunately even public key cryptosystems with short keys like Curve25519 still result in string representations that are prohibitively long for human beings to type. ZeroTier mitigates this usability problem by using a short hash of the public key termed a **ZeroTier address** to refer to a peer's full identity. This short address is also used at the wire level to reduce the size of the packet header. Peers may request full identities based on addresses from one another or (more often) from root servers.
+ZeroTier addresses are very short: only 40 bits or 10 hexadecimal digits, e.g. `89e92ceee5.` This makes them convenient to type, but such a short hash would in a naive implementation introduce a significant risk that an attacker could create a duplicate identity with a different key pair but the same address. With 40 bits an intentional collision would require only an average of about 549,755,813,888 attempts for a 50% chance of colliding. If an attempt requires 0.5ms of CPU time on a typical contemporary desktop or server CPU, this would require about 3,000 CPU-days. Since this type of search is easy to parallelize, it would take only a few days for someone with access to a few thousand CPU cores.
 
-ZeroTier addresses are very short: only 40 bits or 10 hexadecimal digits, e.g. `89e92ceee5.` This makes them convenient to type but would in a naive implementation introduce a significant risk that an attacker could create a duplicate identity with a different key pair but the same address. With 40 bits an intentional collision would require only an average of about 549,755,813,888 attempts (for a 50% chance of colliding). If an attempt requires 0.5ms of CPU time on a typical contemporary desktop or server CPU, this would require about 3,000 CPU-days. Since this type of search is easy to parallelize, it would take only a few days for someone with access to a few thousand CPU cores.
+To provide this short hash with a larger security margin, an intentionally slow one-way "hashcash" or "proof of work" function is required during identity generation. This work function is slow to compute but fast to verify, and an address is not valid unless its work checks out. This gives identity address derivation the following costs:
 
-To increase the security margin of this construction an intentionally slow one-way "hashcash" or "proof of work" function is required during identity generation. This work function is slow to compute but fast to verify. In our system the cost of identity generation is increased to approximately one second per identity key pair per CPU core for a typical desktop or server CPU (e.g. an Intel Core i7 or AMD Ryzen at 2.4ghz). At this rate a search would require about 6.3 million CPU-days for a 50% probability of collision.
+* Type 1 identities: an average of about 500ms per key pair per typical 2.4ghz CPU core, requiring around 3 million CPU-days to reach a 50% collision probability.
+* Type 2 identities: an average of about one second per key pair per typical 2.4ghz CPU core, requiring around 6.3 million CPU-days to reach a 50% collision probability.
 
 While too costly for the vast majority of attackers, this cost may not be prohibitive to a nation-state level attacker or to a criminal with significant funds and/or access to a very large "botnet." It's also possible that FPGA, GPU, or ASIC acceleration could be leveraged to decrease this time in a manner similar to what's been accomplished in the area of cryptocurrency mining.
 
+Fingerprints are full SHA-384 hashes of identity public keys. In base32-encoding they look like this:
+
+```
+bzg7fc3sn46fzyxcxw2ev4c4m2u5fyisb3o4wz5hfmvexbzwk6et3fsglkdcn6nnjobxi3bq7hgxqox3n4u4k
+```
 
+These are too large to type but not to copy/paste, store in databases, or use in scripts and APIs.
 
 ## VL1 Wire Protocol
 

node/Fingerprint.hpp

@@ -63,7 +63,7 @@ public:
 	 *
 	 * @param s Base32 string
 	 */
-	ZT_INLINE void toString(char s[ZT_FINGERPRINT_STRING_BUFFER_LENGTH])
+	ZT_INLINE void toString(char s[ZT_FINGERPRINT_STRING_BUFFER_LENGTH]) const noexcept
 	{
 		uint8_t tmp[48 + 5];
 		address().copyTo(tmp);
@@ -78,7 +78,7 @@ public:
 	 * @param s String to decode
 	 * @return True if string appears to be valid and of the proper length (no other checking is done)
 	 */
-	ZT_INLINE bool fromString(const char *s)
+	ZT_INLINE bool fromString(const char *s) noexcept
 	{
 		uint8_t tmp[48 + 5];
 		if (Utils::b32d(s,tmp,sizeof(tmp)) != sizeof(tmp))

node/Identity.cpp

@@ -112,7 +112,8 @@ bool Identity::generate(const Type t)
 				Utils::storeBigEndian(_pub.t1mimc52,mimc52Delay(&_pub,sizeof(_pub) - sizeof(_pub.t1mimc52),ZT_V1_IDENTITY_MIMC52_VDF_ROUNDS_BASE));
 
 				// Compute SHA384 fingerprint hash of keys and MIMC output and generate address directly from it.
-				_computeHash(); // this sets the address for P384
+				_computeHash();
+				_address.setTo(_fp.hash());
 				if (!_address.isReserved())
 					break;
 			}
@@ -401,6 +402,10 @@ bool Identity::fromString(const char *str)
 	}
 
 	_computeHash();
+	if ((_type == P384)&&(_address != Address(_fp.hash()))) {
+		_address.zero();
+		return false;
+	}
 
 	return true;
 }
@@ -444,6 +449,8 @@ int Identity::unmarshal(const uint8_t *data,const int len) noexcept
 
 	if (len < (1 + ZT_ADDRESS_LENGTH))
 		return -1;
+	_address.setTo(data);
+
 	unsigned int privlen;
 	switch((_type = (Type)data[ZT_ADDRESS_LENGTH])) {
 
@@ -452,7 +459,6 @@ int Identity::unmarshal(const uint8_t *data,const int len) noexcept
 				return -1;
 
 			memcpy(_pub.c25519,data + ZT_ADDRESS_LENGTH + 1,ZT_C25519_PUBLIC_KEY_LEN);
-			_address.setTo(data);
 			_computeHash();
 
 			privlen = data[ZT_ADDRESS_LENGTH + 1 + ZT_C25519_PUBLIC_KEY_LEN];
@@ -474,7 +480,7 @@ int Identity::unmarshal(const uint8_t *data,const int len) noexcept
 
 			memcpy(&_pub,data + ZT_ADDRESS_LENGTH + 1,ZT_IDENTITY_P384_COMPOUND_PUBLIC_KEY_SIZE);
 			_computeHash(); // this sets the address for P384
-			if (_address != Address(data)) // sanity check address in data stream
+			if (_address != Address(_fp.hash())) // this sanity check is possible with V1 identities
 				return -1;
 
 			privlen = data[ZT_ADDRESS_LENGTH + 1 + ZT_IDENTITY_P384_COMPOUND_PUBLIC_KEY_SIZE];
@@ -509,7 +515,6 @@ void Identity::_computeHash()
 
 		case P384:
 			SHA384(_fp._fp.hash,&_pub,sizeof(_pub));
-			_address.setTo(reinterpret_cast<const uint8_t *>(_fp._fp.hash));
 			_fp._fp.address = _address.toInt();
 			break;
 	}

node/Identity.hpp

@@ -174,7 +174,7 @@ public:
 	/**
 	 * @return This identity's address
 	 */
-	ZT_INLINE const Address &address() const noexcept { return _address; }
+	ZT_INLINE Address address() const noexcept { return _address; }
 
 	/**
 	 * Serialize to a more human-friendly string

node/Peer.hpp

@@ -74,7 +74,7 @@ public:
 	/**
 	 * @return This peer's ZT address (short for identity().address())
 	 */
-	ZT_INLINE const Address &address() const noexcept { return _id.address(); }
+	ZT_INLINE Address address() const noexcept { return _id.address(); }
 
 	/**
 	 * @return This peer's identity

node/Tests.cpp

@@ -178,10 +178,7 @@ static const C25519TestVector C25519_TEST_VECTORS[ZT_NUM_C25519_TEST_VECTORS] =
 };
 
 #define IDENTITY_V0_KNOWN_GOOD_0 "8e4df28b72:0:ac3d46abe0c21f3cfe7a6c8d6a85cfcffcb82fbd55af6a4d6350657c68200843fa2e16f9418bbd9702cae365f2af5fb4c420908b803a681d4daef6114d78a2d7:bd8dd6e4ce7022d2f812797a80c6ee8ad180dc4ebf301dec8b06d1be08832bddd63a2f1cfa7b2c504474c75bdc8898ba476ef92e8e2d0509f8441985171ff16e"
-#define IDENTITY_V0_KNOWN_BAD_0 "9e4df28b72:0:ac3d46abe0c21f3cfe7a6c8d6a85cfcffcb82fbd55af6a4d6350657c68200843fa2e16f9418bbd9702cae365f2af5fb4c420908b803a681d4daef6114d78a2d7:bd8dd6e4ce7022d2f812797a80c6ee8ad180dc4ebf301dec8b06d1be08832bddd63a2f1cfa7b2c504474c75bdc8898ba476ef92e8e2d0509f8441985171ff16e"
-
-#define IDENTITY_V1_KNOWN_GOOD_0 "e050dcf19f:1:rmtdbcnpdjhnm6pufarv6yzwexbsbjtuehb4ywr43kwgmon4o4p3gahhd3oyydem2acv2smmnnxropmp5y2rk3op3ss5mmetfdvprhad7ryqonta6wb2jlknflbvjbkpusxtcijmg6tyio4bhbp5dpwirf24rhr4vn4iiue2dtdb5v7iq2i5uaaexjctdvm2pa:wfsu4saaikz3cs23uxixizme45nxyzgtbkht5dxyt7bxffraswf7dszrepg56ois36me3a34t4vzbv4zqpybwumvk2lwfrylk62qcmngowwyxa3e7mlxav7ubgjluxhjokgnnonv5cscjtybh2pk7uromfddsbw6vtwrzsd6yvf5fgsiargq"
-#define IDENTITY_V1_KNOWN_BAD_0 "f050dcf19f:1:rmtdbcnpdjhnm6pufarv6yzwexbsbjtuehb4ywr43kwgmon4o4p3gahhd3oyydem2acv2smmnnxropmp5y2rk3op3ss5mmetfdvprhad7ryqonta6wb2jlknflbvjbkpusxtcijmg6tyio4bhbp5dpwirf24rhr4vn4iiue2dtdb5v7iq2i5uaaexjctdvm2pa:wfsu4saaikz3cs23uxixizme45nxyzgtbkht5dxyt7bxffraswf7dszrepg56ois36me3a34t4vzbv4zqpybwumvk2lwfrylk62qcmngowwyxa3e7mlxav7ubgjluxhjokgnnonv5cscjtybh2pk7uromfddsbw6vtwrzsd6yvf5fgsiargq"
+#define IDENTITY_V1_KNOWN_GOOD_0 "cc6e483c2a:1:uojaacswtakzkpe234q67ti3hunqo5et75qwpej5gnhyofpjduncjogbrof2g72e7rhtp7xdypy2t43ojb2ewva33npnoazum3ifyladgllmc6xe2iwk7mljihrbycoitip647z4vx22vn2v2yimrgudolx4tbu3ip4f6gwt4rvjszupxk35gaalwjfvr3v5gy:apt2gnwj456cu4t5enchipsnx6nj22u6vxizzsu3azahlhqxfqymvatqvtjwtta2v32tbigt6pk7m3ndw766ynsn4hhmxnvkibhajqx5f3ugpiu5yx4xwcz6mbw2vrevxsyh3t346z37elpmgp3xdg4wyqwkyssczqdhfm3w4daq4rt75rvq"
 
 // --------------------------------------------------------------------------------------------------------------------
 
@@ -648,6 +645,8 @@ extern "C" const char *ZTT_general()
 		}
 
 		{
+			char tmp[2048];
+
 			ZT_T_PRINTF("[general] Testing Identity type 0 (C25519)... ");
 			Identity id;
 
@@ -686,7 +685,9 @@ extern "C" const char *ZTT_general()
 			}
 			ZT_T_PRINTF("(marshalled size: %d bytes) ",ms);
 
-			if (!id.fromString(IDENTITY_V0_KNOWN_BAD_0)) {
+			Utils::scopy(tmp,sizeof(tmp),IDENTITY_V0_KNOWN_GOOD_0);
+			tmp[0] = '0';
+			if (!id.fromString(tmp)) {
 				ZT_T_PRINTF("FAILED (error parsing test identity #2)" ZT_EOL_S);
 				return "Identity test failed: parse error";
 			}
@@ -694,13 +695,17 @@ extern "C" const char *ZTT_general()
 				ZT_T_PRINTF("FAILED (validation of known-bad identity returned ok)" ZT_EOL_S);
 				return "Identity test failed: validation of known-bad identity";
 			}
+			ZT_T_PRINTF("OK" ZT_EOL_S);
 
-			ZT_T_PRINTF("OK" ZT_EOL_S "[general] Testing Identity type 1 (P384)... ");
+			{
+				id.generate(Identity::P384);
+				id.toString(true,tmp);
+				ZT_T_PRINTF("[general] Example V1 identity: %s\n",tmp);
+				id.fingerprint().toString(tmp);
+				ZT_T_PRINTF("[general]   Fingerprint: %s" ZT_EOL_S,tmp);
+			}
 
-			//id.generate(Identity::P384);
-			//char tmp[1024];
-			//id.toString(true,tmp);
-			//ZT_T_PRINTF("\n%s\n",tmp);
+			ZT_T_PRINTF("[general] Testing Identity type 1 (P384)... ");
 
 			if (!id.fromString(IDENTITY_V1_KNOWN_GOOD_0)) {
 				ZT_T_PRINTF("FAILED (error parsing test identity #1)" ZT_EOL_S);
@@ -735,9 +740,11 @@ extern "C" const char *ZTT_general()
 			}
 			ZT_T_PRINTF("(marshalled size: %d bytes) ",ms);
 
-			if (!id.fromString(IDENTITY_V1_KNOWN_BAD_0)) {
-				ZT_T_PRINTF("FAILED (error parsing test identity #2)" ZT_EOL_S);
-				return "Identity test failed: parse error";
+			Utils::scopy(tmp,sizeof(tmp),IDENTITY_V1_KNOWN_GOOD_0);
+			tmp[0] = '0';
+			if (id.fromString(tmp)) {
+				ZT_T_PRINTF("FAILED (parse of known-bad identity returned ok)" ZT_EOL_S);
+				return "Identity test failed: parse of known-bad identity";
 			}
 			if (id.locallyValidate()) {
 				ZT_T_PRINTF("FAILED (validation of known-bad identity returned ok)" ZT_EOL_S);
@@ -778,7 +785,7 @@ extern "C" const char *ZTT_crypto()
 		{
 			ZT_T_PRINTF("[crypto] Testing MIMC52 VDF... ");
 			const uint64_t proof = mimc52Delay("",1,1000);
-			if ((!mimc52Verify("",1,1000,proof))||(proof != 0x00036030471c2aec)) {
+			if ((!mimc52Verify("",1,1000,proof))||(proof != 0x000cc1abe2dde7a3)) {
 				ZT_T_PRINTF("FAILED (%.16llx)" ZT_EOL_S,proof);
 				return "MIMC52 failed simple delay/verify test";
 			}