Selaa lähdekoodia

Net 246/cherrypick master (#2342)

* Release v0.20.0 (#2317)

* free tier limit exceeded: status code now 403

* reformat, TODOs

* - nm-certs for zerossl
- added config for email, domain
- updated linux deps

* return {} if no records found for acls/metrics

* Revert "return {} if no records found for acls/metrics"

pushed to wrong branch
This reverts commit 7602e97950a178b141a46bb872dd43e200951e08.

* return {} if no records found for acls/metrics

* add type to enrollement key

* add type to enrollement key

* update version

* - request and mount certs
- handle caddy challenge
- docker fixes
- pull nm-certs.sh

* Revert "add type to enrollement key"

This reverts commit 0cf342dd6e5dcbc1a3b3962350e6dcb7739380df.

* nm-certs.sh
- support EE and new domains
- minor fixes

* shfmt reformat

* add type to APIEnrollementKey

* if -- else to determine type

* spellcheck

* - support EE
- config namespaces
- write config after confirm
- minor fixes

* nm-certs.sh
- config fixes
- crontab symlink

* release workflows

* use forked repo

* Revert "use forked repo"

This reverts commit 730aca7ed88bc01ffbd4c2a66a2e429aafd82da6.

* - fixes
- user msgs

* review comments

* Bump github.com/txn2/txeh from 1.3.0 to 1.4.0

Bumps [github.com/txn2/txeh](https://github.com/txn2/txeh) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/txn2/txeh/releases)
- [Changelog](https://github.com/txn2/txeh/blob/master/goreleaser.yml)
- [Commits](https://github.com/txn2/txeh/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/txn2/txeh
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump alpine from 3.17.2 to 3.17.3

Bumps alpine from 3.17.2 to 3.17.3.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* - nm-certs.sh switched to dockerized certbot
- nm-quick.sh removed certbot from deps

* fixed missing domain

* - shallow clone for local installs
- added certs to other compose files

* missing domain, auto ToS

* fallback to letsencrypt

* removed turris OS

* fix typo

* send host update when deleting relay

* fixed shallow clone for branches

* disable cleanup for tests

* fixed local install

* - fixed cert mounting
- fixed caddy restart in nm-certs.sh
- aligned all configs

* fixed caddy start/stop

* - added NM_SKIP_BUILD
- fixed docker stop

* fixed NM_SKIP_BUILD

* - fixed ServerBrokerEndpoint config (#2283)

- mq credentials in compose

* NET-129: Turn Signal Actions (#2290)

* add signal action field

* add negotiation signal action

* typo fix

* change signal action name

* NET-147 full config for nm-quick.sh (#2291)

* - moved all vars to config
- compose override
- use the config in compose, caddy
- aligned local / remote setup
- proper docker cleanup
- support for a relative installation path

* - config handling
- error handling / env cleanups
- reduced compose files
- misc

* fixed debugs

* fixed UI_IMAGE_TAG / IMAGE_TAG

* Bump alpine from 3.17.3 to 3.18.0 (#2299)

Bumps alpine from 3.17.3 to 3.18.0.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#2298)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/crypto/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Extclient NET-63x (#2286)

* model changes

* additional fields for extclient create

* add DNS to extclient config

* extclient name checks

* update extclient

* nmctl extclient

* final tweaks

* review comments

* add extclientdns to node on ingress creation

* fix to add ingress dns to api (#2296)

---------

Co-authored-by: Aceix <[email protected]>

* versions (#2302)

* Bump golang.org/x/oauth2 from 0.7.0 to 0.8.0 (#2297)

Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Fixed nm-certs relative path (#2311)

* nm-certs permission (#2308)

* nm-certs permission

* single quotes error

* fixed relative path

---------

Co-authored-by: Christopher Blaha <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Matthew R Kasun <[email protected]>
Co-authored-by: Tobias Cudnik <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhishek K <[email protected]>
Co-authored-by: Aceix <[email protected]>
Co-authored-by: Christopher Blaha <[email protected]>

* NET-246: Use the default config as a fallback (#2332)

* - fall back to default values
- nm-quick version
- fixed TURN_PASSWORD

* apk update

* retain existing password

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Matthew R Kasun <[email protected]>
Co-authored-by: Tobias Cudnik <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhishek K <[email protected]>
Co-authored-by: Aceix <[email protected]>
Co-authored-by: Christopher Blaha <[email protected]>
Co-authored-by: Tobias Cudnik <[email protected]>
Alex Feiszli 2 vuotta sitten
vanhempi
commit
90ccb89342
3 muutettua tiedostoa jossa 52 lisäystä ja 35 poistoa
  1. 0 0
      scripts/netmaker.default.env
  2. 1 0
      scripts/nm-certs.sh
  3. 51 35
      scripts/nm-quick.sh

+ 0 - 0
scripts/netmaker.env → scripts/netmaker.default.env


+ 1 - 0
scripts/nm-certs.sh

@@ -47,6 +47,7 @@ EOF
 cat <<EOF >"$SCRIPT_DIR/certbot-entry.sh"
 #!/bin/sh
 # deps
+apk update
 apk add bash curl
 # zerossl
 wget -qO zerossl-bot.sh "https://github.com/zerossl/zerossl-bot/raw/master/zerossl-bot.sh"

+ 51 - 35
scripts/nm-quick.sh

@@ -4,6 +4,7 @@ CONFIG_FILE=netmaker.env
 # location of nm-quick.sh (usually `/root`)
 SCRIPT_DIR=$(dirname "$(realpath "$0")")
 CONFIG_PATH="$SCRIPT_DIR/$CONFIG_FILE"
+NM_QUICK_VERSION="0.1.0"
 LATEST=$(curl -s https://api.github.com/repos/gravitl/netmaker/releases/latest | grep "tag_name" | cut -d : -f 2,3 | tr -d [:space:],\")
 
 if [ $(id -u) -ne 0 ]; then
@@ -11,12 +12,6 @@ if [ $(id -u) -ne 0 ]; then
 	exit 1
 fi
 
-# read the config file
-if [ -f "$CONFIG_PATH" ]; then
-	echo "Reading config from $CONFIG_PATH"
-	source "$CONFIG_PATH"
-fi
-
 unset INSTALL_TYPE
 unset BUILD_TYPE
 unset BUILD_TAG
@@ -25,6 +20,7 @@ unset AUTO_BUILD
 
 # usage - displays usage instructions
 usage() {
+	echo "nm-quick.sh v$NM_QUICK_VERSION"
 	echo "usage: ./nm-quick.sh [-e] [-b buildtype] [-t tag] [-a auto]"
 	echo "  -e      if specified, will install netmaker EE"
 	echo "  -b      type of build; options:"
@@ -141,6 +137,7 @@ set_buildinfo() {
 	echo "  Build Type: $BUILD_TYPE"
 	echo "   Build Tag: $BUILD_TAG"
 	echo "   Image Tag: $IMAGE_TAG"
+	echo "   Installer: v$NM_QUICK_VERSION"
 	echo "-----------------------------------------------------"
 
 }
@@ -303,7 +300,7 @@ save_config() { (
 		save_config_item SERVER_IMAGE_TAG "$IMAGE_TAG"
 	fi
 	# copy entries from the previous config
-	local toCopy=("SERVER_HOST" "MASTER_KEY" "TURN_USERNAME" "MQ_USERNAME" "MQ_PASSWORD"
+	local toCopy=("SERVER_HOST" "MASTER_KEY" "TURN_USERNAME" "TURN_PASSWORD" "MQ_USERNAME" "MQ_PASSWORD"
 		"INSTALL_TYPE" "NODE_ID" "METRICS_EXPORTER" "PROMETHEUS" "DNS_MODE" "NETCLIENT_AUTO_UPDATE" "API_PORT"
 		"CORS_ALLOWED_ORIGIN" "DISPLAY_KEYS" "DATABASE" "SERVER_BROKER_ENDPOINT" "STUN_PORT" "VERBOSITY"
 		"DEFAULT_PROXY_MODE" "TURN_PORT" "USE_TURN" "DEBUG_MODE" "TURN_API_PORT" "REST_BACKEND" "DISABLE_REMOTE_IP_CHECK"
@@ -327,13 +324,20 @@ save_config() { (
 save_config_item() { (
 	local NAME="$1"
 	local VALUE="$2"
-	# echo "NAME $NAME"
-	# echo "VALUE $VALUE"
+	#echo "$NAME=$VALUE"
+	if test -z "$VALUE"; then
+		# load the default for empty values
+		VALUE=$(awk -F'=' "/^$NAME/ { print \$2}"  "$SCRIPT_DIR/netmaker.default.env")
+		# trim quotes for docker
+		VALUE=$(echo "$VALUE" | sed -E "s|^(['\"])(.*)\1$|\2|g")
+		#echo "Default for $NAME=$VALUE"
+	fi
+	# TODO single quote passwords
 	if grep -q "^$NAME=" "$CONFIG_PATH"; then
 		# TODO escape | in the value
 		sed -i "s|$NAME=.*|$NAME=$VALUE|" "$CONFIG_PATH"
 	else
-		echo "$NAME=\"$VALUE\"" >>"$CONFIG_PATH"
+		echo "$NAME=$VALUE" >>"$CONFIG_PATH"
 	fi
 ); }
 
@@ -362,8 +366,7 @@ local_install_setup() { (
 		cp docker/Caddyfile "$SCRIPT_DIR/Caddyfile"
 	fi
 	cp scripts/nm-certs.sh "$SCRIPT_DIR/nm-certs.sh"
-	cp scripts/netmaker.env "$SCRIPT_DIR/netmaker.env"
-	ln -fs "$SCRIPT_DIR/netmaker.env" "$SCRIPT_DIR/.env"
+	cp scripts/netmaker.default.env "$SCRIPT_DIR/netmaker.default.env"
 	cp docker/mosquitto.conf "$SCRIPT_DIR/mosquitto.conf"
 	cp docker/wait.sh "$SCRIPT_DIR/wait.sh"
 	cd ../../
@@ -383,31 +386,31 @@ install_dependencies() {
 
 	OS=$(uname)
 	if [ -f /etc/debian_version ]; then
-		dependencies="git wireguard wireguard-tools dnsutils jq docker.io docker-compose"
+		dependencies="git wireguard wireguard-tools dnsutils jq docker.io docker-compose grep gawk"
 		update_cmd='apt update'
 		install_cmd='apt-get install -y'
 	elif [ -f /etc/alpine-release ]; then
-		dependencies="git wireguard jq docker.io docker-compose"
+		dependencies="git wireguard jq docker.io docker-compose grep gawk"
 		update_cmd='apk update'
 		install_cmd='apk --update add'
 	elif [ -f /etc/centos-release ]; then
-		dependencies="git wireguard jq bind-utils docker.io docker-compose"
+		dependencies="git wireguard jq bind-utils docker.io docker-compose grep gawk"
 		update_cmd='yum update'
 		install_cmd='yum install -y'
 	elif [ -f /etc/fedora-release ]; then
-		dependencies="git wireguard bind-utils jq docker.io docker-compose"
+		dependencies="git wireguard bind-utils jq docker.io docker-compose grep gawk"
 		update_cmd='dnf update'
 		install_cmd='dnf install -y'
 	elif [ -f /etc/redhat-release ]; then
-		dependencies="git wireguard jq docker.io bind-utils docker-compose"
+		dependencies="git wireguard jq docker.io bind-utils docker-compose grep gawk"
 		update_cmd='yum update'
 		install_cmd='yum install -y'
 	elif [ -f /etc/arch-release ]; then
-		dependencies="git wireguard-tools dnsutils jq docker.io docker-compose"
+		dependencies="git wireguard-tools dnsutils jq docker.io docker-compose grep gawk"
 		update_cmd='pacman -Sy'
 		install_cmd='pacman -S --noconfirm'
 	elif [ "${OS}" = "FreeBSD" ]; then
-		dependencies="git wireguard wget jq docker.io docker-compose"
+		dependencies="git wireguard wget jq docker.io docker-compose grep gawk"
 		update_cmd='pkg update'
 		install_cmd='pkg install -y'
 	else
@@ -490,10 +493,12 @@ set_install_vars() {
 
 	NETMAKER_BASE_DOMAIN=nm.$(echo $IP_ADDR | tr . -).nip.io
 	SERVER_HOST=$IP_ADDR
-	MASTER_KEY=$(
-		tr -dc A-Za-z0-9 </dev/urandom | head -c 30
-		echo ''
-	)
+	if test -z "$MASTER_KEY"; then
+		MASTER_KEY=$(
+			tr -dc A-Za-z0-9 </dev/urandom | head -c 30
+			echo ''
+		)
+	fi
 	DOMAIN_TYPE=""
 	echo "-----------------------------------------------------"
 	echo "Would you like to use your own domain for netmaker, or an auto-generated domain?"
@@ -604,13 +609,15 @@ set_install_vars() {
 		MQ_USERNAME="$GET_MQ_USERNAME"
 	fi
 
-	MQ_PASSWORD=$(
-		tr -dc A-Za-z0-9 </dev/urandom | head -c 30
-		echo ''
-	)
+	if test -z "$MQ_PASSWORD"; then
+		MQ_PASSWORD=$(
+			tr -dc A-Za-z0-9 </dev/urandom | head -c 30
+			echo ''
+		)
+	fi
 
 	if [ -z $AUTO_BUILD ]; then
-		select domain_option in "Auto Generated Password" "Input Your Own Password"; do
+		select domain_option in "Auto Generated / Config Password" "Input Your Own Password"; do
 			case $REPLY in
 			1)
 				echo "using random password for mq"
@@ -651,13 +658,15 @@ set_install_vars() {
 		TURN_USERNAME="$GET_TURN_USERNAME"
 	fi
 
-	TURN_PASSWORD=$(
-		tr -dc A-Za-z0-9 </dev/urandom | head -c 30
-		echo ''
-	)
+	if test -z "$TURN_PASSWORD"; then
+		TURN_PASSWORD=$(
+			tr -dc A-Za-z0-9 </dev/urandom | head -c 30
+			echo ''
+		)
+	fi
 
 	if [ -z $AUTO_BUILD ]; then
-		select domain_option in "Auto Generated Password" "Input Your Own Password"; do
+		select domain_option in "Auto Generated / Config Password" "Input Your Own Password"; do
 			case $REPLY in
 			1)
 				echo "using random password for turn"
@@ -734,8 +743,7 @@ install_netmaker() {
 			wget -qO "$SCRIPT_DIR"/docker-compose.override.yml $COMPOSE_OVERRIDE_URL
 		fi
 		wget -qO "$SCRIPT_DIR"/Caddyfile "$CADDY_URL"
-		wget -qO "$SCRIPT_DIR"/netmaker.env "$BASE_URL/scripts/netmaker.env"
-		ln -fs "$SCRIPT_DIR/netmaker.env" "$SCRIPT_DIR/.env"
+		wget -qO "$SCRIPT_DIR"/netmaker.default.env "$BASE_URL/scripts/netmaker.default.env"
 		wget -qO "$SCRIPT_DIR"/mosquitto.conf "$BASE_URL/docker/mosquitto.conf"
 		wget -qO "$SCRIPT_DIR"/nm-certs.sh "$BASE_URL/scripts/nm-certs.sh"
 		wget -qO "$SCRIPT_DIR"/wait.sh "$BASE_URL/docker/wait.sh"
@@ -744,6 +752,8 @@ install_netmaker() {
 	chmod +x "$SCRIPT_DIR"/wait.sh
 	mkdir -p /etc/netmaker
 
+	# link .env to the user config
+	ln -fs "$SCRIPT_DIR/netmaker.env" "$SCRIPT_DIR/.env"
 	save_config
 
 	# Fetch / update certs using certbot
@@ -860,6 +870,12 @@ cleanup() {
 # 1. print netmaker logo
 print_logo
 
+# read the config
+if [ -f "$CONFIG_PATH" ]; then
+	echo "Using config: $CONFIG_PATH"
+	source "$CONFIG_PATH"
+fi
+
 # 2. setup the build instructions
 set_buildinfo