Pārlūkot izejas kodu

Release v0.20.0 (#2304)

* free tier limit exceeded: status code now 403

* reformat, TODOs

* - nm-certs for zerossl
- added config for email, domain
- updated linux deps

* return {} if no records found for acls/metrics

* Revert "return {} if no records found for acls/metrics"

pushed to wrong branch
This reverts commit 7602e97950a178b141a46bb872dd43e200951e08.

* return {} if no records found for acls/metrics

* add type to enrollement key

* add type to enrollement key

* update version

* - request and mount certs
- handle caddy challenge
- docker fixes
- pull nm-certs.sh

* Revert "add type to enrollement key"

This reverts commit 0cf342dd6e5dcbc1a3b3962350e6dcb7739380df.

* nm-certs.sh
- support EE and new domains
- minor fixes

* shfmt reformat

* add type to APIEnrollementKey

* if -- else to determine type

* spellcheck

* - support EE
- config namespaces
- write config after confirm
- minor fixes

* nm-certs.sh
- config fixes
- crontab symlink

* release workflows

* use forked repo

* Revert "use forked repo"

This reverts commit 730aca7ed88bc01ffbd4c2a66a2e429aafd82da6.

* - fixes
- user msgs

* review comments

* Bump github.com/txn2/txeh from 1.3.0 to 1.4.0

Bumps [github.com/txn2/txeh](https://github.com/txn2/txeh) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/txn2/txeh/releases)
- [Changelog](https://github.com/txn2/txeh/blob/master/goreleaser.yml)
- [Commits](https://github.com/txn2/txeh/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/txn2/txeh
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* Bump alpine from 3.17.2 to 3.17.3

Bumps alpine from 3.17.2 to 3.17.3.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

* - nm-certs.sh switched to dockerized certbot
- nm-quick.sh removed certbot from deps

* fixed missing domain

* - shallow clone for local installs
- added certs to other compose files

* missing domain, auto ToS

* fallback to letsencrypt

* removed turris OS

* fix typo

* send host update when deleting relay

* fixed shallow clone for branches

* disable cleanup for tests

* fixed local install

* - fixed cert mounting
- fixed caddy restart in nm-certs.sh
- aligned all configs

* fixed caddy start/stop

* - added NM_SKIP_BUILD
- fixed docker stop

* fixed NM_SKIP_BUILD

* - fixed ServerBrokerEndpoint config (#2283)

- mq credentials in compose

* NET-129: Turn Signal Actions (#2290)

* add signal action field

* add negotiation signal action

* typo fix

* change signal action name

* NET-147 full config for nm-quick.sh (#2291)

* - moved all vars to config
- compose override
- use the config in compose, caddy
- aligned local / remote setup
- proper docker cleanup
- support for a relative installation path

* - config handling
- error handling / env cleanups
- reduced compose files
- misc

* fixed debugs

* fixed UI_IMAGE_TAG / IMAGE_TAG

* Bump alpine from 3.17.3 to 3.18.0 (#2299)

Bumps alpine from 3.17.3 to 3.18.0.

---
updated-dependencies:
- dependency-name: alpine
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#2298)

Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/crypto/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Extclient NET-63x (#2286)

* model changes

* additional fields for extclient create

* add DNS to extclient config

* extclient name checks

* update extclient

* nmctl extclient

* final tweaks

* review comments

* add extclientdns to node on ingress creation

* fix to add ingress dns to api (#2296)

---------

Co-authored-by: Aceix <[email protected]>

* versions (#2302)

* Bump golang.org/x/oauth2 from 0.7.0 to 0.8.0 (#2297)

Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Matthew R Kasun <[email protected]>
Co-authored-by: Tobias Cudnik <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Abhishek K <[email protected]>
Co-authored-by: Aceix <[email protected]>
Alex Feiszli 2 gadi atpakaļ
vecāks
revīzija
4309b450c6
58 mainītis faili ar 1146 papildinājumiem un 1165 dzēšanām
  1. 1 0
      .github/ISSUE_TEMPLATE/bug-report.yml
  2. 2 2
      .github/workflows/deletedroplets.yml
  3. 0 46
      .github/workflows/packages.yml
  4. 19 0
      .github/workflows/pre-release.yml
  5. 0 32
      .github/workflows/pull-request.yml
  6. 0 50
      .github/workflows/release-assets.yml
  7. 0 39
      .github/workflows/release-branch.yml
  8. 4 40
      .github/workflows/release.yml
  9. 0 41
      .github/workflows/upgraderelease.yml
  10. 1 1
      Dockerfile
  11. 1 1
      Dockerfile-quick
  12. 1 1
      README.md
  13. 18 2
      cli/cmd/ext_client/create.go
  14. 8 27
      cli/cmd/ext_client/update.go
  15. 0 2
      cli/cmd/network/create.go
  16. 0 1
      cli/cmd/network/flags.go
  17. 3 9
      cli/functions/ext_client.go
  18. 4 72
      compose/docker-compose-emqx.yml
  19. 22 120
      compose/docker-compose.ee.yml
  20. 1 1
      compose/docker-compose.netclient.yml
  21. 0 121
      compose/docker-compose.reference.yml
  22. 52 53
      compose/docker-compose.yml
  23. 1 1
      controllers/docs.go
  24. 56 29
      controllers/ext_client.go
  25. 1 1
      controllers/limits.go
  26. 6 0
      controllers/network.go
  27. 4 7
      controllers/node.go
  28. 15 3
      controllers/regex.go
  29. 51 0
      controllers/regex_test.go
  30. 15 1
      controllers/relay.go
  31. 32 32
      docker/Caddyfile
  32. 41 38
      docker/Caddyfile-EE
  33. 7 0
      ee/ee_controllers/metrics.go
  34. 1 2
      functions/helpers_test.go
  35. 5 5
      go.mod
  36. 10 34
      go.sum
  37. 1 1
      k8s/client/netclient-daemonset.yaml
  38. 1 1
      k8s/client/netclient.yaml
  39. 1 1
      k8s/server/netmaker-server.yaml
  40. 1 1
      k8s/server/netmaker-ui.yaml
  41. 6 2
      logic/enrollmentkey.go
  42. 22 25
      logic/extpeers.go
  43. 3 2
      logic/gateway.go
  44. 3 1
      main.go
  45. 50 0
      migrate/migrate.go
  46. 3 0
      models/api_node.go
  47. 20 0
      models/enrollment_key.go
  48. 11 1
      models/extclient.go
  49. 16 5
      models/host.go
  50. 0 1
      models/network.go
  51. 1 0
      models/node.go
  52. 6 6
      models/structs.go
  53. 5 13
      release.md
  54. 82 0
      scripts/netmaker.env
  55. 99 0
      scripts/nm-certs.sh
  56. 430 289
      scripts/nm-quick.sh
  57. 2 1
      scripts/nm-upgrade-0-17-1-to-0-19-0.sh
  58. 1 1
      swagger.yaml

+ 1 - 0
.github/ISSUE_TEMPLATE/bug-report.yml

@@ -31,6 +31,7 @@ body:
       label: Version
       description: What version are you running?
       options:
+        - v0.20.0
         - v0.19.0
         - v0.18.7
         - v0.18.6

+ 2 - 2
.github/workflows/deletedroplets.yml

@@ -23,7 +23,7 @@ jobs:
           webhook_token: ${{ secrets.DISCORD_WEBHOOK_TOKEN }}
           color: "#42f545"
           username: "GitHub Bot"
-          message: "${{ github.respository }}: ${{ github.event.workflow_run.name }} was successful"
+          message: "${{ github.repository }}: ${{ github.event.workflow_run.name }} was successful"
           file: ./results/results.log
       - name: discord server message
         uses: appleboy/discord-action@master
@@ -62,7 +62,7 @@ jobs:
           webhook_token: ${{ secrets.DISCORD_WEBHOOK_TOKEN }}
           color: "#990000"
           username: "GitHub Bot"
-          message: "${{ github.respository }}: ${{ github.event.workflow_run.name }} failed"
+          message: "${{ github.repository }}: ${{ github.event.workflow_run.name }} failed"
           file: ./results/results.log
       - name: discord server message
         uses: appleboy/discord-action@master

+ 0 - 46
.github/workflows/packages.yml

@@ -1,46 +0,0 @@
-name: Build linux packages
-on:
-  workflow_call:
-    inputs:
-      version:
-        required: true
-        type: string
-  workflow_dispatch:
-    inputs:
-      version:
-        description: "netmaker version"
-        required: true
-  release:
-    types: [released]
-
-jobs:
-
-  packages:
-    runs-on: ubuntu-latest
-    steps:
-      - name: setup ssh
-        run: |
-          mkdir -p ~/.ssh/
-          echo "$SSH_KEY" > ~/.ssh/id_devops
-          chmod 600 ~/.ssh/id_devops
-          cat >>~/.ssh/config <<END
-          Host *.clustercat.com
-            User root
-            IdentityFile ~/.ssh/id_devops
-            StrictHostKeyChecking no
-          END
-        env:
-          SSH_KEY: ${{ secrets.TESTING_SSH_KEY }}
-      - name: set version
-        run: |
-          if [[ -n "${{ github.event.inputs.version }}" ]]; then
-            # remove everything but digits and . for package (deb, rpm, etc) versions
-            VERSION=$(echo ${{ github.event.inputs.version }} | tr -cd '[:digit:].')
-          else
-            VERSION=$(echo ${{ github.ref.name }} | tr -cd '[:digit:].')
-          fi
-            echo "VERSION=${VERSION}" >> $GITHUB_ENV
-            echo ${VERSION}
-      - name: apt/rpm
-        run: |
-          ssh fileserver.clustercat.com "cd packages/nmcli; export VERSION=${{ ENV.version }}; export REVISION=0; ./buildall.sh "

+ 19 - 0
.github/workflows/pre-release.yml

@@ -0,0 +1,19 @@
+# creates a release from develop
+# creates release branch, generates release assets, publishes docker image and copies release.md to release
+# linux packages are generated and a PR from release branch to master is created 
+name: Release
+
+on:
+  workflow_dispatch:
+    inputs:
+      version:
+        description: "new version number"
+        required: true
+
+jobs:
+
+  release:
+    uses: gravitl/devops/.github/workflows/netmakerPrerelease.yml@master
+    with:
+      version: ${{ inputs.version }}
+    secrets: inherit

+ 0 - 32
.github/workflows/pull-request.yml

@@ -1,32 +0,0 @@
-# creates a PR from release branch to master
-name: Create Release PR to master
-on:
-  workflow_call:
-    inputs:
-      version:
-        required: true
-        type: string
-  workflow_dispatch:
-    inputs:
-      version:
-        description: "netmaker version"
-        required: true
-  release:
-    types: [released]
-
-jobs:
-  pr-to-main:
-    runs-on: ubuntu-latest
-    steps:
-      - name: create pr
-        env:
-          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-        run: |
-          gh api --method POST \
-          -H 'Accept: application/vnd.github+json' -H 'X-GitHub-Api-Version: 2022-11-28'  \
-          /repos/${{ github.repository }}/pulls \
-          -f title='${{ github.event.inputs.version }}' \
-          -f head='release_${{ github.event.inputs.version }}' \
-          -f base="master"
-
-

+ 0 - 50
.github/workflows/release-assets.yml

@@ -1,50 +0,0 @@
-# generates release assets and uploads to release 
-name: Upload Release Assets
-on:
-  workflow_call:
-    inputs:
-      version:
-        required: true
-        type: string
-      prerelease:
-        required: true
-        type: boolean
-  workflow_dispatch:
-    inputs:
-      version:
-        description: "netmaker version"
-        required: true
-      prerelease:
-        required: true
-        type: boolean
-        description: "prerelease"
-jobs:
-  release-assets:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-        with:
-          ref: release_${{ github.event.inputs.version}}
-          fetch-depth: 0
-      - name: Get Tags
-        run: |
-          git fetch --force --tags
-      - name: Setup go
-        uses: actions/setup-go@v4
-        with:
-          go-version: 1.19
-      - name: GoReleaser (full release)
-        if: ${{ github.event.inputs.prerelease == 'false'}}
-        uses: goreleaser/goreleaser-action@v4
-        with:
-          args: release --clean --release-notes release.md
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-      - name: GoReleaser (prerelease)
-        if: ${{ github.event.inputs.prerelease == 'false'}}
-        uses: goreleaser/goreleaser-action@v4
-        with:
-          args: release --clean --release-notes release.md -f .goreleaser.prerelease.yaml
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

+ 0 - 39
.github/workflows/release-branch.yml

@@ -1,39 +0,0 @@
-# creates new branch (release_{{ version }} with tag {{ version }} from develop 
-# will fail if branch or tag already exists on github
-name: Create Release Branch
-on:
-  workflow_call:
-    inputs:
-      version:
-        required: true
-        type: string
-  workflow_dispatch:
-    inputs:
-      version:
-        description: "netmaker version"
-        required: true
-jobs:
-  release-branch:
-    runs-on: ubuntu-latest
-    steps:
-      - name: checkout develop
-        uses: actions/checkout@v3
-        with:
-          ref: develop
-      - name: setup go
-        uses: actions/setup-go@v4
-        with:
-          go-version: 1.19
-      - name: setup git
-        run: |
-          git config user.name "Github Actions"
-          git config user.emaail "[email protected]"
-      - name: create release Branch
-        run: |
-          git switch -c release_${{ github.event.inputs.version }}
-          git tag -f ${{ github.event.inputs.version }}
-          #push branch
-          git push origin release_${{ github.event.inputs.version }}
-          #push tag
-          git push origin ${{ github.event.inputs.version }}
-  

+ 4 - 40
.github/workflows/release.yml

@@ -1,6 +1,6 @@
 # creates a release from develop
 # creates release branch, generates release assets, publishes docker image and copies release.md to release
-# if formal release, linux packages are generated and a PR from release branch to master is created 
+# linux packages are generated and a PR from release branch to master is created 
 name: Release
 
 on:
@@ -9,47 +9,11 @@ on:
       version:
         description: "new version number"
         required: true
-      prerelease:
-        required: true
-        type: boolean
-        description: Is this a pre-release
 
 jobs:
 
-  release-branch:
-    uses: ./.github/workflows/release-branch.yml
-    with:
-      version: ${{ github.event.inputs.version }}
-    secrets: inherit
-
-  
-  release-assets:
-    needs: release-branch
-    uses: ./.github/workflows/release-assets.yml
-    with:
-      version: ${{ github.event.inputs.version }}
-      prerelease: ${{ github.event.inputs.prerelease == 'true' }}
-    secrets: inherit
-
-  docker:
-    needs: release-branch
-    uses: ./.github/workflows/publish-docker.yml
-    with:
-      tag: ${{ github.event.inputs.version }}
-    secrets: inherit
-
-  packages:
-    if: ${{ github.event.inputs.prerelease == 'false' }}
-    needs: release-branch
-    uses: ./.github/workflows/packages.yml
-    with:
-      version: ${{ github.event.inputs.version }}
-    secrets: inherit
-
-  pull-request:
-    if: ${{ github.event.inputs.prerelease == 'false' }}
-    needs: release-branch
-    uses: ./.github/workflows/pull-request.yml
+  release:
+    uses: gravitl/devops/.github/workflows/netmakerRelease.yml@master
     with:
-      version: ${{ github.event.inputs.version }}
+      version: ${{ inputs.version }}
     secrets: inherit

+ 0 - 41
.github/workflows/upgraderelease.yml

@@ -1,41 +0,0 @@
-name: UpgradeRelease
-
-on:
-  workflow_dispatch:
-    inputs:
-      version:
-        description: "release/version to update"
-        required: true
-
-jobs:
-
-  update:
-    runs-on: ubuntu-latest
-    steps:
-      - name: checkout
-        uses: actions/checkout@v3
-        with:
-          ref: release_${{ github.event.inputs.version }}
-          fetch-depth: 0
-      - run: |
-          git fetch --force --tags
-      - name: Setup go
-        uses: actions/setup-go@v4
-        with:
-          go-version: 1.19
-      - name: goreleaser
-        uses: goreleaser/goreleaser-action@v4
-        with:
-          args: release --clean --release-notes release.md -f .goreleaser.update.yaml
-        env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN  }}
-
-  packages:
-    uses: ./.github/workflows/packages.yml
-    with:
-      version: ${{ github.event.inputs.version }}
-
-  pull-request:
-    uses: ./.github/workflows/pull-request.yml
-    with:
-      version: ${{ github.event.inputs.version }}

+ 1 - 1
Dockerfile

@@ -6,7 +6,7 @@ COPY . .
 
 RUN GOOS=linux CGO_ENABLED=1 go build -ldflags="-s -w " -tags ${tags} .
 # RUN go build -tags=ee . -o netmaker main.go
-FROM alpine:3.17.2
+FROM alpine:3.18.0
 
 # add a c lib
 # set the working directory

+ 1 - 1
Dockerfile-quick

@@ -1,5 +1,5 @@
 #first stage - builder
-FROM alpine:3.17.2
+FROM alpine:3.18.0
 ARG version 
 WORKDIR /app
 COPY ./netmaker /root/netmaker

+ 1 - 1
README.md

@@ -17,7 +17,7 @@
 
 <p align="center">
   <a href="https://github.com/gravitl/netmaker/releases">
-    <img src="https://img.shields.io/badge/Version-0.19.0-informational?style=flat-square" />
+    <img src="https://img.shields.io/badge/Version-0.20.0-informational?style=flat-square" />
   </a>
   <a href="https://hub.docker.com/r/gravitl/netmaker/tags">
     <img src="https://img.shields.io/docker/pulls/gravitl/netmaker?label=downloads" />

+ 18 - 2
cli/cmd/ext_client/create.go

@@ -4,10 +4,16 @@ import (
 	"fmt"
 
 	"github.com/gravitl/netmaker/cli/functions"
+	"github.com/gravitl/netmaker/models"
 	"github.com/spf13/cobra"
 )
 
-var extClientID string
+var (
+	extClientID string
+	publicKey   string
+	dns         string
+	allowedips  []string
+)
 
 var extClientCreateCmd = &cobra.Command{
 	Use:   "create [NETWORK NAME] [NODE ID]",
@@ -15,12 +21,22 @@ var extClientCreateCmd = &cobra.Command{
 	Short: "Create an External Client",
 	Long:  `Create an External Client`,
 	Run: func(cmd *cobra.Command, args []string) {
-		functions.CreateExtClient(args[0], args[1], extClientID)
+		extClient := models.CustomExtClient{
+			ClientID:        extClientID,
+			PublicKey:       publicKey,
+			DNS:             dns,
+			ExtraAllowedIPs: allowedips,
+		}
+
+		functions.CreateExtClient(args[0], args[1], extClient)
 		fmt.Println("Success")
 	},
 }
 
 func init() {
 	extClientCreateCmd.Flags().StringVar(&extClientID, "id", "", "ID of the external client")
+	extClientCreateCmd.Flags().StringVar(&publicKey, "public_key", "", "updated public key of the external client")
+	extClientCreateCmd.Flags().StringVar(&dns, "dns", "", "updated DNS of the external client")
+	extClientCreateCmd.Flags().StringSliceVar(&allowedips, "allowedips", []string{}, "updated extra allowed IPs of the external client")
 	rootCmd.AddCommand(extClientCreateCmd)
 }

+ 8 - 27
cli/cmd/ext_client/update.go

@@ -11,15 +11,7 @@ import (
 )
 
 var (
-	extClientUpdateFile    string
-	description            string
-	privateKey             string
-	publicKey              string
-	address                string
-	address6               string
-	ingressGatewayID       string
-	ingressGatewayEndpoint string
-	ownerID                string
+	extClientUpdateFile string
 )
 
 var extClientUpdateCmd = &cobra.Command{
@@ -31,7 +23,7 @@ var extClientUpdateCmd = &cobra.Command{
 		var (
 			network   = args[0]
 			clientID  = args[1]
-			extClient = &models.ExtClient{}
+			extClient = &models.CustomExtClient{}
 		)
 		if extClientUpdateFile != "" {
 			content, err := os.ReadFile(extClientUpdateFile)
@@ -42,30 +34,19 @@ var extClientUpdateCmd = &cobra.Command{
 				log.Fatal(err)
 			}
 		} else {
-			extClient.ClientID = clientID
-			extClient.Description = description
-			extClient.PrivateKey = privateKey
+			extClient.ClientID = extClientID
 			extClient.PublicKey = publicKey
-			extClient.Network = network
-			extClient.Address = address
-			extClient.Address6 = address6
-			extClient.IngressGatewayID = ingressGatewayID
-			extClient.IngressGatewayEndpoint = ingressGatewayEndpoint
-			extClient.OwnerID = ownerID
+			extClient.DNS = dns
 		}
 		functions.PrettyPrint(functions.UpdateExtClient(network, clientID, extClient))
 	},
 }
 
 func init() {
+	extClientUpdateCmd.Flags().StringVar(&extClientID, "id", "", "updated ID of the external client")
 	extClientUpdateCmd.Flags().StringVar(&extClientUpdateFile, "file", "", "Filepath of updated external client definition in JSON")
-	extClientUpdateCmd.Flags().StringVar(&description, "desc", "", "Description of the external client")
-	extClientUpdateCmd.Flags().StringVar(&privateKey, "private_key", "", "Filepath of updated external client definition in JSON")
-	extClientUpdateCmd.Flags().StringVar(&publicKey, "public_key", "", "Filepath of updated external client definition in JSON")
-	extClientUpdateCmd.Flags().StringVar(&address, "ipv4_addr", "", "IPv4 address of the external client")
-	extClientUpdateCmd.Flags().StringVar(&address6, "ipv6_addr", "", "IPv6 address of the external client")
-	extClientUpdateCmd.Flags().StringVar(&ingressGatewayID, "ingress_gateway_id", "", "ID of the ingress gateway")
-	extClientUpdateCmd.Flags().StringVar(&ingressGatewayEndpoint, "ingress_gateway_endpoint", "", "Endpoint of the ingress gateway")
-	extClientUpdateCmd.Flags().StringVar(&ownerID, "owner_id", "", "External Client owner's ID")
+	extClientUpdateCmd.Flags().StringVar(&publicKey, "public_key", "", "updated public key of the external client")
+	extClientUpdateCmd.Flags().StringVar(&dns, "dns", "", "updated DNS of the external client")
+	extClientUpdateCmd.Flags().StringSliceVar(&allowedips, "allowedips", []string{}, "updated extra allowed IPs of the external client")
 	rootCmd.AddCommand(extClientUpdateCmd)
 }

+ 0 - 2
cli/cmd/network/create.go

@@ -45,7 +45,6 @@ var networkCreateCmd = &cobra.Command{
 			if allowManualSignUp {
 				network.AllowManualSignUp = "yes"
 			}
-			network.DefaultExtClientDNS = defaultExtClientDNS
 			network.DefaultMTU = int32(defaultMTU)
 		}
 		functions.PrettyPrint(functions.CreateNetwork(network))
@@ -61,7 +60,6 @@ func init() {
 	networkCreateCmd.Flags().BoolVar(&udpHolePunch, "udp_hole_punch", false, "Enable UDP Hole Punching ?")
 	networkCreateCmd.Flags().BoolVar(&defaultACL, "default_acl", false, "Enable default Access Control List ?")
 	networkCreateCmd.Flags().StringVar(&defaultInterface, "interface", "", "Name of the network interface")
-	networkCreateCmd.Flags().StringVar(&defaultExtClientDNS, "ext_client_dns", "", "IPv4 address of DNS server to be used by external clients")
 	networkCreateCmd.Flags().IntVar(&defaultListenPort, "listen_port", 51821, "Default wireguard port each node will attempt to use")
 	networkCreateCmd.Flags().IntVar(&nodeLimit, "node_limit", 999999999, "Maximum number of nodes that can be associated with this network")
 	networkCreateCmd.Flags().IntVar(&defaultKeepalive, "keep_alive", 20, "Keep Alive in seconds")

+ 0 - 1
cli/cmd/network/flags.go

@@ -12,6 +12,5 @@ var (
 	nodeLimit                 int
 	defaultKeepalive          int
 	allowManualSignUp         bool
-	defaultExtClientDNS       string
 	defaultMTU                int
 )

+ 3 - 9
cli/functions/ext_client.go

@@ -28,14 +28,8 @@ func GetExtClientConfig(networkName, clientID string) string {
 }
 
 // CreateExtClient - create an external client
-func CreateExtClient(networkName, nodeID, extClientID string) {
-	if extClientID != "" {
-		request[any](http.MethodPost, fmt.Sprintf("/api/extclients/%s/%s", networkName, nodeID), &models.CustomExtClient{
-			ClientID: extClientID,
-		})
-	} else {
-		request[any](http.MethodPost, fmt.Sprintf("/api/extclients/%s/%s", networkName, nodeID), nil)
-	}
+func CreateExtClient(networkName, nodeID string, extClient models.CustomExtClient) {
+	request[any](http.MethodPost, fmt.Sprintf("/api/extclients/%s/%s", networkName, nodeID), extClient)
 }
 
 // DeleteExtClient - delete an external client
@@ -44,6 +38,6 @@ func DeleteExtClient(networkName, clientID string) *models.SuccessResponse {
 }
 
 // UpdateExtClient - update an external client
-func UpdateExtClient(networkName, clientID string, payload *models.ExtClient) *models.ExtClient {
+func UpdateExtClient(networkName, clientID string, payload *models.CustomExtClient) *models.ExtClient {
 	return request[models.ExtClient](http.MethodPut, fmt.Sprintf("/api/extclients/%s/%s", networkName, clientID), payload)
 }

+ 4 - 72
compose/docker-compose-emqx.yml

@@ -1,85 +1,17 @@
 version: "3.4"
 
 services:
-  netmaker:
-    container_name: netmaker
-    image: gravitl/netmaker:v0.19.0
-    restart: on-failure
-    volumes:
-      - dnsconfig:/root/config/dnsconfig
-      - sqldata:/root/data
-    environment:
-      BROKER_ENDPOINT: "wss://broker.NETMAKER_BASE_DOMAIN/mqtt"
-      BROKER_TYPE: "emqx"
-      EMQX_REST_ENDPOINT: "http://mq:18083"
-      SERVER_NAME: "NETMAKER_BASE_DOMAIN"
-      STUN_LIST: "stun.NETMAKER_BASE_DOMAIN:3478,stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302"
-      SERVER_HOST: "SERVER_PUBLIC_IP"
-      SERVER_API_CONN_STRING: "api.NETMAKER_BASE_DOMAIN:443"
-      COREDNS_ADDR: "SERVER_PUBLIC_IP"
-      DNS_MODE: "on"
-      SERVER_HTTP_HOST: "api.NETMAKER_BASE_DOMAIN"
-      NETCLIENT_AUTO_UPDATE: "enabled"
-      API_PORT: "8081"
-      MASTER_KEY: "REPLACE_MASTER_KEY"
-      CORS_ALLOWED_ORIGIN: "*"
-      DISPLAY_KEYS: "on"
-      DATABASE: "sqlite"
-      NODE_ID: "netmaker-server-1"
-      SERVER_BROKER_ENDPOINT: "ws://mq:8083/mqtt"
-      STUN_PORT: "3478"      
-      VERBOSITY: "1"
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD"
-      MQ_USERNAME: "REPLACE_MQ_USERNAME"
-      DEFAULT_PROXY_MODE: "off"
-    ports:
-      - "3478:3478/udp"
-  netmaker-ui:
-    container_name: netmaker-ui
-    image: gravitl/netmaker-ui:v0.19.0
-    depends_on:
-      - netmaker
-    links:
-      - "netmaker:api"
-    restart: always
-    environment:
-      BACKEND_URL: "https://api.NETMAKER_BASE_DOMAIN"
-  caddy:
-    image: caddy:2.6.2
-    container_name: caddy
-    restart: unless-stopped
-    volumes:
-      - /root/Caddyfile:/etc/caddy/Caddyfile
-      - caddy_data:/data
-      - caddy_conf:/config
-    ports:
-      - "80:80"
-      - "443:443"
-  coredns:
-    container_name: coredns
-    image: coredns/coredns
-    command: -conf /root/dnsconfig/Corefile
-    depends_on:
-      - netmaker
-    restart: always
-    volumes:
-      - dnsconfig:/root/dnsconfig
   mq:
     container_name: mq
     image: emqx/emqx:5.0.9
     restart: unless-stopped
     environment:
-      EMQX_NAME: "emqx"
-      EMQX_DASHBOARD__DEFAULT_PASSWORD: "REPLACE_MQ_PASSWORD"
-      EMQX_DASHBOARD__DEFAULT_USERNAME: "REPLACE_MQ_USERNAME"
+      - EMQX_NAME: "emqx"
+      - EMQX_DASHBOARD__DEFAULT_PASSWORD=${MQ_PASSWORD}
+      - EMQX_DASHBOARD__DEFAULT_USERNAME=${MQ_USERNAME}
     ports:
       - "1883:1883" # MQTT
       - "8883:8883" # SSL MQTT
       - "8083:8083" # Websockets
       - "18083:18083" # Dashboard/REST_API
-volumes:
-  caddy_data: {}
-  caddy_conf: {}
-  sqldata: {}
-  dnsconfig: {}
-  mosquitto_logs: {}
+

+ 22 - 120
compose/docker-compose.ee.yml

@@ -1,108 +1,29 @@
 version: "3.4"
 
 services:
-  netmaker:
-    container_name: netmaker
-    image: gravitl/netmaker:REPLACE_SERVER_IMAGE_TAG
-    restart: on-failure
-    volumes:
-      - dnsconfig:/root/config/dnsconfig
-      - sqldata:/root/data
-    environment:
-      BROKER_ENDPOINT: "wss://broker.NETMAKER_BASE_DOMAIN"
-      SERVER_NAME: "NETMAKER_BASE_DOMAIN"
-      STUN_LIST: "stun.NETMAKER_BASE_DOMAIN:3478,stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302"
-      SERVER_HOST: "SERVER_PUBLIC_IP"
-      SERVER_API_CONN_STRING: "api.NETMAKER_BASE_DOMAIN:443"
-      COREDNS_ADDR: "SERVER_PUBLIC_IP"
-      DNS_MODE: "on"
-      SERVER_HTTP_HOST: "api.NETMAKER_BASE_DOMAIN"
-      NETCLIENT_AUTO_UPDATE: "enabled"
-      API_PORT: "8081"
-      MASTER_KEY: "REPLACE_MASTER_KEY"
-      CORS_ALLOWED_ORIGIN: "*"
-      DISPLAY_KEYS: "on"
-      DATABASE: "sqlite"
-      NODE_ID: "netmaker-server-1"
-      SERVER_BROKER_ENDPOINT: "ws://mq:1883"
-      MQ_USERNAME: "REPLACE_MQ_USERNAME"
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD"
-      STUN_PORT: "3478"
-      VERBOSITY: "1"
-      METRICS_EXPORTER: "on"
-      LICENSE_KEY: "YOUR_LICENSE_KEY"
-      NETMAKER_ACCOUNT_ID: "YOUR_ACCOUNT_ID"
-      DEFAULT_PROXY_MODE: "off"
-      TURN_SERVER_HOST: "turn.NETMAKER_BASE_DOMAIN"
-      TURN_SERVER_API_HOST: "https://turnapi.NETMAKER_BASE_DOMAIN"
-      TURN_PORT: "3479"
-      TURN_USERNAME: "REPLACE_TURN_USERNAME"
-      TURN_PASSWORD: "REPLACE_TURN_PASSWORD"
-      USE_TURN: "true"
-    ports:
-      - "3478:3478/udp"
-  netmaker-ui:
-    container_name: netmaker-ui
-    image: gravitl/netmaker-ui:REPLACE_UI_IMAGE_TAG
-    depends_on:
-      - netmaker
-    links:
-      - "netmaker:api"
-    restart: always
-    environment:
-      BACKEND_URL: "https://api.NETMAKER_BASE_DOMAIN"
-  caddy:
-    image: caddy:2.6.2
-    container_name: caddy
-    restart: unless-stopped
-    volumes:
-      - /root/Caddyfile:/etc/caddy/Caddyfile
-      - caddy_data:/data
-      - caddy_conf:/config
-    ports:
-      - "80:80"
-      - "443:443"
-  coredns:
-    container_name: coredns
-    image: coredns/coredns
-    command: -conf /root/dnsconfig/Corefile
-    depends_on:
-      - netmaker
-    restart: always
-    volumes:
-      - dnsconfig:/root/dnsconfig
-  mq:
-    container_name: mq
-    image: eclipse-mosquitto:2.0.15-openssl
-    depends_on:
-      - netmaker
-    restart: unless-stopped
-    command: ["/mosquitto/config/wait.sh"]
-    environment:
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD"
-      MQ_USERNAME: "REPLACE_MQ_USERNAME"
-    volumes:
-      - /root/mosquitto.conf:/mosquitto/config/mosquitto.conf
-      - /root/wait.sh:/mosquitto/config/wait.sh
-      - mosquitto_logs:/mosquitto/log
+
   prometheus:
     container_name: prometheus
     image: gravitl/netmaker-prometheus:latest
+    env_file: ./netmaker.env
     environment:
-      NETMAKER_METRICS_TARGET: "netmaker-exporter.NETMAKER_BASE_DOMAIN"
-      LICENSE_KEY: "YOUR_LICENSE_KEY"
+      # config-dependant vars
+      - NETMAKER_METRICS_TARGET=netmaker-exporter.${NM_DOMAIN}
     restart: always
     volumes:
       - prometheus_data:/prometheus
     depends_on:
       - netmaker
+
   grafana:
     container_name: grafana
     image: gravitl/netmaker-grafana:latest
+    env_file: ./netmaker.env
     environment:
-      PROMETHEUS_HOST: "prometheus.NETMAKER_BASE_DOMAIN"
-      NETMAKER_METRICS_TARGET: "netmaker-exporter.NETMAKER_BASE_DOMAIN"
-      LICENSE_KEY: "YOUR_LICENSE_KEY"
+      # config-dependant vars
+      # TODO unify with netmaker-exporter
+      - PROMETHEUS_HOST=prometheus.${NM_DOMAIN}
+      - NETMAKER_METRICS_TARGET=netmaker-exporter.${NM_DOMAIN}
     volumes:
       - grafana_data:/var/lib/grafana
     restart: always
@@ -111,41 +32,22 @@ services:
     depends_on:
       - prometheus
       - netmaker
+
   netmaker-exporter:
     container_name: netmaker-exporter
     image: gravitl/netmaker-exporter:latest
+    env_file: ./netmaker.env
+    environment:
+      # config-dependant vars
+      # TODO unify with grafana
+      - PROMETHEUS_HOST=https://prometheus.${NM_DOMAIN}
+      # The domain/host IP indicating the mq broker address
+      - BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}
+      - API_PORT=${EXPORTER_API_PORT}
     restart: always
     depends_on:
       - netmaker
-    environment:
-      SERVER_BROKER_ENDPOINT: "ws://mq:1883"
-      BROKER_ENDPOINT: "wss://broker.NETMAKER_BASE_DOMAIN"
-      PROMETHEUS: "on"
-      VERBOSITY: "1"
-      API_PORT: "8085"
-      LICENSE_KEY: "YOUR_LICENSE_KEY"
-      PROMETHEUS_HOST: https://prometheus.NETMAKER_BASE_DOMAIN
-  turn:
-    container_name: turn
-    image: gravitl/turnserver:v1.0.0
-    network_mode: "host"
-    volumes:
-      - turn_server:/etc/config
-    environment:
-      DEBUG_MODE: "off"
-      VERBOSITY: "1"
-      TURN_PORT: "3479"
-      TURN_API_PORT: "8089"
-      CORS_ALLOWED_ORIGIN: "*"
-      TURN_SERVER_HOST: "turn.NETMAKER_BASE_DOMAIN"
-      USERNAME: "REPLACE_TURN_USERNAME"
-      PASSWORD: "REPLACE_TURN_PASSWORD"
+
 volumes:
-  caddy_data: {}
-  caddy_conf: {}
-  sqldata: {}
-  dnsconfig: {}
-  mosquitto_logs: {}
-  prometheus_data: {}
-  grafana_data: {}
-  turn_server: {}
+  prometheus_data: { }
+  grafana_data: { }

+ 1 - 1
compose/docker-compose.netclient.yml

@@ -3,7 +3,7 @@ version: "3.4"
 services:
   netclient:
     container_name: netclient
-    image: 'gravitl/netclient:v0.19.0'
+    image: 'gravitl/netclient:v0.20.0'
     hostname: netmaker-1
     network_mode: host
     restart: on-failure

+ 0 - 121
compose/docker-compose.reference.yml

@@ -1,121 +0,0 @@
-version: "3.4"
-
-services:
-  netmaker: # The Primary Server for running Netmaker
-    container_name: netmaker
-    image: gravitl/netmaker:REPLACE_SERVER_IMAGE_TAG
-    restart: on-failure
-    volumes: # Volume mounts necessary for sql, coredns, and mqtt
-      - dnsconfig:/root/config/dnsconfig
-      - sqldata:/root/data
-      - shared_certs:/etc/netmaker
-    environment: # Necessary capabilities to set iptables when running in container
-      BROKER_ENDPOINT: "wss://broker.NETMAKER_BASE_DOMAIN" # The domain/host IP indicating the mq broker address
-      SERVER_NAME: "NETMAKER_BASE_DOMAIN" # The base domain of netmaker
-      SERVER_HOST: "SERVER_PUBLIC_IP" # Set to public IP of machine.
-      SERVER_HTTP_HOST: "api.NETMAKER_BASE_DOMAIN" # Overrides SERVER_HOST if set. Useful for making HTTP available via different interfaces/networks.
-      NETCLIENT_AUTO_UPDATE: "enabled" # Enable auto update of netclient ? ENUM:- enabled,disabled | default: enabled
-      SERVER_API_CONN_STRING: "api.NETMAKER_BASE_DOMAIN:443"
-      COREDNS_ADDR: "SERVER_PUBLIC_IP" # Address of the CoreDNS server. Defaults to SERVER_HOST
-      DNS_MODE: "on" # Enables DNS Mode, meaning all nodes will set hosts file for private dns settings.
-      API_PORT: "8081" # The HTTP API port for Netmaker. Used for API calls / communication from front end. If changed, need to change port of BACKEND_URL for netmaker-ui.
-      REST_BACKEND: "on" # Enables the REST backend (API running on API_PORT at SERVER_HTTP_HOST). Change to "off" to turn off.
-      DISABLE_REMOTE_IP_CHECK: "off" # If turned "on", Server will not set Host based on remote IP check. This is already overridden if SERVER_HOST is set. Turned "off" by default.
-      TELEMETRY: "on" # Whether or not to send telemetry data to help improve Netmaker. Switch to "off" to opt out of sending telemetry.
-      MASTER_KEY: "REPLACE_MASTER_KEY" # The admin master key for accessing the API. Change this in any production installation.
-      CORS_ALLOWED_ORIGIN: "*" # The "allowed origin" for API requests. Change to restrict where API requests can come from with comma-separated URLs. ex:- https://dashboard.netmaker.domain1.com,https://dashboard.netmaker.domain2.com
-      DISPLAY_KEYS: "on" # Show keys permanently in UI (until deleted) as opposed to 1-time display.
-      DATABASE: "sqlite" # Database to use - sqlite, postgres, or rqlite
-      NODE_ID: "netmaker-server-1" # used for HA - identifies this server vs other servers
-      SERVER_BROKER_ENDPOINT: ""ws://mq:1883""  # the address of the mq server. If running from docker compose it will be "mq". Otherwise, need to input address. If using "host networking", it will find and detect the IP of the mq container.
-      MQ_USERNAME: "REPLACE_MQ_USERNAME" # the username to set for MQ access
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD" # the password to set for MQ access
-      STUN_PORT: "3478" # the reachable port of STUN on the server
-      VERBOSITY: "1" # logging verbosity level - 1, 2, or 3
-      # this section is for OAuth
-      AUTH_PROVIDER: "" # "<azure-ad|github|google|oidc>"
-      CLIENT_ID: "" # "<client id of your oauth provider>"
-      CLIENT_SECRET: "" # "<client secret of your oauth provider>"
-      FRONTEND_URL: "" # "https://dashboard.<netmaker base domain>"
-      AZURE_TENANT: "" # "<only for azure, you may optionally specify the tenant for the OAuth>"
-      OIDC_ISSUER: "" # https://oidc.yourprovider.com - URL of oidc provider
-      DEFAULT_PROXY_MODE: "off" # if ON, all new clients will enable proxy by default if OFF, all new clients will disable proxy by default, if AUTO, stick with the existing logic for NAT detection
-      TURN_SERVER_HOST: "turn.NETMAKER_BASE_DOMAIN" # domain for your turn server
-      TURN_SERVER_API_HOST: "https://turnapi.NETMAKER_BASE_DOMAIN" # domain of the turn api server
-      TURN_PORT: "3479" #  port to access turn server
-      TURN_USERNAME: "REPLACE_TURN_USERNAME"  # the username to set for turn api access
-      TURN_PASSWORD: "REPLACE_TURN_PASSWORD" #  the password to set for turn api access
-      USE_TURN: "true" #config for using turn, accepts either true/false
-    ports:
-      - "3478:3478/udp" # the stun port
-  netmaker-ui:  # The Netmaker UI Component
-    container_name: netmaker-ui
-    image: gravitl/netmaker-ui:REPLACE_UI_IMAGE_TAG
-    depends_on:
-      - netmaker
-    links:
-      - "netmaker:api"
-    restart: always
-    environment:
-      BACKEND_URL: "https://api.NETMAKER_BASE_DOMAIN" # URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT
-  caddy: # The reverse proxy that manages traffic for Netmaker
-    image: caddy:2.6.2
-    container_name: caddy
-    restart: unless-stopped
-    volumes:
-      - /root/Caddyfile:/etc/caddy/Caddyfile # Config file for Caddy
-      - caddy_data:/data
-      - caddy_conf:/config
-    ports:
-      - "80:80"
-      - "443:443"
-  coredns: # The DNS Server. CoreDNS can be removed unless doing special advanced use cases
-    container_name: coredns
-    image: coredns/coredns
-    command: -conf /root/dnsconfig/Corefile
-    depends_on:
-      - netmaker
-    restart: always
-    volumes:
-      - dnsconfig:/root/dnsconfig
-  mq: # the mqtt broker for netmaker
-    container_name: mq
-    image: eclipse-mosquitto:2.0.15-openssl
-    depends_on:
-      - netmaker
-    restart: unless-stopped
-    command: ["/mosquitto/config/wait.sh"]
-    environment:
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD" # must be same value as in netmaker env 
-      MQ_USERNAME: "REPLACE_MQ_USERNAME" # must be same value as in netmaker env
-    volumes:
-      - /root/mosquitto.conf:/mosquitto/config/mosquitto.conf
-      - /root/wait.sh:/mosquitto/config/wait.sh
-      - mosquitto_logs:/mosquitto/log
-    ports:
-      - "1883:1883"
-      - "8883:8883"
-  turn:
-    container_name: turn
-    image: gravitl/turnserver:v1.0.0
-    network_mode: "host"
-    volumes:
-      - turn_server:/etc/config
-    environment:
-      DEBUG_MODE: "off"
-      VERBOSITY: "1"
-      TURN_PORT: "3479"
-      TURN_API_PORT: "8089"
-      CORS_ALLOWED_ORIGIN: "*"
-      TURN_SERVER_HOST: "turn.NETMAKER_BASE_DOMAIN"
-      USERNAME: "REPLACE_TURN_USERNAME"
-      PASSWORD: "REPLACE_TURN_PASSWORD"
-      USE_TURN: "true"
-volumes:
-  caddy_data: {} # runtime data for caddy
-  caddy_conf: {} # configuration file for Caddy
-  shared_certs: {} # netmaker certs generated for MQ comms - used by nodes/servers
-  sqldata: {} # storage for embedded sqlite
-  dnsconfig: {} # storage for coredns
-  mosquitto_logs: {} # storage for mqtt logs
-  turn_server: {}

+ 52 - 53
compose/docker-compose.yml

@@ -1,70 +1,69 @@
 version: "3.4"
 
 services:
+
   netmaker:
     container_name: netmaker
-    image: gravitl/netmaker:REPLACE_SERVER_IMAGE_TAG
+    image: gravitl/netmaker:$SERVER_IMAGE_TAG
+    env_file: ./netmaker.env
     restart: on-failure
     volumes:
       - dnsconfig:/root/config/dnsconfig
       - sqldata:/root/data
     environment:
-      BROKER_ENDPOINT: "wss://broker.NETMAKER_BASE_DOMAIN"
-      SERVER_NAME: "NETMAKER_BASE_DOMAIN"
-      STUN_LIST: "stun.NETMAKER_BASE_DOMAIN:3478,stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302"
-      SERVER_HOST: "SERVER_PUBLIC_IP"
-      SERVER_API_CONN_STRING: "api.NETMAKER_BASE_DOMAIN:443"
-      COREDNS_ADDR: "SERVER_PUBLIC_IP"
-      DNS_MODE: "on"
-      SERVER_HTTP_HOST: "api.NETMAKER_BASE_DOMAIN"
-      NETCLIENT_AUTO_UPDATE: "enabled"
-      API_PORT: "8081"
-      MASTER_KEY: "REPLACE_MASTER_KEY"
-      CORS_ALLOWED_ORIGIN: "*"
-      DISPLAY_KEYS: "on"
-      DATABASE: "sqlite"
-      NODE_ID: "netmaker-server-1"
-      SERVER_BROKER_ENDPOINT: "ws://mq:1883"
-      VERBOSITY: "1"
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD"
-      MQ_USERNAME: "REPLACE_MQ_USERNAME"
-      STUN_PORT: "3478"
-      DEFAULT_PROXY_MODE: "off"
-      TURN_SERVER_HOST: "turn.NETMAKER_BASE_DOMAIN"
-      TURN_SERVER_API_HOST: "https://turnapi.NETMAKER_BASE_DOMAIN"
-      TURN_PORT: "3479"
-      TURN_USERNAME: "REPLACE_TURN_USERNAME"
-      TURN_PASSWORD: "REPLACE_TURN_PASSWORD"
-      USE_TURN: "true"
+      # config-dependant vars
+      - STUN_LIST=stun.${NM_DOMAIN}:${STUN_PORT},stun1.netmaker.io:3478,stun2.netmaker.io:3478,stun1.l.google.com:19302,stun2.l.google.com:19302
+      # The domain/host IP indicating the mq broker address
+      - BROKER_ENDPOINT=wss://broker.${NM_DOMAIN}
+      # The base domain of netmaker
+      - SERVER_NAME=${NM_DOMAIN}
+      - SERVER_API_CONN_STRING=api.${NM_DOMAIN}:443
+      # Address of the CoreDNS server. Defaults to SERVER_HOST
+      - COREDNS_ADDR=${SERVER_HOST}
+      # Overrides SERVER_HOST if set. Useful for making HTTP available via different interfaces/networks.
+      - SERVER_HTTP_HOST=api.${NM_DOMAIN}
+      # domain for your turn server
+      - TURN_SERVER_HOST=turn.${NM_DOMAIN}
+      # domain of the turn api server
+      - TURN_SERVER_API_HOST=https://turnapi.${NM_DOMAIN}
     ports:
       - "3478:3478/udp"
+
   netmaker-ui:
     container_name: netmaker-ui
-    image: gravitl/netmaker-ui:REPLACE_UI_IMAGE_TAG
+    image: gravitl/netmaker-ui:$UI_IMAGE_TAG
+    env_file: ./netmaker.env
+    environment:
+      # config-dependant vars
+      # URL where UI will send API requests. Change based on SERVER_HOST, SERVER_HTTP_HOST, and API_PORT
+      BACKEND_URL: "https://api.${NM_DOMAIN}"
     depends_on:
       - netmaker
     links:
       - "netmaker:api"
     restart: always
-    environment:
-      BACKEND_URL: "https://api.NETMAKER_BASE_DOMAIN"
+
   caddy:
     image: caddy:2.6.2
     container_name: caddy
+    env_file: ./netmaker.env
     restart: unless-stopped
     extra_hosts:
       - "host.docker.internal:host-gateway"
     volumes:
-      - /root/Caddyfile:/etc/caddy/Caddyfile
+      - ./Caddyfile:/etc/caddy/Caddyfile
+      - ./certs:/root/certs
       - caddy_data:/data
       - caddy_conf:/config
     ports:
       - "80:80"
       - "443:443"
+
   coredns:
     container_name: coredns
     image: coredns/coredns
     command: -conf /root/dnsconfig/Corefile
+    env_file: ./netmaker.env
     depends_on:
       - netmaker
     restart: always
@@ -73,36 +72,36 @@ services:
   mq:
     container_name: mq
     image: eclipse-mosquitto:2.0.15-openssl
+    env_file: ./netmaker.env
     depends_on:
       - netmaker
     restart: unless-stopped
-    command: ["/mosquitto/config/wait.sh"]
-    environment:
-      MQ_PASSWORD: "REPLACE_MQ_PASSWORD"
-      MQ_USERNAME: "REPLACE_MQ_USERNAME"
+    command: [ "/mosquitto/config/wait.sh" ]
     volumes:
-      - /root/mosquitto.conf:/mosquitto/config/mosquitto.conf
-      - /root/wait.sh:/mosquitto/config/wait.sh
+      - ./mosquitto.conf:/mosquitto/config/mosquitto.conf
+      - ./wait.sh:/mosquitto/config/wait.sh
       - mosquitto_logs:/mosquitto/log
+      - mosquitto_data:/mosquitto/data
+
   turn:
     container_name: turn
     image: gravitl/turnserver:v1.0.0
+    env_file: ./netmaker.env
+    environment:
+      # config-dependant vars
+      - USERNAME=${TURN_USERNAME}
+      - PASSWORD=${TURN_PASSWORD}
+      # domain for your turn server
+      - TURN_SERVER_HOST=turn.${NM_DOMAIN}
     network_mode: "host"
     volumes:
       - turn_server:/etc/config
-    environment:
-      DEBUG_MODE: "off"
-      VERBOSITY: "1"
-      TURN_PORT: "3479"
-      TURN_API_PORT: "8089"
-      CORS_ALLOWED_ORIGIN: "*"
-      TURN_SERVER_HOST: "turn.NETMAKER_BASE_DOMAIN"
-      USERNAME: "REPLACE_TURN_USERNAME"
-      PASSWORD: "REPLACE_TURN_PASSWORD"
+
 volumes:
-  caddy_data: {}
-  caddy_conf: {}
-  sqldata: {}
-  dnsconfig: {}
-  mosquitto_logs: {}
-  turn_server: {}
+  caddy_data: { } # runtime data for caddy
+  caddy_conf: { } # configuration file for Caddy
+  sqldata: { }
+  dnsconfig: { } # storage for coredns
+  mosquitto_logs: { } # storage for mqtt logs
+  mosquitto_data: { } # storage for mqtt data
+  turn_server: { }

+ 1 - 1
controllers/docs.go

@@ -10,7 +10,7 @@
 //
 //	Schemes: https
 //	BasePath: /
-//	Version: 0.19.0
+//	Version: 0.20.0
 //	Host: netmaker.io
 //
 //	Consumes:

+ 56 - 29
controllers/ext_client.go

@@ -4,6 +4,7 @@ import (
 	"encoding/json"
 	"errors"
 	"fmt"
+	"net"
 	"net/http"
 	"strconv"
 
@@ -230,8 +231,10 @@ func getExtClientConf(w http.ResponseWriter, r *http.Request) {
 		}
 	}
 	defaultDNS := ""
-	if network.DefaultExtClientDNS != "" {
-		defaultDNS = "DNS = " + network.DefaultExtClientDNS
+	if client.DNS != "" {
+		defaultDNS = "DNS = " + client.DNS
+	} else if gwnode.IngressDNS != "" {
+		defaultDNS = "DNS = " + gwnode.IngressDNS
 	}
 
 	defaultMTU := 1420
@@ -321,20 +324,13 @@ func createExtClient(w http.ResponseWriter, r *http.Request) {
 	var extclient models.ExtClient
 	var customExtClient models.CustomExtClient
 
-	err := json.NewDecoder(r.Body).Decode(&customExtClient)
-	if err == nil {
-		if customExtClient.ClientID != "" && !validName(customExtClient.ClientID) {
-			logic.ReturnErrorResponse(w, r, logic.FormatError(errInvalidExtClientID, "badrequest"))
-			return
-		}
-		extclient.ClientID = customExtClient.ClientID
-		if len(customExtClient.PublicKey) > 0 {
-			if _, err := wgtypes.ParseKey(customExtClient.PublicKey); err != nil {
-				logic.ReturnErrorResponse(w, r, logic.FormatError(errInvalidExtClientPubKey, "badrequest"))
-				return
-			}
-			extclient.PublicKey = customExtClient.PublicKey
-		}
+	if err := json.NewDecoder(r.Body).Decode(&customExtClient); err != nil {
+		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "badrequest"))
+		return
+	}
+	if err := validateExtClient(&extclient, &customExtClient); err != nil {
+		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "badrequest"))
+		return
 	}
 
 	extclient.Network = networkName
@@ -392,7 +388,7 @@ func createExtClient(w http.ResponseWriter, r *http.Request) {
 				logger.Log(0, "failed to associate client", extclient.ClientID, "to user", userID)
 			}
 			extclient.OwnerID = userID
-			if _, err := logic.UpdateExtClient(extclient.ClientID, extclient.Network, extclient.Enabled, &extclient, extclient.ACLs); err != nil {
+			if err := logic.SaveExtClient(&extclient); err != nil {
 				logger.Log(0, "failed to add owner id", userID, "to client", extclient.ClientID)
 			}
 		}
@@ -426,9 +422,9 @@ func updateExtClient(w http.ResponseWriter, r *http.Request) {
 
 	var params = mux.Vars(r)
 
-	var newExtClient models.ExtClient
+	var update models.CustomExtClient
 	var oldExtClient models.ExtClient
-	err := json.NewDecoder(r.Body).Decode(&newExtClient)
+	err := json.NewDecoder(r.Body).Decode(&update)
 	if err != nil {
 		logger.Log(0, r.Header.Get("user"), "error decoding request body: ",
 			err.Error())
@@ -445,8 +441,8 @@ func updateExtClient(w http.ResponseWriter, r *http.Request) {
 		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "internal"))
 		return
 	}
-	if !validName(newExtClient.ClientID) {
-		logic.ReturnErrorResponse(w, r, logic.FormatError(errInvalidExtClientID, "badrequest"))
+	if err := validateExtClient(&oldExtClient, &update); err != nil {
+		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "badrequest"))
 		return
 	}
 	data, err := database.FetchRecord(database.EXT_CLIENT_TABLE_NAME, key)
@@ -466,7 +462,7 @@ func updateExtClient(w http.ResponseWriter, r *http.Request) {
 
 	// == PRO ==
 	networkName := params["network"]
-	var changedID = newExtClient.ClientID != oldExtClient.ClientID
+	var changedID = update.ClientID != oldExtClient.ClientID
 	if r.Header.Get("ismaster") != "yes" {
 		userID := r.Header.Get("user")
 		_, doesOwn := doesUserOwnClient(userID, params["clientid"], networkName)
@@ -479,17 +475,16 @@ func updateExtClient(w http.ResponseWriter, r *http.Request) {
 		if err := pro.DissociateNetworkUserClient(oldExtClient.OwnerID, networkName, oldExtClient.ClientID); err != nil {
 			logger.Log(0, "failed to dissociate client", oldExtClient.ClientID, "from user", oldExtClient.OwnerID)
 		}
-		if err := pro.AssociateNetworkUserClient(oldExtClient.OwnerID, networkName, newExtClient.ClientID); err != nil {
-			logger.Log(0, "failed to associate client", newExtClient.ClientID, "to user", oldExtClient.OwnerID)
+		if err := pro.AssociateNetworkUserClient(oldExtClient.OwnerID, networkName, update.ClientID); err != nil {
+			logger.Log(0, "failed to associate client", update.ClientID, "to user", oldExtClient.OwnerID)
 		}
 	}
 	// == END PRO ==
 
-	var changedEnabled = (newExtClient.Enabled != oldExtClient.Enabled) || // indicates there was a change in enablement
-		len(newExtClient.ACLs) != len(oldExtClient.ACLs)
+	var changedEnabled = (update.Enabled != oldExtClient.Enabled) // indicates there was a change in enablement
 	// extra var need as logic.Update changes oldExtClient
 	currentClient := oldExtClient
-	newclient, err := logic.UpdateExtClient(newExtClient.ClientID, params["network"], newExtClient.Enabled, &oldExtClient, newExtClient.ACLs)
+	newclient, err := logic.UpdateExtClient(&oldExtClient, &update)
 	if err != nil {
 		logger.Log(0, r.Header.Get("user"),
 			fmt.Sprintf("failed to update ext client [%s], network [%s]: %v",
@@ -497,7 +492,7 @@ func updateExtClient(w http.ResponseWriter, r *http.Request) {
 		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "internal"))
 		return
 	}
-	logger.Log(0, r.Header.Get("user"), "updated ext client", newExtClient.ClientID)
+	logger.Log(0, r.Header.Get("user"), "updated ext client", update.ClientID)
 	if changedEnabled { // need to send a peer update to the ingress node as enablement of one of it's clients has changed
 		if ingressNode, err := logic.GetNodeByID(newclient.IngressGatewayID); err == nil {
 			if err = mq.PublishPeerUpdate(); err != nil {
@@ -509,7 +504,7 @@ func updateExtClient(w http.ResponseWriter, r *http.Request) {
 	json.NewEncoder(w).Encode(newclient)
 	if changedID {
 		go func() {
-			if err := mq.PublishExtClientDNSUpdate(currentClient, newExtClient, networkName); err != nil {
+			if err := mq.PublishExtClientDNSUpdate(currentClient, *newclient, networkName); err != nil {
 				logger.Log(1, "error pubishing dns update for extcient update", err.Error())
 			}
 		}()
@@ -648,3 +643,35 @@ func doesUserOwnClient(username, clientID, network string) (bool, bool) {
 
 	return false, logic.StringSliceContains(netUser.Clients, clientID)
 }
+
+// validateExtClient	Validates the extclient object
+func validateExtClient(extclient *models.ExtClient, customExtClient *models.CustomExtClient) error {
+	//validate clientid
+	if customExtClient.ClientID != "" && !validName(customExtClient.ClientID) {
+		return errInvalidExtClientID
+	}
+	extclient.ClientID = customExtClient.ClientID
+	if len(customExtClient.PublicKey) > 0 {
+		if _, err := wgtypes.ParseKey(customExtClient.PublicKey); err != nil {
+			return errInvalidExtClientPubKey
+		}
+		extclient.PublicKey = customExtClient.PublicKey
+	}
+	//validate extra ips
+	if len(customExtClient.ExtraAllowedIPs) > 0 {
+		for _, ip := range customExtClient.ExtraAllowedIPs {
+			if _, _, err := net.ParseCIDR(ip); err != nil {
+				return errInvalidExtClientExtraIP
+			}
+		}
+		extclient.ExtraAllowedIPs = customExtClient.ExtraAllowedIPs
+	}
+	//validate DNS
+	if customExtClient.DNS != "" {
+		if ip := net.ParseIP(customExtClient.DNS); ip == nil {
+			return errInvalidExtClientDNS
+		}
+		extclient.DNS = customExtClient.DNS
+	}
+	return nil
+}

+ 1 - 1
controllers/limits.go

@@ -20,7 +20,7 @@ const (
 func checkFreeTierLimits(limit_choice int, next http.Handler) http.HandlerFunc {
 	return func(w http.ResponseWriter, r *http.Request) {
 		var errorResponse = models.ErrorResponse{
-			Code: http.StatusUnauthorized, Message: "free tier limits exceeded on networks",
+			Code: http.StatusForbidden, Message: "free tier limits exceeded on networks",
 		}
 
 		if logic.Free_Tier && servercfg.Is_EE { // check that free tier limits not exceeded

+ 6 - 0
controllers/network.go

@@ -170,6 +170,12 @@ func getNetworkACL(w http.ResponseWriter, r *http.Request) {
 	var networkACL acls.ACLContainer
 	networkACL, err := networkACL.Get(acls.ContainerID(netname))
 	if err != nil {
+		if database.IsEmptyRecord(err) {
+			networkACL = acls.ACLContainer{}
+			w.WriteHeader(http.StatusOK)
+			json.NewEncoder(w).Encode(networkACL)
+			return
+		}
 		logger.Log(0, r.Header.Get("user"),
 			fmt.Sprintf("failed to fetch ACLs for network [%s]: %v", netname, err))
 		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "internal"))

+ 4 - 7
controllers/node.go

@@ -520,13 +520,10 @@ func createIngressGateway(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "application/json")
 	nodeid := params["nodeid"]
 	netid := params["network"]
-	type failoverData struct {
-		Failover bool `json:"failover"`
-	}
-	var failoverReqBody failoverData
-	json.NewDecoder(r.Body).Decode(&failoverReqBody)
+	var request models.IngressRequest
+	json.NewDecoder(r.Body).Decode(&request)
 
-	node, err := logic.CreateIngressGateway(netid, nodeid, failoverReqBody.Failover)
+	node, err := logic.CreateIngressGateway(netid, nodeid, request)
 	if err != nil {
 		logger.Log(0, r.Header.Get("user"),
 			fmt.Sprintf("failed to create ingress gateway on node [%s] on network [%s]: %v",
@@ -535,7 +532,7 @@ func createIngressGateway(w http.ResponseWriter, r *http.Request) {
 		return
 	}
 
-	if servercfg.Is_EE && failoverReqBody.Failover {
+	if servercfg.Is_EE && request.Failover {
 		if err = logic.EnterpriseResetFailoverFunc(node.Network); err != nil {
 			logger.Log(1, "failed to reset failover list during failover create", node.ID.String(), node.Network)
 		}

+ 15 - 3
controllers/regex.go

@@ -6,11 +6,23 @@ import (
 )
 
 var (
-	errInvalidExtClientPubKey = errors.New("incorrect ext client public key")
-	errInvalidExtClientID     = errors.New("ext client ID must be alphanumderic and/or dashes")
+	errInvalidExtClientPubKey  = errors.New("incorrect ext client public key")
+	errInvalidExtClientID      = errors.New("ext client ID must be alphanumderic and/or dashes and less that 15 chars")
+	errInvalidExtClientExtraIP = errors.New("ext client extra ip must be a valid cidr")
+	errInvalidExtClientDNS     = errors.New("ext client dns must be a valid ip address")
 )
 
 // allow only dashes and alphaneumeric for ext client and node names
 func validName(name string) bool {
-	return regexp.MustCompile("^[a-zA-Z0-9-]+$").MatchString(name)
+	reg, err := regexp.Compile("^[a-zA-Z0-9-]+$")
+	if err != nil {
+		return false
+	}
+	if !reg.MatchString(name) {
+		return false
+	}
+	if len(name) > 15 {
+		return false
+	}
+	return true
 }

+ 51 - 0
controllers/regex_test.go

@@ -0,0 +1,51 @@
+package controller
+
+import "testing"
+
+// TestValidName tests the validName function
+func TestValidName(t *testing.T) {
+	type args struct {
+		Name string
+	}
+	tests := []struct {
+		Name string
+		Args args
+		Want bool
+	}{
+		{
+			Name: "validName",
+			Args: args{
+				Name: "TestvalidName",
+			},
+			Want: true,
+		},
+		{
+			Name: "invalidName",
+			Args: args{
+				Name: "Test*Name",
+			},
+			Want: false,
+		},
+		{
+			Name: "nametoolong",
+			Args: args{
+				Name: "TestvalidNameTestvalidName",
+			},
+			Want: false,
+		},
+		{
+			Name: "maxlength",
+			Args: args{
+				Name: "123456789012345",
+			},
+			Want: true,
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.Name, func(t *testing.T) {
+			if got := validName(tt.Args.Name); got != tt.Want {
+				t.Errorf("validName() = %v, want %v", got, tt.Want)
+			}
+		})
+	}
+}

+ 15 - 1
controllers/relay.go

@@ -167,7 +167,7 @@ func deleteHostRelay(w http.ResponseWriter, r *http.Request) {
 	w.Header().Set("Content-Type", "application/json")
 	var params = mux.Vars(r)
 	hostid := params["hostid"]
-	relayHost, _, err := logic.DeleteHostRelay(hostid)
+	relayHost, relayed, err := logic.DeleteHostRelay(hostid)
 	if err != nil {
 		logger.Log(0, r.Header.Get("user"), "error decoding request body: ", err.Error())
 		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "badrequest"))
@@ -178,6 +178,20 @@ func deleteHostRelay(w http.ResponseWriter, r *http.Request) {
 		if err := mq.PublishPeerUpdate(); err != nil {
 			logger.Log(0, "fail to publish peer update: ", err.Error())
 		}
+		if err := mq.HostUpdate(&models.HostUpdate{
+			Action: models.UpdateHost,
+			Host:   *relayHost,
+		}); err != nil {
+			logger.Log(0, "failed to send host update: ", relayHost.Name, err.Error())
+		}
+		for _, relayedHost := range relayed {
+			if err := mq.HostUpdate(&models.HostUpdate{
+				Action: models.UpdateHost,
+				Host:   relayedHost,
+			}); err != nil {
+				logger.Log(0, "failed to send host update: ", relayedHost.Name, err.Error())
+			}
+		}
 	}()
 	apiHostData := relayHost.ConvertNMHostToAPI()
 	w.WriteHeader(http.StatusOK)

+ 32 - 32
docker/Caddyfile

@@ -1,51 +1,51 @@
-{
-        # ZeroSSL account
-        # acme_ca https://acme.zerossl.com/v2/DV90
-        email YOUR_EMAIL
-}
-
 # Dashboard
-https://dashboard.NETMAKER_BASE_DOMAIN {
-        # Apply basic security headers
-        header {
-                # Enable cross origin access to *.NETMAKER_BASE_DOMAIN
-                Access-Control-Allow-Origin *.NETMAKER_BASE_DOMAIN
-                # Enable HTTP Strict Transport Security (HSTS)
-                Strict-Transport-Security "max-age=31536000;"
-                # Enable cross-site filter (XSS) and tell browser to block detected attacks
-                X-XSS-Protection "1; mode=block"
-                # Disallow the site to be rendered within a frame on a foreign domain (clickjacking protection)
-                X-Frame-Options "SAMEORIGIN"
-                # Prevent search engines from indexing
-                X-Robots-Tag "none"
-                # Remove the server name
-                -Server
-        }
+https://dashboard.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	# Apply basic security headers
+	header {
+		# Enable cross origin access to *.{$NM_DOMAIN}
+		Access-Control-Allow-Origin *.{$NM_DOMAIN}
+		# Enable HTTP Strict Transport Security (HSTS)
+		Strict-Transport-Security "max-age=31536000;"
+		# Enable cross-site filter (XSS) and tell browser to block detected attacks
+		X-XSS-Protection "1; mode=block"
+		# Disallow the site to be rendered within a frame on a foreign domain (clickjacking protection)
+		X-Frame-Options "SAMEORIGIN"
+		# Prevent search engines from indexing
+		X-Robots-Tag "none"
+		# Remove the server name
+		-Server
+	}
 
-        reverse_proxy http://netmaker-ui
+	reverse_proxy http://netmaker-ui
 }
 
 # API
-https://api.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://netmaker:8081
+https://api.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://netmaker:8081
 }
 
 # STUN
-https://stun.NETMAKER_BASE_DOMAIN {
+https://stun.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
 	reverse_proxy netmaker:3478
 }
 
 # TURN
-https://turn.NETMAKER_BASE_DOMAIN {
+https://turn.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
 	reverse_proxy host.docker.internal:3479
 }
 
-#TURN API
-https://turnapi.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://host.docker.internal:8089
+# TURN API
+https://turnapi.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://host.docker.internal:8089
 }
 
 # MQ
-wss://broker.NETMAKER_BASE_DOMAIN {
-        reverse_proxy ws://mq:8883 # For EMQX websockets use `reverse_proxy ws://mq:8083`
+wss://broker.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy ws://mq:8883 # For EMQX websockets use `reverse_proxy ws://mq:8083`
 }

+ 41 - 38
docker/Caddyfile-EE

@@ -1,66 +1,69 @@
-{
-        # ZeroSSL account
-        acme_ca https://acme.zerossl.com/v2/DV90
-        email YOUR_EMAIL
-}
-
 # Dashboard
-https://dashboard.NETMAKER_BASE_DOMAIN {
-        # Apply basic security headers
-        header {
-                # Enable cross origin access to *.NETMAKER_BASE_DOMAIN
-                Access-Control-Allow-Origin *.NETMAKER_BASE_DOMAIN
-                # Enable HTTP Strict Transport Security (HSTS)
-                Strict-Transport-Security "max-age=31536000;"
-                # Enable cross-site filter (XSS) and tell browser to block detected attacks
-                X-XSS-Protection "1; mode=block"
-                # Disallow the site to be rendered within a frame on a foreign domain (clickjacking protection)
-                X-Frame-Options "SAMEORIGIN"
-                # Prevent search engines from indexing
-                X-Robots-Tag "none"
-                # Remove the server name
-                -Server
-        }
+https://dashboard.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	# Apply basic security headers
+	header {
+		# Enable cross origin access to *.{$NM_DOMAIN}
+		Access-Control-Allow-Origin *.{$NM_DOMAIN}
+		# Enable HTTP Strict Transport Security (HSTS)
+		Strict-Transport-Security "max-age=31536000;"
+		# Enable cross-site filter (XSS) and tell browser to block detected attacks
+		X-XSS-Protection "1; mode=block"
+		# Disallow the site to be rendered within a frame on a foreign domain (clickjacking protection)
+		X-Frame-Options "SAMEORIGIN"
+		# Prevent search engines from indexing
+		X-Robots-Tag "none"
+		# Remove the server name
+		-Server
+	}
 
-        reverse_proxy http://netmaker-ui
+	reverse_proxy http://netmaker-ui
 }
 
 # Netmaker Exporter
-https://netmaker-exporter.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://netmaker-exporter:8085
+https://netmaker-exporter.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://netmaker-exporter:8085
 }
 
 # Prometheus
-https://prometheus.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://prometheus:9090
+https://prometheus.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://prometheus:9090
 }
 
 # Grafana
-https://grafana.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://grafana:3000
+https://grafana.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://grafana:3000
 }
 
 # API
-https://api.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://netmaker:8081
+https://api.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://netmaker:8081
 }
 
 # STUN
-https://stun.NETMAKER_BASE_DOMAIN {
+https://stun.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
 	reverse_proxy netmaker:3478
 }
 
 # TURN
-https://turn.NETMAKER_BASE_DOMAIN {
+https://turn.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
 	reverse_proxy host.docker.internal:3479
 }
 
-#TURN API
-https://turnapi.NETMAKER_BASE_DOMAIN {
-        reverse_proxy http://host.docker.internal:8089
+# TURN API
+https://turnapi.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy http://host.docker.internal:8089
 }
 
 # MQ
-wss://broker.NETMAKER_BASE_DOMAIN {
-        reverse_proxy ws://mq:8883
+wss://broker.{$NM_DOMAIN} {
+	tls /root/certs/fullchain.pem /root/certs/privkey.pem
+	reverse_proxy ws://mq:8883
 }

+ 7 - 0
ee/ee_controllers/metrics.go

@@ -5,6 +5,7 @@ import (
 	"net/http"
 
 	"github.com/gorilla/mux"
+	"github.com/gravitl/netmaker/database"
 	"github.com/gravitl/netmaker/logger"
 	"github.com/gravitl/netmaker/logic"
 	"github.com/gravitl/netmaker/models"
@@ -91,6 +92,12 @@ func getNetworkExtMetrics(w http.ResponseWriter, r *http.Request) {
 
 	clients, err := logic.GetNetworkExtClients(network) // grab all the network ext clients
 	if err != nil {
+		if database.IsEmptyRecord(err) {
+			var metrics struct{}
+			w.WriteHeader(http.StatusOK)
+			json.NewEncoder(w).Encode(metrics)
+			return
+		}
 		logger.Log(1, r.Header.Get("user"), "failed to fetch metrics of ext clients in network", network, err.Error())
 		logic.ReturnErrorResponse(w, r, logic.FormatError(err, "internal"))
 		return

+ 1 - 2
functions/helpers_test.go

@@ -18,8 +18,7 @@ var (
 		NetID: "not-a-network",
 	}
 	testExternalClient = &models.ExtClient{
-		ClientID:    "testExtClient",
-		Description: "ext client for testing",
+		ClientID: "testExtClient",
 	}
 )
 

+ 5 - 5
go.mod

@@ -14,11 +14,11 @@ require (
 	github.com/rqlite/gorqlite v0.0.0-20210514125552-08ff1e76b22f
 	github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e
 	github.com/stretchr/testify v1.8.2
-	github.com/txn2/txeh v1.3.0
-	golang.org/x/crypto v0.8.0
-	golang.org/x/net v0.9.0 // indirect
-	golang.org/x/oauth2 v0.7.0
-	golang.org/x/sys v0.7.0 // indirect
+	github.com/txn2/txeh v1.4.0
+	golang.org/x/crypto v0.9.0
+	golang.org/x/net v0.10.0 // indirect
+	golang.org/x/oauth2 v0.8.0
+	golang.org/x/sys v0.8.0 // indirect
 	golang.org/x/text v0.9.0 // indirect
 	golang.zx2c4.com/wireguard v0.0.0-20220920152132-bb719d3a6e2c // indirect
 	golang.zx2c4.com/wireguard/wgctrl v0.0.0-20220324164955-056925b7df31

+ 10 - 34
go.sum

@@ -6,15 +6,10 @@ cloud.google.com/go/compute/metadata v0.2.1/go.mod h1:jgHgmJd2RKBGzXqF5LR2EZMGxB
 filippo.io/edwards25519 v1.0.0 h1:0wAIcmJUqRdI8IJ/3eGi5/HwXZWPujYXXlkrQogz0Ek=
 filippo.io/edwards25519 v1.0.0/go.mod h1:N1IkdkCkiLB6tki+MYJoSx2JTY9NUlxZE7eHn5EwJns=
 github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
-github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
 github.com/c-robinson/iplib v1.0.6 h1:FfZV9BWNrah3BgLCFl5/nDXe4RbOi/C9n+DeXFOv5CQ=
 github.com/c-robinson/iplib v1.0.6/go.mod h1:i3LuuFL1hRT5gFpBRnEydzw8R6yhGkF4szNDIbF8pgo=
-github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
-github.com/coreos/go-etcd v2.0.0+incompatible/go.mod h1:Jez6KQU2B/sWsbdaef3ED8NzMklzPG4d5KIOhIy30Tk=
 github.com/coreos/go-oidc/v3 v3.5.0 h1:VxKtbccHZxs8juq7RdJntSqtXFtde9YpNpGn0yqgEHw=
 github.com/coreos/go-oidc/v3 v3.5.0/go.mod h1:ecXRtV4romGPeO6ieExAsUK9cb/3fp9hXNz1tlv8PIM=
-github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
-github.com/cpuguy83/go-md2man v1.0.10/go.mod h1:SmD6nW6nTyfqj6ABTjUi3V3JVMnlJmwcJI5acqYI6dE=
 github.com/cpuguy83/go-md2man/v2 v2.0.0-20190314233015-f79a8a8ca69d/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
 github.com/cpuguy83/go-md2man/v2 v2.0.2/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
@@ -28,7 +23,6 @@ github.com/eclipse/paho.mqtt.golang v1.4.2/go.mod h1:JGt0RsEwEX+Xa/agj90YJ9d9DH2
 github.com/felixge/httpsnoop v1.0.1/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
 github.com/felixge/httpsnoop v1.0.3 h1:s/nj+GCswXYzN5v2DpNMuMQYe+0DDwt5WVCU6CWBdXk=
 github.com/felixge/httpsnoop v1.0.3/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
-github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
 github.com/go-jose/go-jose/v3 v3.0.0 h1:s6rrhirfEP/CGIoc6p+PZAeogN2SxKav6Wp7+dyMWVo=
 github.com/go-jose/go-jose/v3 v3.0.0/go.mod h1:RNkWWRld676jZEYoV3+XK8L2ZnNSvIsxFMht0mSX+u8=
 github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
@@ -64,8 +58,6 @@ github.com/guumaster/tablewriter v0.0.10 h1:A0HD94yMdt4usgxBjoEceNeE0XMJ027euoHA
 github.com/guumaster/tablewriter v0.0.10/go.mod h1:p4FRFhyfo0UD9ZLmMRbbJooTUsxo6b80qZTERVDWrH8=
 github.com/hashicorp/go-version v1.6.0 h1:feTTfFNnjP967rlCxM/I9g701jU+RN74YKx2mOkIeek=
 github.com/hashicorp/go-version v1.6.0/go.mod h1:fltr4n8CU8Ke44wwGCBoEymUuxUHl09ZGVZPK5anwXA=
-github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
-github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/josharian/native v1.0.0 h1:Ts/E8zCSEsG17dUqv7joXJFybuMLjQfWE04tsBODTxk=
@@ -81,7 +73,6 @@ github.com/leodido/go-urn v1.2.3 h1:6BE2vPT0lqoz3fmOesHZiaiFh7889ssCo2GMvLCfiuA=
 github.com/leodido/go-urn v1.2.3/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4=
 github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
 github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
-github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
 github.com/matryer/is v1.4.1 h1:55ehd8zaGABKLXQUe2awZ99BD/PTc2ls+KV/dXphgEQ=
 github.com/matryer/is v1.4.1/go.mod h1:8I/i5uYgLzgsgEloJE1U6xx5HkBQpAZvepWuujKwMRU=
 github.com/mattn/go-runewidth v0.0.9/go.mod h1:H031xJmbD/WCDINGzjvQ9THkh0rPKHF+m2gUSrubnMI=
@@ -98,11 +89,8 @@ github.com/mdlayher/socket v0.1.1 h1:q3uOGirUPfAV2MUoaC7BavjQ154J7+JOkTWyiV+intI
 github.com/mdlayher/socket v0.1.1/go.mod h1:mYV5YIZAfHh4dzDVzI8x8tWLWCliuX8Mon5Awbj+qDs=
 github.com/mikioh/ipaddr v0.0.0-20190404000644-d465c8ab6721 h1:RlZweED6sbSArvlE924+mUcZuXKLBHA35U7LN621Bws=
 github.com/mikioh/ipaddr v0.0.0-20190404000644-d465c8ab6721/go.mod h1:Ickgr2WtCLZ2MDGd4Gr0geeCH5HybhRJbonOgQpvSxc=
-github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
-github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
 github.com/olekukonko/tablewriter v0.0.5 h1:P2Ga83D34wi1o9J6Wh1mRuqd4mF/x/lgBS7N7AbDhec=
 github.com/olekukonko/tablewriter v0.0.5/go.mod h1:hPp6KlRPjbx+hW8ykQs1w3UBbZlj6HuIJcUGPhkA7kY=
-github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
 github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
 github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
 github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
@@ -117,48 +105,37 @@ github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZV
 github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
 github.com/rqlite/gorqlite v0.0.0-20210514125552-08ff1e76b22f h1:BSnJgAfHzEp7o8PYJ7YfwAVHhqu7BYUTggcn/LGlUWY=
 github.com/rqlite/gorqlite v0.0.0-20210514125552-08ff1e76b22f/go.mod h1:UW/gxgQwSePTvL1KA8QEHsXeYHP4xkoXgbDdN781p34=
-github.com/russross/blackfriday v1.5.2/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
 github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
 github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
 github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e h1:MRM5ITcdelLK2j1vwZ3Je0FKVCfqOLp5zO6trqMLYs0=
 github.com/skip2/go-qrcode v0.0.0-20200617195104-da1b6568686e/go.mod h1:XV66xRDqSt+GTGFMVlhk3ULuV0y9ZmzeVGR4mloJI3M=
-github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
-github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
-github.com/spf13/cobra v0.0.5/go.mod h1:3K3wKZymM7VvHMDS9+Akkh4K60UwM26emMESw8tLCHU=
 github.com/spf13/cobra v1.7.0 h1:hyqWnYt1ZQShIddO5kBpj3vu05/++x6tJ6dg8EC572I=
 github.com/spf13/cobra v1.7.0/go.mod h1:uLxZILRyS/50WlhOIKD7W6V5bgeIt+4sICxh6uRMrb0=
-github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
-github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
-github.com/spf13/viper v1.3.2/go.mod h1:ZiWeW+zYFKm7srdB9IoDzzZXaJaI5eL9QjNiN/DMA2s=
 github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
 github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
 github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
-github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
 github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
 github.com/stretchr/testify v1.8.2 h1:+h33VjcLVPDHtOdpUCuF+7gSuG3yGIftsP1YvFihtJ8=
 github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
-github.com/txn2/txeh v1.3.0 h1:vnbv63htVMZCaQgLqVBxKvj2+HHHFUzNW7I183zjg3E=
-github.com/txn2/txeh v1.3.0/go.mod h1:O7M6gUTPeMF+vsa4c4Ipx3JDkOYrruB1Wry8QRsMcw8=
-github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
+github.com/txn2/txeh v1.4.0 h1:0tdvpA4HGJrj8X3kmrU6o/JFStI009nKxwDpMK5CnRU=
+github.com/txn2/txeh v1.4.0/go.mod h1:Mgq0hY184zCrDBLgvkIp+9NYGHoYbJcu4xKqUcx1shc=
 github.com/urfave/cli v1.22.5/go.mod h1:Gos4lmkARVdJ6EkW0WaNv/tZAAMe9V7XWyB60NtXRu0=
-github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
 github.com/xtgo/uuid v0.0.0-20140804021211-a0b114877d4c h1:3lbZUMbMiGUW/LMkfsEABsc5zNT9+b1CvsJx47JzJ8g=
 github.com/xtgo/uuid v0.0.0-20140804021211-a0b114877d4c/go.mod h1:UrdRz5enIKZ63MEE3IF9l2/ebyx59GyGgPi+tICQdmM=
 github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
-golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20190911031432-227b76d455e7/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
 golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.0.0-20211108221036-ceb1ce70b4fa/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
 golang.org/x/crypto v0.0.0-20220208050332-20e1d8d225ab/go.mod h1:IxCIyHEi3zRg3s0A5j5BB6A9Jmi73HwBIUl50j+osU4=
-golang.org/x/crypto v0.8.0 h1:pd9TJtTueMTVQXzk8E2XESSMQDj/U7OUu0PqJqPXQjQ=
-golang.org/x/crypto v0.8.0/go.mod h1:mRqEX+O9/h5TFCrQhkgjo2yKi0yYA+9ecGkdQoHrywE=
+golang.org/x/crypto v0.9.0 h1:LF6fAI+IutBocDJ2OT0Q1g8plpYljMZ4+lty+dsqw3g=
+golang.org/x/crypto v0.9.0/go.mod h1:yrmDGqONDYtNj3tH8X9dzUun2m2lzPa9ngI6/RUPGR0=
 golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e h1:+WEEuIdZHnUeJJmEUjyYC2gfUMj69yZXw17EnHg/otA=
 golang.org/x/exp v0.0.0-20220722155223-a9213eeb770e/go.mod h1:Kr81I6Kryrl9sr8s2FK3vxD90NdsKWRuOIl2O4CvYbA=
 golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=
@@ -174,17 +151,16 @@ golang.org/x/net v0.0.0-20220127200216-cd36cc0744dd/go.mod h1:CfG3xpIq0wQ8r1q4Su
 golang.org/x/net v0.0.0-20220722155237-a158d28d115b/go.mod h1:XRhObCWvk6IyKnWLug+ECip1KBveYUHfp+8e9klMJ9c=
 golang.org/x/net v0.3.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
 golang.org/x/net v0.4.0/go.mod h1:MBQ8lrhLObU/6UmLb4fmbmk5OcyYmqtbGd/9yIeKjEE=
-golang.org/x/net v0.9.0 h1:aWJ/m6xSmxWBx+V0XRHTlrYrPG56jKsLdTFmsSsCzOM=
-golang.org/x/net v0.9.0/go.mod h1:d48xBJpPfHeWQsugry2m+kC02ZBRGRgulfHnEXEuWns=
+golang.org/x/net v0.10.0 h1:X2//UzNDwYmtCLn7To6G58Wr6f5ahEAQgKNzv9Y951M=
+golang.org/x/net v0.10.0/go.mod h1:0qNGK6F8kojg2nk9dLZ2mShWaEBan6FAoqfSigmmuDg=
 golang.org/x/oauth2 v0.3.0/go.mod h1:rQrIauxkUhJ6CuwEXwymO2/eh4xz2ZWF1nBkcxS+tGk=
-golang.org/x/oauth2 v0.7.0 h1:qe6s0zUXlPX80/dITx3440hWZ7GwMwgDDyrSGTPJG/g=
-golang.org/x/oauth2 v0.7.0/go.mod h1:hPLQkd9LyjfXTiRohC/41GhcFqxisoUQ99sCUOHO9x4=
+golang.org/x/oauth2 v0.8.0 h1:6dkIjl3j3LtZ/O3sTgZTMsLKSftL/B8Zgq4huOIIUu8=
+golang.org/x/oauth2 v0.8.0/go.mod h1:yr7u4HXZRm1R1kBWqr/xKNqewf0plRYoB7sla+BCIXE=
 golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20210220032951-036812b2e83c/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
 golang.org/x/sync v0.1.0 h1:wsuoTGHzEhffawBOhz5CYhcrV4IdKZbEyZjBMuTp12o=
 golang.org/x/sync v0.1.0/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
-golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
 golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
@@ -200,8 +176,8 @@ golang.org/x/sys v0.0.0-20220207234003-57398862261d/go.mod h1:oPkhp1MJrh7nUepCBc
 golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.3.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.7.0 h1:3jlCCIQZPdOYu1h8BkNvLz8Kgwtae2cagcG/VamtZRU=
-golang.org/x/sys v0.7.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.8.0 h1:EBmGv8NaZBZTWvrbjNoL6HVt+IVy3QDQpJs7VRIw3tU=
+golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
 golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8=
 golang.org/x/term v0.3.0/go.mod h1:q750SLmJuPmVoN1blW3UFBPREJfb1KmY3vwxfr+nFDA=

+ 1 - 1
k8s/client/netclient-daemonset.yaml

@@ -16,7 +16,7 @@ spec:
       hostNetwork: true
       containers:
       - name: netclient
-        image: gravitl/netclient:v0.19.0
+        image: gravitl/netclient:v0.20.0
         env:
         - name: TOKEN
           value: "TOKEN_VALUE"

+ 1 - 1
k8s/client/netclient.yaml

@@ -28,7 +28,7 @@ spec:
       #           - "<node label value>"
       containers:
       - name: netclient
-        image: gravitl/netclient:v0.19.0
+        image: gravitl/netclient:v0.20.0
         env:
         - name: TOKEN
           value: "TOKEN_VALUE"

+ 1 - 1
k8s/server/netmaker-server.yaml

@@ -79,7 +79,7 @@ spec:
           value: "Kubernetes"
         - name: VERBOSITY
           value: "3"
-        image: gravitl/netmaker:v0.19.0
+        image: gravitl/netmaker:v0.20.0
         imagePullPolicy: Always
         name: netmaker
         ports:

+ 1 - 1
k8s/server/netmaker-ui.yaml

@@ -15,7 +15,7 @@ spec:
     spec:
       containers:
       - name: netmaker-ui
-        image: gravitl/netmaker-ui:v0.19.0
+        image: gravitl/netmaker-ui:v0.20.0
         ports:
         - containerPort: 443
         env:

+ 6 - 2
logic/enrollmentkey.go

@@ -41,12 +41,16 @@ func CreateEnrollmentKey(uses int, expiration time.Time, networks, tags []string
 		Unlimited:     unlimited,
 		Networks:      []string{},
 		Tags:          []string{},
+		Type:          models.Undefined,
 	}
 	if uses > 0 {
 		k.UsesRemaining = uses
-	}
-	if !expiration.IsZero() {
+		k.Type = models.Uses
+	} else if !expiration.IsZero() {
 		k.Expiration = expiration
+		k.Type = models.TimeExpiration
+	} else if k.Unlimited {
+		k.Type = models.Unlimited
 	}
 	if len(networks) > 0 {
 		k.Networks = networks

+ 22 - 25
logic/extpeers.go

@@ -174,6 +174,11 @@ func CreateExtClient(extclient *models.ExtClient) error {
 	}
 
 	extclient.LastModified = time.Now().Unix()
+	return SaveExtClient(extclient)
+}
+
+// SaveExtClient - saves an ext client to database
+func SaveExtClient(extclient *models.ExtClient) error {
 	key, err := GetRecordKey(extclient.ClientID, extclient.Network)
 	if err != nil {
 		return err
@@ -188,35 +193,27 @@ func CreateExtClient(extclient *models.ExtClient) error {
 	return SetNetworkNodesLastModified(extclient.Network)
 }
 
-// UpdateExtClient - only supports name changes right now
-func UpdateExtClient(newclientid string, network string, enabled bool, client *models.ExtClient, newACLs map[string]struct{}) (*models.ExtClient, error) {
-	err := DeleteExtClient(network, client.ClientID)
+// UpdateExtClient - updates an ext client with new values
+func UpdateExtClient(old *models.ExtClient, update *models.CustomExtClient) (*models.ExtClient, error) {
+	new := old
+	err := DeleteExtClient(old.Network, old.ClientID)
 	if err != nil {
-		return client, err
+		return new, err
 	}
-	if newclientid != client.ClientID { // name change only
-		client.ClientID = newclientid
-		client.LastModified = time.Now().Unix()
-		data, err := json.Marshal(&client)
-		if err != nil {
-			return nil, err
-		}
-		key, err := GetRecordKey(client.ClientID, client.Network)
-		if err != nil {
-			return nil, err
-		}
-		if err = database.Insert(key, string(data), database.EXT_CLIENT_TABLE_NAME); err != nil {
-			return client, err
-		}
-		return client, nil
+	new.ClientID = update.ClientID
+	if update.PublicKey != "" && old.PublicKey != update.PublicKey {
+		new.PublicKey = update.PublicKey
+	}
+	if update.DNS != "" && update.DNS != old.DNS {
+		new.DNS = update.DNS
+	}
+	if update.Enabled != old.Enabled {
+		new.Enabled = update.Enabled
 	}
-	client.ClientID = newclientid
-	client.Enabled = enabled
-	SetClientACLs(client, newACLs)
-	if err = CreateExtClient(client); err != nil {
-		return client, err
+	if update.ExtraAllowedIPs != nil && StringDifference(old.ExtraAllowedIPs, update.ExtraAllowedIPs) != nil {
+		new.ExtraAllowedIPs = update.ExtraAllowedIPs
 	}
-	return client, err
+	return new, CreateExtClient(new)
 }
 
 // GetExtClientsByID - gets the clients of attached gateway

+ 3 - 2
logic/gateway.go

@@ -96,7 +96,7 @@ func DeleteEgressGateway(network, nodeid string) (models.Node, error) {
 }
 
 // CreateIngressGateway - creates an ingress gateway
-func CreateIngressGateway(netid string, nodeid string, failover bool) (models.Node, error) {
+func CreateIngressGateway(netid string, nodeid string, ingress models.IngressRequest) (models.Node, error) {
 
 	node, err := GetNodeByID(nodeid)
 	if err != nil {
@@ -120,8 +120,9 @@ func CreateIngressGateway(netid string, nodeid string, failover bool) (models.No
 	node.IsIngressGateway = true
 	node.IngressGatewayRange = network.AddressRange
 	node.IngressGatewayRange6 = network.AddressRange6
+	node.IngressDNS = ingress.ExtclientDNS
 	node.SetLastModified()
-	if failover && servercfg.Is_EE {
+	if ingress.Failover && servercfg.Is_EE {
 		node.Failover = true
 	}
 	data, err := json.Marshal(&node)

+ 3 - 1
main.go

@@ -19,6 +19,7 @@ import (
 	"github.com/gravitl/netmaker/logger"
 	"github.com/gravitl/netmaker/logic"
 	"github.com/gravitl/netmaker/logic/pro"
+	"github.com/gravitl/netmaker/migrate"
 	"github.com/gravitl/netmaker/models"
 	"github.com/gravitl/netmaker/mq"
 	"github.com/gravitl/netmaker/netclient/ncutils"
@@ -27,7 +28,7 @@ import (
 	stunserver "github.com/gravitl/netmaker/stun-server"
 )
 
-var version = "v0.19.0"
+var version = "v0.20.0"
 
 // Start DB Connection and start API Request Handler
 func main() {
@@ -74,6 +75,7 @@ func initialize() { // Client Mode Prereq Check
 		logger.FatalLog("Error connecting to database: ", err.Error())
 	}
 	logger.Log(0, "database successfully connected")
+	migrate.Run()
 
 	logic.SetJWTSecret()
 

+ 50 - 0
migrate/migrate.go

@@ -0,0 +1,50 @@
+package migrate
+
+import (
+	"encoding/json"
+
+	"github.com/gravitl/netmaker/database"
+	"github.com/gravitl/netmaker/logger"
+	"github.com/gravitl/netmaker/models"
+)
+
+// Run - runs all migrations
+func Run() {
+	updateEnrollmentKeys()
+}
+
+func updateEnrollmentKeys() {
+	rows, err := database.FetchRecords(database.ENROLLMENT_KEYS_TABLE_NAME)
+	if err != nil {
+		return
+	}
+	for _, row := range rows {
+		var key models.EnrollmentKey
+		if err = json.Unmarshal([]byte(row), &key); err != nil {
+			continue
+		}
+		if key.Type != models.Undefined {
+			logger.Log(2, "migration: enrollment key type already set")
+			continue
+		} else {
+			logger.Log(2, "migration: updating enrollment key type")
+			if key.Unlimited {
+				key.Type = models.Unlimited
+			} else if key.UsesRemaining > 0 {
+				key.Type = models.Uses
+			} else if !key.Expiration.IsZero() {
+				key.Type = models.TimeExpiration
+			}
+		}
+		data, err := json.Marshal(key)
+		if err != nil {
+			logger.Log(0, "migration: marshalling enrollment key: "+err.Error())
+			continue
+		}
+		if err = database.Insert(key.Value, string(data), database.ENROLLMENT_KEYS_TABLE_NAME); err != nil {
+			logger.Log(0, "migration: inserting enrollment key: "+err.Error())
+			continue
+		}
+
+	}
+}

+ 3 - 0
models/api_node.go

@@ -32,6 +32,7 @@ type ApiNode struct {
 	RelayAddrs              []string `json:"relayaddrs"`
 	FailoverNode            string   `json:"failovernode"`
 	DNSOn                   bool     `json:"dnson"`
+	IngressDns              string   `json:"ingressdns"`
 	Server                  string   `json:"server"`
 	InternetGateway         string   `json:"internetgateway"`
 	Connected               bool     `json:"connected"`
@@ -61,6 +62,7 @@ func (a *ApiNode) ConvertToServerNode(currentNode *Node) *Node {
 	convertedNode.IngressGatewayRange = currentNode.IngressGatewayRange
 	convertedNode.IngressGatewayRange6 = currentNode.IngressGatewayRange6
 	convertedNode.DNSOn = a.DNSOn
+	convertedNode.IngressDNS = a.IngressDns
 	convertedNode.EgressGatewayRequest = currentNode.EgressGatewayRequest
 	convertedNode.EgressGatewayNatEnabled = currentNode.EgressGatewayNatEnabled
 	convertedNode.PersistentKeepalive = time.Second * time.Duration(a.PersistentKeepalive)
@@ -148,6 +150,7 @@ func (nm *Node) ConvertToAPINode() *ApiNode {
 		apiNode.FailoverNode = ""
 	}
 	apiNode.DNSOn = nm.DNSOn
+	apiNode.IngressDns = nm.IngressDNS
 	apiNode.Server = nm.Server
 	apiNode.InternetGateway = nm.InternetGateway.String()
 	if isEmptyAddr(apiNode.InternetGateway) {

+ 20 - 0
models/enrollment_key.go

@@ -4,6 +4,21 @@ import (
 	"time"
 )
 
+const (
+	Undefined KeyType = iota
+	TimeExpiration
+	Uses
+	Unlimited
+)
+
+// KeyType - the type of enrollment key
+type KeyType int
+
+// String - returns the string representation of a KeyType
+func (k KeyType) String() string {
+	return [...]string{"Undefined", "TimeExpiration", "Uses", "Unlimited"}[k]
+}
+
 // EnrollmentToken - the tokenized version of an enrollmentkey;
 // to be used for host registration
 type EnrollmentToken struct {
@@ -23,6 +38,7 @@ type EnrollmentKey struct {
 	Unlimited     bool      `json:"unlimited"`
 	Tags          []string  `json:"tags"`
 	Token         string    `json:"token,omitempty"` // B64 value of EnrollmentToken
+	Type          KeyType   `json:"type"`
 }
 
 // APIEnrollmentKey - used to create enrollment keys via API
@@ -32,6 +48,7 @@ type APIEnrollmentKey struct {
 	Networks      []string `json:"networks"`
 	Unlimited     bool     `json:"unlimited"`
 	Tags          []string `json:"tags"`
+	Type          KeyType  `json:"type"`
 }
 
 // RegisterResponse - the response to a successful enrollment register
@@ -51,6 +68,9 @@ func (k *EnrollmentKey) IsValid() bool {
 	if !k.Expiration.IsZero() && time.Now().Before(k.Expiration) {
 		return true
 	}
+	if k.Type == Undefined {
+		return false
+	}
 
 	return k.Unlimited
 }

+ 11 - 1
models/extclient.go

@@ -3,12 +3,13 @@ package models
 // ExtClient - struct for external clients
 type ExtClient struct {
 	ClientID               string              `json:"clientid" bson:"clientid"`
-	Description            string              `json:"description" bson:"description"`
 	PrivateKey             string              `json:"privatekey" bson:"privatekey"`
 	PublicKey              string              `json:"publickey" bson:"publickey"`
 	Network                string              `json:"network" bson:"network"`
+	DNS                    string              `json:"dns" bson:"dns"`
 	Address                string              `json:"address" bson:"address"`
 	Address6               string              `json:"address6" bson:"address6"`
+	ExtraAllowedIPs        []string            `json:"extraallowedips" bson:"extraallowedips"`
 	IngressGatewayID       string              `json:"ingressgatewayid" bson:"ingressgatewayid"`
 	IngressGatewayEndpoint string              `json:"ingressgatewayendpoint" bson:"ingressgatewayendpoint"`
 	LastModified           int64               `json:"lastmodified" bson:"lastmodified"`
@@ -16,3 +17,12 @@ type ExtClient struct {
 	OwnerID                string              `json:"ownerid" bson:"ownerid"`
 	ACLs                   map[string]struct{} `json:"acls,omitempty" bson:"acls,omitempty"`
 }
+
+// CustomExtClient - struct for CustomExtClient params
+type CustomExtClient struct {
+	ClientID        string   `json:"clientid,omitempty"`
+	PublicKey       string   `json:"publickey,omitempty"`
+	DNS             string   `json:"dns,omitempty"`
+	ExtraAllowedIPs []string `json:"extraallowedips,omitempty"`
+	Enabled         bool     `json:"enabled,omitempty"`
+}

+ 16 - 5
models/host.go

@@ -121,6 +121,16 @@ const (
 	UpdateKeys = "UPDATE_KEYS"
 )
 
+// SignalAction - turn peer signal action
+type SignalAction string
+
+const (
+	// Disconnect - action to stop using turn connection
+	Disconnect SignalAction = "DISCONNECT"
+	// ConnNegotiation - action to negotiate connection between peers
+	ConnNegotiation SignalAction = "CONNECTION_NEGOTIATION"
+)
+
 // HostUpdate - struct for host update
 type HostUpdate struct {
 	Action HostMqAction
@@ -137,11 +147,12 @@ type HostTurnRegister struct {
 
 // Signal - struct for signalling peer
 type Signal struct {
-	Server            string `json:"server"`
-	FromHostPubKey    string `json:"from_host_pubkey"`
-	TurnRelayEndpoint string `json:"turn_relay_addr"`
-	ToHostPubKey      string `json:"to_host_pubkey"`
-	Reply             bool   `json:"reply"`
+	Server            string       `json:"server"`
+	FromHostPubKey    string       `json:"from_host_pubkey"`
+	TurnRelayEndpoint string       `json:"turn_relay_addr"`
+	ToHostPubKey      string       `json:"to_host_pubkey"`
+	Reply             bool         `json:"reply"`
+	Action            SignalAction `json:"action"`
 }
 
 // RegisterMsg - login message struct for hosts to join via SSO login

+ 0 - 1
models/network.go

@@ -23,7 +23,6 @@ type Network struct {
 	IsIPv4              string                `json:"isipv4" bson:"isipv4" validate:"checkyesorno"`
 	IsIPv6              string                `json:"isipv6" bson:"isipv6" validate:"checkyesorno"`
 	DefaultUDPHolePunch string                `json:"defaultudpholepunch" bson:"defaultudpholepunch" validate:"checkyesorno"`
-	DefaultExtClientDNS string                `json:"defaultextclientdns" bson:"defaultextclientdns"`
 	DefaultMTU          int32                 `json:"defaultmtu" bson:"defaultmtu"`
 	DefaultACL          string                `json:"defaultacl" bson:"defaultacl" yaml:"defaultacl" validate:"checkyesorno"`
 	ProSettings         *promodels.ProNetwork `json:"prosettings,omitempty" bson:"prosettings,omitempty" yaml:"prosettings,omitempty"`

+ 1 - 0
models/node.go

@@ -69,6 +69,7 @@ type CommonNode struct {
 	IsEgressGateway     bool          `json:"isegressgateway" yaml:"isegressgateway"`
 	EgressGatewayRanges []string      `json:"egressgatewayranges" bson:"egressgatewayranges" yaml:"egressgatewayranges"`
 	IsIngressGateway    bool          `json:"isingressgateway" yaml:"isingressgateway"`
+	IngressDNS          string        `json:"ingressdns" yaml:"ingressdns"`
 	DNSOn               bool          `json:"dnson" yaml:"dnson"`
 	PersistentKeepalive time.Duration `json:"persistentkeepalive" yaml:"persistentkeepalive"`
 }

+ 6 - 6
models/structs.go

@@ -14,12 +14,6 @@ const (
 	PLACEHOLDER_TOKEN_TEXT = "ACCESS_TOKEN"
 )
 
-// CustomExtClient - struct for CustomExtClient params
-type CustomExtClient struct {
-	ClientID  string `json:"clientid"`
-	PublicKey string `json:"publickey,omitempty"`
-}
-
 // AuthParams - struct for auth params
 type AuthParams struct {
 	MacAddress string `json:"macaddress"`
@@ -170,6 +164,12 @@ type HostRelayRequest struct {
 	RelayedHosts []string `json:"relayed_hosts"`
 }
 
+// IngressRequest - ingress request struct
+type IngressRequest struct {
+	ExtclientDNS string `json:"extclientdns"`
+	Failover     bool   `json:"failover"`
+}
+
 // ServerUpdateData - contains data to configure server
 // and if it should set peers
 type ServerUpdateData struct {

+ 5 - 13
release.md

@@ -1,23 +1,15 @@
-# Netmaker v0.19.0
+
+# Netmaker v0.20.0
 
 ## whats new
+- New UI
+- revamped compose-files and install scripts
 - TURN
-- dependency updates
-- internet gateways (0.0.0.0/0) for egress
-- deprecated editing of network parameters
-- allow extra ips for extclient (not enabled in UI)
     
 ## whats fixed
-- unbiased random string
-- get traffic keys on pull
-- CI updates
-- install/update script updates
-- firewall checks
--  
-- 
+- Caddy does not handle netmaker exporter well for EE
 
 ## known issues
-- Caddy does not handle netmaker exporter well for EE
 - Migration causes a listen port of 0 for some upgraded hosts
 - Docker clients can not re-join after deletion
 - Innacurate Ext Client Metrics 

+ 82 - 0
scripts/netmaker.env

@@ -0,0 +1,82 @@
+# Email used for SSL certificates
+NM_EMAIL=
+# The base domain of netmaker
+NM_DOMAIN=
+# Public IP of machine
+SERVER_HOST=
+# The admin master key for accessing the API. Change this in any production installation.
+MASTER_KEY=
+# The username to set for turn api access
+TURN_USERNAME=
+# The password to set for turn api access
+TURN_PASSWORD=
+# The username to set for MQ access
+MQ_USERNAME=
+# The password to set for MQ access
+MQ_PASSWORD=
+INSTALL_TYPE=
+NETMAKER_ACCOUNT_ID=
+LICENSE_KEY=
+SERVER_IMAGE_TAG=
+UI_IMAGE_TAG=
+# used for HA - identifies this server vs other servers
+NODE_ID="netmaker-server-1"
+METRICS_EXPORTER="off"
+PROMETHEUS="off"
+# Enables DNS Mode, meaning all nodes will set hosts file for private dns settings
+DNS_MODE="on"
+# Enable auto update of netclient ? ENUM:- enabled,disabled | default=enabled
+NETCLIENT_AUTO_UPDATE="enabled"
+# The HTTP API port for Netmaker. Used for API calls / communication from front end.
+# If changed, need to change port of BACKEND_URL for netmaker-ui.
+API_PORT="8081"
+EXPORTER_API_PORT="8085"
+# The "allowed origin" for API requests. Change to restrict where API requests can come from with comma-separated
+# URLs. ex:- https://dashboard.netmaker.domain1.com,https://dashboard.netmaker.domain2.com
+CORS_ALLOWED_ORIGIN="*"
+# Show keys permanently in UI (until deleted) as opposed to 1-time display.
+DISPLAY_KEYS="on"
+# Database to use - sqlite, postgres, or rqlite
+DATABASE="sqlite"
+# The address of the mq server. If running from docker compose it will be "mq". Otherwise, need to input address.
+# If using "host networking", it will find and detect the IP of the mq container.
+SERVER_BROKER_ENDPOINT="ws://mq:1883"
+# The reachable port of STUN on the server
+STUN_PORT="3478"
+# Logging verbosity level - 1, 2, or 3
+VERBOSITY="1"
+# If ON, all new clients will enable proxy by default
+# If OFF, all new clients will disable proxy by default
+# If AUTO, stick with the existing logic for NAT detection
+DEFAULT_PROXY_MODE="off"
+# Port to access turn server
+TURN_PORT="3479"
+# Config for using turn, accepts either true/false
+USE_TURN="true"
+DEBUG_MODE="off"
+TURN_API_PORT="8089"
+# Enables the REST backend (API running on API_PORT at SERVER_HTTP_HOST).
+# Change to "off" to turn off.
+REST_BACKEND="on"
+# If turned "on", Server will not set Host based on remote IP check.
+# This is already overridden if SERVER_HOST is set. Turned "off" by default.
+DISABLE_REMOTE_IP_CHECK="off"
+# Whether or not to send telemetry data to help improve Netmaker. Switch to "off" to opt out of sending telemetry.
+TELEMETRY="on"
+###
+#
+# OAuth section
+#
+###
+# "<azure-ad|github|google|oidc>"
+AUTH_PROVIDER=
+# "<client id of your oauth provider>"
+CLIENT_ID=
+# "<client secret of your oauth provider>"
+CLIENT_SECRET=
+# "https://dashboard.<netmaker base domain>"
+FRONTEND_URL=
+# "<only for azure, you may optionally specify the tenant for the OAuth>"
+AZURE_TENANT=
+# https://oidc.yourprovider.com - URL of oidc provider
+OIDC_ISSUER=

+ 99 - 0
scripts/nm-certs.sh

@@ -0,0 +1,99 @@
+#!/bin/bash
+
+CONFIG_FILE=netmaker.env
+SCRIPT_DIR=$(dirname "$(realpath "$0")")
+
+# get and check the config
+if [ ! -f "$SCRIPT_DIR/$CONFIG_FILE" ]; then
+	echo "Config file missing"
+	exit 1
+fi
+source "$SCRIPT_DIR/$CONFIG_FILE"
+if [ -z "$NM_DOMAIN" ] || [ -z "$NM_EMAIL" ]; then
+	echo "Config not valid"
+	exit 1
+fi
+
+# TODO make sure this doesnt break, parse `certbot certificates` if yes
+CERT_DIR="$SCRIPT_DIR/letsencrypt/live/stun.$NM_DOMAIN"
+
+echo "Setting up SSL certificates..."
+
+# preserve the env state
+RESTART_CADDY=false
+if [ -n "$(docker ps | grep caddy)" ]; then
+	echo "Caddy is running, stopping for now..."
+	RESTART_CADDY=true
+	docker-compose -f /root/docker-compose.yml stop caddy
+fi
+
+CERTBOT_PARAMS=$(cat <<EOF
+certonly --standalone \
+	--non-interactive --agree-tos \
+	-m "$NM_EMAIL" \
+	-d "stun.$NM_DOMAIN" \
+	-d "api.$NM_DOMAIN" \
+	-d "broker.$NM_DOMAIN" \
+	-d "dashboard.$NM_DOMAIN" \
+	-d "turn.$NM_DOMAIN" \
+	-d "turnapi.$NM_DOMAIN" \
+	-d "netmaker-exporter.$NM_DOMAIN" \
+	-d "grafana.$NM_DOMAIN" \
+	-d "prometheus.$NM_DOMAIN"
+EOF
+)
+
+# generate an entrypoint for zerossl-certbot
+cat <<EOF >"$SCRIPT_DIR/certbot-entry.sh"
+#!/bin/sh
+# deps
+apk add bash curl
+# zerossl
+wget -qO zerossl-bot.sh "https://github.com/zerossl/zerossl-bot/raw/master/zerossl-bot.sh"
+chmod +x zerossl-bot.sh
+# request the certs
+./zerossl-bot.sh "$CERTBOT_PARAMS"
+EOF
+chmod +x certbot-entry.sh
+
+# request certs
+sudo docker run -it --rm --name certbot \
+	-p 80:80 -p 443:443 \
+	-v "$SCRIPT_DIR/certbot-entry.sh:/opt/certbot/certbot-entry.sh" \
+	-v "$SCRIPT_DIR/letsencrypt:/etc/letsencrypt" \
+	--entrypoint "/opt/certbot/certbot-entry.sh" \
+	certbot/certbot
+
+# clean up
+rm "$SCRIPT_DIR/certbot-entry.sh"
+
+# check if successful
+if [ ! -f "$CERT_DIR"/fullchain.pem ]; then
+	# fallback to letsencrypt-certbot
+	sudo docker run -it --rm --name certbot \
+		-p 80:80 -p 443:443 \
+		-v "$SCRIPT_DIR/letsencrypt:/etc/letsencrypt" \
+		--entrypoint "/opt/certbot/certbot-entry.sh" \
+		certbot/certbot "$CERTBOT_PARAMS"
+	if [ ! -f "$CERT_DIR"/fullchain.pem ]; then
+		echo "Missing file: $CERT_DIR/fullchain.pem"
+		echo "SSL certificates failed"
+		exit 1
+	fi
+fi
+
+# copy for mounting
+mkdir -p certs
+cp -L "$CERT_DIR/fullchain.pem" "$SCRIPT_DIR/certs/fullchain.pem"
+cp -L "$CERT_DIR/privkey.pem" "$SCRIPT_DIR/certs/privkey.pem"
+
+echo "SSL certificates ready"
+
+# preserve the env state
+if [ "$RESTART_CADDY" = true ]; then
+	echo "Starting Caddy..."
+	docker-compose -f /root/docker-compose.yml start caddy
+fi
+
+# install crontab
+ln -sfn "$SCRIPT_DIR"/nm-certs.sh /etc/cron.monthly/nm-certs.sh

+ 430 - 289
scripts/nm-quick.sh

@@ -1,29 +1,20 @@
 #!/bin/bash
 
+CONFIG_FILE=netmaker.env
+# location of nm-quick.sh (usually `/root`)
+SCRIPT_DIR=$(dirname "$(realpath "$0")")
+CONFIG_PATH="$SCRIPT_DIR/$CONFIG_FILE"
 LATEST=$(curl -s https://api.github.com/repos/gravitl/netmaker/releases/latest | grep "tag_name" | cut -d : -f 2,3 | tr -d [:space:],\")
 
-print_logo() {(
-cat << "EOF"
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-                                                                                         
- __   __     ______     ______   __    __     ______     __  __     ______     ______    
-/\ "-.\ \   /\  ___\   /\__  _\ /\ "-./  \   /\  __ \   /\ \/ /    /\  ___\   /\  == \   
-\ \ \-.  \  \ \  __\   \/_/\ \/ \ \ \-./\ \  \ \  __ \  \ \  _"-.  \ \  __\   \ \  __<   
- \ \_\\"\_\  \ \_____\    \ \_\  \ \_\ \ \_\  \ \_\ \_\  \ \_\ \_\  \ \_____\  \ \_\ \_\ 
-  \/_/ \/_/   \/_____/     \/_/   \/_/  \/_/   \/_/\/_/   \/_/\/_/   \/_____/   \/_/ /_/ 
-                                                                                                                                                                                                 
-
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
-EOF
-)}
-
 if [ $(id -u) -ne 0 ]; then
-   echo "This script must be run as root"
-   exit 1
+	echo "This script must be run as root"
+	exit 1
+fi
+
+# read the config file
+if [ -f "$CONFIG_PATH" ]; then
+	echo "Reading config from $CONFIG_PATH"
+	source "$CONFIG_PATH"
 fi
 
 unset INSTALL_TYPE
@@ -33,53 +24,52 @@ unset IMAGE_TAG
 unset AUTO_BUILD
 
 # usage - displays usage instructions
-usage () {
-    echo "usage: ./nm-quick.sh [-e] [-b buildtype] [-t tag] [-a auto]"
-    echo "  -e      if specified, will install netmaker EE"
-    echo "  -b      type of build; options:"
+usage() {
+	echo "usage: ./nm-quick.sh [-e] [-b buildtype] [-t tag] [-a auto]"
+	echo "  -e      if specified, will install netmaker EE"
+	echo "  -b      type of build; options:"
 	echo "          \"version\" - will install a specific version of Netmaker using remote git and dockerhub"
-	echo "          \"local\": - will install by cloning repo and and building images from git"
+	echo "          \"local\": - will install by cloning repo and building images from git"
 	echo "          \"branch\": - will install a specific branch using remote git and dockerhub"
-    echo "  -t      tag of build; if buildtype=version, tag=version. If builtype=branch or builtype=local, tag=branch"
-    echo "  -a      auto-build; skip prompts and use defaults, if none provided"
-    echo "examples:"
+	echo "  -t      tag of build; if buildtype=version, tag=version. If builtype=branch or builtype=local, tag=branch"
+	echo "  -a      auto-build; skip prompts and use defaults, if none provided"
+	echo "examples:"
 	echo "          nm-quick.sh -e -b version -t $LATEST"
-	echo "          nm-quick.sh -e -b local -t feature_v0.17.2_newfeature"	
+	echo "          nm-quick.sh -e -b local -t feature_v0.17.2_newfeature"
 	echo "          nm-quick.sh -e -b branch -t develop"
-    exit 1
+	exit 1
 }
 
-while getopts evab:t: flag
-do
+while getopts evab:t: flag; do
 	case "${flag}" in
-		e) 
-			INSTALL_TYPE="ee"
-			;;
-		v) 
+	e)
+		INSTALL_TYPE="ee"
+		;;
+	v)
+		usage
+		exit 0
+		;;
+	a)
+		AUTO_BUILD="on"
+		;;
+	b)
+		BUILD_TYPE=${OPTARG}
+		if [[ ! "$BUILD_TYPE" =~ ^(version|local|branch)$ ]]; then
+			echo "error: $BUILD_TYPE is invalid"
+			echo "valid options: version, local, branch"
 			usage
-			exit 0
-			;;
-		a) 
-			AUTO_BUILD="on"
-			;;			
-		b) 
-			BUILD_TYPE=${OPTARG}
-			if [[ ! "$BUILD_TYPE" =~ ^(version|local|branch)$ ]]; then
-				echo "error: $BUILD_TYPE is invalid"
-				echo "valid options: version, local, branch"
-				usage
-				exit 1
-			fi
-			;;
-		t) 
-			BUILD_TAG=${OPTARG}
-			;;
+			exit 1
+		fi
+		;;
+	t)
+		BUILD_TAG=${OPTARG}
+		;;
 	esac
 done
 
 # print_logo - prints the netmaker logo
 print_logo() {
-cat << "EOF"
+	cat <<"EOF"
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
@@ -111,11 +101,11 @@ set_buildinfo() {
 
 	if [ -z "$BUILD_TAG" ] && [ ! -z "$BUILD_TYPE" ]; then
 		echo "error: must specify build tag when build type \"$BUILD_TYPE\" is specified"
-		usage		
+		usage
 		exit 1
 	fi
 
-	IMAGE_TAG=$(sed 's/\//-/g' <<< "$BUILD_TAG")
+	IMAGE_TAG=$(sed 's/\//-/g' <<<"$BUILD_TAG")
 
 	if [ "$1" = "ce" ]; then
 		INSTALL_TYPE="ce"
@@ -131,48 +121,48 @@ set_buildinfo() {
 		echo "EE will require you to create an account at https://dashboard.license.netmaker.io"
 		echo "-----------------------------------------------------"
 		select install_option in "Community Edition" "Enterprise Edition"; do
-		case $REPLY in
+			case $REPLY in
 			1)
-			echo "installing Netmaker CE"
-			INSTALL_TYPE="ce"
-			break
-			;;      
+				echo "installing Netmaker CE"
+				INSTALL_TYPE="ce"
+				break
+				;;
 			2)
-			echo "installing Netmaker EE"
-			INSTALL_TYPE="ee"
-			break
-			;;
-			*) echo "invalid option $REPLY";;
-		esac
+				echo "installing Netmaker EE"
+				INSTALL_TYPE="ee"
+				break
+				;;
+			*) echo "invalid option $REPLY" ;;
+			esac
 		done
 	fi
 	echo "-----------Build Options-----------------------------"
-	echo "    EE or CE: $INSTALL_TYPE";
-	echo "  Build Type: $BUILD_TYPE";
-	echo "   Build Tag: $BUILD_TAG";
-	echo "   Image Tag: $IMAGE_TAG";
+	echo "    EE or CE: $INSTALL_TYPE"
+	echo "  Build Type: $BUILD_TYPE"
+	echo "   Build Tag: $BUILD_TAG"
+	echo "   Image Tag: $IMAGE_TAG"
 	echo "-----------------------------------------------------"
 
 }
 
 # install_yq - install yq if not present
 install_yq() {
-	if ! command -v yq &> /dev/null; then
-		wget -O /usr/bin/yq https://github.com/mikefarah/yq/releases/download/v4.31.1/yq_linux_$(dpkg --print-architecture)
+	if ! command -v yq &>/dev/null; then
+		wget -qO /usr/bin/yq https://github.com/mikefarah/yq/releases/download/v4.31.1/yq_linux_$(dpkg --print-architecture)
 		chmod +x /usr/bin/yq
 	fi
 	set +e
-	if ! command -v yq &> /dev/null; then
+	if ! command -v yq &>/dev/null; then
 		set -e
-		wget -O /usr/bin/yq https://github.com/mikefarah/yq/releases/download/v4.31.1/yq_linux_amd64
+		wget -qO /usr/bin/yq https://github.com/mikefarah/yq/releases/download/v4.31.1/yq_linux_amd64
 		chmod +x /usr/bin/yq
 	fi
 	set -e
-	if ! command -v yq &> /dev/null; then
+	if ! command -v yq &>/dev/null; then
 		echo "failed to install yq. Please install yq and try again."
 		echo "https://github.com/mikefarah/yq/#install"
 		exit 1
-	fi	
+	fi
 }
 
 # setup_netclient - adds netclient to docker-compose
@@ -182,21 +172,44 @@ setup_netclient() {
 	netclient uninstall
 	set -e
 
-	wget -O netclient https://github.com/gravitl/netclient/releases/download/$LATEST/netclient-linux-amd64
+	# TODO arm support
+	wget -qO netclient https://github.com/gravitl/netclient/releases/download/$LATEST/netclient-linux-amd64
 	chmod +x netclient
 	./netclient install
+	echo "Register token: $TOKEN"
 	netclient register -t $TOKEN
 
-	echo "waiting for client to become available"
-	wait_seconds 10 
+	echo "waiting for netclient to become available"
+	local found=false
+	local file=/etc/netclient/nodes.yml
+	for ((a = 1; a <= 90; a++)); do
+		if [ -f "$file" ]; then
+			found=true
+			break
+		fi
+		sleep 1
+	done
+
+	if [ "$found" = false ]; then
+		echo "Error - $file not present"
+		exit 1
+	fi
 }
 
 # configure_netclient - configures server's netclient as a default host and an ingress gateway
 configure_netclient() {
 
 	NODE_ID=$(sudo cat /etc/netclient/nodes.yml | yq -r .netmaker.commonnode.id)
+	if [ "$NODE_ID" = "" ] || [ "$NODE_ID" = "null" ]; then
+		echo "Error obtaining NODE_ID for the new network"
+		exit 1
+	fi
 	echo "register complete. New node ID: $NODE_ID"
 	HOST_ID=$(sudo cat /etc/netclient/netclient.yml | yq -r .host.id)
+	if [ "$HOST_ID" = "" ] || [ "$HOST_ID" = "null" ]; then
+		echo "Error obtaining HOST_ID for the new network"
+		exit 1
+	fi
 	echo "making host a default"
 	echo "Host ID: $HOST_ID"
 	# set as a default host
@@ -210,69 +223,162 @@ configure_netclient() {
 # setup_nmctl - pulls nmctl and makes it executable
 setup_nmctl() {
 
-	wget -O /usr/bin/nmctl https://github.com/gravitl/netmaker/releases/download/$LATEST/nmctl-linux-amd64
-
-    chmod +x /usr/bin/nmctl
-    echo "using server api.$NETMAKER_BASE_DOMAIN"
-    echo "using master key $MASTER_KEY"
-    nmctl context set default --endpoint="https://api.$NETMAKER_BASE_DOMAIN" --master_key="$MASTER_KEY"
-    nmctl context use default
-    RESP=$(nmctl network list)
-    if [[ $RESP == *"unauthorized"* ]]; then
-        echo "Unable to properly configure NMCTL, exiting..."
-        exit 1
-    fi
+	# TODO arm support
+	local URL="https://github.com/gravitl/netmaker/releases/download/$LATEST/nmctl-linux-amd64"
+	echo "Downloading nmctl..."
+	wget -qO /usr/bin/nmctl "$URL"
+
+	if [ ! -f /usr/bin/nmctl ]; then
+		echo "Error downloading nmctl from '$URL'"
+		exit 1
+	fi
+
+	chmod +x /usr/bin/nmctl
+	echo "using server api.$NETMAKER_BASE_DOMAIN"
+	echo "using master key $MASTER_KEY"
+	nmctl context set default --endpoint="https://api.$NETMAKER_BASE_DOMAIN" --master_key="$MASTER_KEY"
+	nmctl context use default
+	RESP=$(nmctl network list)
+	if [[ $RESP == *"unauthorized"* ]]; then
+		echo "Unable to properly configure NMCTL, exiting..."
+		exit 1
+	fi
 }
 
 # wait_seconds - wait for the specified period of time
-wait_seconds() {(
-  for ((a=1; a <= $1; a++))
-  do
-    echo ". . ."
-    sleep 1
-  done
-)}
+wait_seconds() { (
+	for ((a = 1; a <= $1; a++)); do
+		echo ". . ."
+		sleep 1
+	done
+); }
 
 # confirm - get user input to confirm that they want to perform the next step
-confirm() {(
-  if [ "$AUTO_BUILD" = "on" ]; then
-	return 0
-  fi
-  while true; do
-      read -p 'Does everything look right? [y/n]: ' yn
-      case $yn in
-          [Yy]* ) override="true"; break;;
-          [Nn]* ) echo "exiting..."; exit 1;;
-          * ) echo "Please answer yes or no.";;
-      esac
-  done
-)}
+confirm() { (
+	if [ "$AUTO_BUILD" = "on" ]; then
+		return 0
+	fi
+	while true; do
+		read -p 'Does everything look right? [y/n]: ' yn
+		case $yn in
+		[Yy]*)
+			override="true"
+			break
+			;;
+		[Nn]*)
+			echo "exiting..."
+			exit 1
+			# TODO start from the beginning instead
+			;;
+		*) echo "Please answer yes or no." ;;
+		esac
+	done
+) }
+
+save_config() { (
+	echo "Saving the config to $CONFIG_PATH"
+	touch "$CONFIG_PATH"
+	save_config_item NM_EMAIL "$EMAIL"
+	save_config_item NM_DOMAIN "$NETMAKER_BASE_DOMAIN"
+	save_config_item UI_IMAGE_TAG "$IMAGE_TAG"
+	if [ "$BUILD_TYPE" = "local" ]; then
+		save_config_item UI_IMAGE_TAG "$LATEST"
+	else
+		save_config_item UI_IMAGE_TAG "$IMAGE_TAG"
+	fi
+	# version-specific entries
+	if [ "$INSTALL_TYPE" = "ee" ]; then
+		save_config_item NETMAKER_ACCOUNT_ID "$ACCOUNT_ID"
+		save_config_item LICENSE_KEY "$LICENSE_KEY"
+		save_config_item METRICS_EXPORTER "on"
+		save_config_item PROMETHEUS "on"
+		if [ "$BUILD_TYPE" = "version" ]; then
+			save_config_item SERVER_IMAGE_TAG "$IMAGE_TAG-ee"
+		else
+			save_config_item SERVER_IMAGE_TAG "$IMAGE_TAG"
+		fi
+	else
+		save_config_item METRICS_EXPORTER "off"
+		save_config_item PROMETHEUS "off"
+		save_config_item SERVER_IMAGE_TAG "$IMAGE_TAG"
+	fi
+	# copy entries from the previous config
+	local toCopy=("SERVER_HOST" "MASTER_KEY" "TURN_USERNAME" "MQ_USERNAME" "MQ_PASSWORD"
+		"INSTALL_TYPE" "NODE_ID" "METRICS_EXPORTER" "PROMETHEUS" "DNS_MODE" "NETCLIENT_AUTO_UPDATE" "API_PORT"
+		"CORS_ALLOWED_ORIGIN" "DISPLAY_KEYS" "DATABASE" "SERVER_BROKER_ENDPOINT" "STUN_PORT" "VERBOSITY"
+		"DEFAULT_PROXY_MODE" "TURN_PORT" "USE_TURN" "DEBUG_MODE" "TURN_API_PORT" "REST_BACKEND" "DISABLE_REMOTE_IP_CHECK"
+		"TELEMETRY" "AUTH_PROVIDER" "CLIENT_ID" "CLIENT_SECRET" "FRONTEND_URL" "AZURE_TENANT" "OIDC_ISSUER"
+		"EXPORTER_API_PORT")
+	for name in "${toCopy[@]}"; do
+		save_config_item $name "${!name}"
+	done
+	# preserve debug entries
+	if test -n "$NM_SKIP_BUILD"; then
+		save_config_item NM_SKIP_BUILD "$NM_SKIP_BUILD"
+	fi
+	if test -n "$NM_SKIP_CLONE"; then
+		save_config_item NM_SKIP_CLONE "$NM_SKIP_CLONE"
+	fi
+	if test -n "$NM_SKIP_DEPS"; then
+		save_config_item NM_SKIP_DEPS "$NM_SKIP_DEPS"
+	fi
+); }
+
+save_config_item() { (
+	local NAME="$1"
+	local VALUE="$2"
+	# echo "NAME $NAME"
+	# echo "VALUE $VALUE"
+	if grep -q "^$NAME=" "$CONFIG_PATH"; then
+		# TODO escape | in the value
+		sed -i "s|$NAME=.*|$NAME='$VALUE'|" "$CONFIG_PATH"
+	else
+		echo "$NAME=\"$VALUE\"" >>"$CONFIG_PATH"
+	fi
+); }
 
 # local_install_setup - builds artifacts based on specified branch locally to use in install
-local_install_setup() {(
-	rm -rf netmaker-tmp
-	mkdir netmaker-tmp
-	cd netmaker-tmp
-	git clone https://www.github.com/gravitl/netmaker
+local_install_setup() { (
+	if test -z "$NM_SKIP_CLONE"; then
+		rm -rf netmaker-tmp
+		mkdir netmaker-tmp
+		cd netmaker-tmp
+		git clone --single-branch --depth=1 --branch=$BUILD_TAG https://www.github.com/gravitl/netmaker
+	else
+		cd netmaker-tmp
+		echo "Skipping git clone on NM_SKIP_CLONE"
+	fi
 	cd netmaker
-	git checkout $BUILD_TAG
-	git pull origin $BUILD_TAG
-	docker build --no-cache --build-arg version=$IMAGE_TAG -t gravitl/netmaker:$IMAGE_TAG .
+	if test -z "$NM_SKIP_BUILD"; then
+		docker build --no-cache --build-arg version=$IMAGE_TAG -t gravitl/netmaker:$IMAGE_TAG .
+	else
+		echo "Skipping build on NM_SKIP_BUILD"
+	fi
+	cp compose/docker-compose.yml "$SCRIPT_DIR/docker-compose.yml"
 	if [ "$INSTALL_TYPE" = "ee" ]; then
-		cp compose/docker-compose.ee.yml /root/docker-compose.yml 
-		cp docker/Caddyfile-EE /root/Caddyfile
+		cp compose/docker-compose.ee.yml "$SCRIPT_DIR/docker-compose.override.yml"
+		cp docker/Caddyfile-EE "$SCRIPT_DIR/Caddyfile"
 	else
-		cp compose/docker-compose.yml /root/docker-compose.yml 
-		cp docker/Caddyfile /root/Caddyfile
+		cp docker/Caddyfile "$SCRIPT_DIR/Caddyfile"
 	fi
-	cp docker/mosquitto.conf /root/mosquitto.conf
-	cp docker/wait.sh /root/wait.sh
+	cp scripts/nm-certs.sh "$SCRIPT_DIR/nm-certs.sh"
+	cp scripts/netmaker.env "$SCRIPT_DIR/netmaker.env"
+	ln -fs "$SCRIPT_DIR/netmaker.env" "$SCRIPT_DIR/.env"
+	cp docker/mosquitto.conf "$SCRIPT_DIR/mosquitto.conf"
+	cp docker/wait.sh "$SCRIPT_DIR/wait.sh"
 	cd ../../
-	rm -rf netmaker-tmp
-)}
+	if test -z "$NM_SKIP_CLONE"; then
+		rm -rf netmaker-tmp
+	fi
+); }
 
-# install_dependencies - install necessary packages to run netmaker 
+# install_dependencies - install necessary packages to run netmaker
 install_dependencies() {
+
+	if test -n "$NM_SKIP_DEPS"; then
+		return
+	fi
+
 	echo "checking dependencies..."
 
 	OS=$(uname)
@@ -297,29 +403,21 @@ install_dependencies() {
 		update_cmd='yum update'
 		install_cmd='yum install -y'
 	elif [ -f /etc/arch-release ]; then
-			dependecies="git wireguard-tools dnsutils jq docker.io docker-compose"
+		dependencies="git wireguard-tools dnsutils jq docker.io docker-compose"
 		update_cmd='pacman -Sy'
 		install_cmd='pacman -S --noconfirm'
 	elif [ "${OS}" = "FreeBSD" ]; then
 		dependencies="git wireguard wget jq docker.io docker-compose"
 		update_cmd='pkg update'
 		install_cmd='pkg install -y'
-	elif [ -f /etc/turris-version ]; then
-		dependencies="git wireguard-tools bash jq docker.io docker-compose"
-		OS="TurrisOS"
-		update_cmd='opkg update'	
-		install_cmd='opkg install'
-	elif [ -f /etc/openwrt_release ]; then
-		dependencies="git wireguard-tools bash jq docker.io docker-compose"
-		OS="OpenWRT"
-		update_cmd='opkg update'	
-		install_cmd='opkg install'
 	else
 		install_cmd=''
 	fi
 
 	if [ -z "${install_cmd}" ]; then
-			echo "OS unsupported for automatic dependency install"
+		echo "OS unsupported for automatic dependency install"
+		# TODO shouldnt exit, check if deps available, if not
+		#  ask the user to install manually and continue when ready
 		exit 1
 	fi
 
@@ -345,7 +443,7 @@ install_dependencies() {
 					echo "  " FAILED TO INSTALL $1
 					echo "  " This may break functionality.
 				fi
-			fi	
+			fi
 		else
 			if [ "${OS}" = "OpenWRT" ] || [ "${OS}" = "TurrisOS" ]; then
 				is_installed=$(opkg list-installed $1 | grep $1)
@@ -379,7 +477,7 @@ install_dependencies() {
 	echo "-----------------------------------------------------"
 	echo "dependency check complete"
 	echo "-----------------------------------------------------"
-} 
+}
 set -e
 
 # set_install_vars - sets the variables that will be used throughout installation
@@ -391,36 +489,38 @@ set_install_vars() {
 	fi
 
 	NETMAKER_BASE_DOMAIN=nm.$(echo $IP_ADDR | tr . -).nip.io
-	COREDNS_IP=$(ip route get 1 | sed -n 's/^.*src \([0-9.]*\) .*$/\1/p')
-	SERVER_PUBLIC_IP=$IP_ADDR
-	MASTER_KEY=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 30 ; echo '')
+	SERVER_HOST=$IP_ADDR
+	MASTER_KEY=$(
+		tr -dc A-Za-z0-9 </dev/urandom | head -c 30
+		echo ''
+	)
 	DOMAIN_TYPE=""
 	echo "-----------------------------------------------------"
 	echo "Would you like to use your own domain for netmaker, or an auto-generated domain?"
-	echo "To use your own domain, add a Wildcard DNS record (e.x: *.netmaker.example.com) pointing to $SERVER_PUBLIC_IP"
+	echo "To use your own domain, add a Wildcard DNS record (e.x: *.netmaker.example.com) pointing to $SERVER_HOST"
 	echo "IMPORTANT: Due to the high volume of requests, the auto-generated domain has been rate-limited by the certificate provider."
 	echo "For this reason, we STRONGLY RECOMMEND using your own domain. Using the auto-generated domain may lead to a failed installation due to rate limiting."
 	echo "-----------------------------------------------------"
 
 	if [ "$AUTO_BUILD" = "on" ]; then
-			DOMAIN_TYPE="auto"
+		DOMAIN_TYPE="auto"
 	else
 		select domain_option in "Auto Generated ($NETMAKER_BASE_DOMAIN)" "Custom Domain (e.x: netmaker.example.com)"; do
-		case $REPLY in
+			case $REPLY in
 			1)
-			echo "using $NETMAKER_BASE_DOMAIN for base domain"
-			DOMAIN_TYPE="auto"
-			break
-			;;      
+				echo "using $NETMAKER_BASE_DOMAIN for base domain"
+				DOMAIN_TYPE="auto"
+				break
+				;;
 			2)
-			read -p "Enter Custom Domain (make sure  *.domain points to $SERVER_PUBLIC_IP first): " domain
-			NETMAKER_BASE_DOMAIN=$domain
-			echo "using $NETMAKER_BASE_DOMAIN"
-			DOMAIN_TYPE="custom"
-			break
-			;;
-			*) echo "invalid option $REPLY";;
-		esac
+				read -p "Enter Custom Domain (make sure  *.domain points to $SERVER_HOST first): " domain
+				NETMAKER_BASE_DOMAIN=$domain
+				echo "using $NETMAKER_BASE_DOMAIN"
+				DOMAIN_TYPE="custom"
+				break
+				;;
+			*) echo "invalid option $REPLY" ;;
+			esac
 		done
 	fi
 
@@ -433,7 +533,7 @@ set_install_vars() {
 	echo "             broker.$NETMAKER_BASE_DOMAIN"
 	echo "               stun.$NETMAKER_BASE_DOMAIN"
 	echo "               turn.$NETMAKER_BASE_DOMAIN"
-	echo "               turnapi.$NETMAKER_BASE_DOMAIN"
+	echo "            turnapi.$NETMAKER_BASE_DOMAIN"
 
 	if [ "$INSTALL_TYPE" = "ee" ]; then
 		echo "         prometheus.$NETMAKER_BASE_DOMAIN"
@@ -444,7 +544,7 @@ set_install_vars() {
 	echo "-----------------------------------------------------"
 
 	if [[ "$DOMAIN_TYPE" == "custom" ]]; then
-		echo "before continuing, confirm DNS is configured correctly, with records pointing to $SERVER_PUBLIC_IP"
+		echo "before continuing, confirm DNS is configured correctly, with records pointing to $SERVER_HOST"
 		confirm
 	fi
 
@@ -471,15 +571,21 @@ set_install_vars() {
 
 	unset GET_EMAIL
 	unset RAND_EMAIL
-	RAND_EMAIL="$(echo $RANDOM | md5sum  | head -c 16)@email.com"
+	RAND_EMAIL="$(echo $RANDOM | md5sum | head -c 16)@email.com"
+	# suggest the prev email or a random one
+	EMAIL_SUGGESTED=${NM_EMAIL:-$RAND_EMAIL}
 	if [ -z $AUTO_BUILD ]; then
-		read -p "Email Address for Domain Registration (click 'enter' to use $RAND_EMAIL): " GET_EMAIL
+		read -p "Email Address for Domain Registration (click 'enter' to use $EMAIL_SUGGESTED): " GET_EMAIL
 	fi
 	if [ -z "$GET_EMAIL" ]; then
-	echo "using rand email"
-	EMAIL="$RAND_EMAIL"
+		EMAIL="$EMAIL_SUGGESTED"
+		if [ "$EMAIL" = "$NM_EMAIL" ]; then
+			echo "using config email"
+		else
+			echo "using rand email"
+		fi
 	else
-	EMAIL="$GET_EMAIL"
+		EMAIL="$GET_EMAIL"
 	fi
 
 	wait_seconds 1
@@ -492,40 +598,42 @@ set_install_vars() {
 		read -p "MQ Username (click 'enter' to use 'netmaker'): " GET_MQ_USERNAME
 	fi
 	if [ -z "$GET_MQ_USERNAME" ]; then
-	echo "using default username for mq"
-	MQ_USERNAME="netmaker"
+		echo "using default username for mq"
+		MQ_USERNAME="netmaker"
 	else
-	MQ_USERNAME="$GET_MQ_USERNAME"
+		MQ_USERNAME="$GET_MQ_USERNAME"
 	fi
 
-	MQ_PASSWORD=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 30 ; echo '')
+	MQ_PASSWORD=$(
+		tr -dc A-Za-z0-9 </dev/urandom | head -c 30
+		echo ''
+	)
 
-	if [ -z $AUTO_BUILD ]; then  
+	if [ -z $AUTO_BUILD ]; then
 		select domain_option in "Auto Generated Password" "Input Your Own Password"; do
 			case $REPLY in
 			1)
-			echo "using random password for mq"
-			break
-			;;      
+				echo "using random password for mq"
+				break
+				;;
 			2)
-			while true
-			do
-				echo "Enter your Password For MQ: " 
-				read -s GET_MQ_PASSWORD
-				echo "Enter your password again to confirm: "
-				read -s CONFIRM_MQ_PASSWORD
-				if [ ${GET_MQ_PASSWORD} != ${CONFIRM_MQ_PASSWORD} ]; then
-					echo "wrong password entered, try again..."
-					continue
-				fi
-				MQ_PASSWORD="$GET_MQ_PASSWORD"
-				echo "MQ Password Saved Successfully!!"
+				while true; do
+					echo "Enter your Password For MQ: "
+					read -s GET_MQ_PASSWORD
+					echo "Enter your password again to confirm: "
+					read -s CONFIRM_MQ_PASSWORD
+					if [ ${GET_MQ_PASSWORD} != ${CONFIRM_MQ_PASSWORD} ]; then
+						echo "wrong password entered, try again..."
+						continue
+					fi
+					MQ_PASSWORD="$GET_MQ_PASSWORD"
+					echo "MQ Password Saved Successfully!!"
+					break
+				done
 				break
-			done
-			break
-			;;
-			*) echo "invalid option $REPLY";;
-		esac
+				;;
+			*) echo "invalid option $REPLY" ;;
+			esac
 		done
 	fi
 
@@ -537,40 +645,42 @@ set_install_vars() {
 		read -p "TURN Username (click 'enter' to use 'netmaker'): " GET_TURN_USERNAME
 	fi
 	if [ -z "$GET_TURN_USERNAME" ]; then
-	echo "using default username for mq"
-	TURN_USERNAME="netmaker"
+		echo "using default username for TURN"
+		TURN_USERNAME="netmaker"
 	else
-	TURN_USERNAME="$GET_TURN_USERNAME"
+		TURN_USERNAME="$GET_TURN_USERNAME"
 	fi
 
-	TURN_PASSWORD=$(tr -dc A-Za-z0-9 </dev/urandom | head -c 30 ; echo '')
+	TURN_PASSWORD=$(
+		tr -dc A-Za-z0-9 </dev/urandom | head -c 30
+		echo ''
+	)
 
-	if [ -z $AUTO_BUILD ]; then  
+	if [ -z $AUTO_BUILD ]; then
 		select domain_option in "Auto Generated Password" "Input Your Own Password"; do
 			case $REPLY in
 			1)
-			echo "using random password for turn"
-			break
-			;;      
+				echo "using random password for turn"
+				break
+				;;
 			2)
-			while true
-			do
-				echo "Enter your Password For TURN: " 
-				read -s GET_TURN_PASSWORD
-				echo "Enter your password again to confirm: "
-				read -s CONFIRM_TURN_PASSWORD
-				if [ ${GET_TURN_PASSWORD} != ${CONFIRM_TURN_PASSWORD} ]; then
-					echo "wrong password entered, try again..."
-					continue
-				fi
-				TURN_PASSWORD="$GET_TURN_PASSWORD"
-				echo "TURN Password Saved Successfully!!"
+				while true; do
+					echo "Enter your Password For TURN: "
+					read -s GET_TURN_PASSWORD
+					echo "Enter your password again to confirm: "
+					read -s CONFIRM_TURN_PASSWORD
+					if [ ${GET_TURN_PASSWORD} != ${CONFIRM_TURN_PASSWORD} ]; then
+						echo "wrong password entered, try again..."
+						continue
+					fi
+					TURN_PASSWORD="$GET_TURN_PASSWORD"
+					echo "TURN Password Saved Successfully!!"
+					break
+				done
 				break
-			done
-			break
-			;;
-			*) echo "invalid option $REPLY";;
-		esac
+				;;
+			*) echo "invalid option $REPLY" ;;
+			esac
 		done
 	fi
 
@@ -581,7 +691,7 @@ set_install_vars() {
 	echo "-----------------------------------------------------------------"
 	echo "        domain: $NETMAKER_BASE_DOMAIN"
 	echo "         email: $EMAIL"
-	echo "     public ip: $SERVER_PUBLIC_IP"
+	echo "     public ip: $SERVER_HOST"
 	if [ "$INSTALL_TYPE" = "ee" ]; then
 		echo "       license: $LICENSE_KEY"
 		echo "    account id: $ACCOUNT_ID"
@@ -590,8 +700,11 @@ set_install_vars() {
 	echo "Confirm Settings for Installation"
 	echo "- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -"
 
-	confirm
+	if [ ! "$BUILD_TYPE" = "local" ]; then
+		IMAGE_TAG="$LATEST"
+	fi
 
+	confirm
 }
 
 # install_netmaker - sets the config files and starts docker-compose
@@ -605,53 +718,45 @@ install_netmaker() {
 
 	echo "Pulling config files..."
 
+	if [ "$BUILD_TYPE" = "local" ]; then
+		local_install_setup
+	else
+		local BASE_URL="https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG"
 
-	COMPOSE_URL="https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG/compose/docker-compose.yml" 
-	CADDY_URL="https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG/docker/Caddyfile"
-	if [ "$INSTALL_TYPE" = "ee" ]; then
-		COMPOSE_URL="https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG/compose/docker-compose.ee.yml" 
-		CADDY_URL="https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG/docker/Caddyfile-EE"
-	fi
-	if [ ! "$BUILD_TYPE" = "local" ]; then
-		wget -O /root/docker-compose.yml $COMPOSE_URL && wget -O /root/mosquitto.conf https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG/docker/mosquitto.conf && wget -O /root/Caddyfile $CADDY_URL
-		wget -O /root/wait.sh https://raw.githubusercontent.com/gravitl/netmaker/$BUILD_TAG/docker/wait.sh
+		local COMPOSE_URL="$BASE_URL/compose/docker-compose.yml"
+		local CADDY_URL="$BASE_URL/docker/Caddyfile"
+		if [ "$INSTALL_TYPE" = "ee" ]; then
+			local COMPOSE_OVERRIDE_URL="$BASE_URL/compose/docker-compose.ee.yml"
+			local CADDY_URL="$BASE_URL/docker/Caddyfile-EE"
+		fi
+		wget -qO "$SCRIPT_DIR"/docker-compose.yml $COMPOSE_URL
+		if test -n "$COMPOSE_OVERRIDE_URL"; then
+			wget -qO "$SCRIPT_DIR"/docker-compose.override.yml $COMPOSE_OVERRIDE_URL
+		fi
+		wget -qO "$SCRIPT_DIR"/Caddyfile "$CADDY_URL"
+		wget -qO "$SCRIPT_DIR"/netmaker.env "$BASE_URL/scripts/netmaker.env"
+		ln -fs "$SCRIPT_DIR/netmaker.env" "$SCRIPT_DIR/.env"
+		wget -qO "$SCRIPT_DIR"/mosquitto.conf "$BASE_URL/docker/mosquitto.conf"
+		wget -qO "$SCRIPT_DIR"/nm-certs.sh "$BASE_URL/scripts/nm-certs.sh"
+		wget -qO "$SCRIPT_DIR"/wait.sh "$BASE_URL/docker/wait.sh"
 	fi
 
-	chmod +x /root/wait.sh
+	chmod +x "$SCRIPT_DIR"/wait.sh
 	mkdir -p /etc/netmaker
 
-	echo "Setting docker-compose and Caddyfile..."
+	save_config
 
-	sed -i "s/SERVER_PUBLIC_IP/$SERVER_PUBLIC_IP/g" /root/docker-compose.yml
-	sed -i "s/NETMAKER_BASE_DOMAIN/$NETMAKER_BASE_DOMAIN/g" /root/Caddyfile
-	sed -i "s/NETMAKER_BASE_DOMAIN/$NETMAKER_BASE_DOMAIN/g" /root/docker-compose.yml
-	sed -i "s/REPLACE_MASTER_KEY/$MASTER_KEY/g" /root/docker-compose.yml
-	sed -i "s/YOUR_EMAIL/$EMAIL/g" /root/Caddyfile
-	sed -i "s/REPLACE_MQ_USERNAME/$MQ_USERNAME/g" /root/docker-compose.yml 
-	sed -i "s/REPLACE_MQ_PASSWORD/$MQ_PASSWORD/g" /root/docker-compose.yml
-	sed -i "s/REPLACE_TURN_USERNAME/$TURN_USERNAME/g" /root/docker-compose.yml 
-	sed -i "s/REPLACE_TURN_PASSWORD/$TURN_PASSWORD/g" /root/docker-compose.yml
-
-	if [ "$INSTALL_TYPE" = "ee" ]; then
-		sed -i "s~YOUR_LICENSE_KEY~$LICENSE_KEY~g" /root/docker-compose.yml
-		sed -i "s/YOUR_ACCOUNT_ID/$ACCOUNT_ID/g" /root/docker-compose.yml
-	fi
-
-	if [ "$BUILD_TYPE" = "version" ] && [ "$INSTALL_TYPE" = "ee" ]; then
-		sed -i "s/REPLACE_SERVER_IMAGE_TAG/$IMAGE_TAG-ee/g" /root/docker-compose.yml
-	else
-		sed -i "s/REPLACE_SERVER_IMAGE_TAG/$IMAGE_TAG/g" /root/docker-compose.yml
-	fi
-
-	if [ "$BUILD_TYPE" = "local" ]; then
-		sed -i "s/REPLACE_UI_IMAGE_TAG/$LATEST/g" /root/docker-compose.yml
-	else
-		sed -i "s/REPLACE_UI_IMAGE_TAG/$IMAGE_TAG/g" /root/docker-compose.yml
-	fi
+	# Fetch / update certs using certbot
+	"$SCRIPT_DIR"/nm-certs.sh
 
 	echo "Starting containers..."
 
-	docker-compose -f /root/docker-compose.yml up -d
+	# increase the timeouts
+	export DOCKER_CLIENT_TIMEOUT=120
+	export COMPOSE_HTTP_TIMEOUT=120
+
+	# start docker and rebuild containers / networks
+	docker-compose -f "$SCRIPT_DIR"/docker-compose.yml up -d --force-recreate
 
 	wait_seconds 2
 
@@ -661,25 +766,24 @@ install_netmaker() {
 test_connection() {
 
 	echo "Testing Caddy setup (please be patient, this may take 1-2 minutes)"
-	for i in 1 2 3 4 5 6 7 8
-	do
-	curlresponse=$(curl -vIs https://api.${NETMAKER_BASE_DOMAIN} 2>&1)
-
-	if [[ "$i" == 8 ]]; then
-	echo "    Caddy is having an issue setting up certificates, please investigate (docker logs caddy)"
-	echo "    Exiting..."
-	exit 1
-	elif [[ "$curlresponse" == *"failed to verify the legitimacy of the server"* ]]; then
-	echo "    Certificates not yet configured, retrying..."
-
-	elif [[ "$curlresponse" == *"left intact"* ]]; then
-	echo "    Certificates ok"
-	break
-	else
-	secs=$(($i*5+10))
-	echo "    Issue establishing connection...retrying in $secs seconds..."       
-	fi
-	sleep $secs
+	for i in 1 2 3 4 5 6 7 8; do
+		curlresponse=$(curl -vIs https://api.${NETMAKER_BASE_DOMAIN} 2>&1)
+
+		if [[ "$i" == 8 ]]; then
+			echo "    Caddy is having an issue setting up certificates, please investigate (docker logs caddy)"
+			echo "    Exiting..."
+			exit 1
+		elif [[ "$curlresponse" == *"failed to verify the legitimacy of the server"* ]]; then
+			echo "    Certificates not yet configured, retrying..."
+
+		elif [[ "$curlresponse" == *"left intact"* ]]; then
+			echo "    Certificates ok"
+			break
+		else
+			secs=$(($i * 5 + 10))
+			echo "    Issue establishing connection...retrying in $secs seconds..."
+		fi
+		sleep $secs
 	done
 
 }
@@ -689,16 +793,28 @@ setup_mesh() {
 
 	wait_seconds 5
 
-	echo "Creating netmaker network (10.101.0.0/16)"
+	local networkCount=$(nmctl network list -o json | jq '. | length')
 
-	nmctl network create --name netmaker --ipv4_addr 10.101.0.0/16
+	# add a network if none present
+	if [ "$networkCount" -lt 1 ]; then
+		echo "Creating netmaker network (10.101.0.0/16)"
 
-	wait_seconds 5
+		# TODO causes "Error Status: 400 Response: {"Code":400,"Message":"could not find any records"}"
+		nmctl network create --name netmaker --ipv4_addr 10.101.0.0/16
+
+		wait_seconds 5
+	fi
 
-	echo "Creating netmaker enrollment key"
+	echo "Obtaining a netmaker enrollment key..."
 
-	tokenJson=$(nmctl enrollment_key create --unlimited --networks netmaker)
-	TOKEN=$(jq -r '.token' <<< ${tokenJson})
+	local tokenJson=$(nmctl enrollment_key create --unlimited --networks netmaker)
+	TOKEN=$(jq -r '.token' <<<${tokenJson})
+	if test -z "$TOKEN"; then
+		echo "Error creating an enrollment key"
+		exit 1
+	else
+		echo "Enrollment key ready"
+	fi
 
 	wait_seconds 3
 
@@ -714,6 +830,32 @@ print_success() {
 	echo "-----------------------------------------------------------------"
 }
 
+cleanup() {
+	# remove the existing netclient's instance from the existing network
+	if command -v nmctl >/dev/null 2>&1; then
+		local node_id=$(netclient list | jq '.[0].node_id' 2>/dev/null)
+		# trim doublequotes
+		node_id="${node_id//\"/}"
+		if test -n "$node_id"; then
+			echo "De-registering the existing netclient..."
+			nmctl node delete netmaker $node_id >/dev/null 2>&1
+		fi
+	fi
+
+	echo "Stopping all containers..."
+	local containers=("mq" "netmaker-ui" "coredns" "turn" "caddy" "netmaker" "netmaker-exporter" "prometheus" "grafana")
+	for name in "${containers[@]}"; do
+		local running=$(docker ps | grep -w "$name")
+		local exists=$(docker ps -a | grep -w "$name")
+		if test -n "$running"; then
+			docker stop "$name" 1>/dev/null
+		fi
+		if test -n "$exists"; then
+			docker rm "$name" 1>/dev/null
+		fi
+	done
+}
+
 # 1. print netmaker logo
 print_logo
 
@@ -728,16 +870,15 @@ install_dependencies
 # 4. install yq if necessary
 install_yq
 
-# 5. if running a local build, clone git and build artifacts
-if [ "$BUILD_TYPE" = "local" ]; then
-	local_install_setup
-fi
-
 set -e
 
 # 6. get user input for variables
 set_install_vars
 
+set +e
+cleanup
+set -e
+
 # 7. get and set config files, startup docker-compose
 install_netmaker
 

+ 2 - 1
scripts/nm-upgrade-0-17-1-to-0-19-0.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-LATEST="v0.19.0"
+LATEST="v0.20.0"
 INSTALL_PATH="/root"
 
 trap restore_old_netmaker_instructions
@@ -27,6 +27,7 @@ backup_v17_files() {
 }
 
 backup_volumes() {
+  # TODO backup to /root/nm-backup
   cp -r /var/lib/docker/volumes/root_caddy_conf/ /var/lib/docker/volumes/root_caddy_conf-backup/
   cp -r /var/lib/docker/volumes/root_caddy_data/ /var/lib/docker/volumes/root_caddy_data-backup/
   cp -r /var/lib/docker/volumes/root_dnsconfig/ /var/lib/docker/volumes/root_dnsconfig-backup/

+ 1 - 1
swagger.yaml

@@ -704,7 +704,7 @@ info:
 
         API calls must be authenticated via a header of the format -H “Authorization: Bearer <YOUR_SECRET_KEY>” There are two methods to obtain YOUR_SECRET_KEY: 1. Using the masterkey. By default, this value is “secret key,” but you should change this on your instance and keep it secure. This value can be set via env var at startup or in a config file (config/environments/< env >.yaml). See the [Netmaker](https://docs.netmaker.org/index.html) documentation for more details. 2. Using a JWT received for a node. This can be retrieved by calling the /api/nodes/<network>/authenticate endpoint, as documented below.
     title: Netmaker
-    version: 0.19.0
+    version: 0.20.0
 paths:
     /api/dns:
         get: