Fix Github actions duplicate builds

We were building for any commits AND pull requests. This change makes it
so we only build for commits to master AND pull requests.

.github/workflows/test.yml

@@ -1,5 +1,9 @@
 name: Build and test
-on: [push, pull_request]
+on:
+  push:
+    branches:
+      - master
+  pull_request:
 jobs:
 
   test-linux: