Browse Source

SCons: Default `num_jobs` to max CPUs minus 1 if not specified

This doesn't change the behavior when `--jobs`/`-j` is specified as a
command-line argument or in `SCONSFLAGS`.

The SCons hack used to know if `num_jobs` was set by the user is derived
from the MongoDB setup.

We use `os.cpu_count()` for portability (available since Python 3.4).

With 4 CPUs or less, we use the max. With more than 4 we use max - 1 to
preserve some bandwidth for the user's other programs.
Rémi Verschelde 3 years ago
parent
commit
ea21122575
3 changed files with 20 additions and 2 deletions
  1. 1 1
      .github/actions/godot-build/action.yml
  2. 1 1
      .github/workflows/linux_builds.yml
  3. 18 0
      SConstruct

+ 1 - 1
.github/actions/godot-build/action.yml

@@ -35,5 +35,5 @@ runs:
       run: |
         echo "Building with flags:" ${{ env.SCONSFLAGS }}
         if ! ${{ inputs.tools }}; then rm -rf editor; fi  # Ensure we don't include editor code.
-        scons p=${{ inputs.platform }} target=${{ inputs.target }} tools=${{ inputs.tools }} tests=${{ inputs.tests }} --jobs=2 ${{ env.SCONSFLAGS }}
+        scons p=${{ inputs.platform }} target=${{ inputs.target }} tools=${{ inputs.tools }} tests=${{ inputs.tests }} ${{ env.SCONSFLAGS }}
         ls -l bin/

+ 1 - 1
.github/workflows/linux_builds.yml

@@ -212,7 +212,7 @@ jobs:
         if: ${{ matrix.godot-cpp-test }}
         run: |
           cd godot-cpp/test
-          scons target=${{ matrix.target }} -j2
+          scons target=${{ matrix.target }}
           cd ../..
 
       - name: Prepare artifact

+ 18 - 0
SConstruct

@@ -399,6 +399,24 @@ if selected_platform in platform_list:
 
     env = env_base.Clone()
 
+    # Default num_jobs to local cpu count if not user specified.
+    # SCons has a peculiarity where user-specified options won't be overridden
+    # by SetOption, so we can rely on this to know if we should use our default.
+    initial_num_jobs = env.GetOption("num_jobs")
+    altered_num_jobs = initial_num_jobs + 1
+    env.SetOption("num_jobs", altered_num_jobs)
+    if env.GetOption("num_jobs") == altered_num_jobs:
+        cpu_count = os.cpu_count()
+        if cpu_count is None:
+            print("Couldn't auto-detect CPU count to configure build parallelism. Specify it with the -j argument.")
+        else:
+            safer_cpu_count = cpu_count if cpu_count <= 4 else cpu_count - 1
+            print(
+                "Auto-detected %d CPU cores available for build parallelism. Using %d cores by default. You can override it with the -j argument."
+                % (cpu_count, safer_cpu_count)
+            )
+            env.SetOption("num_jobs", safer_cpu_count)
+
     if env["compiledb"]:
         # Generating the compilation DB (`compile_commands.json`) requires SCons 4.0.0 or later.
         from SCons import __version__ as scons_raw_version