o3de CLI enable-gem dependency resolution (#14885)

* compatibility checks multiple gem versions

includes unit tests

Signed-off-by: Alex Peterson <[email protected]>

* get most compatible object, fix manifest test

- manifest.get_registered() will now return the most compatible object by name with optional version specifier
- get_gems_json_data_by_name return format changed and the test needs to be updated

Signed-off-by: Alex Peterson <[email protected]>

* enable_gems accepts names with version specifiers

Signed-off-by: Alex Peterson <[email protected]>

* Disable gem supports optional version specifiers in gem names

Signed-off-by: Alex Peterson <[email protected]>

* fix enabled_gem_file argument

Signed-off-by: Alex Peterson <[email protected]>

* fix adding gem names not removing existing version

Signed-off-by: Alex Peterson <[email protected]>

* add dependency resolution when enabling gems

Signed-off-by: Alex Peterson <[email protected]>

* Remove error about engine name not being unique

Engine names no longer have to be unique

Signed-off-by: Alex Peterson <[email protected]>

* fix spelling

Signed-off-by: Alex Peterson <[email protected]>

* CMake uses gem dependency resolution

Also fixes a bug in the resolver where gems with the higherst version were not preferred and multiple mappings could be returned

Signed-off-by: Alex Peterson <[email protected]>

* remove unnecessary sha and comments

Signed-off-by: Alex Peterson <[email protected]>

* minor comment fixes, remove unused code

Signed-off-by: Alex Peterson <[email protected]>

* Avoid resolving dependencies multiple times

Signed-off-by: Alex Peterson <[email protected]>

* Don't force engine registration

Signed-off-by: alexpete <[email protected]>

* Normalize file path when caching

Signed-off-by: Alex Peterson <[email protected]>

* fix function comments

Signed-off-by: Alex Peterson <[email protected]>

* be explicit about excluding optional gems

Signed-off-by: Alex Peterson <[email protected]>

* python string.format is so Python 3.5

Signed-off-by: Alex Peterson <[email protected]>

* fix falsey key error, simplify cache path list

Signed-off-by: alexpete <[email protected]>

* use stdout for resolution output instead of file

Signed-off-by: alexpete <[email protected]>

* make indent consistent, fix var name

Signed-off-by: alexpete <[email protected]>

* simpler Python

Signed-off-by: alexpete <[email protected]>

* ly_ to o3de_ cmake function names

Signed-off-by: alexpete <[email protected]>

* remove # noqa

Signed-off-by: alexpete <[email protected]>

* remove enable_gems.cmake

still need to update tests

Signed-off-by: alexpete <[email protected]>

* remove cmake.add_dependency and update tests

because enabled_gems.cmake is now deprecated we no longer add gem names to it or error if a gem name is not added to it, instead we check in the 'gem_names' field in project.json

Signed-off-by: Alex Peterson <[email protected]>

* use engine.json instead of ly_enable_gems()

gem dependencies are now listed in engine.json 'gem_names' field instead of in misc CMakeList.txt files using ly_enable_gems()

Signed-off-by: Alex Peterson <[email protected]>

* enabled_gems.cmake is now an optional file

Signed-off-by: Alex Peterson <[email protected]>

* fix register, cache subdir results,  fix templates

Signed-off-by: Alex Peterson <[email protected]>

* fix comment

Signed-off-by: Alex Peterson <[email protected]>

* strip whitespace when extracting version spec

Signed-off-by: Alex Peterson <[email protected]>

* Bumping engine version for enabled_gems.cmake

From this point on, enabled_gems.cmake is deprecated and gem dependency resolution is enabled

Signed-off-by: Alex Peterson <[email protected]>

* fix gem catalog relying on enabled_gems.cmake

The Project Manager was relying exclusively on enabled_gems.cmake to know what gems were enabled in a project.  These changes update Gem Catalog to select the gems a project uses in the Gem Catalog.

Signed-off-by: Alex Peterson <[email protected]>

* fix missing errors when add/remove project fails

Signed-off-by: Alex Peterson <[email protected]>

* Fix MockPythonBindings

Signed-off-by: Alex Peterson <[email protected]>

* QHash needed for non-unity builds

Signed-off-by: Alex Peterson <[email protected]>

* fix circular cmake <-> manifest dependency

resolved by moving `get_enabled_gems` and `get_enabled_gem_cmake_file` into manifest.py

Signed-off-by: Alex Peterson <[email protected]>

* deactivate any gem when no version specifier given

Signed-off-by: Alex Peterson <[email protected]>

* fix error handling when invalid project path given

Signed-off-by: Alex Peterson <[email protected]>

* remove unnecessary resolve()

Paths are already absolute at this point

Signed-off-by: Alex Peterson <[email protected]>

---------

Signed-off-by: Alex Peterson <[email protected]>
Signed-off-by: alexpete <[email protected]>

AutomatedTesting/Gem/Code/CMakeLists.txt

@@ -43,14 +43,6 @@ ly_create_alias(NAME AutomatedTesting.Clients  NAMESPACE Gem TARGETS Gem::Automa
 ly_create_alias(NAME AutomatedTesting.Servers  NAMESPACE Gem TARGETS Gem::AutomatedTesting)
 ly_create_alias(NAME AutomatedTesting.Unified  NAMESPACE Gem TARGETS Gem::AutomatedTesting)
 
-
-################################################################################
-# Gem dependencies
-################################################################################
-
-# Enable the enabled_gems for the Project:
-ly_enable_gems(PROJECT_NAME AutomatedTesting GEM_FILE enabled_gems.cmake)
-
 # Add project to the list server projects to create the AutomatedTesting.ServerLauncher
 if(PAL_TRAIT_BUILD_SERVER_SUPPORTED)
     set_property(GLOBAL APPEND PROPERTY LY_LAUNCHER_SERVER_PROJECTS AutomatedTesting)

AutomatedTesting/Gem/Code/enabled_gems.cmake

@@ -1,71 +0,0 @@
-#
-# Copyright (c) Contributors to the Open 3D Engine Project.
-# For complete copyright and license terms please see the LICENSE at the root of this distribution.
-#
-# SPDX-License-Identifier: Apache-2.0 OR MIT
-#
-#
-
-set(ENABLED_GEMS
-    ImGui
-    ScriptEvents
-    ExpressionEvaluation
-    Gestures
-    CertificateManager
-    DebugDraw
-    SceneProcessing
-    GraphCanvas
-    InAppPurchases
-    AutomatedTesting
-    EditorPythonBindings
-    QtForPython
-    PythonAssetBuilder
-    Metastream
-    Camera
-    EMotionFX
-    AtomTressFX
-    EditorModeFeedback
-    PhysX
-    CameraFramework
-    StartingPointMovement
-    StartingPointCamera
-    ScriptCanvas
-    ScriptCanvasPhysics
-    ScriptCanvasTesting
-    LyShineExamples
-    StartingPointInput
-    PhysXDebug
-    WhiteBox
-    FastNoise
-    SurfaceData
-    GradientSignal
-    Vegetation
-    GraphModel
-    LandscapeCanvas
-    NvCloth
-    Maestro
-    TextureAtlas
-    LmbrCentral
-    LyShine
-    HttpRequestor
-    Atom
-    AWSCore
-    AWSClientAuth
-    AWSMetrics
-    PythonCoverage
-    PrefabBuilder
-    AudioSystem
-    Terrain
-    Profiler
-    Multiplayer
-    TestAssetBuilder
-    DevTextures
-    PrimitiveAssets
-    Stars
-    RecastNavigation
-    ScriptAutomation
-    DiffuseProbeGrid
-    PythonCoverage
-    RemoteTools
-    Atom_TestData
-)

AutomatedTesting/project.json

@@ -1,7 +1,7 @@
 {
     "project_name": "AutomatedTesting",
     "product_name": "AutomatedTesting",
-    "version": "1.0.0",
+    "version": "1.1.0",
     "executable_name": "AutomatedTestingLauncher",
     "modules": [],
     "project_id": "{D816AFAE-4BB7-4FEF-88F4-E2B786DCF29D}",
@@ -11,5 +11,65 @@
         "Gem"
     ],
     "gem_names": [
+        "Atom",
+        "AtomTressFX",
+        "Atom_TestData",
+        "AudioSystem",
+        "AWSCore",
+        "AWSClientAuth",
+        "AWSMetrics",
+        "AutomatedTesting",
+        "Camera",
+        "CameraFramework",
+        "CertificateManager",
+        "DebugDraw",
+        "DevTextures",
+        "DiffuseProbeGrid",
+        "EditorModeFeedback",
+        "EditorPythonBindings",
+        "EMotionFX",
+        "ExpressionEvaluation",
+        "FastNoise",
+        "Gestures",
+        "GradientSignal",
+        "GraphCanvas",
+        "GraphModel",
+        "HttpRequestor",
+        "ImGui",
+        "InAppPurchases",
+        "LandscapeCanvas",
+        "LmbrCentral",
+        "LyShine",
+        "LyShineExamples",
+        "Maestro",
+        "Metastream",
+        "Multiplayer",
+        "NvCloth",
+        "PhysX",
+        "PhysXDebug",
+        "PrefabBuilder",
+        "PrimitiveAssets",
+        "Profiler",
+        "PythonAssetBuilder",
+        "PythonCoverage",
+        "QtForPython",
+        "RecastNavigation",
+        "RemoteTools",
+        "SceneProcessing",
+        "ScriptAutomation",
+        "ScriptCanvas",
+        "ScriptCanvasPhysics",
+        "ScriptCanvasTesting",
+        "ScriptEvents",
+        "Stars",
+        "StartingPointCamera",
+        "StartingPointInput",
+        "StartingPointMovement",
+        "SurfaceData",
+        "Terrain",
+        "TestAssetBuilder",
+        "TextureAtlas",
+        "Vegetation",
+        "WhiteBox"
     ]
 }

Code/Tools/ProjectManager/Source/Application.cpp

@@ -180,7 +180,6 @@ namespace O3DE::ProjectManager
 
     bool Application::RegisterEngine(bool interactive)
     {
-        // get this engine's info
         auto engineInfoOutcome = m_pythonBindings->GetEngineInfo();
         if (!engineInfoOutcome)
         {
@@ -203,43 +202,9 @@ namespace O3DE::ProjectManager
             return true;
         }
 
-        // check if an engine with this name is already registered and has a valid engine.json
-        auto existingEngineResult = m_pythonBindings->GetEngineInfo(engineInfo.m_name);
-        if (existingEngineResult)
-        {
-            if (!interactive)
-            {
-                AZ_Error("Project Manager", false, "An engine with the name %s is already registered with the path %s",
-                    engineInfo.m_name.toUtf8().constData(), engineInfo.m_path.toUtf8().constData());
-                return false;
-            }
-
-            // get the updated engine name unless the user wants to cancel
-            bool okPressed = false;
-            const EngineInfo& otherEngineInfo = existingEngineResult.GetValue();
-
-            engineInfo.m_name = QInputDialog::getText(nullptr,
-                QObject::tr("Engine '%1' already registered").arg(engineInfo.m_name),
-                QObject::tr("An engine named '%1' is already registered.<br /><br />"
-                            "<b>Current path</b><br />%2<br/><br />"
-                            "<b>New path</b><br />%3<br /><br />"
-                            "Press 'OK' to force registration, or provide a new engine name below.<br />"
-                            "Alternatively, press `Cancel` to close the Project Manager and resolve the issue manually.")
-                            .arg(engineInfo.m_name, otherEngineInfo.m_path, engineInfo.m_path),
-                QLineEdit::Normal,
-                engineInfo.m_name,
-                &okPressed);
-
-            if (!okPressed)
-            {
-                // user elected not to change the name or force registration
-                return false;
-            }
-        }
-
-        // always force register in case there is an engine registered in o3de_manifest.json, but
-        // the engine.json is missing or corrupt in which case GetEngineInfo() fails
-        constexpr bool forceRegistration = true;
+        // We no longer force registration because we no longer require that only one engine can
+        // be registered with each engine name
+        constexpr bool forceRegistration = false;
         auto registerOutcome = m_pythonBindings->SetEngineInfo(engineInfo, forceRegistration);
         if (!registerOutcome)
         {

Code/Tools/ProjectManager/Source/CreateProjectCtrl.cpp

@@ -256,8 +256,7 @@ namespace O3DE::ProjectManager
             auto result = PythonBindingsInterface::Get()->CreateProject(projectTemplatePath, projectInfo);
             if (result.IsSuccess())
             {
-                // automatically register the project
-                PythonBindingsInterface::Get()->AddProject(projectInfo.m_path);
+                // don't need to register here, the project is already registered in CreateProject()
 
                 const ProjectGemCatalogScreen::ConfiguredGemsResult gemResult = m_projectGemCatalogScreen->ConfigureGemsForProject(projectInfo.m_path);
                 if (gemResult == ProjectGemCatalogScreen::ConfiguredGemsResult::Failed)

Code/Tools/ProjectManager/Source/GemCatalog/GemCatalogScreen.cpp

@@ -7,6 +7,7 @@
  */
 
 #include <GemCatalog/GemCatalogScreen.h>
+#include <AzCore/Dependency/Dependency.h>
 #include <PythonBindingsInterface.h>
 #include <GemCatalog/GemCatalogHeaderWidget.h>
 #include <GemCatalog/GemFilterWidget.h>
@@ -600,24 +601,39 @@ namespace O3DE::ProjectManager
             m_notificationsEnabled = false;
 
             // Gather enabled gems for the given project.
-            const auto& enabledGemNamesResult = PythonBindingsInterface::Get()->GetEnabledGemNames(projectPath);
+            constexpr bool includeDependencies = false;
+            const auto& enabledGemNamesResult = PythonBindingsInterface::Get()->GetEnabledGems(projectPath, includeDependencies);
             if (enabledGemNamesResult.IsSuccess())
             {
-                const QVector<AZStd::string>& enabledGemNames = enabledGemNamesResult.GetValue();
-                for (const AZStd::string& enabledGemName : enabledGemNames)
+                const auto& enabledGemNames = enabledGemNamesResult.GetValue();
+                for (auto itr = enabledGemNames.cbegin(); itr != enabledGemNames.cend(); itr++)
                 {
-                    const QModelIndex modelIndex = m_gemModel->FindIndexByNameString(enabledGemName.c_str());
+                    const QString& gemNameWithSpecifier = itr.key();
+                    const QString& gemPath = itr.value(); 
+
+                    AZ::Dependency<AZ::SemanticVersion::parts_count> dependency;
+                    auto parseOutcome = dependency.ParseVersions({ gemNameWithSpecifier.toUtf8().constData() });
+                    const QString& gemName = parseOutcome ? dependency.GetName().c_str() : gemNameWithSpecifier; 
+
+                    // First, try to find the gem by path
+                    QModelIndex modelIndex = m_gemModel->FindIndexByPath(gemPath);
+                    if (!modelIndex.isValid())
+                    {
+                        // Fall back to lookup by name 
+                        modelIndex = m_gemModel->FindIndexByNameString(gemName);
+                    }
+
                     if (modelIndex.isValid())
                     {
                         GemModel::SetWasPreviouslyAdded(*m_gemModel, modelIndex, true);
                         GemModel::SetIsAdded(*m_gemModel, modelIndex, true);
                     }
                     // ${Name} is a special name used in templates and is not really an error
-                    else if (enabledGemName != "${Name}")
+                    else if (gemName != "${Name}")
                     {
                         AZ_Warning("ProjectManager::GemCatalog", false,
                             "Cannot find entry for gem with name '%s'. The CMake target name probably does not match the specified name in the gem.json.",
-                            enabledGemName.c_str());
+                            gemName.toUtf8().constData());
                     }
                 }
             }

Code/Tools/ProjectManager/Source/GemCatalog/GemModel.cpp

@@ -68,6 +68,10 @@ namespace O3DE::ProjectManager
 
         const QModelIndex modelIndex = index(rowCount()-1, 0);
         m_nameToIndexMap[gemInfo.m_name] = modelIndex;
+        if (!gemInfo.m_path.isEmpty())
+        {
+            m_pathToIndexMap[gemInfo.m_path] = modelIndex;
+        }
 
         return modelIndex;
     }
@@ -223,6 +227,17 @@ namespace O3DE::ProjectManager
         return {};
     }
 
+    QModelIndex GemModel::FindIndexByPath(const QString& path) const
+    {
+        const auto iterator = m_pathToIndexMap.find(path);
+        if (iterator != m_pathToIndexMap.end())
+        {
+            return iterator.value();
+        }
+
+        return {};
+    }
+
     QStringList GemModel::GetDependingGems(const QModelIndex& modelIndex)
     {
         return modelIndex.data(RoleDependingGems).toStringList();

Code/Tools/ProjectManager/Source/GemCatalog/GemModel.h

@@ -62,6 +62,7 @@ namespace O3DE::ProjectManager
         void UpdateGemDependencies();
 
         QModelIndex FindIndexByNameString(const QString& nameString) const;
+        QModelIndex FindIndexByPath(const QString& path) const;
         QVector<Tag> GetDependingGemTags(const QModelIndex& modelIndex);
         bool HasDependentGems(const QModelIndex& modelIndex) const;
 
@@ -126,6 +127,7 @@ namespace O3DE::ProjectManager
         QStringList GetDependingGems(const QModelIndex& modelIndex);
 
         QHash<QString, QModelIndex> m_nameToIndexMap;
+        QHash<QString, QModelIndex> m_pathToIndexMap;
         QItemSelectionModel* m_selectionModel = nullptr;
         QHash<QString, QSet<QModelIndex>> m_gemDependencyMap;
         QHash<QString, QSet<QModelIndex>> m_gemReverseDependencyMap;

Code/Tools/ProjectManager/Source/ProjectUtils.cpp

@@ -293,20 +293,30 @@ namespace O3DE::ProjectManager
             QString path = QDir::toNativeSeparators(QFileDialog::getExistingDirectory(parent, QObject::tr("Select Project Directory")));
             if (!path.isEmpty())
             {
-                return RegisterProject(path);
+                return RegisterProject(path, parent);
             }
 
             return false;
         }
 
-        bool RegisterProject(const QString& path)
+        bool RegisterProject(const QString& path, QWidget* parent)
         {
-            return PythonBindingsInterface::Get()->AddProject(path);
+            if (auto result = PythonBindingsInterface::Get()->AddProject(path); !result)
+            {
+                DisplayDetailedError("Failed to add project", result, parent);
+                return false;
+            }
+            return true;
         }
 
-        bool UnregisterProject(const QString& path)
+        bool UnregisterProject(const QString& path, QWidget* parent)
         {
-            return PythonBindingsInterface::Get()->RemoveProject(path);
+            if (auto result = PythonBindingsInterface::Get()->RemoveProject(path); !result)
+            {
+                DisplayDetailedError("Failed to unregister project", result, parent);
+                return false;
+            }
+            return true;
         }
 
         bool CopyProjectDialog(const QString& origPath, ProjectInfo& newProjectInfo, QWidget* parent)

Code/Tools/ProjectManager/Source/ProjectUtils.h

@@ -22,8 +22,8 @@ namespace O3DE::ProjectManager
     namespace ProjectUtils
     {
         bool AddProjectDialog(QWidget* parent = nullptr);
-        bool RegisterProject(const QString& path);
-        bool UnregisterProject(const QString& path);
+        bool RegisterProject(const QString& path, QWidget* parent = nullptr);
+        bool UnregisterProject(const QString& path, QWidget* parent = nullptr);
         bool CopyProjectDialog(const QString& origPath, ProjectInfo& newProjectInfo, QWidget* parent = nullptr);
         bool CopyProject(const QString& origPath, const QString& newPath, QWidget* parent, bool skipRegister = false, bool showProgress = true);
         bool DeleteProjectFiles(const QString& path, bool force = false);

Code/Tools/ProjectManager/Source/PythonBindings.cpp

@@ -326,7 +326,6 @@ namespace O3DE::ProjectManager
             }
 
             // import required modules
-            m_cmake = pybind11::module::import("o3de.cmake");
             m_register = pybind11::module::import("o3de.register");
             m_manifest = pybind11::module::import("o3de.manifest");
             m_engineTemplate = pybind11::module::import("o3de.engine_template");
@@ -688,37 +687,29 @@ namespace O3DE::ProjectManager
         return AZ::Success(AZStd::move(gems));
     }
 
-    AZ::Outcome<QVector<AZStd::string>, AZStd::string> PythonBindings::GetEnabledGemNames(const QString& projectPath) const
+    AZ::Outcome<QHash<QString, QString>, AZStd::string> PythonBindings::GetEnabledGems(const QString& projectPath, bool includeDependencies) const
     {
-        // Retrieve the path to the cmake file that lists the enabled gems.
-        pybind11::str enabledGemsFilename;
+        QHash<QString, QString> enabledGems;
         auto result = ExecuteWithLockErrorHandling([&]
         {
-            enabledGemsFilename = m_cmake.attr("get_enabled_gem_cmake_file")(
-                pybind11::none(), // project_name
-                QString_To_Py_Path(projectPath)); // project_path
-        });
-        if (!result.IsSuccess())
-        {
-            return AZ::Failure<AZStd::string>(result.GetError().c_str());
-        }
-
-        // Retrieve the actual list of names from the cmake file.
-        QVector<AZStd::string> gemNames;
-        result = ExecuteWithLockErrorHandling([&]
-        {
-            const auto pyGemNames = m_cmake.attr("get_enabled_gems")(enabledGemsFilename);
-            for (auto gemName : pyGemNames)
+            auto enabledGemsData = m_projectManagerInterface.attr("get_enabled_gems")(QString_To_Py_Path(projectPath), includeDependencies);
+            if (pybind11::isinstance<pybind11::dict>(enabledGemsData))
             {
-                gemNames.push_back(Py_To_String(gemName));
+                for (auto item : pybind11::dict(enabledGemsData))
+                {
+                    enabledGems.insert(Py_To_String(item.first), Py_To_String(item.second));
+                }
             }
         });
+
         if (!result.IsSuccess())
         {
             return AZ::Failure<AZStd::string>(result.GetError().c_str());
         }
-
-        return AZ::Success(AZStd::move(gemNames));
+        else
+        {
+            return AZ::Success(AZStd::move(enabledGems));
+        }
     }
 
     AZ::Outcome<void, AZStd::string> PythonBindings::GemRegistration(const QString& gemPath, const QString& projectPath, bool remove)
@@ -779,56 +770,46 @@ namespace O3DE::ProjectManager
         return GemRegistration(gemPath, projectPath, /*remove*/true);
     }
 
-    bool PythonBindings::AddProject(const QString& path)
+    IPythonBindings::DetailedOutcome PythonBindings::AddProject(const QString& path)
     {
+        using namespace pybind11::literals;
         bool registrationResult = false;
-        bool result = ExecuteWithLock(
-            [&]
-        {
-            auto projectPath = QString_To_Py_Path(path);
-            auto pythonRegistrationResult = m_register.attr("register")(pybind11::none(), projectPath);
+        bool result = ExecuteWithLock([&] {
+            auto pythonRegistrationResult = m_register.attr("register")(
+                "project_path"_a = QString_To_Py_Path(path));
 
             // Returns an exit code so boolify it then invert result
             registrationResult = !pythonRegistrationResult.cast<bool>();
         });
 
-        return result && registrationResult;
+        if (!result || !registrationResult)
+        {
+            return AZ::Failure<IPythonBindings::ErrorPair>(GetErrorPair());
+        }
+
+        return AZ::Success();
     }
 
-    bool PythonBindings::RemoveProject(const QString& path)
+    IPythonBindings::DetailedOutcome PythonBindings::RemoveProject(const QString& path)
     {
         using namespace pybind11::literals;
-
         bool registrationResult = false;
-        bool result = ExecuteWithLock(
-            [&]
-        {
+        bool result = ExecuteWithLock([&] {
             auto pythonRegistrationResult = m_register.attr("register")(
-                "engine_path"_a                  = pybind11::none(),
-                "project_path"_a                 = QString_To_Py_Path(path),
-                "gem_path"_a                     = pybind11::none(),
-                "external_subdir_path"_a         = pybind11::none(),
-                "template_path"_a                = pybind11::none(),
-                "restricted_path"_a              = pybind11::none(),
-                "repo_uri"_a                     = pybind11::none(),
-                "default_engines_folder"_a       = pybind11::none(),
-                "default_projects_folder"_a      = pybind11::none(),
-                "default_gems_folder"_a          = pybind11::none(),
-                "default_templates_folder"_a     = pybind11::none(),
-                "default_restricted_folder"_a    = pybind11::none(),
-                "default_third_party_folder"_a   = pybind11::none(),
-                "external_subdir_engine_path"_a  = pybind11::none(),
-                "external_subdir_project_path"_a = pybind11::none(),
-                "external_subdir_gem_path"_a     = pybind11::none(),
-                "remove"_a                       = true,
-                "force"_a                        = false
+                "project_path"_a = QString_To_Py_Path(path),
+                "remove"_a       = true
                 );
 
             // Returns an exit code so boolify it then invert result
             registrationResult = !pythonRegistrationResult.cast<bool>();
         });
 
-        return result && registrationResult;
+        if (!result || !registrationResult)
+        {
+            return AZ::Failure<IPythonBindings::ErrorPair>(GetErrorPair());
+        }
+
+        return AZ::Success();
     }
 
     AZ::Outcome<ProjectInfo> PythonBindings::CreateProject(const QString& projectTemplatePath, const ProjectInfo& projectInfo, bool registerProject)
@@ -1315,16 +1296,17 @@ namespace O3DE::ProjectManager
         if (templateInfo.IsValid())
         {
             QString templateProjectPath = QDir(templateInfo.m_path).filePath("Template");
-            auto enabledGemNames = GetEnabledGemNames(templateProjectPath);
-            if (enabledGemNames)
+            constexpr bool includeDependencies = false;
+            auto enabledGems = GetEnabledGems(templateProjectPath, includeDependencies);
+            if (enabledGems)
             {
-                for (auto gem : enabledGemNames.GetValue())
+                for (auto gemName : enabledGems.GetValue().keys())
                 {
                     // Exclude the template ${Name} placeholder for the list of included gems
                     // That Gem gets created with the project
-                    if (!gem.contains("${Name}"))
+                    if (!gemName.contains("${Name}"))
                     {
-                        templateInfo.m_includedGems.push_back(Py_To_String(gem.c_str()));
+                        templateInfo.m_includedGems.push_back(gemName);
                     }
                 }
             }

Code/Tools/ProjectManager/Source/PythonBindings.h

@@ -49,7 +49,8 @@ namespace O3DE::ProjectManager
         AZ::Outcome<GemInfo> GetGemInfo(const QString& path, const QString& projectPath = {}) override;
         AZ::Outcome<QVector<GemInfo>, AZStd::string> GetEngineGemInfos() override;
         AZ::Outcome<QVector<GemInfo>, AZStd::string> GetAllGemInfos(const QString& projectPath) override;
-        AZ::Outcome<QVector<AZStd::string>, AZStd::string> GetEnabledGemNames(const QString& projectPath) const override;
+        AZ::Outcome<QHash<QString /*gem name with specifier*/, QString /* gem path */>, AZStd::string> GetEnabledGems(
+            const QString& projectPath, bool includeDependencies) const override;
         AZ::Outcome<void, AZStd::string> RegisterGem(const QString& gemPath, const QString& projectPath = {}) override;
         AZ::Outcome<void, AZStd::string> UnregisterGem(const QString& gemPath, const QString& projectPath = {}) override;
 
@@ -59,8 +60,8 @@ namespace O3DE::ProjectManager
         AZ::Outcome<QVector<ProjectInfo>> GetProjects() override;
         AZ::Outcome<QVector<ProjectInfo>, AZStd::string> GetProjectsForRepo(const QString& repoUri) override;
         AZ::Outcome<QVector<ProjectInfo>, AZStd::string> GetProjectsForAllRepos() override;
-        bool AddProject(const QString& path) override;
-        bool RemoveProject(const QString& path) override;
+        DetailedOutcome AddProject(const QString& path) override;
+        DetailedOutcome RemoveProject(const QString& path) override;
         AZ::Outcome<void, AZStd::string> UpdateProject(const ProjectInfo& projectInfo) override;
         AZ::Outcome<void, AZStd::string> AddGemToProject(const QString& gemPath, const QString& projectPath) override;
         AZ::Outcome<void, AZStd::string> RemoveGemFromProject(const QString& gemPath, const QString& projectPath) override;
@@ -119,7 +120,6 @@ namespace O3DE::ProjectManager
         pybind11::handle m_gemProperties;
         pybind11::handle m_engineTemplate;
         pybind11::handle m_engineProperties;
-        pybind11::handle m_cmake;
         pybind11::handle m_register;
         pybind11::handle m_manifest;
         pybind11::handle m_enableGemProject;

Code/Tools/ProjectManager/Source/PythonBindingsInterface.h

@@ -19,6 +19,11 @@
 #include <ProjectTemplateInfo.h>
 #include <GemRepo/GemRepoInfo.h>
 
+#if !defined(Q_MOC_RUN)
+#include <QHash>
+#endif
+
+
 namespace O3DE::ProjectManager
 {
     //! Interface used to interact with the o3de cli python functions
@@ -146,10 +151,14 @@ namespace O3DE::ProjectManager
 
         /**
          * Get a list of all enabled gem names for a given project.
-         * @param[in] projectPath Absolute file path to the project.
-         * @return A list of gem names of all the enabled gems for a given project or a error message on failure.
+         * @param projectPath Absolute file path to the project.
+         * @param includeDependencies Whether to return gem dependencies or only gems listed in project.json
+         *                            and the deprecated enabled_gems.cmake file if it exists
+         * @return A QHash of gem names with optional version specifiers and gem paths of all
+           the enabled gems for a given project or a error message on failure.
          */
-        virtual AZ::Outcome<QVector<AZStd::string>, AZStd::string> GetEnabledGemNames(const QString& projectPath) const = 0;
+        virtual AZ::Outcome<QHash<QString /*gem name with specifier*/, QString /* gem path */>, AZStd::string> GetEnabledGems(
+            const QString& projectPath, bool includeDependencies = true) const = 0;
 
         /**
          * Registers the gem to the specified project, or to the o3de_manifest.json if no project path is given
@@ -208,21 +217,21 @@ namespace O3DE::ProjectManager
         /**
          * Adds existing project on disk
          * @param path the absolute path to the project
-         * @return true on success, false on failure
+         * @return An outcome with the success flag as well as an error message in case of a failure.
          */
-        virtual bool AddProject(const QString& path) = 0;
+        virtual DetailedOutcome AddProject(const QString& path) = 0;
 
         /**
          * Adds existing project on disk
          * @param path the absolute path to the project
-         * @return true on success, false on failure
+         * @return An outcome with the success flag as well as an error message in case of a failure.
          */
-        virtual bool RemoveProject(const QString& path) = 0;
+        virtual DetailedOutcome RemoveProject(const QString& path) = 0;
 
         /**
          * Update a project
          * @param projectInfo the info to use to update the project 
-         * @return true on success, false on failure
+         * @return An outcome with the success flag as well as an error message in case of a failure.
          */
         virtual AZ::Outcome<void, AZStd::string> UpdateProject(const ProjectInfo& projectInfo) = 0;
 

Code/Tools/ProjectManager/tests/MockPythonBindings.h

@@ -29,7 +29,7 @@ namespace O3DE::ProjectManager
         MOCK_METHOD2(GetGemInfo, AZ::Outcome<GemInfo>(const QString&, const QString&));
         MOCK_METHOD0(GetEngineGemInfos, AZ::Outcome<QVector<GemInfo>, AZStd::string>());
         MOCK_METHOD1(GetAllGemInfos, AZ::Outcome<QVector<GemInfo>, AZStd::string>(const QString&));
-        MOCK_CONST_METHOD1(GetEnabledGemNames, AZ::Outcome<QVector<AZStd::string>, AZStd::string>(const QString&));
+        MOCK_CONST_METHOD2(GetEnabledGems, AZ::Outcome<QHash<QString, QString>, AZStd::string>(const QString&, bool includeDependencies));
         MOCK_METHOD2(RegisterGem, AZ::Outcome<void, AZStd::string>(const QString&, const QString&));
         MOCK_METHOD2(UnregisterGem, AZ::Outcome<void, AZStd::string>(const QString&, const QString&));
 
@@ -37,8 +37,8 @@ namespace O3DE::ProjectManager
         MOCK_METHOD3(CreateProject, AZ::Outcome<ProjectInfo>(const QString&, const ProjectInfo&, bool));
         MOCK_METHOD1(GetProject, AZ::Outcome<ProjectInfo>(const QString&));
         MOCK_METHOD0(GetProjects, AZ::Outcome<QVector<ProjectInfo>>());
-        MOCK_METHOD1(AddProject, bool(const QString&));
-        MOCK_METHOD1(RemoveProject, bool(const QString&));
+        MOCK_METHOD1(AddProject, DetailedOutcome(const QString&));
+        MOCK_METHOD1(RemoveProject, DetailedOutcome(const QString&));
         MOCK_METHOD1(UpdateProject, AZ::Outcome<void, AZStd::string>(const ProjectInfo&));
         MOCK_METHOD2(AddGemToProject, AZ::Outcome<void, AZStd::string>(const QString&, const QString&));
         MOCK_METHOD2(RemoveGemFromProject, AZ::Outcome<void, AZStd::string>(const QString&, const QString&));

Gems/Atom/Asset/Shader/Code/CMakeLists.txt

@@ -101,7 +101,6 @@ ly_add_target(
 
 # The Atom_Asset_Shader is a required gem for Builders in order to process the assets that come WITHOUT
 # the Atom_Feature_Common required gem
-ly_enable_gems(GEMS Atom_Asset_Shader)
 
 ################################################################################
 # Tests

Gems/Atom/Bootstrap/Code/CMakeLists.txt

@@ -46,4 +46,3 @@ ly_create_alias(NAME Atom_Bootstrap.Servers NAMESPACE Gem TARGETS Gem::Atom_Boot
 ly_create_alias(NAME Atom_Bootstrap.Unified NAMESPACE Gem TARGETS Gem::Atom_Bootstrap)
 
 # The Atom_Bootstrap gem is responsible for making the NativeWindow handle in the launcher applications
-ly_enable_gems(GEMS Atom_Bootstrap)

Gems/AtomLyIntegration/CommonFeatures/Code/CMakeLists.txt

@@ -189,4 +189,3 @@ endif()
 # due to the AZ::Render::EditorDirectionalLightComponent, AZ::Render::EditorMeshComponent,
 # AZ::Render::EditorGridComponent, AZ::Render::EditorHDRiSkyboxComponent,
 # AZ::Render::EditorImageBasedLightComponent being saved as part of the DefaultLevel.prefab
-ly_enable_gems(GEMS AtomLyIntegration_CommonFeatures)

Gems/Camera/Code/CMakeLists.txt

@@ -69,4 +69,3 @@ if (PAL_TRAIT_BUILD_HOST_TOOLS)
 endif()
 
 # The DefaultPrefab contains an EditorCameraComponent which makes this gem required
-ly_enable_gems(GEMS Camera)

Gems/Maestro/Code/CMakeLists.txt

@@ -81,8 +81,6 @@ if (PAL_TRAIT_BUILD_HOST_TOOLS)
 endif()
 
 # Maestro is still used by the CrySystem Level System, CSystem::SystemInit and TrackView
-ly_enable_gems(GEMS Maestro)
-
 
 ################################################################################
 # Tests

Gems/Meshlets/ASV_GPU_and_CPU_Demo/Readme.txt

@@ -172,12 +172,11 @@ is not the case, simply replace with the directory name of your active project.
             }
     Remark: this is NOT required in this case as Meshlets can process regular Atom meshes
 
-3. Enable Meshlets gem for the active project - AtomSampleViewer/Gem/code/enabled_gems.cmake
-            (
-                set(ENABLED_GEMS
+3. Enable Meshlets gem for the active project - AtomSampleViewer/project.json
+            "gem_names": [
                 ...
-                Meshlets
-            )
+                "Meshlets"
+            ]
 
 4. Add a build dependency on the meshlets gem - AtomSampleViewer/Gem/Code/CMakeLists.txt
         ly_add_target(

Gems/Prefab/PrefabBuilder/CMakeLists.txt

@@ -45,8 +45,6 @@ ly_add_target(
 ly_create_alias(NAME PrefabBuilder.Tools NAMESPACE Gem TARGETS Gem::PrefabBuilder.Builders)
 
 # we automatically add this gem, if it is present, to all our known set of builder applications:
-ly_enable_gems(GEMS PrefabBuilder)
-
 if(PAL_TRAIT_BUILD_TESTS_SUPPORTED)
     ly_add_target(
         NAME PrefabBuilder.Tests ${PAL_TRAIT_TEST_TARGET_TYPE}

Gems/SceneProcessing/Code/CMakeLists.txt

@@ -69,7 +69,6 @@ if (PAL_TRAIT_BUILD_HOST_TOOLS)
     ly_create_alias(NAME SceneProcessing.Tools    NAMESPACE Gem TARGETS Gem::SceneProcessing.Editor)
 
     # SceneProcessing Gem is only used in Tools and builders and is a requirement for the Editor and AssetProcessor
-    ly_enable_gems(GEMS SceneProcessing)
 endif()
 
 ################################################################################

Templates/DefaultProject/Template/Gem/${NameLower}_files.cmake

@@ -10,5 +10,4 @@ set(FILES
     Include/${Name}/${Name}Bus.h
     Source/${Name}SystemComponent.cpp
     Source/${Name}SystemComponent.h
-    enabled_gems.cmake
 )

Templates/DefaultProject/Template/Gem/CMakeLists.txt

@@ -98,6 +98,3 @@ o3de_find_ancestor_project_root(project_path project_name "${CMAKE_CURRENT_SOURC
 if (NOT project_name)
     set(project_name ${Name})
 endif()
-
-# Enable the specified list of gems from GEM_FILE or GEMS list for this specific project:
-ly_enable_gems(PROJECT_NAME ${project_name} GEM_FILE enabled_gems.cmake)

Templates/DefaultProject/Template/Gem/enabled_gems.cmake

@@ -1,33 +0,0 @@
-# {BEGIN_LICENSE}
-# Copyright (c) Contributors to the Open 3D Engine Project.
-# For complete copyright and license terms please see the LICENSE at the root of this distribution.
-#
-# SPDX-License-Identifier: Apache-2.0 OR MIT
-#
-# {END_LICENSE}
-
-set(ENABLED_GEMS
-    ${Name}
-    Atom
-    AudioSystem
-    AWSCore
-    CameraFramework
-    DebugDraw
-    EditorPythonBindings
-    EMotionFX
-    GameState
-    ImGui
-    LandscapeCanvas
-    LyShine
-    PhysX
-    PrimitiveAssets
-    PrefabBuilder
-    SaveData
-    ScriptCanvasPhysics
-    ScriptEvents
-    StartingPointInput
-    TextureAtlas
-    WhiteBox
-    DiffuseProbeGrid
-    Compression
-)

Templates/DefaultProject/Template/project.json

@@ -17,5 +17,30 @@
     "external_subdirectories": [
         "Gem"
     ],
-    "restricted": "${Name}"
+    "restricted": "${Name}",
+    "gem_names": [
+        "${Name}",
+        "Atom",
+        "AudioSystem",
+        "AWSCore",
+        "CameraFramework",
+        "Compression",
+        "DebugDraw",
+        "DiffuseProbeGrid",
+        "EditorPythonBindings",
+        "EMotionFX",
+        "GameState",
+        "ImGui",
+        "LandscapeCanvas",
+        "LyShine",
+        "PhysX",
+        "PrimitiveAssets",
+        "PrefabBuilder",
+        "SaveData",
+        "ScriptCanvasPhysics",
+        "ScriptEvents",
+        "StartingPointInput",
+        "TextureAtlas",
+        "WhiteBox"
+    ]
 }

Templates/DefaultProject/template.json

@@ -130,10 +130,6 @@
             "file": "Gem/Source/${Name}SystemComponent.h",
             "isTemplated": true
         },
-        {
-            "file": "Gem/enabled_gems.cmake",
-            "isTemplated": true
-        },
         {
             "file": "Gem/gem.json",
             "isTemplated": true

Templates/MinimalProject/Template/Gem/${NameLower}_files.cmake

@@ -10,5 +10,4 @@ set(FILES
     Include/${Name}/${Name}Bus.h
     Source/${Name}SystemComponent.cpp
     Source/${Name}SystemComponent.h
-    enabled_gems.cmake
 )

Templates/MinimalProject/Template/Gem/CMakeLists.txt

@@ -98,6 +98,3 @@ o3de_find_ancestor_project_root(project_path project_name "${CMAKE_CURRENT_SOURC
 if (NOT project_name)
     set(project_name ${Name})
 endif()
-
-# Enable the specified list of gems from GEM_FILE or GEMS list for this specific project:
-ly_enable_gems(PROJECT_NAME ${project_name} GEM_FILE enabled_gems.cmake)

Templates/MinimalProject/Template/Gem/enabled_gems.cmake

@@ -1,14 +0,0 @@
-# {BEGIN_LICENSE}
-# Copyright (c) Contributors to the Open 3D Engine Project.
-# For complete copyright and license terms please see the LICENSE at the root of this distribution.
-#
-# SPDX-License-Identifier: Apache-2.0 OR MIT
-#
-# {END_LICENSE}
-
-set(ENABLED_GEMS
-    ${Name}
-    Atom
-    CameraFramework
-    ImGui
-)

Templates/MinimalProject/Template/project.json

@@ -16,5 +16,11 @@
     "engine": "o3de",
     "external_subdirectories": [
         "Gem"
+    ],
+    "gem_names": [
+        "${Name}",
+        "Atom",
+        "CameraFramework",
+        "ImGui"
     ]
 }

Templates/MinimalProject/template.json

@@ -126,10 +126,6 @@
             "file": "Gem/Source/${Name}SystemComponent.h",
             "isTemplated": true
         },
-        {
-            "file": "Gem/enabled_gems.cmake",
-            "isTemplated": true
-        },
         {
             "file": "Gem/gem.json",
             "isTemplated": true

cmake/FileUtil.cmake

@@ -141,6 +141,20 @@ function(ly_file_read path content)
     set_property(DIRECTORY APPEND PROPERTY CMAKE_CONFIGURE_DEPENDS ${path})
 endfunction()
 
+#! o3de_file_read_cache: wrap ly_file_read but store the file in a cache to avoid 
+#  extra reads.
+function(o3de_file_read_cache path content)
+    unset(file_content)
+    cmake_path(SET path "${path}")
+    cmake_path(NORMAL_PATH path)
+    set(file_cache_var_name "O3DE_FILE_CACHE_${path}")
+    get_property(file_content GLOBAL PROPERTY ${file_cache_var_name})
+    if(NOT file_content)
+        ly_file_read(${path} file_content)
+        set_property(GLOBAL PROPERTY ${file_cache_var_name} ${file_content})
+    endif()
+    set(${content} ${file_content} PARENT_SCOPE)
+endfunction()
 
 #! ly_get_last_path_segment_concat_sha256 : Concatenates the last path segment of the absolute path
 # with the first 8 characters of the absolute path SHA256 hash to make a unique relative path segment

cmake/O3DEJson.cmake

@@ -49,7 +49,7 @@ function(o3de_read_json_array read_output_array input_json_path array_key)
 endfunction()
 
 function(o3de_read_json_key output_value input_json_path key)
-    ly_file_read(${input_json_path} manifest_json_data)
+    o3de_file_read_cache(${input_json_path} manifest_json_data)
     string(JSON value ERROR_VARIABLE manifest_json_error GET ${manifest_json_data} ${key})
     if(manifest_json_error)
         message(WARNING "Error reading field at key ${key} in file \"${input_json_path}\" : ${manifest_json_error}")
@@ -57,3 +57,27 @@ function(o3de_read_json_key output_value input_json_path key)
     endif()
     set(${output_value} ${value} PARENT_SCOPE)
 endfunction()
+
+#! o3de_read_json_keys: read multiple json keys at once. More efficient
+# than using o3de_read_json_key multiple times.
+# \arg:input_json_path - the path to the json file 
+# \args: pairs of 'key' and 'output_value'
+# e.g. To read the key1 and key2 values
+# o3de_read_json_keys(c:/myfile.json 'key1' out_key1_value 'key2' out_key2_value)
+function(o3de_read_json_keys input_json_path)
+    o3de_file_read_cache(${input_json_path} manifest_json_data)
+    unset(key)
+    foreach(arg IN LISTS ARGN)
+        if(NOT DEFINED key)
+            set(key ${arg})
+        else()
+            string(JSON value ERROR_VARIABLE manifest_json_error GET ${manifest_json_data} ${key})
+            if(manifest_json_error)
+                message(WARNING "Error reading field at key ${key} in file \"${input_json_path}\" : ${manifest_json_error}")
+            else()
+                set(${arg} ${value} PARENT_SCOPE)
+            endif()
+            unset(key)
+        endif()
+    endforeach()
+endfunction()

cmake/PAL.cmake

@@ -51,9 +51,9 @@ function(o3de_read_manifest o3de_manifest_json_data)
     endif()
 endfunction()
 
-
 #! o3de_find_gem_with_registered_external_subdirs: Query the path of a gem using its name
-#
+#  IMPORTANT NOTE: This does not take into account any gem versions or dependency resolution, 
+#  which is fine if you don't need it and just want speed.
 # \arg:gem_name the gem name to find
 # \arg:output_gem_path the path of the gem to set
 # \arg:registered_external_subdirs a list of external subdirectories registered accross

cmake/Subdirectories.cmake

@@ -8,6 +8,8 @@
 
 include_guard()
 
+set(O3DE_DISABLE_GEM_DEPENDENCY_RESOLUTION FALSE CACHE BOOL "Option to forcibly disable the resolution of gem dependencies")
+
 ################################################################################
 # Subdirectory processing
 ################################################################################
@@ -32,11 +34,16 @@ function(add_o3de_object_gem_json_external_subdirectories object_type object_nam
     if(EXISTS ${gem_json_path})
         o3de_read_json_external_subdirs(gem_external_subdirs ${gem_path}/gem.json)
         # Read the gem_name from the gem.json and map it to the gem path
-        o3de_read_json_key(gem_name "${gem_path}/gem.json" "gem_name")
-        if (gem_name)
-            set_property(GLOBAL PROPERTY "@GEMROOT:${gem_name}@" "${gem_path}")
+        o3de_read_json_key(gem_name_with_version_specifier "${gem_path}/gem.json" "gem_name")
+        if(NOT gem_name_with_version_specifier)
+            MESSAGE(FATAL_ERROR "Failed to read the gem name from '${gem_path/gem.json}' or the gem name is empty.")
+            return()
         endif()
 
+        # Remove any version specifier
+        o3de_get_name_and_version_specifier(${gem_name_with_version_specifier} gem_name spec_op spec_version)
+        set_property(GLOBAL PROPERTY "@GEMROOT:${gem_name}@" "${gem_path}")
+
         # Push the gem name onto the visited set
         list(APPEND ${visited_gem_name_set_ref} ${gem_name})
         foreach(gem_external_subdir IN LISTS gem_external_subdirs)
@@ -153,17 +160,27 @@ endfunction()
 #! A fatal error is logged indicating that is not gem could not be found in the list of external subdirectories
 function(query_gem_paths_from_external_subdirs output_gem_dirs gem_names registered_external_subdirs)
     if (gem_names)
-        foreach(gem_name IN LISTS gem_names)
+        foreach(gem_name_with_version_specifier IN LISTS gem_names)
             unset(gem_path)
+
+            # Remove the version specifier from the gem name before fetching properties
+            o3de_get_name_and_version_specifier(${gem_name_with_version_specifier} gem_name spec_op spec_version)
+
             get_property(gem_optional GLOBAL PROPERTY ${gem_name}_OPTIONAL)
-            o3de_find_gem_with_registered_external_subdirs(${gem_name} gem_path "${registered_external_subdirs}")
+            get_property(gem_path GLOBAL PROPERTY "@GEMROOT:${gem_name}@")
+
             if (gem_path)
                 list(APPEND gem_dirs ${gem_path})
             elseif(NOT gem_optional)
-                list(JOIN registered_external_subdirs "\n" external_subdirs_formatted)
+                # Sort the list so it is easier to search visually
+                list(SORT registered_external_subdirs COMPARE NATURAL CASE INSENSITIVE ORDER ASCENDING)
+                # Indent the text to be easier to read. If the indent is removed CMake will add an
+                # additional newline automatically because "non-indented text is formatted in 
+                # line-wrapped paragraphs delimited by newlines"
+                list(JOIN registered_external_subdirs "\n  " external_subdirs_formatted)
                 message(SEND_ERROR "The gem \"${gem_name}\""
-                " could not be found in any gem.json from the following list of registered external subdirectories:\n"
-                "${external_subdirs_formatted}")
+                " could not be found in any gem.json from the following list of registered external subdirectories:"
+                "\n  ${external_subdirs_formatted}")
                 break()
             endif()
         endforeach()
@@ -171,6 +188,63 @@ function(query_gem_paths_from_external_subdirs output_gem_dirs gem_names registe
     set(${output_gem_dirs} ${gem_dirs} PARENT_SCOPE)
 endfunction()
 
+#! Use cmake.py to get a resolved list of gem names and paths for this object (engine or project)
+#! If dependencies are resolved successfully, save each gem's resolved path in a global property
+#! named "@GEMROOT:${gem_name}@"
+function(resolve_gem_dependencies object_type object_path)
+
+    # Avoid resolving dependencies for the same object type and path multiple times
+    get_property(resolved_dependencies GLOBAL PROPERTY "O3DE_RESOLVED_GEM_DEPENDENCIES_${object_type}_${object_path}")
+    if(resolved_dependencies)
+        return()
+    endif()
+
+    set(ENV{PYTHONNOUSERSITE} 1)
+    string(TOLOWER ${object_type} object_type_lower)
+    execute_process(COMMAND 
+        ${LY_PYTHON_CMD} "${LY_ROOT_FOLDER}/scripts/o3de/o3de/cmake.py" --${object_type_lower}-path "${object_path}"
+        WORKING_DIRECTORY ${LY_ROOT_FOLDER}
+        RESULT_VARIABLE O3DE_CLI_RESULT
+        OUTPUT_VARIABLE resolved_gem_dependency_output 
+        ERROR_VARIABLE O3DE_CLI_OUT
+        )
+
+    if(O3DE_CLI_RESULT)
+        message(WARNING "Dependecy resolution failed\n  Error: ${O3DE_CLI_OUT}")
+        return()
+    endif()
+
+    # Strip any whitespace which might be included in the first or last elements of the list
+    string(STRIP "${resolved_gem_dependency_output}" resolved_gem_dependency_output)
+
+    unset(gem_name)
+    foreach(entry IN LISTS resolved_gem_dependency_output)
+        if(NOT DEFINED gem_name)
+            # The first entry is the gem name
+            set(gem_name ${entry})
+        else()
+            # The next entry after every gem name is the gem path
+            cmake_path(SET gem_path "${entry}")
+
+            get_property(current_gem_path GLOBAL PROPERTY "@GEMROOT:${gem_name}@")
+            if(current_gem_path)
+                cmake_path(SET current_gem_path "${current_gem_path}")
+                cmake_path(COMPARE "${gem_path}" NOT_EQUAL "${current_gem_path}" paths_are_different)
+                if (paths_are_different)
+                    message(VERBOSE "Multiple paths were found for the same gem '${gem_name}'.\n  Current:'${current_gem_path}'\n  New:'${gem_path}'")
+                endif()
+            else()
+                message(VERBOSE "New path found for gem '${gem_name}' ${current_gem_path}")
+            endif()
+
+            set_property(GLOBAL PROPERTY "@GEMROOT:${gem_name}@" "${gem_path}")
+            unset(gem_name)
+        endif()
+    endforeach()
+
+    set_property(GLOBAL PROPERTY "O3DE_RESOLVED_GEM_DEPENDENCIES_${object_type}_${object_path}" TRUE)
+endfunction()
+
 #! Queries the list of gem names against the list of ALL registered external subdirectories
 #! in order to determine the paths corresponding to the gem names
 function(add_registered_gems_to_external_subdirs output_gem_dirs gem_names)
@@ -255,27 +329,46 @@ function(get_all_external_subdirectories_for_o3de_object output_subdirs object_t
     # These gems are registered in the users o3de_manifest.json
     o3de_read_json_array(initial_gem_names ${object_path}/${object_json_filename} "gem_names")
     set(gem_names "")
-    foreach(gem_name IN LISTS initial_gem_names)
+
+    # Gem dependency resolution can be disabled to speed up configuration 
+    # for projects where it is not needed
+    if(initial_gem_names AND NOT O3DE_DISABLE_GEM_DEPENDENCY_RESOLUTION)
+        # Resolve gem dependency names to gem paths before adding them to external subdirectories 
+        resolve_gem_dependencies(${object_type} "${object_path}")
+    endif()
+
+    foreach(gem_name_with_version_specifier IN LISTS initial_gem_names)
+
         # Use the ERROR_VARIABLE to catch the common case when it's a simple string and not a json type.
-        string(JSON json_type ERROR_VARIABLE json_error TYPE ${gem_name})
+        string(JSON json_type ERROR_VARIABLE json_error TYPE ${gem_name_with_version_specifier})
         set(gem_optional FALSE)
         if(${json_type} STREQUAL "OBJECT")
-            string(JSON gem_optional GET ${gem_name} "optional")
-            string(JSON gem_name GET ${gem_name} "name")
+            string(JSON gem_optional GET ${gem_name_with_version_specifier} "optional")
+            string(JSON gem_name_with_version_specifier GET ${gem_name_with_version_specifier} "name")
         endif()
 
+        # Remove any version specifier from the gem name
+        o3de_get_name_and_version_specifier(${gem_name_with_version_specifier} gem_name spec_op spec_version)
+
         # Set a global "optional" property on the gem name
         set_property(GLOBAL PROPERTY "${gem_name}_OPTIONAL" ${gem_optional})
+
         # Build the gem_names list with extracted names
-        list(APPEND gem_names ${gem_name})
+        list(APPEND gem_names ${gem_name_with_version_specifier})
     endforeach()
 
+    # Ensure all gems from "gem_names" are included in the settings registry 
+    # file used to load runtime gems libraries
+    ly_enable_gems(GEMS ${gem_names} PROJECT_NAME ${object_name})
+
     add_registered_gems_to_external_subdirs(object_gem_reference_dirs "${gem_names}")
     list(APPEND subdirs_for_object ${object_gem_reference_dirs})
 
     # Also append the array the "external_subdirectories" from each gem referenced through the "gem_names"
     # field
-    foreach(gem_name IN LISTS gem_names)
+    foreach(gem_name_with_version_specifier IN LISTS gem_names)
+        # Remove any version specifier from the gem name e.g. 'atom>=1.2.3' becomes 'atom'
+        o3de_get_name_and_version_specifier(${gem_name_with_version_specifier} gem_name spec_op spec_version)
         get_property(gem_real_external_subdirs GLOBAL PROPERTY O3DE_EXTERNAL_SUBDIRS_GEM_${gem_name})
         list(APPEND subdirs_for_object ${gem_real_external_subdirs})
     endforeach()
@@ -300,8 +393,16 @@ endfunction()
 #! plus all external subdirectories that every active project provides("external_subdirectories")
 #! or references("gem_names")
 function(get_external_subdirectories_in_use output_subdirs)
+    get_property(all_external_subdirs GLOBAL PROPERTY O3DE_ALL_EXTERNAL_SUBDIRECTORIES)
+    if(all_external_subdirs)
+        # This function has already run, use the calculated list of external subdirs
+        set(${output_subdirs} ${all_external_subdirs} PARENT_SCOPE)
+        return()
+    endif()
+
     # Gather the list of external subdirectories set through the O3DE_EXTERNAL_SUBDIRS Cache Variable
     get_property(all_external_subdirs CACHE O3DE_EXTERNAL_SUBDIRS PROPERTY VALUE)
+
     # Append the list of external subdirectories from the engine.json
     get_all_external_subdirectories_for_o3de_object(engine_external_subdirs "ENGINE" "" ${LY_ROOT_FOLDER} "engine.json")
     list(APPEND all_external_subdirs ${engine_external_subdirs})
@@ -321,10 +422,14 @@ function(get_external_subdirectories_in_use output_subdirs)
     # are ordered before that gem, so they are parsed first.
     reorder_dependent_gems_before_external_subdirs(all_external_subdirs "${all_external_subdirs}")
     list(REMOVE_DUPLICATES all_external_subdirs)
+
+    # Store in a global property so we don't re-calculate this list again
+    set_property(GLOBAL PROPERTY O3DE_ALL_EXTERNAL_SUBDIRECTORIES "${all_external_subdirs}")
+
     set(${output_subdirs} ${all_external_subdirs} PARENT_SCOPE)
 endfunction()
 
-#! Visit all external subdirectories that is in use by the engine and each project
+#! Visit all external subdirectories that are in use by the engine and each project
 #! This visits "external_subdirectories" listed in the engine.json,
 #! the "external_subdirectories" listed in the each LY_PROJECTS project.json,
 #! and the "external_subdirectories" listed o3de_manifest.json in which the engine.json/project.json

cmake/Version.cmake

@@ -6,6 +6,8 @@
 #
 #
 
+include_guard()
+
 string(TIMESTAMP current_year "%Y")
 set(O3DE_COPYRIGHT_YEAR ${current_year} CACHE STRING "Open 3D Engine's copyright year")
 
@@ -14,6 +16,83 @@ ly_file_read("${LY_ROOT_FOLDER}/engine.json" tmp_json_data)
 set_property(GLOBAL PROPERTY O3DE_ENGINE_JSON_DATA ${tmp_json_data})
 unset(tmp_json_data)
 
+#! o3de_get_name_and_version_specifier: Parse the dependency name, and optional version 
+# operator and version from the supplied input string. 
+# \arg:input - the input string e.g. o3de==1.0.0
+# \arg:dependency_name - the name to the left of any optional version specifier
+# \arg:operator - the version specifier operator e.g. == 
+# \arg:version - the version specifier version e.g. 1.0.0
+function(o3de_get_name_and_version_specifier input dependency_name operator version)
+    if("${input}" MATCHES "^(.*)(~=|==|!=|<=|>=|<|>|===)(.*)$")
+        if(${CMAKE_MATCH_COUNT} GREATER_EQUAL 1)
+            string(STRIP ${CMAKE_MATCH_1} _dependency_name)
+            set(${dependency_name} ${_dependency_name} PARENT_SCOPE)
+        endif()
+        if(${CMAKE_MATCH_COUNT} GREATER_EQUAL 2)
+            set(${operator} ${CMAKE_MATCH_2} PARENT_SCOPE)
+        endif()
+        if(${CMAKE_MATCH_COUNT} GREATER_EQUAL 3)
+            string(STRIP ${CMAKE_MATCH_3} _version)
+            set(${version} ${_version} PARENT_SCOPE)
+        endif()
+    else()
+        # format unknown, assume it's just the dependency name
+        set(${dependency_name} ${input} PARENT_SCOPE)
+    endif()
+endfunction()
+
+#! o3de_get_version_compatible: Check if the input version is compatible based on
+# the operator and specifier version provided
+# \arg:version - input version to check  e.g. 1.0.0
+# \arg:op - the version specifier operator e.g. ==
+# \arg:specifier_version - the version part of the version specifier  e.g. 1.2.0
+# \arg:is_compatible - TRUE if version is compatible otherwise FALSE 
+function(o3de_get_version_compatible version op specifier_version is_compatible)
+    set(contains_version FALSE)
+
+    if(op STREQUAL "==" AND version VERSION_EQUAL specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL "!=" AND NOT version VERSION_EQUAL specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL "<=" AND version VERSION_LESS_EQUAL specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL ">=" AND version VERSION_GREATER_EQUAL specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL "<" AND version VERSION_LESS specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL ">" AND version VERSION_GREATER specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL "===" AND version STREQUAL specifier_version)
+        set(contains_version TRUE)
+    elseif(op STREQUAL "~=")
+        # compatible versions have an equivalent combination of >= and == 
+        # e.g. ~=2.2 is equivalent to >=2.2,==2.*
+        if(version VERSION_GREATER_EQUAL specifier_version)
+            string(REPLACE "." ";" specifier_version_part_list ${specifier_version})
+            list(LENGTH specifier_version_part_list list_length)
+            if(list_length LESS 2)
+                # truncating would leave nothing to compare 
+                set(contains_version TRUE)
+            else()
+                # trim the last version part because CMake doesn't support '*'
+                math(EXPR truncated_length "${list_length} - 1")
+                list(SUBLIST specifier_version_part_list 0 ${truncated_length} specifier_version)
+                string(REPLACE ";" "." specifier_version "${specifier_version}")
+                string(REPLACE "." ";" version_part_list ${version})
+                list(SUBLIST version_part_list 0 ${truncated_length} version)
+                string(REPLACE ";" "." version "${version}")
+
+                # compare the truncated versions
+                if(version VERSION_EQUAL specifier_version)
+                    set(contains_version TRUE)
+                endif()
+            endif()
+        endif()
+    endif()
+
+    set(${is_compatible} ${contains_version} PARENT_SCOPE)
+endfunction()
+
 #! o3de_read_engine_default: Read a field from engine.json or use the default if not found
 # \arg:output_value - name of output variable to set 
 # \arg:key - name of field in engine.json 

engine.json

@@ -4,7 +4,7 @@
     "O3DEVersion": "0.1.0.0",
     "O3DEBuildNumber": 0,
     "display_version":"00.00",
-    "version":"1.0.0",
+    "version":"1.1.0",
     "api_versions": {
         "editor":"1.0.0",
         "framework":"1.0.0",
@@ -101,6 +101,15 @@
         "Gems/VirtualGamepad",
         "Gems/WhiteBox"
     ],
+    "gem_names": [
+        "AtomShader",
+        "Atom_Bootstrap",
+        "CommonFeaturesAtom",
+        "Camera",
+        "Maestro",
+        "PrefabBuilder",
+        "SceneProcessing"
+    ],
     "projects": [
         "AutomatedTesting"
     ],

python/requirements.txt

@@ -338,3 +338,6 @@ iniconfig==1.1.1 \
 toml==0.10.2 \
     --hash=sha256:806143ae5bfb6a3c6e736a764057db0e6a0e05e338b5630894a5f779cabb4f9b \
     --hash=sha256:b3bda1d108d5dd99f4a20d24d9c348e91c4db7ab1b749200bded2f839ccbe68f
+resolvelib==0.9.0 \
+    --hash=sha256:40ab05117c3281b1b160105e10075094c5ab118315003c922b77673a365290e1 \
+    --hash=sha256:597adcbdf81d62d0cde55d90faa8e79187ec0f18e5012df30bd7a751b26343ae

scripts/o3de/o3de/cmake.py

@@ -9,11 +9,12 @@
 Contains methods for query CMake gem target information
 """
 
+import argparse
 import logging
-import os
 import pathlib
+import sys
 
-from o3de import manifest, utils
+from o3de import manifest, utils, compatibility
 
 logger = logging.getLogger('o3de.cmake')
 logging.basicConfig(format=utils.LOG_FORMAT)
@@ -21,83 +22,9 @@ logging.basicConfig(format=utils.LOG_FORMAT)
 enable_gem_start_marker = 'set(ENABLED_GEMS'
 enable_gem_end_marker = ')'
 
-
-def add_gem_dependency(cmake_file: pathlib.Path,
-                       gem_name: str) -> int:
-    """
-    adds a gem dependency to a cmake file
-    :param cmake_file: path to the cmake file
-    :param gem_name: name of the gem
-    :return: 0 for success or non 0 failure code
-    """
-    if not cmake_file.is_file():
-        logger.error(f'Failed to locate cmake file {str(cmake_file)}')
-        return 1
-
-    # on a line by basis, see if there already is {gem_name}
-    # find the first occurrence of a gem, copy its formatting and replace
-    # the gem name with the new one and append it
-    t_data = []
-    added = False
-    start_marker_line_index = None
-    end_marker_line_index = None
-    with cmake_file.open('r') as s:
-        in_gem_list = False
-        line_index = 0
-        for line in s:
-            parsed_line = line.strip()
-            if parsed_line.startswith(enable_gem_start_marker):
-                # Skip pass the 'set(ENABLED_GEMS' marker just in case their are gems declared on the same line
-                parsed_line = parsed_line[len(enable_gem_start_marker):]
-                # Set the flag to indicate that we are in the ENABLED_GEMS variable
-                in_gem_list = True
-                start_marker_line_index = line_index
-
-            if in_gem_list:
-                # Since we are inside the ENABLED_GEMS variable determine if the line has the end_marker of ')'
-                if parsed_line.endswith(enable_gem_end_marker):
-                    # Strip away the line end marker
-                    parsed_line = parsed_line[:-len(enable_gem_end_marker)]
-                    # Set the flag to indicate that we are no longer in the ENABLED_GEMS variable after this line
-                    in_gem_list = False
-                    end_marker_line_index = line_index
-
-                # Split the rest of the line on whitespace just in case there are multiple gems in a line
-                gem_name_list = map(lambda gem_name: gem_name.strip('"'), parsed_line.split())
-                if gem_name in gem_name_list:
-                    logger.info(f'{gem_name} is already enabled in file {str(cmake_file)}.')
-                    return 0
-
-            t_data.append(line)
-            line_index += 1
-
-    indent = 4
-    if start_marker_line_index:
-        # Make sure if there is a enable gem start marker, there is an end marker as well
-        if not end_marker_line_index:
-            logger.error(f'The Enable Gem start marker of "{enable_gem_start_marker}" has been found, but not the'
-                         f' Enable Gem end marker of "{enable_gem_end_marker}"')
-            return 1
-
-        # Insert the gem before the ')' end marker
-        end_marker_partition = list(t_data[end_marker_line_index].rpartition(enable_gem_end_marker))
-        end_marker_partition[1] = f'{" " * indent}{gem_name}\n' + end_marker_partition[1]
-        t_data[end_marker_line_index] = ''.join(end_marker_partition)
-        added = True
-
-    # if we didn't add, then create a new set(ENABLED_GEMS) variable
-    # add a new gem, if empty the correct format is 1 tab=4spaces
-    if not added:
-        t_data.append('\n')
-        t_data.append(f'{enable_gem_start_marker}\n')
-        t_data.append(f'{" "  * indent}{gem_name}\n')
-        t_data.append(f'{enable_gem_end_marker}\n')
-
-    # write the cmake
-    with cmake_file.open('w') as s:
-        s.writelines(t_data)
-
-    return 0
+# The need for `enabled_gems.cmake` is deprecated
+# Functionality still exists to retrieve and remove gems from `enabled_gems.cmake`
+# but gems should only be added to `project.json` by the o3de CLI
 
 def remove_gem_dependency(cmake_file: pathlib.Path,
                           gem_name: str) -> int:
@@ -168,78 +95,129 @@ def remove_gem_dependency(cmake_file: pathlib.Path,
     return 0
 
 
-def get_enabled_gems(cmake_file: pathlib.Path) -> set:
+def resolve_gem_dependency_paths(
+        engine_path:pathlib.Path,
+        project_path:pathlib.Path,
+        resolved_gem_dependencies_output_path:pathlib.Path or None):
     """
-    Gets a list of enabled gems from the cmake file
-    :param cmake_file: path to the cmake file
-    :return: set of gem targets found
+    Resolves gem dependencies for the given engine and project and
+    writes the output to the path provided.  This is used during CMake
+    configuration because writing a CMake depencency resolver would be
+    difficult and Python already has a solver with unit tests.
+    :param engine_path: optional path to the engine, if not provided, the project's engine will be determined 
+    :param project_path: optional path to the project, if not provided the engine path must be provided
+    :param resolved_gem_dependencies_output_path: optional path to a file that will be written 
+        containing a CMake list of gem names and paths.  If not provided, the list is written to STDOUT.
+    :return: 0 for success or non 0 failure code
     """
-    cmake_file = pathlib.Path(cmake_file).resolve()
-
-    if not cmake_file.is_file():
-        logger.error(f'Failed to locate cmake file {cmake_file}')
-        return set()
 
-    gem_target_set = set()
-    with cmake_file.open('r') as s:
-        in_gem_list = False
-        for line in s:
-            line = line.strip()
-            if line.startswith(enable_gem_start_marker):
-                # Set the flag to indicate that we are in the ENABLED_GEMS variable
-                in_gem_list = True
-                # Skip pass the 'set(ENABLED_GEMS' marker just in case their are gems declared on the same line
-                line = line[len(enable_gem_start_marker):]
-            if in_gem_list:
-                # Since we are inside the ENABLED_GEMS variable determine if the line has the end_marker of ')'
-                if line.endswith(enable_gem_end_marker):
-                    # Strip away the line end marker
-                    line = line[:-len(enable_gem_end_marker)]
-                    # Set the flag to indicate that we are no longer in the ENABLED_GEMS variable after this line
-                    in_gem_list = False
-                # Split the rest of the line on whitespace just in case there are multiple gems in a line
-                gem_name_list = list(map(lambda gem_name: gem_name.strip('"'), line.split()))
-                gem_target_set.update(gem_name_list)
+    if not engine_path and not project_path:
+        logger.error(f'project path or engine path are required to resolve dependencies')
+        return 1
 
-    return gem_target_set
+    if not engine_path:
+        engine_path = manifest.get_project_engine_path(project_path=project_path)
+        if not engine_path:
+            engine_path = manifest.get_this_engine_path()
+            if not engine_path:
+                logger.error('Failed to find a valid engine path for the project at '
+                             f'"{project_path}" which is required to resolve gem dependencies.')
+                return 1
+
+            logger.warning('Failed to determine the correct engine for the project at '
+                           f'"{project_path}", falling back to this engine at {engine_path}.')
+    
+    engine_json_data = manifest.get_engine_json_data(engine_path=engine_path)
+    if not engine_json_data:
+        logger.error('Failed to retrieve engine json data for the engine at '
+                     f'"{engine_path}" which is required to resolve gem dependencies.')
+        return 1
 
+    if project_path:
+        project_json_data = manifest.get_project_json_data(project_path=project_path)
+        if not project_json_data:
+            logger.error('Failed to retrieve project json data for the project at '
+                        f'"{project_path}" which is required to resolve gem dependencies.')
+            return 1
+        active_gem_names = project_json_data.get('gem_names',[])
+        enabled_gems_file = manifest.get_enabled_gem_cmake_file(project_path=project_path)
+        if enabled_gems_file.is_file():
+            active_gem_names.extend(manifest.get_enabled_gems(enabled_gems_file))
+    else:
+        active_gem_names = engine_json_data.get('gem_names',[])
+
+    # some gem name entries will be dictionaries - convert to a set of strings 
+    gem_names_with_optional_gems = utils.get_gem_names_set(active_gem_names, include_optional=True)
+    if not gem_names_with_optional_gems:
+        logger.info(f'No gem names were found to use as input to resolve gem dependencies.')
+        if resolved_gem_dependencies_output_path:
+            with resolved_gem_dependencies_output_path.open('w') as output:
+                output.write('')
+        return 0
+
+    all_gems_json_data = manifest.get_gems_json_data_by_name(engine_path=engine_path, 
+                                                             project_path=project_path, 
+                                                             include_manifest_gems=True, 
+                                                             include_engine_gems=True)
+
+    # First try to resolve with optional gems
+    results, errors = compatibility.resolve_gem_dependencies(gem_names_with_optional_gems, 
+                                                             all_gems_json_data, 
+                                                             engine_json_data, 
+                                                             include_optional=True)
+    if errors:
+        logger.warning('Failed to resolve dependencies with optional gems, trying without optional gems.')
+
+        # Try without optional gems
+        gem_names_without_optional = utils.get_gem_names_set(active_gem_names, include_optional=False)
+        results, errors = compatibility.resolve_gem_dependencies(gem_names_without_optional, 
+                                                                 all_gems_json_data, 
+                                                                 engine_json_data,
+                                                                 include_optional=False)
+
+    if errors:
+        logger.error(f'Failed to resolve dependencies:\n  ' + '\n  '.join(errors))
+        return 1
 
-def get_enabled_gem_cmake_file(project_name: str = None,
-                                project_path: str or pathlib.Path = None,
-                                platform: str = 'Common') -> pathlib.Path or None:
-    """
-    get the standard cmake file name for a particular type of dependency
-    :param gem_name: name of the gem, resolves gem_path
-    :param gem_path: path of the gem
-    :return: list of gem targets
-    """
-    if not project_name and not project_path:
-        logger.error(f'Must supply either a Project Name or Project Path.')
-        return None
-
-    if project_name and not project_path:
-        project_path = manifest.get_registered(project_name=project_name)
-
-    project_path = pathlib.Path(project_path).resolve()
-    enable_gem_filename = "enabled_gems.cmake"
-
-    if platform == 'Common':
-        possible_project_enable_gem_filename_paths = [
-            pathlib.Path(project_path / 'Gem' / enable_gem_filename),
-            pathlib.Path(project_path / 'Gem/Code' / enable_gem_filename),
-            pathlib.Path(project_path / 'Code' / enable_gem_filename)
-        ]
-        for possible_project_enable_gem_filename_path in possible_project_enable_gem_filename_paths:
-            if possible_project_enable_gem_filename_path.is_file():
-                return possible_project_enable_gem_filename_path.resolve()
-        return possible_project_enable_gem_filename_paths[0].resolve()
+    # make a list of <gem_name>;<gem_path> for cmake
+    gem_paths = sorted(f"{gem.gem_json_data['gem_name'].strip()};{gem.gem_json_data['path'].resolve().as_posix()}" for _, gem in results.items())
+    # use dict to remove duplicates and preserve order so it's easier to read/debug
+    gem_paths = list(dict.fromkeys(gem_paths))
+    # join everything with a ';' character which is a list entry delimiter in CMake
+    # so the keys and values are all list entries
+    gem_paths_list = ';'.join(gem_paths)
+
+    if resolved_gem_dependencies_output_path:
+        with resolved_gem_dependencies_output_path.open('w') as output:
+            output.write(gem_paths_list)
     else:
-        possible_project_platform_enable_gem_filename_paths = [
-            pathlib.Path(project_path / 'Gem/Platform' / platform / enable_gem_filename),
-            pathlib.Path(project_path / 'Gem/Code/Platform' / platform / enable_gem_filename),
-            pathlib.Path(project_path / 'Code/Platform' / platform / enable_gem_filename)
-        ]
-        for possible_project_platform_enable_gem_filename_path in possible_project_platform_enable_gem_filename_paths:
-            if possible_project_platform_enable_gem_filename_path.is_file():
-                return possible_project_platform_enable_gem_filename_path.resolve()
-        return possible_project_platform_enable_gem_filename_paths[0].resolve()
+        print(gem_paths_list)
+
+    return 0
+
+def _resolve_gem_dependency_paths(args: argparse) -> int:
+    return resolve_gem_dependency_paths(
+                            engine_path=args.engine_path,
+                            project_path=args.project_path,
+                            resolved_gem_dependencies_output_path=args.gem_paths_output_file
+                             )
+
+def add_parser_args(parser):
+    group = parser.add_argument_group("resolve gem dependencies")
+    group.add_argument('-pp', '--project-path', type=pathlib.Path, required=False,
+                       help='The path to the project.')
+    group.add_argument('-ep', '--engine-path', type=pathlib.Path, required=False,
+                       help='The path to the engine.')
+    group.add_argument('-gpof', '--gem-paths-output-file', type=pathlib.Path, required=False,
+                       help='The path to the resolved gem paths output file. If not provided, the list will be output to STDOUT.')
+    parser.set_defaults(func=_resolve_gem_dependency_paths)
+
+def main():
+    the_parser = argparse.ArgumentParser()
+    add_parser_args(the_parser)
+    the_args = the_parser.parse_args()
+    ret = the_args.func(the_args) if hasattr(the_args, 'func') else 1
+    sys.exit(ret)
+
+if __name__ == "__main__":
+    main()

scripts/o3de/o3de/compatibility.py

@@ -13,6 +13,14 @@ from packaging.specifiers import SpecifierSet
 import pathlib
 import logging
 from o3de import manifest, utils, cmake, validation
+from collections import namedtuple
+from resolvelib import (
+    AbstractProvider,
+    BaseReporter,
+    InconsistentCandidate,
+    ResolutionImpossible,
+    Resolver,
+)
 
 logger = logging.getLogger('o3de.compatibility')
 logging.basicConfig(format=utils.LOG_FORMAT)
@@ -99,19 +107,19 @@ def get_project_engine_incompatible_objects(project_path:pathlib.Path, engine_pa
 
     # verify project -> gem -> engine compatibility
     active_gem_names = project_json_data.get('gem_names',[])
-    enabled_gems_file = cmake.get_enabled_gem_cmake_file(project_path=project_path)
-    active_gem_names.extend(cmake.get_enabled_gems(enabled_gems_file))
+    enabled_gems_file = manifest.get_enabled_gem_cmake_file(project_path=project_path)
+    if enabled_gems_file and enabled_gems_file.is_file():
+        active_gem_names.extend(manifest.get_enabled_gems(enabled_gems_file))
     active_gem_names = utils.get_gem_names_set(active_gem_names)
 
     # it's much more efficient to get all gem data once than to query them by name one by one
     all_gems_json_data = manifest.get_gems_json_data_by_name(engine_path, project_path, include_manifest_gems=True)
 
-    for gem_name in active_gem_names:
-        if gem_name not in all_gems_json_data:
-            logger.warning(f'Skipping compatibility check for {gem_name} because no gem.json data was found for it. '
-                'Please verify this gem is registered.')
-            continue
-        incompatible_objects.update(get_gem_engine_incompatible_objects(all_gems_json_data[gem_name], engine_json_data, all_gems_json_data))
+    # Dependency resolution takes into account gem and engine requirements so if 
+    # it succeeds, all is well
+    _, errors = resolve_gem_dependencies(active_gem_names, all_gems_json_data, engine_json_data)
+    if errors:
+        incompatible_objects.update(errors)
 
     return incompatible_objects
 
@@ -133,7 +141,8 @@ def get_incompatible_gem_dependencies(gem_json_data:dict, all_gems_json_data:dic
 
 def get_gem_project_incompatible_objects(gem_path:pathlib.Path, 
                                         gem_json_data:dict, 
-                                        project_path:pathlib.Path
+                                        project_path:pathlib.Path,
+                                        gem_name:str = None
                                         ) -> set:
     """
     Returns any incompatible objects for this gem and project.
@@ -141,6 +150,7 @@ def get_gem_project_incompatible_objects(gem_path:pathlib.Path,
     :param project_path: path to the project
     :param all_gems_json_data: optional dictionary containing data for all gems to use in compatibility checks. 
     If not provided, uses all gems from the manifest, engine and project.
+    :param gem_name: optional gem name with version specifier to use for gem dependency resolution 
     """
     # early out if this project has no assigned engine
     engine_path = manifest.get_project_engine_path(project_path=project_path)
@@ -153,22 +163,40 @@ def get_gem_project_incompatible_objects(gem_path:pathlib.Path,
         logger.error(f'Failed to load project.json data from {project_path} needed for checking compatibility')
         return set(f'project.json (missing)') 
 
-    # in the future we should check if the gem and project depend on conflicting engines and/or apis
-    # we need some way of doing version specifier overlap checks
-
     engine_json_data = manifest.get_engine_json_data(engine_path=engine_path)
     if not engine_json_data:
-        logger.error(f'Failed to load engine.json data based on the engine field in project.json or detect the engine from the current folder')
+        logger.error('Failed to load engine.json data based on the engine field in project.json or detect the engine from the current folder')
         return set(f'engine.json (missing)') 
 
     # Include the gem_path for the gem we are adding so it 
     # and any gems in 'external_subdirectories' it has will be considered 
     all_gems_json_data = manifest.get_gems_json_data_by_name(engine_path, project_path, 
         external_subdirectories=[gem_path], include_manifest_gems=True)
+    
+    # Verify we can resolve all dependencies after adding this new gem
+    active_gem_names = engine_json_data.get('gem_names',[])
+    active_gem_names.extend(project_json_data.get('gem_names',[]))
+    enabled_gems_file = manifest.get_enabled_gem_cmake_file(project_path=project_path)
+    if enabled_gems_file and enabled_gems_file.is_file():
+        active_gem_names.extend(manifest.get_enabled_gems(enabled_gems_file))
+    active_gem_names = utils.get_gem_names_set(active_gem_names)
+
+    if not gem_name:
+        gem_name = gem_json_data['gem_name']
+        gem_version = gem_json_data.get('version')
+        if gem_version:
+            # try to match the name and version from this specific gem json data
+            gem_name = f'{gem_name}=={gem_version}'
+        else:
+            # match any gem with this name
+            gem_name = f'{gem_name}>=0.0.0'
+
+    active_gem_names.add(gem_name)
 
-    # compatibility will be based on the engine the project uses and the gems visible to
-    # the engine and project
-    return get_gem_engine_incompatible_objects(gem_json_data, engine_json_data, all_gems_json_data)
+    # Dependency resolution takes into account gem and engine requirements so if 
+    # it succeeds, all is well
+    _, errors = resolve_gem_dependencies(active_gem_names, all_gems_json_data, engine_json_data)
+    return errors if errors else set()
 
 
 def get_gem_engine_incompatible_objects(gem_json_data:dict, engine_json_data:dict, all_gems_json_data:dict) -> set:
@@ -219,9 +247,13 @@ def get_incompatible_objects_for_engine(object_json_data:dict, engine_json_data:
 
         incompatible_apis = set()
         for api_version_specifier in engine_api_version_specifiers:
-            api_name, unused_version_specifiers = utils.get_object_name_and_optional_version_specifier(api_version_specifier)
-            if not has_compatible_version([api_version_specifier], api_name, engine_api_versions.get(api_name,'')):
-                incompatible_apis.add(api_version_specifier)
+            api_name, _ = utils.get_object_name_and_optional_version_specifier(api_version_specifier)
+            engine_api_version = engine_api_versions.get(api_name,'')
+            if not has_compatible_version([api_version_specifier], api_name, engine_api_version):
+                if engine_api_version:
+                    incompatible_apis.add(f"The engine '{api_name}' version '{engine_api_version}' doesn't satisfy requirement '{api_version_specifier}'")
+                else:
+                    incompatible_apis.add(f"The engine doesn't satisfy the requirement '{api_version_specifier}'")
         
         # if an object is compatible with all APIs then it's compatible even
         # if that engine is not listed in the `compatible_engines` field
@@ -235,10 +267,11 @@ def get_incompatible_objects_for_engine(object_json_data:dict, engine_json_data:
 
 def get_incompatible_gem_version_specifiers(gem_json_data:dict, all_gems_json_data:dict, checked_specifiers:set) -> set:
     """
-    Returns a set of gem version specifiers that are not compatible with the gem's provided
-    If a gem_version_specifier_list entry only has a gem name, it is assumed compatible with every gem version with that name.
-    :param gem_version_specifier_list: a list of gem names and (optional)version specifiers
+    Returns a set of gem version specifiers that are not compatible with the provided gem's dependencies
+    If a dependency entry only has a gem name, it is assumed compatible with every gem version with that name.
+    :param gem_json_data: gem json data with optional 'dependencies' list
     :param all_gems_json_data: json data of all gems to use for compatibility checks
+    :param checked_specifiers: a set of all gem specifiers already checked to avoid cycles
     """
     gem_version_specifier_list = gem_json_data.get('dependencies')
     if not gem_version_specifier_list:
@@ -250,14 +283,6 @@ def get_incompatible_gem_version_specifiers(gem_json_data:dict, all_gems_json_da
 
     incompatible_gem_version_specifiers = set()
 
-    # helper function to check dependency tree for incompatible gem dependencies
-    def get_gem_dependency_version_specifiers(gem_name):
-        gem_dependencies = all_gems_json_data[gem_name].get('dependencies')
-        if gem_dependencies:
-            incompatible_dependency_specifiers = get_incompatible_gem_version_specifiers(all_gems_json_data[gem_name], all_gems_json_data, checked_specifiers)
-            if incompatible_dependency_specifiers:
-                incompatible_gem_version_specifiers.update(incompatible_dependency_specifiers)
-
     for gem_version_specifier in gem_version_specifier_list:
         if gem_version_specifier in checked_specifiers:
             continue
@@ -269,17 +294,36 @@ def get_incompatible_gem_version_specifiers(gem_json_data:dict, all_gems_json_da
             incompatible_gem_version_specifiers.add(f"{gem_json_data['gem_name']} is missing the dependency {gem_version_specifier}")
             continue
 
-        if not version_specifier:
-            # when no version specifier is provided we assume compatibility with any version
-            get_gem_dependency_version_specifiers(gem_name)
-            continue
+        all_candidate_incompatible_gem_version_specifiers = set() 
+        # check every version of this gem and only record incompatible specifiers
+        # when we don't find one that is compatible
+        found_candidate = False
+        for candidate_gem_json_data in all_gems_json_data[gem_name]:
+            if version_specifier:
+                gem_version = candidate_gem_json_data.get('version')
+                if gem_version and not has_compatible_version([gem_version_specifier], gem_name, gem_version):
+                    if not found_candidate:
+                        all_candidate_incompatible_gem_version_specifiers.add(f"{gem_json_data['gem_name']} depends on {gem_version_specifier} but {gem_name} version {gem_version} was found")
+                    continue
+                else:
+                    found_candidate = True
+            else:
+                found_candidate = True
+
+            if found_candidate:
+                # clear all previous incompatible errors because we found a candidate
+                # we only want to show dependency incompatibility 
+                all_candidate_incompatible_gem_version_specifiers = set()
+
+            # check dependencies recursively 
+            candidate_incompatible_gem_version_specifiers = get_incompatible_gem_version_specifiers(candidate_gem_json_data, all_gems_json_data, checked_specifiers)
+            if candidate_incompatible_gem_version_specifiers:
+                all_candidate_incompatible_gem_version_specifiers.update(candidate_incompatible_gem_version_specifiers)
+            else:
+                all_candidate_incompatible_gem_version_specifiers = set()
+                break
         
-        gem_version = all_gems_json_data[gem_name].get('version')
-        if gem_version and not has_compatible_version([gem_version_specifier], gem_name, gem_version):
-            incompatible_gem_version_specifiers.add(f"{gem_json_data['gem_name']} depends on {gem_version_specifier} but {gem_name} version {gem_version} was found")
-            continue
-
-        get_gem_dependency_version_specifiers(gem_name)
+        incompatible_gem_version_specifiers.update(all_candidate_incompatible_gem_version_specifiers)
 
     return incompatible_gem_version_specifiers
 
@@ -326,3 +370,177 @@ def has_compatible_version(name_and_version_specifier_list:list, object_name:str
             pass
 
     return False 
+
+class GemRequirement(namedtuple("GemRequirement", ["name", "specifier"])):
+    def __repr__(self):
+        return f"<GemRequirement({self.name}{self.specifier})>"
+
+    def identify(self):
+        # IMPORTANT don't use the specifier or we will get multiple mappings
+        # for gems instead of unique mappings for each gem name
+        return f"GemRequirement:{self.name}"
+
+    def failure_reason(self, object_name):
+        return f'{object_name} requires {self.name}{self.specifier}'
+
+class EngineRequirement(namedtuple("EngineRequirement", ["gem_json_data"])):
+    def __repr__(self):
+        return f"<EngineRequirement({self.gem_json_data.get('compatible_engines','')}{self.gem_json_data.get('engine_api_dependencies','')})>"
+
+    def identify(self):
+        # Use a singular identifier because there is only one engine being considered
+        return "EngineRequirement"
+
+    def failure_reason(self, engine_json_data):
+        gem_name = self.gem_json_data['gem_name']
+        incompatible_engine_objects = get_incompatible_objects_for_engine(self.gem_json_data, engine_json_data)
+        incompatible_engine_objects = [f'{gem_name} is incompatible because: {error}' for error in incompatible_engine_objects]
+        return f'\n'.join(incompatible_engine_objects)
+
+class EngineCandidate(namedtuple("Candidate",["name", "version","requirements", "engine_json_data"])):
+    def __repr__(self):
+        return f"<Engine {self.name}=={self.version}>"
+
+    def identify(self):
+        return repr(self)
+
+    def __hash__(self) -> int:
+        return hash(('engine', self.name, self.version))
+
+class GemCandidate(namedtuple("Candidate",["name", "version","requirements","gem_json_data"])):
+    def __repr__(self):
+        return f"<Gem {self.name}=={self.version}>"
+
+    def identify(self):
+        # IMPORTANT identify with just the name and don't include a specifier
+        # or we will get conflicting mappings. Use a prefix to avoid collisions with Engines
+        #return f"Gem:{self.name}"
+        return repr(self)
+
+    def __hash__(self) -> int:
+        return hash(('gem', self.name, self.version))
+
+class GemDependencyProvider(AbstractProvider):
+    def __init__(self, candidates):
+        self.candidates = candidates
+
+    def identify(self, requirement_or_candidate):
+        return requirement_or_candidate.identify()
+
+    def get_preference(self, identifier, resolutions, candidates, information, **_):
+        # This function could be used in the future to make good choices 
+        # to speed up dependency resolution when selecting the order in which
+        # candidates are evaluated
+        return 0
+
+    def find_matches(self, identifier, requirements, incompatibilities):
+        # Returns a list of candidates that satisfy the requirements
+        # sorted by version so we prefer higher version numbers
+        name = identifier
+        return sorted(
+            (c
+            for c in self.candidates
+            if all(self.is_satisfied_by(r, c) for r in requirements[name])
+            and all(c.version != i.version for i in incompatibilities[name])),
+            key=lambda candidate: candidate.version,
+            reverse=True
+        )
+
+    def is_satisfied_by(self, requirement, candidate):
+        if isinstance(candidate, EngineCandidate) and isinstance(requirement, EngineRequirement):
+            incompatible_engine_objects = get_incompatible_objects_for_engine(requirement.gem_json_data, candidate.engine_json_data)
+            # If the gem has engine dependencies and we fail to satisfy them
+            # incompatible_engine_objects will contain the set of every failed
+            # dependency.  We could store these on the EngineRequirement, but 
+            # won't need it if a valid candidate is found.
+            return not incompatible_engine_objects
+        elif isinstance(candidate, GemCandidate) and isinstance(requirement,GemRequirement):
+            return (
+                candidate.name == requirement.name
+                and candidate.version in requirement.specifier
+            )
+        else:
+            # GemCandidates do no satisfy EngineRequirements
+            # EngineCandidates do not satisfy GemRequirements
+            return False
+
+    def get_dependencies(self, candidate):
+        return candidate.requirements
+
+def resolve_gem_dependencies(gem_names:list, all_gem_json_data:dict, engine_json_data:dict, include_optional=False) -> bool:
+    # Start with the engine candidate using a version of 0.0.0 for any
+    # engine that has no version field (older engine)
+    candidates = [
+        EngineCandidate(engine_json_data.get('engine_name'), engine_json_data.get('version','0.0.0'), [], engine_json_data)
+    ] 
+
+    # Add all gem candidates and their requirements
+    for gem_name, gem_versions_json_data in all_gem_json_data.items():
+        for gem_json_data in gem_versions_json_data:
+            requirements = [] 
+
+            # If the version field exists but is empty use '0.0.0'
+            # This gives us a preference for gems with version fields
+            gem_version = gem_json_data.get('version','0.0.0') or '0.0.0'
+            if gem_version[:1] == '$':
+                # Special case where this version is a variable in a template
+                # and looks like '${Version}' or similar
+                gem_version = '0.0.0'
+            gem_dependencies = gem_json_data.get('dependencies',[])
+
+            # Add gem requirements
+            gem_dependency_names = utils.get_gem_names_set(gem_dependencies, include_optional=include_optional)
+            for gem_dependency in gem_dependency_names:
+                dep_name, dep_version_specifier = utils.get_object_name_and_optional_version_specifier(gem_dependency)
+                if not dep_version_specifier:
+                    dep_version_specifier = ">=0.0.0"
+
+                requirements.append(GemRequirement(dep_name, SpecifierSet(dep_version_specifier)))
+            
+            # Add engine requirements. Technically "compatible_engines" should not
+            # be used for incompatibility, and this should be addressed in the future.
+            compatible_engines = gem_json_data.get('compatible_engines',[])
+            engine_api_dependencies = gem_json_data.get('engine_api_dependencies',[])
+            if compatible_engines or engine_api_dependencies:
+                requirements.append(EngineRequirement(gem_json_data))
+
+            candidate = GemCandidate(gem_name, Version(gem_version), requirements, gem_json_data)
+            candidates.append(candidate)
+
+    provider = GemDependencyProvider(candidates)
+    reporter = BaseReporter()
+    resolver = Resolver(provider=provider, reporter=reporter)
+
+    # Add all project gem dependency requirements
+    project_gem_requirements = set()
+    for gem_name in gem_names:
+        dep_name, dep_version_specifier = utils.get_object_name_and_optional_version_specifier(gem_name)
+        if not dep_version_specifier:
+            dep_version_specifier = ">=0.0.0"
+        project_gem_requirements.add(GemRequirement(dep_name, SpecifierSet(dep_version_specifier)))
+
+    result_mapping = None
+    errors = None
+    try:
+        result = resolver.resolve(requirements=project_gem_requirements)
+
+        # Remove any EngineCandidates that may appear in the mappings
+        result_mapping = {k: v for k, v in result.mapping.items() if isinstance(v, GemCandidate)}
+    except InconsistentCandidate as e:
+        # An error exists in our dependency resolver provider
+        # need to fix find_matches() and/or is_satisfied_by().
+        errors = set([f'An error exists in the dependency resolver provider resulting in an inconsistent candidate {e.candidate} with criterion {e.criterion}'])
+    except ResolutionImpossible as e:
+        reason = 'The following dependency requirements could not be satisfied:'
+        errors = set()
+        for cause in e.causes:
+            reason += '\n'
+            if isinstance(cause.requirement, EngineRequirement):
+                reason += cause.requirement.failure_reason(engine_json_data)
+            else:
+                # If cause.parent isn't set it's a project gem dependency
+                reason += cause.requirement.failure_reason(cause.parent if cause.parent else 'The project')
+
+        errors.add(reason) 
+
+    return result_mapping, errors

scripts/o3de/o3de/disable_gem.py

@@ -79,21 +79,38 @@ def disable_gem_in_project(gem_name: str = None,
         logger.error(f'Could not read gem.json content under {gem_path}.')
         return 1
 
-    if not enabled_gem_file:
-        enabled_gem_file = cmake.get_enabled_gem_cmake_file(project_path=project_path)
+    gem_name = gem_name or gem_json_data.get('gem_name','')
+    gem_name_without_specifier, version_specifier = utils.get_object_name_and_optional_version_specifier(gem_name)
+    ret_val = 0
 
-    # make sure this is a project has an enabled gems file
-    if not enabled_gem_file.is_file():
-        logger.error(f'Enabled gem file {enabled_gem_file} is not present.')
+    # Remove the gem from the deprecated enabled_gems.cmake file
+    gem_enabled_in_cmake = False
+    if not enabled_gem_file:
+        enabled_gem_file = manifest.get_enabled_gem_cmake_file(project_path=project_path)
+    if enabled_gem_file.is_file():
+        # enabled_gems.cmake does not support gem names with specifiers
+        gem_found_in_cmake = gem_name_without_specifier in manifest.get_enabled_gems(enabled_gem_file)
+        if gem_found_in_cmake:
+            ret_val = cmake.remove_gem_dependency(enabled_gem_file, gem_name_without_specifier)
+
+    # Remove the name of the gem from the project.json "gem_names" field 
+    project_json_data = manifest.get_project_json_data(project_path=project_path)
+    if not project_json_data:
+        logger.error(f'Could not read project.json content under {project_path}.')
         return 1
+    gem_names = project_json_data.get('gem_names',[])
 
-    # remove the gem
-    error_code = cmake.remove_gem_dependency(enabled_gem_file, gem_json_data['gem_name'])
+    # Remove the gem with the exact match, or any entry with the gem name if 
+    # they don't include a version specifier 
+    gem_found_in_gem_names = gem_name in gem_names or \
+        (not version_specifier and utils.contains_object_name(gem_name_without_specifier, gem_names)) 
+
+    if not gem_found_in_cmake and not gem_found_in_gem_names:
+        # No gems found to remove with this name
+        return 1
 
-    # Remove the name of the gem from the project.json "gem_names" field if the gem is neither
-    # registered with the project.json nor engine.json
     ret_val = project_properties.edit_project_props(project_path,
-                                                    delete_gem_names=gem_json_data['gem_name']) or error_code
+                                                    delete_gem_names=gem_name) or ret_val
 
     return ret_val
 

scripts/o3de/o3de/enable_gem.py

@@ -6,7 +6,7 @@
 #
 #
 """
-Contains command to add a gem to a project's enabled_gem.cmake file
+Contains command to add a gem to a project's project.json file
 """
 
 import argparse
@@ -15,7 +15,7 @@ import os
 import pathlib
 import sys
 
-from o3de import cmake, compatibility, manifest, project_properties, utils
+from o3de import compatibility, manifest, project_properties, utils
 
 logging.basicConfig(format=utils.LOG_FORMAT)
 logger = logging.getLogger('o3de.enable_gem')
@@ -26,17 +26,15 @@ def enable_gem_in_project(gem_name: str = None,
                           gem_path: pathlib.Path = None,
                           project_name: str = None,
                           project_path: pathlib.Path = None,
-                          enabled_gem_file: pathlib.Path = None,
                           force: bool = False,
                           dry_run: bool = False,
                           optional: bool = False) -> int:
     """
-    enable a gem in a projects enabled_gems.cmake file
-    :param gem_name: name of the gem to add
+    enable a gem in a projects project.json file
+    :param gem_name: name of the gem to add with optional version specifier, e.g. atom>=1.2.3
     :param gem_path: path to the gem to add
     :param project_name: name of to the project to add the gem to
     :param project_path: path to the project to add the gem to
-    :param enabled_gem_file: if this dependency goes/is in a specific file
     :param force: bypass version compatibility checks 
     :param dry_run: check version compatibility without modifying anything
     :param optional: mark the gem as optional
@@ -69,7 +67,7 @@ def enable_gem_in_project(gem_name: str = None,
     if gem_name and not gem_path:
         gem_path = manifest.get_registered(gem_name=gem_name, project_path=project_path)
     if not gem_path:
-        logger.error(f'Unable to locate gem path from the registered manifest.json files:'
+        logger.error(f'Unable to locate "{gem_name}" from the registered gems in:'
                      f' {str(pathlib.Path( "~/.o3de/o3de_manifest.json").expanduser())},'
                      f' {project_path / "project.json"}, engine.json')
         return 1
@@ -86,26 +84,6 @@ def enable_gem_in_project(gem_name: str = None,
         logger.error(f'Could not read gem.json content under {gem_path}.')
         return 1
 
-    if enabled_gem_file:
-        # make sure this project has an enabled gems file
-        if not enabled_gem_file.is_file():
-            logger.error(f'Enabled gem file {enabled_gem_file} is not present.')
-            return 1
-        project_enabled_gem_file = enabled_gem_file
-
-    else:
-        # Find the path to enabled gem file.
-        # It will be created if it doesn't exist
-        project_enabled_gem_file = cmake.get_enabled_gem_cmake_file(project_path=project_path)
-        if not project_enabled_gem_file.is_file():
-            project_enabled_gem_file.touch()
-
-    # Before adding the gem_dependency check if the gem is registered in either the project or engine manifest
-    buildable_gems = manifest.get_engine_gems()
-    buildable_gems.extend(manifest.get_project_gems(project_path))
-    # Convert each path to pathlib.Path object and filter out duplicates using dict.fromkeys
-    buildable_gems = list(dict.fromkeys(map(lambda gem_path_string: pathlib.Path(gem_path_string), buildable_gems)))
-
     # check compatibility
     if force:
         logger.info(f'Bypassing version compatibility check for {gem_json_data["gem_name"]}.')
@@ -114,8 +92,8 @@ def enable_gem_in_project(gem_name: str = None,
         # because most gems depend on engine gems which would not be found 
         if manifest.get_project_engine_path(project_path):
             # Note: we don't remove gems that are not active or dependencies
-            # because they will be implicitely found and activated via cmake 
-            incompatible_objects = compatibility.get_gem_project_incompatible_objects(gem_path, gem_json_data, project_path)
+            # because they will be implicitly found and activated via cmake 
+            incompatible_objects = compatibility.get_gem_project_incompatible_objects(gem_path, gem_json_data, project_path, gem_name=gem_name)
             if incompatible_objects:
                 logger.error(f'{gem_json_data["gem_name"]} has the following dependency compatibility issues and '
                     'requires the --force parameter to activate:\n  '+ 
@@ -126,29 +104,23 @@ def enable_gem_in_project(gem_name: str = None,
             logger.info(f'{gem_json_data["gem_name"]} is compatible with this project')
             return 0
 
-    ret_val = 0
-    # If the gem is not part of buildable set, it's gem_name should be registered to the "gem_names" field
-    if gem_path not in buildable_gems:
-        ret_val = project_properties.edit_project_props(project_path, new_gem_names=gem_json_data['gem_name'],
-                                                        is_optional_gem=optional)
-
-    # add the gem if it is registered in either the project.json or engine.json
-    ret_val = ret_val or cmake.add_gem_dependency(project_enabled_gem_file, gem_json_data['gem_name'])
+    # include the version specifier if provided e.g. gem==1.2.3
+    gem_name = gem_name or gem_json_data['gem_name']
 
-    return ret_val
+    return project_properties.edit_project_props(proj_path=project_path,
+                                                 new_gem_names=gem_name,
+                                                 is_optional_gem=optional)
 
 
 def add_explicit_gem_activation_for_all_paths(gem_root_folders: list,
                                               project_name: str = None,
-                                              project_path: pathlib.Path = None,
-                                              enabled_gem_file: pathlib.Path = None) -> int:
+                                              project_path: pathlib.Path = None) -> int:
     """
     walks each gem root folder directory structure and adds explicit
     activation of the gems to the project
     :param gem_root_folders: name of the gem to add
     :param project_name: name of to the project to add the gem to
     :param project_path: path to the project to add the gem to
-    :param enabled_gem_file: if this dependency goes/is in a specific file
     :return: 0 for success or non 0 failure code
     """
     if not gem_root_folders:
@@ -176,8 +148,7 @@ def add_explicit_gem_activation_for_all_paths(gem_root_folders: list,
         # Run the command to add explicit activation even if previous calls failed
         ret_val = enable_gem_in_project(gem_path=gem_dir,
                                         project_name=project_name,
-                                        project_path=project_path,
-                                        enabled_gem_file=enabled_gem_file) or ret_val
+                                        project_path=project_path) or ret_val
 
     return ret_val
 
@@ -187,18 +158,16 @@ def _run_enable_gem_in_project(args: argparse) -> int:
         return add_explicit_gem_activation_for_all_paths(
             args.all_gem_paths,
             args.project_name,
-            args.project_path,
-            args.enabled_gem_file
+            args.project_path
         )
     else:
-        return enable_gem_in_project(args.gem_name,
-                                     args.gem_path,
-                                     args.project_name,
-                                     args.project_path,
-                                     args.enabled_gem_file,
-                                     args.force,
-                                     args.dry_run,
-                                     args.optional
+        return enable_gem_in_project(gem_name=args.gem_name,
+                                     gem_path=args.gem_path,
+                                     project_name=args.project_name,
+                                     project_path=args.project_path,
+                                     force=args.force,
+                                     dry_run=args.dry_run,
+                                     optional=args.optional
                                      )
 
 
@@ -222,12 +191,9 @@ def add_parser_args(parser):
     group.add_argument('-gp', '--gem-path', type=pathlib.Path, required=False,
                        help='The path to the gem.')
     group.add_argument('-gn', '--gem-name', type=str, required=False,
-                       help='The name of the gem.')
+                       help='The name of the gem (e.g. "atom"). May also include a version specifier, e.g. "atom>=1.2.3"')
     group.add_argument('-agp', '--all-gem-paths', type=pathlib.Path, nargs='*', required=False,
                        help='Explicitly activates all gems in the path recursively.')
-    parser.add_argument('-egf', '--enabled-gem-file', type=pathlib.Path, required=False,
-                        help='The cmake enabled_gem file in which the gem names are specified.'
-                        'If not specified it will assume enabled_gems.cmake')
     group = parser.add_mutually_exclusive_group(required=False)
     group.add_argument('-f', '--force', required=False, action='store_true', default=False,
                        help='Bypass version compatibility checks')

scripts/o3de/o3de/manifest.py

@@ -13,6 +13,8 @@ import json
 import logging
 import os
 import pathlib
+from packaging.version import Version
+from collections import deque
 
 from o3de import validation, utils, repo, compatibility
 
@@ -299,6 +301,164 @@ def get_engine_templates() -> list:
 def get_project_gems(project_path: pathlib.Path) -> list:
     return get_gems_from_external_subdirectories(get_project_external_subdirectories(project_path))
 
+def get_enabled_gem_cmake_file(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                platform: str = 'Common') -> pathlib.Path or None:
+    """
+    get the standard cmake file name for a particular type of dependency
+    :param gem_name: name of the gem, resolves gem_path
+    :param gem_path: path of the gem
+    :return: list of gem targets
+    """
+    if not project_name and not project_path:
+        logger.error(f'Must supply either a Project Name or Project Path.')
+        return None
+
+    if project_name and not project_path:
+        project_path = get_registered(project_name=project_name)
+
+    project_path = pathlib.Path(project_path).resolve()
+    enable_gem_filename = "enabled_gems.cmake"
+
+    if platform == 'Common':
+        possible_project_enable_gem_filename_paths = [
+            pathlib.Path(project_path / 'Gem' / enable_gem_filename),
+            pathlib.Path(project_path / 'Gem/Code' / enable_gem_filename),
+            pathlib.Path(project_path / 'Code' / enable_gem_filename)
+        ]
+        for possible_project_enable_gem_filename_path in possible_project_enable_gem_filename_paths:
+            if possible_project_enable_gem_filename_path.is_file():
+                return possible_project_enable_gem_filename_path.resolve()
+        return possible_project_enable_gem_filename_paths[0].resolve()
+    else:
+        possible_project_platform_enable_gem_filename_paths = [
+            pathlib.Path(project_path / 'Gem/Platform' / platform / enable_gem_filename),
+            pathlib.Path(project_path / 'Gem/Code/Platform' / platform / enable_gem_filename),
+            pathlib.Path(project_path / 'Code/Platform' / platform / enable_gem_filename)
+        ]
+        for possible_project_platform_enable_gem_filename_path in possible_project_platform_enable_gem_filename_paths:
+            if possible_project_platform_enable_gem_filename_path.is_file():
+                return possible_project_platform_enable_gem_filename_path.resolve()
+        return possible_project_platform_enable_gem_filename_paths[0].resolve()
+
+
+def get_enabled_gems(cmake_file: pathlib.Path) -> set:
+    """
+    Gets a list of enabled gems from the cmake file
+    :param cmake_file: path to the cmake file
+    :return: set of gem targets found
+    """
+    cmake_file = pathlib.Path(cmake_file).resolve()
+
+    if not cmake_file.is_file():
+        logger.error(f'Failed to locate cmake file {cmake_file}')
+        return set()
+
+    enable_gem_start_marker = 'set(ENABLED_GEMS'
+    enable_gem_end_marker = ')'
+
+    gem_target_set = set()
+    with cmake_file.open('r') as s:
+        in_gem_list = False
+        for line in s:
+            line = line.strip()
+            if line.startswith(enable_gem_start_marker):
+                # Set the flag to indicate that we are in the ENABLED_GEMS variable
+                in_gem_list = True
+                # Skip pass the 'set(ENABLED_GEMS' marker just in case their are gems declared on the same line
+                line = line[len(enable_gem_start_marker):]
+            if in_gem_list:
+                # Since we are inside the ENABLED_GEMS variable determine if the line has the end_marker of ')'
+                if line.endswith(enable_gem_end_marker):
+                    # Strip away the line end marker
+                    line = line[:-len(enable_gem_end_marker)]
+                    # Set the flag to indicate that we are no longer in the ENABLED_GEMS variable after this line
+                    in_gem_list = False
+                # Split the rest of the line on whitespace just in case there are multiple gems in a line
+                gem_name_list = list(map(lambda gem_name: gem_name.strip('"'), line.split()))
+                gem_target_set.update(gem_name_list)
+
+    return gem_target_set
+
+
+
+def get_project_enabled_gems(project_path: pathlib.Path, include_dependencies:bool = True) -> dict or None:
+    """
+    Returns a dictionary of "<gem name with optional specifier>":"<gem path>"
+    Example: {"gemA>=1.2.3":"c:/gemA", "gemB":"c:/gemB"}
+    :param project_path The path to the project
+    :param include_gem_dependencies True to include all gem dependencies, otherwise just return
+    gems listed in project.json and the deprecated enabled_gems.json
+    """
+    project_json_data = get_project_json_data(project_path=project_path)
+    active_gem_names = project_json_data.get('gem_names',[])
+    enabled_gems_file = get_enabled_gem_cmake_file(project_path=project_path)
+    if enabled_gems_file and enabled_gems_file.is_file():
+        active_gem_names.extend(get_enabled_gems(enabled_gems_file))
+
+    gem_names_with_optional_gems = utils.get_gem_names_set(active_gem_names, include_optional=True)
+    if not gem_names_with_optional_gems:
+        return {}
+    
+    # We have the gem names but not the resolved paths yet
+    result = {gem_name: None for gem_name in gem_names_with_optional_gems}
+
+    engine_path = get_project_engine_path(project_path=project_path)
+    if not engine_path:
+        engine_path = get_this_engine_path()
+        if not engine_path:
+            logger.error('Failed to find an engine path for the project at '
+                            f'"{project_path}" which is required to resolve gem dependencies.')
+            return result
+
+        logger.warning('Failed to determine the correct engine for the project at '
+                        f'"{project_path}", falling back to this engine at {engine_path}.')
+    
+    engine_json_data = get_engine_json_data(engine_path=engine_path)
+    if not engine_json_data:
+        logger.error('Failed to retrieve engine json data for the engine at '
+                     f'"{engine_path}" which is required to resolve gem dependencies.')
+        return result 
+
+    all_gems_json_data = get_gems_json_data_by_name(engine_path=engine_path, 
+                                                    project_path=project_path, 
+                                                    include_manifest_gems=True, 
+                                                    include_engine_gems=True)
+
+    # we need a mapping of gem name to gem name with version specifier because
+    # the resolver will remove the version specifier
+    gem_names_with_version_specifiers = {}
+    for gem_name_with_specifier in gem_names_with_optional_gems:
+        gem_name_only, _ = utils.get_object_name_and_optional_version_specifier(gem_name_with_specifier)
+        gem_names_with_version_specifiers[gem_name_only] = gem_name_with_specifier
+
+    # Try to resolve with optional gems
+    resolved_gems, errors = compatibility.resolve_gem_dependencies(gem_names_with_optional_gems, 
+                                                             all_gems_json_data, 
+                                                             engine_json_data, 
+                                                             include_optional=True)
+    if errors:
+        # Try without optional gems
+        gem_names_without_optional = utils.get_gem_names_set(active_gem_names, include_optional=False)
+        resolved_gems, errors = compatibility.resolve_gem_dependencies(gem_names_without_optional, 
+                                                                 all_gems_json_data, 
+                                                                 engine_json_data,
+                                                                 include_optional=False)
+    if not errors:
+        for _, gem in resolved_gems.items():
+            gem_name = gem.gem_json_data['gem_name']
+            gem_name_with_specifier = gem_names_with_version_specifiers.get(gem_name,gem_name)
+            if gem_name_with_specifier in result or include_dependencies:
+                result[gem_name_with_specifier] = gem.gem_json_data['path'].as_posix()
+    else:
+        # Likely there is no resolution because gems are missing or wrong version
+        # Provide the paths for the gems that are available
+        for gem_name in result.keys():
+            gem_path = get_most_compatible_gem(gem_name, all_gems_json_data)
+            if gem_path:
+                result[gem_name] = gem_path.resolve().as_posix()
+    return result
+
 
 def get_project_external_subdirectories(project_path: pathlib.Path) -> list:
     project_object = get_project_json_data(project_path=project_path)
@@ -515,7 +675,24 @@ def get_gems_json_data_by_name(engine_path:pathlib.Path = None,
         get_gem_external_subdirectories(gem_path, list(), all_gems_json_data)
 
     # convert from being keyed on gem_path to gem_name and store the paths
-    utils.replace_dict_keys_with_value_key(all_gems_json_data, value_key='gem_name', replaced_key_name='path')
+    # resulting dictionary format will look like
+    # {
+    #     '<gem name>': [
+    #         {'gem_name':'<gem name>', 'version':'<version>', 'path':'<path>'}
+    #     ],
+    # }
+    #     e.g.
+    # {
+    #     'gem1': [
+    #         {'gem_name':'gem1', 'version':'1.0.0'},
+    #         {'gem_name':'gem1', 'version':'2.0.0'},
+    #     ],
+    #     'gem2': [
+    #         {'gem_name':'gem2', 'version':'1.0.0'},
+    #         {'gem_name':'gem2', 'version':'2.0.0'},
+    #     ],
+    # }
+    utils.replace_dict_keys_with_value_key(all_gems_json_data, value_key='gem_name', replaced_key_name='path', place_values_in_list=True)
 
     return all_gems_json_data
 
@@ -741,6 +918,89 @@ def get_repo_path(repo_uri: str, cache_folder: str or pathlib.Path = None) -> pa
     return cache_file
 
 
+def get_most_compatible_gem(gem_name: str, 
+                            gem_json_data_by_name: dict or None) -> pathlib.Path or None:
+    """
+    Optimized version of get_most_compatible_object() for gems when we have already
+    opened all the gem.json files
+    :param gem_name The gem name with optional version specifier, example: o3de>=1.2.3
+    :param gem_json_data_by_name Gem data from get_gems_json_data_by_name()
+    """
+    gem_name_with_version_specifier = gem_name
+    gem_name, version_specifier = utils.get_object_name_and_optional_version_specifier(gem_name)
+    if not gem_name in gem_json_data_by_name:
+        return None
+
+    matching_paths = deque()
+    most_compatible_version = Version('0.0.0')
+    for gem_json_data in gem_json_data_by_name.get(gem_name, {}):
+        if version_specifier:
+            candidate_version = gem_json_data.get('version','0.0.0')
+            if compatibility.has_compatible_version([gem_name_with_version_specifier], gem_name, candidate_version):
+                if not matching_paths:
+                    matching_paths.appendleft(gem_json_data['path'])
+                    most_compatible_version = Version(candidate_version)
+                elif Version(candidate_version) > most_compatible_version:
+                    matching_paths.appendleft(gem_json_data['path'])
+                    most_compatible_version = Version(candidate_version)
+                else:
+                    matching_paths.append(gem_json_data['path'])
+        else:
+            matching_paths.append(gem_json_data['path'])
+
+    return None if not matching_paths else matching_paths[0]
+
+
+def get_most_compatible_object(object_name: str, 
+                              object_typename: str, 
+                              object_validator: callable, 
+                              name_key: str, 
+                              objects: list) -> pathlib.Path or None:
+    """
+    Looks for the most compatible object based on object_name which may contain a version specifier.
+    Example: o3de>=1.2.3
+
+    :param object_name: Name of the object with optional version specifier 
+    :param object_typename: Type of object e.g. 'engine','project' or 'gem' 
+    :param object_validator: Validator to use for json file 
+    :param name_key: Object name key inside the object's json file e.g. 'engine_name' 
+    :param objects: List of paths to search
+    :param gem_json_data_by_name
+    """
+    matching_paths = deque()
+    most_compatible_version = Version('0.0.0')
+    object_name, version_specifier = utils.get_object_name_and_optional_version_specifier(object_name)
+    for object in objects:
+        if isinstance(object, dict):
+            path = pathlib.Path(object['path']).resolve()
+        else:
+            path = pathlib.Path(object).resolve()
+
+        json_data = get_json_data(object_typename, path, object_validator)
+        if json_data:
+            candidate_name = json_data.get(name_key,'')
+            if version_specifier:
+                candidate_version = json_data.get('version','0.0.0')
+                if compatibility.has_compatible_version([object_name + version_specifier], candidate_name, candidate_version):
+                    if not matching_paths:
+                        matching_paths.appendleft(path)
+                        most_compatible_version = Version(candidate_version)
+                    elif Version(candidate_version) > most_compatible_version:
+                        matching_paths.appendleft(path)
+                        most_compatible_version = Version(candidate_version)
+                    else:
+                        matching_paths.append(path)
+            elif candidate_name == object_name:
+                matching_paths.append(path)
+    if matching_paths:
+        best_candidate_path = matching_paths[0]
+        if len(matching_paths) > 1:
+            matches = "\n".join(map(str,matching_paths))
+            logger.warning(f"Multiple matches found for: '{object_name}'\n{matches}\nMost compatible match: '{best_candidate_path}'")
+        return best_candidate_path
+
+    return None
+
 def get_registered(engine_name: str = None,
                    project_name: str = None,
                    gem_name: str = None,
@@ -775,54 +1035,11 @@ def get_registered(engine_name: str = None,
     """
     json_data = load_o3de_manifest()
 
-    # check global first then this engine
     if isinstance(engine_name, str):
-        engines = get_manifest_engines()
-        matching_engine_paths = []
-        for engine in engines:
-            if isinstance(engine, dict):
-                engine_path = pathlib.Path(engine['path']).resolve()
-            else:
-                engine_path = pathlib.Path(engine).resolve()
-
-            engine_json = engine_path / 'engine.json'
-            if not pathlib.Path(engine_json).is_file():
-                logger.warning(f'{engine_json} does not exist')
-            else:
-                with engine_json.open('r') as f:
-                    try:
-                        engine_json_data = json.load(f)
-                    except json.JSONDecodeError as e:
-                        logger.warning(f'{engine_json} failed to load: {str(e)}')
-                    else:
-                        this_engines_name = engine_json_data.get('engine_name','')
-                        if this_engines_name == engine_name:
-                            matching_engine_paths.append(engine_path)
-        if matching_engine_paths:
-            engine_path = matching_engine_paths[0]
-            if len(matching_engine_paths) > 1:
-                engines = "\n".join(map(str,matching_engine_paths))
-                logger.warning(f"Multiple engines were found that match: '{engine_name}'\n{engines}\nSelecting first engine: '{engine_path}'")
-            return engine_path
-        
+        return get_most_compatible_object(engine_name, 'engine', validation.valid_o3de_engine_json, 'engine_name', get_manifest_engines())
 
     elif isinstance(project_name, str):
-        projects = get_all_projects()
-        for project_path in projects:
-            project_path = pathlib.Path(project_path).resolve()
-            project_json = project_path / 'project.json'
-            if not pathlib.Path(project_json).is_file():
-                logger.warning(f'{project_json} does not exist')
-            else:
-                with project_json.open('r') as f:
-                    try:
-                        project_json_data = json.load(f)
-                    except json.JSONDecodeError as e:
-                        logger.warning(f'{project_json} failed to load: {str(e)}')
-                    else:
-                        this_projects_name = project_json_data['project_name']
-                        if this_projects_name == project_name:
-                            return project_path
+        return get_most_compatible_object(project_name, 'project', validation.valid_o3de_project_json, 'project_name', get_all_projects())
 
     elif isinstance(gem_name, str):
         gems = []
@@ -839,22 +1056,7 @@ def get_registered(engine_name: str = None,
                 for registered_project_path in registered_project_paths:
                     gems.extend(get_all_gems(registered_project_path))
                 gems = list(dict.fromkeys(gems))
-
-        for gem_path in gems:
-            gem_path = pathlib.Path(gem_path).resolve()
-            gem_json = gem_path / 'gem.json'
-            if not pathlib.Path(gem_json).is_file():
-                logger.warning(f'{gem_json} does not exist')
-            else:
-                with gem_json.open('r') as f:
-                    try:
-                        gem_json_data = json.load(f)
-                    except json.JSONDecodeError as e:
-                        logger.warning(f'{gem_json} failed to load: {str(e)}')
-                    else:
-                        this_gems_name = gem_json_data['gem_name']
-                        if this_gems_name == gem_name:
-                            return gem_path
+        return get_most_compatible_object(gem_name, 'gem', validation.valid_o3de_gem_json, 'gem_name', gems)
 
     elif isinstance(template_name, str):
         templates = []

scripts/o3de/o3de/project_manager_interface.py

@@ -105,7 +105,7 @@ def create_project(project_info: dict, template_path: str):
     pass
 
 
-def get_enabled_gem_names(project_path: str) -> list:
+def get_enabled_gems(project_path: str, include_dependencies: bool = True) -> list:
     """
         Call get_enabled_gem_cmake_file for project_path
 
@@ -115,7 +115,7 @@ def get_enabled_gem_names(project_path: str) -> list:
 
         :return list of strs of enable gems for project.
     """
-    return list()
+    return manifest.get_project_enabled_gems(project_path=project_path, include_dependencies=include_dependencies)
 
 
 def get_project_info(project_path: str) -> dict or None:

scripts/o3de/o3de/project_properties.py

@@ -43,16 +43,28 @@ def _edit_gem_names(proj_json: dict,
         add_list = new_gem_names.split() if isinstance(new_gem_names, str) else new_gem_names
         if is_optional_gem:
             add_list = [dict(name=gem_name, optional=True) for gem_name in add_list]
+
+        def is_version_of_gem(candidate: str or dict, gem_name) -> bool:
+            candidate_gem_name = candidate if isinstance(candidate, str) else candidate.get('name')
+            candidate_gem_name_only, _ = utils.get_object_name_and_optional_version_specifier(candidate_gem_name)
+            return candidate_gem_name_only == gem_name
+
+        # remove any versions of the existing gem first
+        if proj_json.get('gem_names',[]):
+            for new_gem_name in add_list:
+                new_gem_name = new_gem_name if isinstance(new_gem_name,str) else new_gem_name['name']
+                gem_name_only, _ = utils.get_object_name_and_optional_version_specifier(new_gem_name)
+                proj_json['gem_names'] = [gem for gem in proj_json['gem_names'] if not is_version_of_gem(gem, gem_name_only)]
+            
         proj_json.setdefault('gem_names', []).extend(add_list)
 
     if delete_gem_names:
         removal_list = delete_gem_names.split() if isinstance(delete_gem_names, str) else delete_gem_names
         if 'gem_names' in proj_json:
             def in_list(gem: str or dict, remove_list: list) -> bool:
-                if isinstance(gem, dict):
-                    return gem.get('name', '') in remove_list
-                else:
-                    return gem in remove_list
+                gem_name_with_specifier = gem.get('name', '') if isinstance(gem, dict) else gem
+                gem_name, _ = utils.get_object_name_and_optional_version_specifier(gem_name_with_specifier)
+                return gem_name_with_specifier in remove_list or gem_name in remove_list 
 
             proj_json['gem_names'] = [gem for gem in proj_json['gem_names'] if not in_list(gem, removal_list)]
 

scripts/o3de/o3de/utils.py

@@ -349,14 +349,30 @@ def find_ancestor_dir_containing_file(target_file_name: pathlib.PurePath, start_
     return ancestor_file.parent if ancestor_file else None
 
 
-def get_gem_names_set(gems: list) -> set:
+def get_gem_names_set(gems: list, include_optional:bool = True) -> set:
     """
     For working with the 'gem_names' lists in project.json
     Returns a set of gem names in a list of gems
     :param gems: The original list of gems, strings or small dicts (json objects)
+    :param include_optional: If false, exclude optional gems
     :return: A set of gem name strings
     """
-    return set([gem['name'] if isinstance(gem, dict) else gem for gem in gems])
+    return set([gem['name'] if isinstance(gem, dict) and (include_optional or not gem.get('optional', False)) else gem for gem in gems])
+
+
+def contains_object_name(object_name:str, candidates:list) -> bool:
+    """
+    Returns True if any item in the list of candidates contains object_name with or
+    without a version specifier
+    :param object_name: The object name to search for 
+    :param candidates: The list of candidate object names with optional version specifiers 
+    :return: True if a match is found 
+    """
+    for candidate in candidates:
+        candidate_name, _ = get_object_name_and_optional_version_specifier(candidate)
+        if candidate_name == object_name:
+            return True
+    return False
 
 
 def remove_gem_duplicates(gems: list) -> list:
@@ -474,6 +490,15 @@ def get_object_name_and_version_specifier(input:str) -> (str, str) or None:
     return match.group("object_name").strip(), match.group("version_specifier").strip()
 
 
+def object_name_found(input:str, match:str) -> bool:
+    """
+    Returns True if the object name in the input string matches match 
+    :param input: The input string
+    :param match: The object name to match
+    """
+    object_name, _ = get_object_name_and_optional_version_specifier(input)
+    return object_name == match
+
 def get_object_name_and_optional_version_specifier(input:str):
     """
     Returns an object name and optional version specifier 
@@ -485,7 +510,7 @@ def get_object_name_and_optional_version_specifier(input:str):
         return input, None
 
 
-def replace_dict_keys_with_value_key(input:dict, value_key:str, replaced_key_name:str = None):
+def replace_dict_keys_with_value_key(input:dict, value_key:str, replaced_key_name:str = None, place_values_in_list:bool = False):
     """
     Takes a dictionary of dictionaries and replaces the keys with the value of 
     a specific value key.
@@ -494,6 +519,7 @@ def replace_dict_keys_with_value_key(input:dict, value_key:str, replaced_key_nam
     :param input: A dictionary of key->value pairs where every value is a dictionary that has a value_key
     :param value_key: The value's key to replace the current key with
     :param replaced_key_name: (Optional) A key name under which to store the replaced key in value
+    :param place_values_in_list: (Optional) Put the values in a list, useful when the new key is not unique
     """
 
     # we cannot iterate over the dict while deleting entries
@@ -515,4 +541,9 @@ def replace_dict_keys_with_value_key(input:dict, value_key:str, replaced_key_nam
         del input[key]
 
         # replace with an entry keyed on value_key's value
-        input[value[value_key]] = value
+        if place_values_in_list:
+            entries = input.get(value[value_key], [])
+            entries.append(value)
+            input[value[value_key]] = entries
+        else:
+            input[value[value_key]] = value

scripts/o3de/tests/test_cmake.py

@@ -11,139 +11,15 @@ import pytest
 import pathlib
 from unittest.mock import patch
 
-from o3de import cmake
-
-
-class TestGetEnabledGems:
-    @pytest.mark.parametrize(
-        "enable_gems_cmake_data, expected_set", [
-            pytest.param("""
-                # Comment
-                set(ENABLED_GEMS foo bar baz)
-            """, set(['foo', 'bar', 'baz'])),
-            pytest.param("""
-                        # Comment
-                        set(ENABLED_GEMS
-                            foo
-                            bar
-                            baz
-                        )
-                    """, set(['foo', 'bar', 'baz'])),
-            pytest.param("""
-                    # Comment
-                    set(ENABLED_GEMS
-                        foo
-                        bar
-                        baz)
-                """, set(['foo', 'bar', 'baz'])),
-            pytest.param("""
-                        # Comment
-                        set(ENABLED_GEMS
-                            foo bar
-                            baz)
-                    """, set(['foo', 'bar', 'baz'])),
-            pytest.param("""
-                        # Comment
-                        set(RANDOM_VARIABLE TestGame, TestProject Test Engine)
-                        set(ENABLED_GEMS HelloWorld IceCream
-                            foo
-                            baz bar
-                            baz baz baz baz baz morebaz lessbaz
-                        )
-                        Random Text
-                    """, set(['HelloWorld', 'IceCream', 'foo', 'bar', 'baz', 'morebaz', 'lessbaz'])),
-        ]
-    )
-    def test_get_enabled_gems(self, enable_gems_cmake_data, expected_set):
-        enabled_gems_set = set()
-        with patch('pathlib.Path.resolve', return_value=pathlib.Path('enabled_gems.cmake')) as pathlib_is_resolve_mock,\
-                patch('pathlib.Path.is_file', return_value=True) as pathlib_is_file_mock,\
-                patch('pathlib.Path.open', return_value=io.StringIO(enable_gems_cmake_data)) as pathlib_open_mock:
-            enabled_gems_set = cmake.get_enabled_gems(pathlib.Path('enabled_gems.cmake'))
-
-        assert enabled_gems_set == expected_set
-
-
-class TestAddGemDependency:
-    @pytest.mark.parametrize(
-        "enable_gems_cmake_data, expected_set, expected_return", [
-            pytest.param("""
-                # Comment
-                set(ENABLED_GEMS foo bar baz)
-            """, set(['foo', 'bar', 'baz', 'TestGem']), 0),
-            pytest.param("""
-                        # Comment
-                        set(ENABLED_GEMS
-                            foo
-                            bar
-                            baz
-                        )
-                    """, set(['foo', 'bar', 'baz', 'TestGem']), 0),
-            pytest.param("""
-                    # Comment
-                    set(ENABLED_GEMS
-                        foo
-                        bar
-                        baz)
-                """, set(['foo', 'bar', 'baz', 'TestGem']), 0),
-            pytest.param("""
-                        # Comment
-                        set(ENABLED_GEMS
-                            foo bar
-                            baz)
-                    """, set(['foo', 'bar', 'baz', 'TestGem']), 0),
-            pytest.param("""
-                """, set(['TestGem']), 0),
-            pytest.param("""
-                        # Comment
-                        set(RANDOM_VARIABLE TestGame, TestProject Test Engine)
-                        set(ENABLED_GEMS HelloWorld IceCream
-                            foo
-                            baz bar
-                            baz baz baz baz baz morebaz lessbaz
-                        )
-                        Random Text
-                    """, set(['HelloWorld', 'IceCream', 'foo', 'bar', 'baz', 'morebaz', 'lessbaz', 'TestGem']),
-                         0),
-            pytest.param("""
-                        set(ENABLED_GEMS foo bar baz
-                """, set(['foo', 'bar', 'baz']), 1),
-        ]
-    )
-    def test_add_gem_dependency(self, enable_gems_cmake_data, expected_set, expected_return):
-        enabled_gems_set = set()
-        add_gem_return = None
-
-        class StringBufferIOWrapper(io.StringIO):
-            def __init__(self):
-                nonlocal enable_gems_cmake_data
-                super().__init__(enable_gems_cmake_data)
-            def __enter__(self):
-                return super().__enter__()
-            def __exit__(self, exc_type, exc_val, exc_tb):
-                nonlocal enable_gems_cmake_data
-                enable_gems_cmake_data = super().getvalue()
-                super().__exit__(exc_tb, exc_val, exc_tb)
-
-
-        with patch('pathlib.Path.resolve', return_value=pathlib.Path('enabled_gems.cmake')) as pathlib_is_resolve_mock,\
-                patch('pathlib.Path.is_file', return_value=True) as pathlib_is_file_mock,\
-                patch('pathlib.Path.open', side_effect=lambda mode: StringBufferIOWrapper()) as pathlib_open_mock:
-
-            add_gem_return = cmake.add_gem_dependency(pathlib.Path('enabled_gems.cmake'), 'TestGem')
-            enabled_gems_set = cmake.get_enabled_gems(pathlib.Path('enabled_gems.cmake'))
-
-        assert add_gem_return == expected_return
-        assert enabled_gems_set == expected_set
-
+from o3de import cmake, manifest
 
 class TestRemoveGemDependency:
     @pytest.mark.parametrize(
-        "enable_gems_cmake_data, expected_set, expected_return", [
+        "enable_gems_cmake_data, gem_name, expected_set, expected_return", [
             pytest.param("""
                 # Comment
                 set(ENABLED_GEMS foo bar baz TestGem)
-            """, set(['foo', 'bar', 'baz']), 0),
+            """, 'TestGem', set(['foo', 'bar', 'baz']), 0),
             pytest.param("""
                         # Comment
                         set(ENABLED_GEMS
@@ -152,7 +28,7 @@ class TestRemoveGemDependency:
                             baz
                             TestGem
                         )
-                    """, set(['foo', 'bar', 'baz']), 0),
+                    """, 'TestGem', set(['foo', 'bar', 'baz']), 0),
             pytest.param("""
                     # Comment
                     set(ENABLED_GEMS
@@ -160,13 +36,13 @@ class TestRemoveGemDependency:
                         bar
                         baz
                         TestGem)
-                """, set(['foo', 'bar', 'baz']), 0),
+                """, 'TestGem', set(['foo', 'bar', 'baz']), 0),
             pytest.param("""
                         # Comment
                         set(ENABLED_GEMS
                             foo bar
                             baz TestGem)
-                    """, set(['foo', 'bar', 'baz']), 0),
+                    """, 'TestGem', set(['foo', 'bar', 'baz']), 0),
             pytest.param("""
                         # Comment
                         set(ENABLED_GEMS
@@ -177,7 +53,7 @@ class TestRemoveGemDependency:
                             baz
                         )
                         Random Text
-                    """, set(['foo', 'bar', 'baz']),
+                    """, 'TestGem', set(['foo', 'bar', 'baz']),
                          0),
             pytest.param("""
                         set(ENABLED_GEMS
@@ -186,19 +62,19 @@ class TestRemoveGemDependency:
                             baz
                             "TestGem"
                         )
-                """, set(['foo', 'bar', 'baz']), 0),
+                """, 'TestGem', set(['foo', 'bar', 'baz']), 0),
             pytest.param("""
-            """, set(), 1),
+            """, 'TestGem', set(), 1),
             pytest.param("""
                 set(ENABLED_GEMS
                     foo
                     bar
                     baz
                 )
-                """, set(['foo', 'bar', 'baz']), 1),
+                """, 'TestGem', set(['foo', 'bar', 'baz']), 1)
         ]
     )
-    def test_remove_gem_dependency(self, enable_gems_cmake_data, expected_set, expected_return):
+    def test_remove_gem_dependency(self, enable_gems_cmake_data, gem_name, expected_set, expected_return):
         enabled_gems_set = set()
         add_gem_return = None
 
@@ -218,8 +94,120 @@ class TestRemoveGemDependency:
                 patch('pathlib.Path.is_file', return_value=True) as pathlib_is_file_mock,\
                 patch('pathlib.Path.open', side_effect=lambda mode: StringBufferIOWrapper()) as pathlib_open_mock:
 
-            add_gem_return = cmake.remove_gem_dependency(pathlib.Path('enabled_gems.cmake'), 'TestGem')
-            enabled_gems_set = cmake.get_enabled_gems(pathlib.Path('enabled_gems.cmake'))
+            add_gem_return = cmake.remove_gem_dependency(pathlib.Path('enabled_gems.cmake'), gem_name=gem_name)
+            enabled_gems_set = manifest.get_enabled_gems(pathlib.Path('enabled_gems.cmake'))
 
         assert add_gem_return == expected_return
         assert enabled_gems_set == expected_set
+
+
+class TestResolveGemDependencyPaths:
+
+    @staticmethod
+    def resolve(self):
+        return self
+
+    @pytest.mark.parametrize(
+        "engine_gem_names, project_gem_names, all_gems_json_data, expected_gem_paths, expected_result", [
+            # When no version specifiers are provided expect the gem dependency is found
+            pytest.param([], ["gemA"],{
+                "gemA":[
+                    {"gem_name":"gemA", "path":pathlib.Path('gemAPath')},
+                ]
+            }, 'gemA;gemAPath', 0),
+            # When multiple gem versions exists, expect the one with the highest version is selected
+            pytest.param([], ["gemA","gemB"],{
+                "gemA":[
+                    {"gem_name":"gemA", "path":pathlib.Path('gemAPath')},
+                ],
+                "gemB":[
+                    {"gem_name":"gemB", "version":"0.0.0","path":pathlib.Path('gemB0Path')},
+                    {"gem_name":"gemB", "version":"2.0.0","path":pathlib.Path('gemB2Path')},
+                    {"gem_name":"gemB", "version":"1.0.0","path":pathlib.Path('gemB1Path')}
+                ]
+            }, 'gemA;gemAPath;gemB;gemB2Path', 0),
+            # When a specific version is requested expect it is found
+            pytest.param([], ["gemA==1.2.3","gemB==2.3.4"],{
+                "gemA":[
+                    {"gem_name":"gemA", "version":"1.2.3", "path":pathlib.Path('gemA1Path')},
+                    {"gem_name":"gemA", "version":"2.3.4", "path":pathlib.Path('gemA2Path')},
+                ],
+                "gemB":[
+                    {"gem_name":"gemB", "version":"1.2.3","path":pathlib.Path('gemB1Path')},
+                    {"gem_name":"gemB", "version":"2.3.4","path":pathlib.Path('gemB2Path')},
+                ]
+            }, 'gemA;gemA1Path;gemB;gemB2Path', 0),
+            # When no project gems are provided expect engine gems are used
+            pytest.param(["gemA==1.2.3"], [],{
+                "gemA":[
+                    {"gem_name":"gemA", "version":"1.2.3", "path":pathlib.Path('gemA1Path')},
+                    {"gem_name":"gemA", "version":"2.3.4", "path":pathlib.Path('gemA2Path')},
+                ]
+            }, 'gemA;gemA1Path', 0) 
+        ]
+    )
+    def test_resolve_gem_dependency_paths(self, engine_gem_names, project_gem_names, all_gems_json_data, expected_gem_paths, expected_result):
+        engine_path = pathlib.PurePath('c:/o3de')
+        project_path = pathlib.PurePath('c:/o3de')
+        resolved_gem_paths = ''
+
+        def get_project_engine_path(project_path:str or pathlib.Path) -> pathlib.Path or None:
+            return engine_path
+
+        def get_this_engine_path():
+            return engine_path
+
+        def get_engine_json_data(engine_name:str = None, engine_path:pathlib.Path = None):
+            return {
+                "engine_name":"o3de",
+                "gem_names": engine_gem_names
+            } 
+
+        def get_project_json_data(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                user: bool = False) -> dict or None:
+            return {
+                "project_name":"o3de_project",
+                "engine":"o3de",
+                "gem_names": project_gem_names
+            } 
+
+        class StringBufferIOWrapper(io.StringIO):
+            def __exit__(self, exc_type, exc_val, exc_tb):
+                nonlocal resolved_gem_paths
+                resolved_gem_paths = super().getvalue()
+                super().__exit__(exc_tb, exc_val, exc_tb)
+
+        def get_enabled_gem_cmake_file(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                platform: str = 'Common'):
+            return pathlib.Path() 
+
+        def get_enabled_gems(cmake_file: pathlib.Path) -> set:
+            return set() 
+
+        def get_gems_json_data_by_name(engine_path:pathlib.Path = None, 
+                                       project_path: pathlib.Path = None, 
+                                       include_manifest_gems: bool = False,
+                                       include_engine_gems: bool = False,
+                                       external_subdirectories: list = None
+                                       ) -> dict:
+            return all_gems_json_data
+
+        with patch('pathlib.Path.is_file', return_value=True) as pathlib_is_file_mock,\
+                patch('pathlib.Path.resolve', self.resolve) as pathlib_is_resolve_mock, \
+                patch('o3de.manifest.get_project_engine_path', side_effect=get_project_engine_path) as get_project_engine_path_patch,\
+                patch('o3de.manifest.get_this_engine_path', side_effect=get_this_engine_path) as get_this_engine_path_patch,\
+                patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as get_engine_json_data_patch,\
+                patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) as get_project_json_data_patch,\
+                patch('o3de.manifest.get_gems_json_data_by_name', side_effect=get_gems_json_data_by_name) as get_gems_json_data_by_name_patch,\
+                patch('o3de.manifest.get_enabled_gem_cmake_file', side_effect=get_enabled_gem_cmake_file) as get_enabled_gem_cmake_patch, \
+                patch('o3de.manifest.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch,\
+                patch('pathlib.Path.open', side_effect=lambda mode: StringBufferIOWrapper()) as pathlib_open_mock:
+            result = cmake.resolve_gem_dependency_paths(
+                                        engine_path=engine_path if engine_gem_names else None, 
+                                        project_path=project_path if project_gem_names else None, 
+                                        resolved_gem_dependencies_output_path=pathlib.Path('out'))
+
+        assert result == expected_result
+        assert resolved_gem_paths == expected_gem_paths

scripts/o3de/tests/test_compatibility.py

@@ -0,0 +1,183 @@
+#
+# Copyright (c) Contributors to the Open 3D Engine Project.
+# For complete copyright and license terms please see the LICENSE at the root of this distribution.
+#
+# SPDX-License-Identifier: Apache-2.0 OR MIT
+#
+#
+
+import pytest
+import pathlib
+
+from o3de import compatibility 
+
[email protected](
+    "gem_json_data, all_gem_json_data, expected_number_incompatible", [
+        # when no dependencies or versions exist expect compatible
+        pytest.param({'gem_name':'gemA'}, {'gemA':[{'gem_name':'gemA'}]}, 0),
+        # when dependency exists with no versions specifier expect compatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB'}],
+                'gemB':[{'gem_name':'gemB'}]
+            },
+            0),
+        # when dependency is missing expect incompatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB'}]
+            },
+            1),
+        # when dependency exists with version specifier expect compatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB==1.0.0']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB==1.0.0'}],
+                'gemB':[
+                    {'gem_name':'gemB','version':'1.0.0'},
+                    {'gem_name':'gemB','version':'2.0.0'}
+                    ]
+            },
+            0),
+        # when multiple compatible gems exist with versions expect compatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB>1.0.0']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB>1.0.0'}],
+                'gemB':[
+                    {'gem_name':'gemB','version':'2.0.0'}, # compatible
+                    {'gem_name':'gemB','version':'3.0.0'}  # also compatible
+                    ]
+            },
+            0),
+        # when dependency with expected version missing expect incompatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB==1.0.0']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB==1.0.0'}],
+                'gemB':[{'gem_name':'gemB','version':'2.0.0'}] # not compatible
+            },
+            1),
+        # when nested dependency exists with version specifier expect compatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB==1.0.0']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB==1.0.0'}],
+                'gemB':[
+                    {'gem_name':'gemB','version':'1.0.0', 'dependencies':['gemC==1.0.0']},
+                    {'gem_name':'gemB','version':'2.0.0', 'dependencies':['gemC==2.0.0']}
+                    ],
+                'gemC':[
+                    {'gem_name':'gemC','version':'1.0.0'}, # compatible
+                    {'gem_name':'gemC','version':'2.0.0'}
+                    ]
+            },
+            0),
+        # when nested dependency with requested version not found expect incompatible
+        pytest.param({'gem_name':'gemA','dependencies':['gemB==1.0.0']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB==1.0.0'}],
+                'gemB':[
+                    {'gem_name':'gemB','version':'1.0.0', 'dependencies':['gemC==1.0.0']},
+                    {'gem_name':'gemB','version':'2.0.0', 'dependencies':['gemC==2.0.0']}
+                    ],
+                'gemC':[
+                    {'gem_name':'gemC','version':'3.0.0'}, # not compatible 
+                    {'gem_name':'gemC','version':'4.0.0'} # not compatible 
+                    ]
+            },
+            2),
+        # when multiple compatible options exist expect compatible 
+        pytest.param({'gem_name':'gemA','dependencies':['gemB']}, 
+            {
+                'gemA':[{'gem_name':'gemA', 'dependencies':'gemB'}],
+                'gemB':[
+                    {'gem_name':'gemB','version':'1.0.0', 'dependencies':['gemC']},
+                    {'gem_name':'gemB','version':'2.0.0', 'dependencies':['gemC==2.0.0']}
+                    ],
+                'gemC':[
+                    {'gem_name':'gemC','version':'1.0.0'}, # compatible
+                    {'gem_name':'gemC','version':'2.0.0'}  # also compatible
+                    ]
+            },
+            0),
+    ],
+)
+def test_get_incompatible_gem_version_specifiers(gem_json_data, all_gem_json_data, expected_number_incompatible):
+    result = compatibility.get_incompatible_gem_version_specifiers(gem_json_data, all_gem_json_data, set())
+    # Test number of incompatible version specifiers, because we don't want these 
+    # tests to fail if the error message changes
+    assert len(result) == expected_number_incompatible
+
+
[email protected](
+    "gem_names, all_gem_json_data, expected_result", [
+        # when the gem exists and no version specifiers are provided expect found
+        pytest.param(['gemA'], {'gemA':[{'gem_name':'gemA','gem_path':pathlib.PurePath()}]}, ['gemA==0.0.0']),
+        # when the gem dependency doesn't exist expect failure
+        pytest.param(['gemA'], {}, []),
+        # when the gem dependency with correct version doesn't exist expect failure
+        pytest.param(['gemA~=1.2.0'], {'gemA':[{'gem_name':'gemA','version':'2.4.0'}]}, []),
+        # when two gems exist with different versions expect higher version is selected 
+        pytest.param(['gemA'], {
+            'gemA':[
+                {'gem_name':'gemA','version':'3.2.3'},
+                {'gem_name':'gemA','version':'20.3.4'},
+                {'gem_name':'gemA','version':'0.1.2'}
+            ]}, ['gemA==20.3.4']),
+        # when the gem sub dependency exists and no version specifiers are provided expect found
+        pytest.param(['gemA'], {
+            'gemA':[{'gem_name':'gemA','dependencies':['gemB']}],
+            'gemB':[{'gem_name':'gemB'}]
+            }, ['gemA==0.0.0','gemB==0.0.0']),
+        # when the gem sub dependency doesn't exist expect failure 
+        pytest.param(['gemA'], {
+            'gemA':[{'gem_name':'gemA','dependencies':['gemB']}]
+            }, []),
+        # when the gem is compatible with the engine expect it is found 
+        pytest.param(['gemA'], {
+            'gemA':[{'gem_name':'gemA','compatible_engines':['o3de==1.0.0']}]
+            }, ['gemA==0.0.0']),
+        # when the gem is not compatible with the engine expect failure 
+        pytest.param(['gemA'], {
+            'gemA':[{'gem_name':'gemA','compatible_engines':['o3de==2.0.0']}]
+            }, []),
+        # when a gem that is compatible with the engine exists expect it is found 
+        pytest.param(['gemA'], {
+            'gemA':[
+                {'gem_name':'gemA',"version":"1.0.0",'compatible_engines':['o3de==1.0.0']},
+                {'gem_name':'gemA',"version":"2.0.0",'compatible_engines':['o3de==2.0.0']}
+                ]
+            }, ['gemA==1.0.0']),
+        # when the circular dependency exists expect still succeeds 
+        pytest.param(['gemA'], {
+            'gemA':[{'gem_name':'gemA','dependencies':['gemB']}],
+            'gemB':[{'gem_name':'gemB','dependencies':['gemA']}]
+            }, ['gemA==0.0.0','gemB==0.0.0']),
+        # when the two versions of a gem exist, but only one solution expect version gemC==1.0.0 used
+        pytest.param(['gemA','gemB'], { 
+            'gemA':[
+                # gemA can use gemC 1.0.0 or 2.0.0
+                {'gem_name':'gemA','version':'1.0.0','dependencies':['gemC>=1.0.0']}
+                ],
+            'gemB':[
+                # gemB can only use gemC 1.0.0
+                {'gem_name':'gemB','version':'4.3.2','dependencies':['gemC==1.0.0']}
+                ],
+            'gemC':[
+                {'gem_name':'gemC','version':'1.0.0'}, # <-- should be selected
+                {'gem_name':'gemC','version':'2.0.0'}
+                ]
+            }, ['gemA==1.0.0','gemB==4.3.2','gemC==1.0.0'])
+    ]
+)
+def test_resolve_gem_dependencies(gem_names, all_gem_json_data, expected_result):
+    engine_json_data = {
+        'engine_name':'o3de',
+        'version':'1.0.0'
+    }
+    result, errors = compatibility.resolve_gem_dependencies(gem_names, all_gem_json_data, engine_json_data)
+    if result:
+        result_set = set()
+        for _, candidate in result.items():
+            result_set.add(f'{candidate.name}=={candidate.version}')
+        assert result_set == set(expected_result)
+    else:
+        assert not expected_result
+        assert errors

scripts/o3de/tests/test_disable_gem.py

@@ -11,7 +11,7 @@ import pytest
 import pathlib
 from unittest.mock import patch
 
-from o3de import cmake, disable_gem, enable_gem
+from o3de import manifest, disable_gem, enable_gem
 
 
 TEST_ENGINE_JSON_PAYLOAD = '''
@@ -116,24 +116,46 @@ def init_disable_gem_data(request):
 
 @pytest.mark.usefixtures('init_disable_gem_data')
 class TestDisableGemCommand:
-    @pytest.mark.parametrize("gem_path, project_path, gem_registered_with_project, gem_registered_with_engine,"
-                             "expected_result", [
-        pytest.param(pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), False, True, 0),
-        pytest.param(pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), False, False, 0),
-        pytest.param(pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), True, False, 0),
-        pytest.param(pathlib.PurePath('TestGem'), pathlib.PurePath('TestProject'), False, False, 0),
+    @staticmethod
+    def resolve(self):
+        return self
+
+    @pytest.mark.parametrize("enable_gem_name, enable_gem_version, disable_gem_name, test_gem_path, "
+                             "project_path, gem_registered_with_project, "
+                             "gem_registered_with_engine, enabled_in_cmake, expected_result", [
+        pytest.param(None, None, None, pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), False, True, True, 0),
+        pytest.param(None, None, None, pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), False, False, False, 0),
+        pytest.param(None, None, None, pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), True, False, False, 0),
+        pytest.param(None, None, None, pathlib.PurePath('TestGem'), pathlib.PurePath('TestProject'), False, False, False, 0),
+        # when requested to remove by name with no version expect success
+        pytest.param('TestGem', None, 'TestGem', None, pathlib.PurePath('TestProject'), False, False, True, 0),
+        # when requested to remove a gem with matching version expect success
+        pytest.param('TestGem==1.0.0', None, 'TestGem==1.0.0', None, pathlib.PurePath('TestProject'), False, False, False, 0),
+        # when a gem specifier is included but the gem doesn't match, expect failure
+        pytest.param('TestGem', 'None', 'TestGem==1.0.0', None, pathlib.PurePath('TestProject'), False, False, False, 1),
+        # when a gem name with no specifier is provided, expect any gem with that name is removed 
+        pytest.param('TestGem==1.2.3', '1.2.3', 'TestGem', None, pathlib.PurePath('TestProject'), False, False, False, 0),
         ]
     )
-    def test_disable_gem_registers_gem_name_with_project_json(self, gem_path, project_path, gem_registered_with_project,
-                                                             gem_registered_with_engine, expected_result):
+    def test_disable_gem_registers_gem_name_with_project_json(self, enable_gem_name, enable_gem_version, disable_gem_name, 
+                                                              test_gem_path, project_path, gem_registered_with_project,
+                                                              gem_registered_with_engine, enabled_in_cmake, expected_result):
 
         project_gem_dependencies = []
+        default_gem_path = pathlib.PurePath('TestGem')
 
-        def get_registered_path(engine_name: str = None, project_name: str = None, gem_name: str = None) -> pathlib.Path or None:
+        def get_registered(engine_name: str = None,
+                        project_name: str = None,
+                        gem_name: str = None,
+                        template_name: str = None,
+                        default_folder: str = None,
+                        repo_name: str = None,
+                        restricted_name: str = None,
+                        project_path: pathlib.Path = None) -> pathlib.Path or None:
             if project_name:
                 return project_path
             elif gem_name:
-                return gem_path
+                return test_gem_path if test_gem_path else default_gem_path
             elif engine_name:
                 return pathlib.PurePath('o3de')
             return None
@@ -155,24 +177,29 @@ class TestDisableGemCommand:
 
         def get_gem_json_data(gem_name: str = None, gem_path: str or pathlib.Path = None,
                             project_path: pathlib.Path = None) -> dict or None:
-            return self.disable_gem.gem_data
+            gem_json_data = self.disable_gem.gem_data
+            if enable_gem_version:
+                gem_json_data['version'] = enable_gem_version
+            return gem_json_data
 
         def get_engine_json_data(engine_name:str = None, engine_path:pathlib.Path = None):
             return self.disable_gem.engine_data
 
         def get_project_gems(project_path: pathlib.Path):
-            return [pathlib.Path(gem_path).resolve()] if gem_registered_with_project else []
+            gem_path = test_gem_path if test_gem_path else default_gem_path
+            return [gem_path] if gem_registered_with_project else []
 
         def get_engine_gems():
-            return [pathlib.Path(gem_path).resolve()] if gem_registered_with_engine else []
-
-        def add_gem_dependency(enable_gem_cmake_file: pathlib.Path, gem_name: str):
-            project_gem_dependencies.append(gem_name)
-            return 0
+            gem_path = test_gem_path if test_gem_path else default_gem_path
+            return [gem_path] if gem_registered_with_engine else []
 
         def remove_gem_dependency(enable_gem_cmake_file: pathlib.Path, gem_name: str):
-            project_gem_dependencies.remove(gem_name)
-            return 0
+            if gem_name in project_gem_dependencies:
+                project_gem_dependencies.remove(gem_name)
+                return 0
+            # If the gem was enabled in enabled_gems.cmake return an 
+            # error because it wasn't found in the list of dependencies
+            return 1 if enabled_in_cmake else 0
 
         def get_enabled_gems(enable_gem_cmake_file: pathlib.Path) -> list:
             return project_gem_dependencies
@@ -187,46 +214,45 @@ class TestDisableGemCommand:
                 patch('o3de.manifest.load_o3de_manifest', side_effect=load_o3de_manifest) as load_o3de_manifest_patch, \
                 patch('o3de.manifest.save_o3de_manifest', side_effect=save_o3de_manifest) as save_o3de_manifest_patch,\
                 patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as get_engine_json_data_patch,\
-                patch('o3de.manifest.get_registered', side_effect=get_registered_path) as get_registered_patch,\
+                patch('o3de.manifest.get_registered', side_effect=get_registered) as get_registered_patch,\
                 patch('o3de.manifest.get_gem_json_data', side_effect=get_gem_json_data) as get_gem_json_data_patch,\
                 patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) as get_gem_json_data_patch,\
                 patch('o3de.manifest.get_project_gems', side_effect=get_project_gems) as get_project_gems_patch,\
                 patch('o3de.manifest.get_engine_gems', side_effect=get_engine_gems) as get_engine_gems_patch,\
-                patch('o3de.cmake.add_gem_dependency', side_effect=add_gem_dependency) as add_gem_dependency_patch, \
+                patch('pathlib.Path.resolve', new=self.resolve) as pathlib_is_resolve_mock,\
                 patch('o3de.cmake.remove_gem_dependency',
                       side_effect=remove_gem_dependency) as remove_gem_dependency_patch, \
-                patch('o3de.cmake.get_enabled_gems',
+                patch('o3de.manifest.get_enabled_gems',
                       side_effect=get_enabled_gems) as get_enabled_gems, \
                 patch('o3de.validation.valid_o3de_gem_json', return_value=True) as valid_gem_json_patch:
 
             # Clear out any "gem_names" from the previous iterations
             self.disable_gem.project_data.pop('gem_names', None)
 
-            # First enable the gem
-            assert enable_gem.enable_gem_in_project(gem_path=gem_path, project_path=project_path) == 0
+            # Enable the gem
+            assert enable_gem.enable_gem_in_project(gem_name=enable_gem_name, gem_path=test_gem_path, project_path=project_path) == 0
+
+            gem_json = get_gem_json_data(gem_name=enable_gem_name, gem_path=test_gem_path, project_path=project_path)
+            expected_gem_name = enable_gem_name or gem_json['gem_name']
+
+            if enabled_in_cmake:
+                # Simulate the gem existing in the deprecated `enabled_gems.cmake` file
+                project_gem_dependencies.append(expected_gem_name)
 
-            # Check that the gem is enabled
-            gem_json = get_gem_json_data(gem_path=gem_path, project_path=project_path)
+            # Check that the gem is enabled in project.json
             project_json = get_project_json_data(project_path=project_path)
-            enabled_gems_list = cmake.get_enabled_gems(project_path / "Gem/enabled_gems.cmake")
-            assert gem_json.get('gem_name', '') in enabled_gems_list
-
-            # If the gem that is neither registered in the project.json nor engine.json,
-            # then it must appear in the "gem_names" field.
-            if not gem_registered_with_engine and not gem_registered_with_project:
-                assert gem_json.get('gem_name', '') in project_json.get('gem_names', [])
-            else:
-                assert gem_json.get('gem_name', '') not in project_json.get('gem_names', [])
-
-            # Now disable the gem
-            result = disable_gem.disable_gem_in_project(gem_path=gem_path, project_path=project_path)
+            assert expected_gem_name in project_json.get('gem_names', [])
+
+            # Disable the gem
+            result = disable_gem.disable_gem_in_project(gem_name=disable_gem_name, gem_path=test_gem_path, project_path=project_path)
             assert result == expected_result
 
-            # Refresh the enabled_gems list and check for removal of the gem
-            gem_json = get_gem_json_data(gem_path=gem_path, project_path=project_path)
-            project_json = get_project_json_data(project_path=project_path)
-            enabled_gems_list = cmake.get_enabled_gems(project_path / "Gem/enabled_gems.cmake")
-            assert gem_json.get('gem_name', '') not in enabled_gems_list
+            expected_gem_name = disable_gem_name or expected_gem_name
 
-            # If gem name should no longer appear in the "gem_names" field
-            assert gem_json.get('gem_name', '') not in project_json.get('gem_names', [])
+            if enabled_in_cmake:
+                # The gem name should no longer exist in enabled_gems.cmake 
+                assert expected_gem_name not in manifest.get_enabled_gems(project_path / "Gem/enabled_gems.cmake")
+
+            # The gem name should no longer appear in the "gem_names" field
+            project_json = get_project_json_data(project_path=project_path)
+            assert expected_gem_name not in project_json.get('gem_names', [])

scripts/o3de/tests/test_enable_gem.py

@@ -116,6 +116,10 @@ def init_enable_gem_data(request):
 
 @pytest.mark.usefixtures('init_enable_gem_data')
 class TestEnableGemCommand:
+    @staticmethod
+    def resolve(self):
+        return self
+
     @pytest.mark.parametrize("gem_path, project_path, gem_registered_with_project, gem_registered_with_engine,"
                              "optional, expected_result", [
         pytest.param(pathlib.PurePath('TestProject/TestGem'), pathlib.PurePath('TestProject'), False, True, False, 0),
@@ -168,16 +172,13 @@ class TestEnableGemCommand:
             return self.enable_gem.engine_data
 
         def get_project_gems(project_path: pathlib.Path):
-            return [pathlib.Path(gem_path).resolve()] if gem_registered_with_project else []
+            return [gem_path] if gem_registered_with_project else []
 
         def get_enabled_gems(cmake_file: pathlib.Path) -> set:
             return set() 
 
         def get_engine_gems():
-            return [pathlib.Path(gem_path).resolve()] if gem_registered_with_engine else []
-
-        def add_gem_dependency(enable_gem_cmake_file: pathlib.Path, gem_name: str):
-            return 0
+            return [gem_path] if gem_registered_with_engine else []
 
         def is_file(path : pathlib.Path) -> bool:
             if path.match("*/user/project.json"):
@@ -195,8 +196,8 @@ class TestEnableGemCommand:
                 patch('o3de.manifest.get_project_gems', side_effect=get_project_gems) as get_project_gems_patch,\
                 patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as get_engine_json_data_patch,\
                 patch('o3de.manifest.get_engine_gems', side_effect=get_engine_gems) as get_engine_gems_patch,\
-                patch('o3de.cmake.add_gem_dependency', side_effect=add_gem_dependency) as add_gem_dependency_patch,\
-                patch('o3de.cmake.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch,\
+                patch('o3de.manifest.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch,\
+                patch('pathlib.Path.resolve', new=self.resolve) as pathlib_is_resolve_mock,\
                 patch('o3de.validation.valid_o3de_gem_json', return_value=True) as valid_gem_json_patch:
 
             self.enable_gem.project_data.pop('gem_names', None)
@@ -207,10 +208,7 @@ class TestEnableGemCommand:
             project_json = get_project_json_data(project_path=project_path)
             gem_name = gem_json.get('gem_name', '')
             gem = gem_name if not optional else {'name':gem_name, 'optional':optional}
-            if not gem_registered_with_engine and not gem_registered_with_project:
-                assert gem in project_json.get('gem_names', [])
-            else:
-                assert gem not in project_json.get('gem_names', [])
+            assert gem in project_json.get('gem_names', [])
 
     @pytest.mark.parametrize("gem_registered_with_project, gem_registered_with_engine, "
                              "gem_version, gem_dependencies, gem_json_data_by_name, dry_run, force, "
@@ -307,8 +305,12 @@ class TestEnableGemCommand:
                                         external_subdirectories: list = None
                                         ) -> dict:
             all_gems_json_data = {}
+            # include the gem we're enabling
+            gem_json_data = get_gem_json_data()
+            all_gems_json_data[gem_json_data['gem_name']] = [gem_json_data]
+
             for gem_name in gem_json_data_by_name.keys():
-                all_gems_json_data[gem_name] = get_gem_json_data(gem_name=gem_name)
+                all_gems_json_data[gem_name] = [get_gem_json_data(gem_name=gem_name)]
             return all_gems_json_data
 
         def get_enabled_gem_cmake_file(project_name: str = None,
@@ -353,19 +355,17 @@ class TestEnableGemCommand:
             return gem_data
 
         def get_project_gems(project_path: pathlib.Path):
-            return [pathlib.Path(gem_path).resolve()] if gem_registered_with_project else []
+            return [gem_path] if gem_registered_with_project else []
 
         def get_enabled_gems(cmake_file: pathlib.Path) -> set:
             return set() 
 
         def get_engine_gems():
-            gem_paths = list(map(lambda path:pathlib.Path(path).resolve(), gem_json_data_by_name.keys()))
+            gem_paths = list(map(lambda path:path, gem_json_data_by_name.keys()))
             if gem_registered_with_engine:
-                gem_paths.append(pathlib.Path(gem_path).resolve())
+                gem_paths.append(gem_path)
             return gem_paths
 
-        def add_gem_dependency(enable_gem_cmake_file: pathlib.Path, gem_name: str):
-            return 0
 
         def is_file(path : pathlib.Path) -> bool:
             if path.match("*/user/project.json"):
@@ -383,9 +383,9 @@ class TestEnableGemCommand:
                 patch('o3de.manifest.get_project_gems', side_effect=get_project_gems) as get_project_gems_patch,\
                 patch('o3de.manifest.get_engine_gems', side_effect=get_engine_gems) as get_engine_gems_patch,\
                 patch('o3de.manifest.get_gems_json_data_by_name', side_effect=get_gems_json_data_by_name) as get_gems_json_data_by_name_patch,\
-                patch('o3de.cmake.add_gem_dependency', side_effect=add_gem_dependency) as add_gem_dependency_patch,\
-                patch('o3de.cmake.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch,\
-                patch('o3de.cmake.get_enabled_gem_cmake_file', side_effect=get_enabled_gem_cmake_file) as get_enabled_gem_cmake_patch, \
+                patch('o3de.manifest.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch,\
+                patch('o3de.manifest.get_enabled_gem_cmake_file', side_effect=get_enabled_gem_cmake_file) as get_enabled_gem_cmake_patch, \
+                patch('pathlib.Path.resolve', new=self.resolve) as pathlib_is_resolve_mock,\
                 patch('o3de.validation.valid_o3de_gem_json', return_value=True) as valid_gem_json_patch:
 
             self.enable_gem.project_data.pop('gem_names', None)
@@ -397,7 +397,130 @@ class TestEnableGemCommand:
                 project_json = get_project_json_data(project_path=project_path)
                 gem_name = gem_json.get('gem_name', '')
                 gem = gem_name if not is_optional_gem else {'name':gem_name, 'optional':is_optional_gem}
-                if not gem_registered_with_engine and not gem_registered_with_project and not dry_run:
+                if not dry_run:
                     assert gem in project_json.get('gem_names', [])
                 else:
                     assert gem not in project_json.get('gem_names', [])
+
+    @pytest.mark.parametrize("gem_name, gem_json_data_by_path, force,"
+                             "expected_result", [
+        # when no version specifier used and gem exists expect it is found
+        pytest.param("GemA", {pathlib.PurePath('GemA'):{"gem_name":"GemA"}}, False, 0),
+        # when version specifier used and gem exists with version expect it is found
+        pytest.param("GemA==1.0", {pathlib.PurePath('GemA'):{"gem_name":"GemA","version":"1.0.0"}}, False, 0),
+        # when version specifier used and gem version doesn't exists expect it is not found
+        pytest.param("GemA==1.0", {pathlib.PurePath('GemA'):{"gem_name":"GemA","version":"2.0.0"}}, False, 1),
+        # when version specifier used and multiple gem versions exist, expect gem activated 
+        pytest.param("GemA>1.0", {
+            pathlib.PurePath('GemA1'):{"gem_name":"GemA","version":"2.0.0"},
+            pathlib.PurePath('GemA2'):{"gem_name":"GemA","version":"3.0.0"},
+            }, False, 0),
+        # when dependencies cannot be satisfied expect fails, the following requires gemC 1.0.0 and 2.0.0
+        pytest.param("GemA", {
+            pathlib.PurePath('GemA'):{"gem_name":"GemA","version":"1.0.0","dependencies":['GemB==3.0.0','GemC==2.0.0']},
+            pathlib.PurePath('GemB'):{"gem_name":"GemB","version":"3.0.0","dependencies":['GemC==1.0.0']},
+            pathlib.PurePath('GemC1'):{"gem_name":"GemC","version":"1.0.0"},
+            pathlib.PurePath('GemC2'):{"gem_name":"GemC","version":"2.0.0"},
+            }, False, 1),
+        ],
+    )
+    def test_enable_gem_with_version_specifier_checks_compatibility(self, gem_name, gem_json_data_by_path, 
+                                                    force, expected_result):
+        project_path = pathlib.PurePath('TestProject')
+        engine_path = pathlib.PurePath('o3de')
+
+        def get_manifest_engines() -> list:
+            return [engine_path]
+
+        def get_project_engine_path(project_path:str or pathlib.Path) -> pathlib.Path or None:
+            return engine_path
+
+        def load_o3de_manifest(manifest_path: pathlib.Path = None) -> dict:
+            if not manifest_path:
+                return json.loads(TEST_O3DE_MANIFEST_JSON_PAYLOAD)
+            return None
+
+        def save_o3de_manifest(new_project_data: dict, manifest_path: pathlib.Path = None) -> bool:
+            if manifest_path == project_path / 'project.json':
+                self.enable_gem.project_data = new_project_data
+            return True
+
+        def get_gem_json_data(gem_name: str = None, gem_path: str or pathlib.Path = None,
+                            project_path: pathlib.Path = None) -> dict or None:
+            gem_data = self.enable_gem.gem_data.copy()
+            if gem_path in gem_json_data_by_path:
+                gem_data.update(gem_json_data_by_path[gem_path])
+            return gem_data
+
+        def get_gems_json_data_by_name( engine_path:pathlib.Path = None, 
+                                        project_path: pathlib.Path = None, 
+                                        include_manifest_gems: bool = False,
+                                        include_engine_gems: bool = False,
+                                        external_subdirectories: list = None
+                                        ) -> dict:
+            all_gems_json_data = {}
+            for gem_path in gem_json_data_by_path.keys():
+                gem_json_data = get_gem_json_data(gem_path=gem_path)
+                gem_name = gem_json_data.get('gem_name','')
+
+                entries = all_gems_json_data.get(gem_name, [])
+                entries.append(gem_json_data)
+                all_gems_json_data[gem_name] = entries
+
+            return all_gems_json_data
+        def is_file(path : pathlib.Path) -> bool:
+            if path.match("*/user/project.json"):
+                return False
+            return True
+
+        def get_project_json_data(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                user: bool = False) -> dict or None:
+            return self.enable_gem.project_data
+
+        def get_engine_json_data(engine_name:str = None, engine_path:pathlib.Path = None):
+            return self.enable_gem.engine_data
+
+        def get_project_gems(project_path: pathlib.Path):
+            return []
+
+        def get_enabled_gems(cmake_file: pathlib.Path) -> set:
+            return set() 
+
+        def get_engine_gems():
+            return []
+        
+        def get_all_gems(project_path: pathlib.Path = None) -> list:
+            return list(gem_json_data_by_path.keys())
+
+        def get_json_data(object_typename: str,
+                        object_path: str or pathlib.Path,
+                        object_validator: callable) -> dict or None:
+            return gem_json_data_by_path.get(object_path)
+
+
+        with patch('pathlib.Path.is_dir', return_value=True) as pathlib_is_dir_patch,\
+            patch('pathlib.Path.is_file', new=is_file) as pathlib_is_file_patch, \
+            patch('o3de.manifest.load_o3de_manifest', side_effect=load_o3de_manifest) as load_o3de_manifest_patch, \
+            patch('o3de.manifest.save_o3de_manifest', side_effect=save_o3de_manifest) as save_o3de_manifest_patch,\
+            patch('o3de.manifest.get_manifest_engines', side_effect=get_manifest_engines) as get_manifest_engines_patch,\
+            patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as get_engine_json_data_patch,\
+            patch('o3de.manifest.get_json_data', side_effect=get_json_data) as get_json_data_patch,\
+            patch('o3de.manifest.get_gem_json_data', side_effect=get_gem_json_data) as get_gem_json_data_patch,\
+            patch('o3de.manifest.get_gems_json_data_by_name', side_effect=get_gems_json_data_by_name) as get_gems_json_data_by_name_patch,\
+            patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) as get_gem_json_data_patch,\
+            patch('o3de.manifest.get_project_gems', side_effect=get_project_gems) as get_project_gems_patch,\
+            patch('o3de.manifest.get_project_engine_path', side_effect=get_project_engine_path) as get_project_engine_path_patch,\
+            patch('o3de.manifest.get_engine_gems', side_effect=get_engine_gems) as get_engine_gems_patch,\
+            patch('o3de.manifest.get_all_gems', side_effect=get_all_gems) as get_all_gems_patch,\
+            patch('o3de.manifest.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch,\
+            patch('pathlib.Path.resolve', new=self.resolve) as pathlib_is_resolve_mock,\
+            patch('o3de.validation.valid_o3de_gem_json', return_value=True) as valid_gem_json_patch:
+
+            # remove any existing 'gem_names'
+            self.enable_gem.project_data.pop('gem_names', None)
+            result = enable_gem.enable_gem_in_project(gem_name=gem_name, project_path=project_path, force=force)
+            assert result == expected_result
+            if expected_result == 0:
+                project_json = get_project_json_data(project_path=project_path)
+                assert gem_name in project_json.get('gem_names', [])

scripts/o3de/tests/test_manifest.py

@@ -110,6 +110,56 @@ TEST_O3DE_MANIFEST_JSON_PAYLOAD = '''
 }
 '''
 
+class TestGetEnabledGems:
+    @pytest.mark.parametrize(
+        "enable_gems_cmake_data, expected_set", [
+            pytest.param("""
+                # Comment
+                set(ENABLED_GEMS foo bar baz)
+            """, set(['foo', 'bar', 'baz'])),
+            pytest.param("""
+                        # Comment
+                        set(ENABLED_GEMS
+                            foo
+                            bar
+                            baz
+                        )
+                    """, set(['foo', 'bar', 'baz'])),
+            pytest.param("""
+                    # Comment
+                    set(ENABLED_GEMS
+                        foo
+                        bar
+                        baz)
+                """, set(['foo', 'bar', 'baz'])),
+            pytest.param("""
+                        # Comment
+                        set(ENABLED_GEMS
+                            foo bar
+                            baz)
+                    """, set(['foo', 'bar', 'baz'])),
+            pytest.param("""
+                        # Comment
+                        set(RANDOM_VARIABLE TestGame, TestProject Test Engine)
+                        set(ENABLED_GEMS HelloWorld IceCream
+                            foo
+                            baz bar
+                            baz baz baz baz baz morebaz lessbaz
+                        )
+                        Random Text
+                    """, set(['HelloWorld', 'IceCream', 'foo', 'bar', 'baz', 'morebaz', 'lessbaz'])),
+        ]
+    )
+    def test_get_enabled_gems(self, enable_gems_cmake_data, expected_set):
+        enabled_gems_set = set()
+        with patch('pathlib.Path.resolve', return_value=pathlib.Path('enabled_gems.cmake')) as pathlib_is_resolve_mock,\
+                patch('pathlib.Path.is_file', return_value=True) as pathlib_is_file_mock,\
+                patch('pathlib.Path.open', return_value=io.StringIO(enable_gems_cmake_data)) as pathlib_open_mock:
+            enabled_gems_set = manifest.get_enabled_gems(pathlib.Path('enabled_gems.cmake'))
+
+        assert enabled_gems_set == expected_set
+
+
 @pytest.mark.parametrize("valid_project_json_paths, valid_gem_json_paths", [
     pytest.param([pathlib.Path('D:/o3de/Templates/DefaultProject/Template/project.json')],
                  [pathlib.Path('D:/o3de/Templates/DefaultGem/Template/gem.json')])
@@ -368,6 +418,140 @@ class TestGetAllGems:
 
             assert self.cycle_detected == expected_cycle_detected
 
+class TestGetProjectEnabledGems:
+
+    project_path = pathlib.Path("project")
+
+    @staticmethod
+    def get_this_engine_path() -> pathlib.Path:
+        return pathlib.Path('D:/o3de/o3de')
+
+    @staticmethod
+    def resolve(self):
+        return self
+
+    @staticmethod
+    def as_posix(self):
+        return self
+
+    @pytest.mark.parametrize("""project_gem_names, cmake_gem_names, 
+                                all_gems_json_data, include_dependencies,
+                                expected_result""", [
+            # When gems are provided without version specifiers expect they are found
+            pytest.param(
+                ['GemA'], ['GemB'],
+                {
+                    'GemA':[{'gem_name':'GemA','path':pathlib.Path('c:/GemA')}],
+                    'GemB':[{'gem_name':'GemB','path':pathlib.Path('c:/GemB')}]
+                },
+                True,
+                {'GemA':'c:/GemA', 'GemB':'c:/GemB'}
+            ),
+            # When dependencies exist they are not included if include_dependencies is False 
+            pytest.param(
+                ['GemA'], ['GemB'],
+                {
+                    'GemA':[{'gem_name':'GemA','path':pathlib.Path('c:/GemA'), 'dependencies':['GemC']}],
+                    'GemB':[{'gem_name':'GemB','path':pathlib.Path('c:/GemB')}],
+                    'GemC':[{'gem_name':'GemC','path':pathlib.Path('c:/GemC')}]
+                },
+                False,
+                {'GemA':'c:/GemA', 'GemB':'c:/GemB'}
+            ),
+            # When dependencies exist they are included if include_dependencies is True 
+            pytest.param(
+                ['GemA'], ['GemB'],
+                {
+                    'GemA':[{'gem_name':'GemA','path':pathlib.Path('c:/GemA'), 'dependencies':['GemC']}],
+                    'GemB':[{'gem_name':'GemB','path':pathlib.Path('c:/GemB')}],
+                    'GemC':[{'gem_name':'GemC','path':pathlib.Path('c:/GemC')}]
+                },
+                True,
+                {'GemA':'c:/GemA', 'GemB':'c:/GemB', 'GemC':'c:/GemC'}
+            ),
+            # When a mix of gems are provided with and without version specifiers expect they are found
+            pytest.param(
+                ['GemA>=1.0.0'], ['GemB'],
+                {
+                    'GemA':[
+                        {'gem_name':'GemA','version':'1.0.0', 'path':pathlib.Path('c:/GemA1')},
+                        {'gem_name':'GemA','version':'2.0.0', 'path':pathlib.Path('c:/GemA2')}
+                    ],
+                    'GemB':[{'gem_name':'GemB','path':pathlib.Path('c:/GemB')}]
+                },
+                True,
+                {'GemA>=1.0.0':'c:/GemA2', 'GemB':'c:/GemB'}
+            ),
+            # When no gems are installed expect the names are returned without paths
+            pytest.param(
+                ['GemA>=1.0.0'], ['GemB==2.0.0'],
+                {},
+                True,
+                {'GemA>=1.0.0':None, 'GemB==2.0.0':None}
+            ),
+            # When some gems are missing expect their paths are none 
+            pytest.param(
+                ['GemA<3.0.0'], ['GemB==2.0.0'],
+                {
+                    'GemA':[
+                        {'gem_name':'GemA','version':'2.0.0','path':pathlib.Path('c:/GemA2')}, # <-- correct
+                        {'gem_name':'GemA','version':'3.0.0','path':pathlib.Path('c:/GemA3')},
+                    ]
+                    # no GemB
+                },
+                True,
+                {'GemA<3.0.0':'c:/GemA2', 'GemB==2.0.0':None}
+            ),
+        ]
+    )
+    def test_get_project_enabled_gems(self, project_gem_names, cmake_gem_names, 
+                                      all_gems_json_data, include_dependencies, expected_result):
+        def get_project_json_data(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                user: bool = False) -> dict or None:
+            project_json_data = json.loads(TEST_PROJECT_JSON_PAYLOAD)
+            project_json_data['gem_names'] = project_gem_names
+            return project_json_data
+        
+        def get_engine_json_data(engine_name: str = None,
+                                engine_path: str or pathlib.Path = None) -> dict or None:
+            return json.loads(TEST_ENGINE_JSON_PAYLOAD)
+
+        def get_enabled_gem_cmake_file(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                platform: str = 'Common'):
+            return None if not cmake_gem_names else pathlib.Path('enabled_gems.cmake')
+
+        def get_enabled_gems(cmake_file: pathlib.Path) -> set:
+            return cmake_gem_names 
+
+        def get_gems_json_data_by_name( engine_path:pathlib.Path = None, 
+                                        project_path: pathlib.Path = None, 
+                                        include_manifest_gems: bool = False,
+                                        include_engine_gems: bool = False,
+                                        external_subdirectories: list = None
+                                        ) -> dict:
+            return all_gems_json_data
+
+        def get_project_engine_path(project_path: pathlib.Path) -> pathlib.Path or None:
+            return None
+
+        with patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) \
+                as get_engine_json_data_patch,\
+            patch('o3de.manifest.get_gems_json_data_by_name', side_effect=get_gems_json_data_by_name) \
+                as get_gems_json_data_by_name_patch,\
+            patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) \
+                as get_project_json_data_patch, \
+            patch('o3de.manifest.get_project_engine_path', side_effect=get_project_engine_path) \
+                as get_project_engine_path_patch,\
+            patch('o3de.manifest.get_enabled_gem_cmake_file', side_effect=get_enabled_gem_cmake_file) \
+                as get_enabled_gem_cmake_file_patch, \
+            patch('o3de.manifest.get_enabled_gems', side_effect=get_enabled_gems) as get_enabled_gems_patch, \
+            patch('pathlib.Path.resolve', self.resolve) as resolve_patch, \
+            patch('pathlib.Path.is_file', return_value=True) as is_file_patch:
+
+            assert expected_result == manifest.get_project_enabled_gems(self.project_path, include_dependencies)
+
 class TestManifestGetRegistered:
     @staticmethod
     def get_this_engine_path() -> pathlib.Path:
@@ -391,41 +575,137 @@ class TestManifestGetRegistered:
             pytest.param('InvalidProject', pathlib.Path('Templates/DefaultProject'), None)
     ])
     def test_get_registered_template(self, template_name, relative_template_path, expected_path):
-            def get_engine_json_data(engine_name: str = None,
-                                    engine_path: str or pathlib.Path = None) -> dict or None:
-                engine_payload = json.loads(TEST_ENGINE_JSON_PAYLOAD)
-                if expected_path:
-                    engine_payload['templates'] = [relative_template_path]
-                return engine_payload
-
-            def get_gem_json_data(gem_name: str = None,
-                                    gem_path: str or pathlib.Path = None) -> dict or None:
-                gem_payload = json.loads(TEST_GEM_JSON_PAYLOAD)
-                return gem_payload
-
-            def get_project_json_data(project_name: str = None,
-                                    project_path: str or pathlib.Path = None,
-                                    user: bool = False) -> dict or None:
-                project_payload = json.loads(TEST_PROJECT_JSON_PAYLOAD)
-                return project_payload
-
-            def load_o3de_manifest(manifest_path: pathlib.Path = None) -> dict:
-                manifest_payload = json.loads(TEST_O3DE_MANIFEST_JSON_PAYLOAD)
-                manifest_payload['projects'] = []
-                return manifest_payload
-
-            with patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as _1, \
-                patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) as _2, \
-                patch('o3de.manifest.get_gem_json_data', side_effect=get_gem_json_data) as _3, \
-                patch('o3de.manifest.load_o3de_manifest', side_effect=load_o3de_manifest) as _4, \
-                patch('pathlib.Path.resolve', self.resolve) as _5, \
-                patch('pathlib.Path.samefile', self.samefile) as _6, \
-                patch('pathlib.Path.open', return_value=io.StringIO(TEST_PROJECT_TEMPLATE_JSON_PAYLOAD)) as _7, \
-                patch('pathlib.Path.is_file', self.is_file) as _8,\
-                patch('o3de.manifest.get_this_engine_path', side_effect=self.get_this_engine_path) as _9: 
-
-                path = manifest.get_registered(template_name=template_name)
-                assert path == expected_path
+        def get_engine_json_data(engine_name: str = None,
+                                engine_path: str or pathlib.Path = None) -> dict or None:
+            engine_payload = json.loads(TEST_ENGINE_JSON_PAYLOAD)
+            if expected_path:
+                engine_payload['templates'] = [relative_template_path]
+            return engine_payload
+
+        def get_gem_json_data(gem_name: str = None,
+                                gem_path: str or pathlib.Path = None) -> dict or None:
+            gem_payload = json.loads(TEST_GEM_JSON_PAYLOAD)
+            return gem_payload
+
+        def get_project_json_data(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                user: bool = False) -> dict or None:
+            project_payload = json.loads(TEST_PROJECT_JSON_PAYLOAD)
+            return project_payload
+
+        def load_o3de_manifest(manifest_path: pathlib.Path = None) -> dict:
+            manifest_payload = json.loads(TEST_O3DE_MANIFEST_JSON_PAYLOAD)
+            manifest_payload['projects'] = []
+            return manifest_payload
+
+        with patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as _1, \
+            patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) as _2, \
+            patch('o3de.manifest.get_gem_json_data', side_effect=get_gem_json_data) as _3, \
+            patch('o3de.manifest.load_o3de_manifest', side_effect=load_o3de_manifest) as _4, \
+            patch('pathlib.Path.resolve', self.resolve) as _5, \
+            patch('pathlib.Path.samefile', self.samefile) as _6, \
+            patch('pathlib.Path.open', return_value=io.StringIO(TEST_PROJECT_TEMPLATE_JSON_PAYLOAD)) as _7, \
+            patch('pathlib.Path.is_file', self.is_file) as _8,\
+            patch('o3de.manifest.get_this_engine_path', side_effect=self.get_this_engine_path) as _9: 
+
+            path = manifest.get_registered(template_name=template_name)
+            assert path == expected_path
+
[email protected]("test_object_typename", [
+        pytest.param('engine'),
+        pytest.param('project'),
+        pytest.param('gem')
+    ])
+class TestManifestGetRegisteredVersionedObject:
+    @staticmethod
+    def get_this_engine_path() -> pathlib.Path:
+        return pathlib.Path('D:/o3de/o3de')
+
+    @staticmethod
+    def is_file(self) -> bool:
+        # use a simple suffix check to avoid hitting the actual file system
+        return self.suffix != ''
+
+    @staticmethod
+    def resolve(self):
+        return self
+
+    @pytest.mark.parametrize("object_name, json_data_by_path, expected_path", [
+            # when no version information exists and name matches expect object not found
+            pytest.param('object-name', {pathlib.PurePath('object1'):{"name":"object-name"}}, pathlib.PurePath('object1'), ),
+            # when version information exists and version specifier provided expect correct engine found
+            pytest.param('object-name==1.0.0', {
+                pathlib.PurePath('object1'):{"name":"object-name","version":"1.0.0"},
+                pathlib.PurePath('object2'):{"name":"object-name","version":"2.0.0"}
+                }, pathlib.PurePath('object1'), ),
+            # when version information exists and version specifier provided expect correct engine found
+            pytest.param('object-name>=1.0.0', {
+                pathlib.PurePath('object1'):{"name":"object-name","version":"1.0.0"},
+                pathlib.PurePath('object2'):{"name":"object-name","version":"2.0.0"}
+                }, pathlib.PurePath('object2'), ),
+            # when version information exists and version specifier matches multiple engines, expect first match returned 
+            pytest.param('object-name==1.0.0', {
+                pathlib.PurePath('object1'):{"name":"object-name","version":"1.0.0"},
+                pathlib.PurePath('object2'):{"name":"object-name","version":"1.0.0"}
+                }, pathlib.PurePath('object1'), ),
+            # when engine is not found expect none is returned 
+            pytest.param('object-missing', {pathlib.PurePath('object1'):{"name":"object-name"}}, None ),
+    ])
+    def test_get_registered_object_with_version(self, test_object_typename, object_name, json_data_by_path, expected_path):
+        def get_json_data(object_typename: str,
+                        object_path: str or pathlib.Path,
+                        object_validator: callable) -> dict or None:
+            if object_typename == test_object_typename:
+                if test_object_typename == 'engine':
+                    payload = json.loads(TEST_ENGINE_JSON_PAYLOAD)
+                elif test_object_typename == 'project':
+                    payload = json.loads(TEST_PROJECT_JSON_PAYLOAD)
+                elif test_object_typename == 'gem':
+                    payload = json.loads(TEST_GEM_JSON_PAYLOAD)
+                object_path = pathlib.PurePath(object_path)
+                payload.update(json_data_by_path.get(object_path, {}))
+                # change '<object>_name' field value set in 'name' 
+                # e.g. 'engine_name' or 'project_name' gets value from 'name'
+                if 'name' in payload:
+                    payload[object_typename + '_name'] = payload['name']
+                return payload
+
+        def get_engine_json_data(engine_name: str = None,
+                                engine_path: str or pathlib.Path = None) -> dict or None:
+            return get_json_data('engine', engine_path, None)
+
+        def get_gem_json_data(gem_name: str = None,
+                            gem_path: str or pathlib.Path = None) -> dict or None:
+            return get_json_data('gem', gem_path, None)
+
+        def get_project_json_data(project_name: str = None,
+                                project_path: str or pathlib.Path = None,
+                                user: bool = False) -> dict or None:
+            return get_json_data('project', project_path, None)
+
+        def load_o3de_manifest(manifest_path: pathlib.Path = None) -> dict:
+            manifest_payload = json.loads(TEST_O3DE_MANIFEST_JSON_PAYLOAD)
+            if test_object_typename == 'gem':
+                manifest_payload['external_subdirectories'] = [p.as_posix() for p in json_data_by_path.keys()]
+            else:
+                manifest_payload[test_object_typename + 's'] = [p.as_posix() for p in json_data_by_path.keys()]
+            return manifest_payload
+
+        with patch('o3de.manifest.get_engine_json_data', side_effect=get_engine_json_data) as _1, \
+            patch('o3de.manifest.get_project_json_data', side_effect=get_project_json_data) as _2, \
+            patch('o3de.manifest.get_json_data', side_effect=get_json_data) as _2, \
+            patch('o3de.manifest.get_gem_json_data', side_effect=get_gem_json_data) as _3, \
+            patch('o3de.manifest.load_o3de_manifest', side_effect=load_o3de_manifest) as _4, \
+            patch('pathlib.Path.resolve', self.resolve) as _5, \
+            patch('pathlib.Path.is_file', self.is_file) as _8,\
+            patch('o3de.manifest.get_this_engine_path', side_effect=self.get_this_engine_path) as _9: 
+
+            engine_name = object_name if test_object_typename == 'engine' else None
+            project_name = object_name if test_object_typename == 'project' else None
+            gem_name = object_name if test_object_typename == 'gem' else None
+
+            path = manifest.get_registered(engine_name=engine_name, project_name=project_name, gem_name=gem_name)
+            assert path == expected_path
 
 class TestManifestProjects:
     @staticmethod
@@ -541,6 +821,8 @@ class TestManifestGetGemsJsonData:
     engine_external_path = pathlib.Path("engine_gem1")
     project_external_path = pathlib.Path("project_gem1")
     gem_external_path = pathlib.Path("external_gem1")
+    gem_version1_external_path = pathlib.Path("external_gem_v1")
+    gem_version2_external_path = pathlib.Path("external_gem_v2")
 
     @staticmethod
     def resolve(self):
@@ -550,24 +832,31 @@ class TestManifestGetGemsJsonData:
                             "external_subdirectories, expected_result", [
             # when engine_path provided, expect engine gems
             pytest.param(pathlib.Path('C:/engine1'), None, False, False, list(), 
-                {'engine_gem1':{'gem_name':'engine_gem1', 'path':pathlib.Path('engine_gem1')}}),
+                {'engine_gem1':[{'gem_name':'engine_gem1', 'path':pathlib.Path('engine_gem1')}]}),
             # when project_path provided, expect project gems
             pytest.param(None, pathlib.Path('C:/project1'), False, False, list(),
-                {'project_gem1':{'gem_name':'project_gem1', 'path':pathlib.Path('project_gem1')}}),
+                {'project_gem1':[{'gem_name':'project_gem1', 'path':pathlib.Path('project_gem1')}]}),
             # when manifest gems are requested expect manifest gems 
             pytest.param(None, None, True, False, list(),
-                {'manifest_gem1':{'gem_name':'manifest_gem1', 'path':pathlib.Path('manifest_gem1')}}),
+                {'manifest_gem1':[{'gem_name':'manifest_gem1', 'path':pathlib.Path('manifest_gem1')}]}),
             # when engine gems are requested expect engine gems 
             pytest.param(None, None, False, True, list(),
-                {'engine_gem1':{'gem_name':'engine_gem1', 'path':pathlib.Path('engine_gem1')}}),
+                {'engine_gem1':[{'gem_name':'engine_gem1', 'path':pathlib.Path('engine_gem1')}]}),
             # when project_path provided and engine gems are requested expect both 
             pytest.param(None, pathlib.Path('C:/project1'), False, True, list(),
-                {'project_gem1':{'gem_name':'project_gem1', 'path':pathlib.Path('project_gem1')},
-                 'engine_gem1':{'gem_name':'engine_gem1', 'path':pathlib.Path('engine_gem1')}}),
+                {'project_gem1':[{'gem_name':'project_gem1', 'path':pathlib.Path('project_gem1')}],
+                 'engine_gem1':[{'gem_name':'engine_gem1', 'path':pathlib.Path('engine_gem1')}]}),
             # when external subdirectories are provided (recursive), expect they are found 
             pytest.param(None, None, False, False, ['external_gem1'],
-                {'external_gem1':{'gem_name':'external_gem1', 'external_subdirectories':['external_sub_gem2'], 'path':pathlib.Path('external_gem1')},
-                'external_sub_gem2':{'gem_name':'external_sub_gem2', 'path':pathlib.Path('external_gem1/external_sub_gem2')}
+                {'external_gem1':[{'gem_name':'external_gem1', 'external_subdirectories':['external_sub_gem2'], 'path':pathlib.Path('external_gem1')}],
+                'external_sub_gem2':[{'gem_name':'external_sub_gem2', 'path':pathlib.Path('external_gem1/external_sub_gem2')}]
+                }),
+            # when a gem with multiple version exists, expect they are both found
+            pytest.param(None, None, False, False, ['external_gem_v1','external_gem_v2'],
+                {'versioned_gem':[
+                    {'gem_name':'versioned_gem', 'version':'1.0.0', 'path':pathlib.Path('external_gem_v1')},
+                    {'gem_name':'versioned_gem', 'version':'2.0.0', 'path':pathlib.Path('external_gem_v2')},
+                    ],
                 }),
         ]
     )
@@ -599,6 +888,10 @@ class TestManifestGetGemsJsonData:
                 return {'gem_name':'external_gem1', 'external_subdirectories':['external_sub_gem2']}
             elif gem_path == self.gem_external_path / 'external_sub_gem2':
                 return {'gem_name':'external_sub_gem2'}
+            elif gem_path == self.gem_version1_external_path:
+                return {'gem_name':'versioned_gem','version':'1.0.0'}
+            elif gem_path == self.gem_version2_external_path:
+                return {'gem_name':'versioned_gem','version':'2.0.0'}
             return  {}
 
         with patch('o3de.manifest.get_gem_json_data', side_effect=get_gem_json_data) as get_gem_json_data_patch, \

scripts/o3de/tests/test_project_manager_interface.py

@@ -11,7 +11,7 @@ from unittest.mock import patch
 from inspect import signature
 import pathlib
 
-from o3de import manifest, engine_properties, register, cmake, engine_template, enable_gem, disable_gem, project_properties, repo, download
+from o3de import manifest, engine_properties, register, engine_template, enable_gem, disable_gem, project_properties, repo, download
 
 # If any tests are failing in this, this means the interface Project Manager depends on has changed.
 # This likely means that some Project Manager functionality has been broken.
@@ -249,7 +249,7 @@ def test_remove_invalid_o3de_projects():
 
 # cmake interface
 def test_get_enabled_gem_cmake_file():
-    sig = signature(cmake.get_enabled_gem_cmake_file)
+    sig = signature(manifest.get_enabled_gem_cmake_file)
     assert len(sig.parameters) >= 2
 
     project_path = list(sig.parameters.values())[1]
@@ -259,7 +259,7 @@ def test_get_enabled_gem_cmake_file():
     assert sig.return_annotation == pathlib.Path
 
 def test_get_enabled_gems():
-    sig = signature(cmake.get_enabled_gems)
+    sig = signature(manifest.get_enabled_gems)
     assert len(sig.parameters) >= 1
 
     cmake_file = list(sig.parameters.values())[0]

scripts/o3de/tests/test_project_properties.py

@@ -49,6 +49,13 @@ TEST_PROJECT_JSON_PAYLOAD = '''
     "icon_path": "preview.png",
     "engine": "o3de-install",
     "restricted_name": "projects",
+    "gem_names": [
+        "ExistingGemA",
+        {
+            "name":"ExistingGemB",
+            "optional": true
+        }
+    ],
     "external_subdirectories": [
         "D:/TestGem"
     ]
@@ -86,7 +93,18 @@ class TestEditProjectProperties:
                     'ProjNameA1', 'ProjNameB', 'ProjID', 'Origin', 
                     'Display', 'Summary', 'Icon', '1.0.0.0', 
                     'A B C', 'B', 'D E F', ['D','E','F'],
-                    'GemA GemB GemB', ['GemA'], None, ['GemB'],
+                    'GemA GemB GemB', ['GemA'], None, ['ExistingGemA',{'name':'ExistingGemB','optional':True},'GemB'],
+                    'NewEngineName',
+                    'o3de>=1.0', 'o3de-sdk==2205.01', None, ['o3de>=1.0'],
+                    ['editor==2.3.4'], None, None, ['framework==1.2.3','editor==2.3.4'],
+                    False, True, pathlib.Path('D:/TestEngine'), pathlib.Path('cmake/CustomEngineFinder.cmake'),
+                    0),
+        # when adding the same gem with a different version, expect only the final gem version remains
+        pytest.param(pathlib.Path('E:/TestProject'),
+                    'ProjNameA1', 'ProjNameB', 'ProjID', 'Origin', 
+                    'Display', 'Summary', 'Icon', '1.0.0.0', 
+                    'A B C', 'B', 'D E F', ['D','E','F'],
+                    'ExistingGemA==1.0.0 GemB', None, None, ['ExistingGemA==1.0.0',{'name':'ExistingGemB','optional':True},'GemB'],
                     'NewEngineName',
                     'o3de>=1.0', 'o3de-sdk==2205.01', None, ['o3de>=1.0'],
                     ['editor==2.3.4'], None, None, ['framework==1.2.3','editor==2.3.4'],
@@ -96,7 +114,7 @@ class TestEditProjectProperties:
                     'ProjNameA2', 'ProjNameB', 'ProjID', 'Origin', 
                     'Display', 'Summary', 'Icon', '1.0.0.0', 
                     'A B C', 'B', 'D E F', ['D','E','F'],
-                    'GemA GemB GemB', ['GemA'], None, ['GemB'],
+                    'GemA GemB GemB', ['GemA'], None, ['GemB','ExistingGemA',{'name':'ExistingGemB','optional':True}],
                     'o3de-sdk',
                     'c==4.3.2.1', None, 'a>=0.1 b==1.0,==2.0', ['a>=0.1', 'b==1.0,==2.0'],
                     ['launcher==3.4.5'], ['framework==1.2.3'], None, ['launcher==3.4.5'],
@@ -106,7 +124,7 @@ class TestEditProjectProperties:
                     'ProjNameA3', 'ProjNameB', 'ProjID', 'Origin', 
                     'Display', 'Summary', 'Icon', '1.0.0.0', 
                     'A B C', 'B', 'D E F', ['D','E','F'],
-                    'GemA GemB GemB', ['GemA'], None, ['GemB'],
+                    'GemA GemB GemB', ['GemA'], None, ['GemB','ExistingGemA',{'name':'ExistingGemB','optional':True}],
                     'o3de-install',
                     None, 'o3de-sdk==2205.01', None, [],
                     None, None, ['framework==9.8.7'], ['framework==9.8.7'],
@@ -116,7 +134,7 @@ class TestEditProjectProperties:
                     'ProjNameA4', 'ProjNameB', 'ProjID', 'Origin', 
                     'Display', 'Summary', 'Icon', '1.0.0.0', 
                     'A B C', 'B', 'D E F', ['D','E','F'],
-                    None, None, '', [],
+                    None, None, '', ['ExistingGemA',{'name':'ExistingGemB','optional':True}],
                     'o3de-custom==1.0.0',
                     None, None, [], [],
                     None, None, [], [],
@@ -126,7 +144,7 @@ class TestEditProjectProperties:
                     'ProjNameA5', 'ProjNameB', 'ProjID', 'Origin', 
                     'Display', 'Summary', 'Icon', '1.0.0.0', 
                     'A B C', 'B', 'D E F', ['D','E','F'],
-                    'GemA GemB GemB', ['GemA'], None, ['GemB'],
+                    'GemA GemB GemB', ['GemA'], None, ['GemB','ExistingGemA',{'name':'ExistingGemB','optional':True}],
                     None,
                     None, None, 'invalid', ['b==1.0,==2.0'], # invalid version
                     None, None, None, ['framework==1.2.3'],
@@ -136,7 +154,7 @@ class TestEditProjectProperties:
                     'ProjNameA6', 'ProjNameB', 'IDB', 'OriginB', 
                     'DisplayB', 'SummaryB', 'IconB', '1.0.0.0',
                     'A B C', 'B', 'D E F', ['D','E','F'],
-                    ['GemA','GemB'], None, ['GemC'], ['GemC'],
+                    ['GemA','GemB'], None, ['GemC'], ['GemC','ExistingGemA',{'name':'ExistingGemB','optional':True}],
                     None,
                     None, None, 'o3de-sdk==2205.1', ['o3de-sdk==2205.1'],
                     None, None, None, ['framework==1.2.3'],
@@ -147,7 +165,7 @@ class TestEditProjectProperties:
                     'ProjNameA4', 'ProjNameB', 'ProjID', 'Origin', 
                     'Display', 'Summary', 'Icon', '1.0.0.0', 
                     'A', None, None, ['A', 'TestProject'],
-                    ['GemA'], None, '', [{'name':'GemA','optional':True}],
+                    ['GemA'], None, '', ['ExistingGemA',{'name':'ExistingGemB','optional':True},{'name':'GemA','optional':True}],
                     'o3de~=1.2',
                     None, None, [], [],
                     None, None, [], [],
@@ -234,7 +252,7 @@ class TestEditProjectProperties:
                 assert set(self.project_json.data.get('user_tags', [])) == set(expected_tags)
 
                 for gem in self.project_json.data.get('gem_names', []):
-                    assert(gem in expected_gem_names)
+                    assert gem in expected_gem_names
 
                 assert set(self.project_json.data.get('compatible_engines', [])) == set(expected_compatible_engines)
                 assert set(self.project_json.data.get('engine_api_dependencies', [])) == set(expected_engine_api_dependencies)

scripts/o3de/tests/test_register.py

@@ -392,7 +392,7 @@ class TestRegisterProject:
                                         ) -> dict:
             all_gems_json_data = {}
             for gem_name in registered_gem_versions.keys():
-                all_gems_json_data[gem_name] = get_gem_json_data(gem_name=gem_name)
+                all_gems_json_data[gem_name] = [get_gem_json_data(gem_name=gem_name)]
             return all_gems_json_data
 
         def get_gem_json_data(gem_name: str = None, gem_path: str or pathlib.Path = None,
@@ -481,9 +481,9 @@ class TestRegisterProject:
             patch('pathlib.Path.is_dir', return_value=True) as _8,\
             patch('o3de.validation.valid_o3de_project_json', return_value=True) as _9, \
             patch('o3de.utils.backup_file', return_value=True) as _10, \
-            patch('o3de.cmake.get_enabled_gem_cmake_file', side_effect=get_enabled_gem_cmake_file) as _11, \
+            patch('o3de.manifest.get_enabled_gem_cmake_file', side_effect=get_enabled_gem_cmake_file) as _11, \
             patch('o3de.manifest.get_gems_json_data_by_name', side_effect=get_gems_json_data_by_name) as get_gems_json_data_by_name_patch,\
-            patch('o3de.cmake.get_enabled_gems', side_effect=get_enabled_gems) as _12:
+            patch('o3de.manifest.get_enabled_gems', side_effect=get_enabled_gems) as _12:
 
             result = register.register(project_path=TestRegisterProject.project_path, force=force, dry_run=dry_run)