Migrating basic Multiplayer testing workflows (#80)

Signed-off-by: robhaco <[email protected]>
Co-authored-by: Pip Potter <[email protected]>

testing-guidance/workflow-tests/multiplayer/basic-multiplayer-workflow-tests.md

@@ -0,0 +1,29 @@
+# Multiplayer Testing
+
+These tests are designed to cover the basic functionality related to running a level from a Multiplayer project.
+
+General Documentation for this feature set: [https://docs.o3de.org/docs/user-guide/gems/reference/multiplayer/](https://docs.o3de.org/docs/user-guide/gems/reference/multiplayer/).
+
+## Prerequisites
+
+*   MultiplayerSample project is cloned and built ( [https://github.com/o3de/o3de-multiplayersample](https://github.com/o3de/o3de-multiplayersample) )
+    * If using this approach, the SampleBase level can be used for most tests.
+    * Ensure the ServerLauncher is built, as well as the Editor and ClientLauncher (if testing via the client launcher).
+*   Alternatively: A project has been created and built that includes the Multiplayer Gem and subsequent changes have been made to enable functionality ( [https://docs.o3de.org/docs/user-guide/gems/reference/multiplayer/multiplayer-gem/configuration/](https://docs.o3de.org/docs/user-guide/gems/reference/multiplayer/multiplayer-gem/configuration/) )
+    *   If using this approach, a new level should be created in the project, The level should utilize a player spawner and network prefabs, as these will act as identifiers for whether multiplayer connections are working as intended.
+
+## Common Issues
+
+*   Most issues will be caught through log errors, which can be seen in the Editor and Launcher consoles or in logs after running tests
+*   In addition, network prefabs can be a visual clue that something isn't working as expected (objects disappearing or not appearing in the first place)
+* Issues connecting to the server are caused either by:
+       1. Processes that remain after closing applications - this can result in errors when trying to launch subsequent times.
+       2. The ServerLauncher has not been built or has not been built against the same commit as Editor/ClientLauncher.
+
+## Workflows
+
+| Workflow                        | Requests                                                                                                                                                                                                                                                                                                                                                    | What To Watch Out For                                                                                                                                                                   |
+|---------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| Editor Game Mode                | *   Run the network enabled level (network prefabs, spawners, etc.) in game mode in Editor<br>*   Leave game mode                                                                                                                                                                                                                                           | *   Network player prefabs are never spawned<br>*   Error logs in Editor Console<br>*   Connection drops<br>*   Server process doesn't shut down                                        |
+| Server/Game Launchers           | *   Launch the server launch and configure to use the network enabled level<br>*   Launch the client/game launcher and connect to the server                                                                                                                                                                                                                | *   Network player prefabs are never spawned<br>*   Error logs in either console (~)<br>*   Connection drops                                                                            |
+| Auto Component Creation/Editing | *   Edit an existing auto component or create a new one (existing example: MultiplayerSample\\Gem\\Code\\Source\\AutoGen\\NetworkHealthComponent.AutoComponent.xml )<br>*   Compile the project<br>*   Add the new/edited component to an entity in the level<br>*   Check the level in Editor game mode<br>*   Check the level in Server and Game Launcher | *   Compile errors that don't seem to result from user error<br>*   Component doesn't display in Component List in Editor<br>*   Error logs in Editor, Server or Game Launcher consoles |

testing-guidance/workflow-tests/multiplayer/multi-machine-multiplayer-workflow-tests.md

@@ -0,0 +1,74 @@
+# Testing Multiplayer Launchers on Multiple Machines
+
+This test is intended to verify successful connection between separate machines on the same network.  Ideally this is completed with a Linux machine as server and any number of Windows machines as clients, but it can be completed with any combination.
+
+## Prerequisites
+
+**All machines used for this test should be on the same Network**
+
+**Acquire and Compile MultiplayerSample project** ( [https://github.com/o3de/o3de-multiplayersample](https://github.com/o3de/o3de-multiplayersample) )
+
+**Set Up for the Server Machine:**
+
+These steps only need to be done on the 1 machine that will be acting as the server/host for the test
+
+### Linux:
+
+1.  Open a terminal window
+2.  Enter “sudo ufw allow from any to any port 33450 proto udp”
+3.  Enter “sudo ufw status verbose”
+4.  Check that rule(s) exist for port 33450 with action “ALLOW IN”
+5.  Enter “ifconfig”
+6.  Note the inet ip address found in the first block of information, this address will be used by the clients to connect to the server machine
+
+Cleaning up after testing:
+
+1.  Open a terminal window
+2.  Enter “sudo ufw status numbered”
+3.  In the list, note the number for the first entry of your created 33450 rules
+4.  Enter “sudo ufw delete #” where ‘#’ is the number assigned to the rule
+5.  Press ‘y’ to confirm
+6.  Repeat steps 2-5 to remove the second entry (the number for the entry will have updated after deleting the first)
+
+### Windows:
+
+1.  Open Windows Defender Firewall
+2.  Click Advanced Settings
+3.  Click Inbound Rules
+4.  Click New Rule
+5.  Click Port, then Next
+6.  Click UDP
+7.  If it’s not already selected, select Specific local ports and enter 33450, then Next
+8.  Click Allow the connection, then Next
+9.  Select Network types you’d like to allow connection over (Public shouldn’t be necessary, but you can leave it as the default), then Next
+10.  Name the rule, then Finish
+11.  Open a cmd window
+12.  Enter “ipconfig”
+13.  Note the IPv4 address, this will be used by the clients to connect to the server machine
+
+Cleaning up after testing:
+
+1.  Open Windows Defender Firewall
+2.  Click Advanced Settings
+3.  Click Inbound Rules
+4.  Find the rule with the name that was given
+5.  Right-Click the rule and Select Delete
+6.  Confirm the dialog
+
+## What To Watch For
+
+1.  Number of clients connected
+2.  Performance on clients (FPS, input response)
+3.  Logs from clients and server after testing
+
+## Testing
+
+1.  Server machine has completed the above prerequisites
+2.  All machines have built MultiplayerSample project(for minimum compiling - the server machine should at least have MultiplayerSample.ServerLauncher built and the clients should have MultiplayerSample.GameLauncher) - [https://github.com/o3de/o3de-multiplayersample](https://github.com/o3de/o3de-multiplayersample)
+3.  Server machine launches MutiplayerSample.ServerLauncher
+    1.  If server.cfg is being used and is still the default entries of “host” and “LoadLevel Levels/SampleBase/SampleBase.spawnable” then nothing more should need to be done on this machine
+    2.  If server.cfg is not being used, the user will need to press “~” to bring up the console after launching the launcher and enter “host” followed by “loadlevel samplebase”
+4.  Client machines launch MultiplayerSample.GameLauncher
+    1.  If client.cfg is being used, add an entry before the default “connect” entry that says cl\_serveraddr ###.###.#.### where the numbers are the IP address of the server machine that was noted during server set up.  This needs to be done **before** launching the game launcher
+    2.  If client.cfg is not being used, once the game launcher is up the user will need to press “~” to bring up the console and enter “cl\_serveraddr ###.###.#.###” (the noted server machine ip address) followed by “connect”
+5.  At this point the client machines should have loaded the level that the Server machine defined and can move around in the level

testing-guidance/workflow-tests/remote-tools-connection-gem/remote-tools-connection-gem-workflow-tests.md

@@ -0,0 +1,26 @@
+# Remote Tools Connection Gem
+
+This gem enables connection to the Editor to other remote tools. Currently the gem is only used to connect LuaIDE tools.  As the Gem is created in an extensible way, O3DE may support connections from other tools, but the following are tests for the initial needs/functionality of the gem.
+
+## Common Issues
+
+*   Connection failure
+*   Connection drop
+*   Tool instability
+
+## Workflows
+
+| Workflow                                                                  | Steps                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | What to look for                                                                           |
+|---------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------|
+| LuaIDE launched from Editor tools menu and connected to running Editor    | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Launch the LuaIDE from the tools menu (Tools > Lua Editor)<br>5.  Connect LuaIDE to Editor (Target > Editor)                                                                                                                                                                                                                                          | LuaIDE connects to the running Editor instance                                             |
+| LuaIDE launched standalone and connects to launched Editor                | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  In the file explorer, navigate to and launch LuaIDE.exe from the project build directory (not the Engine)<br>5.  Connect LuaIDE to Editor (Target > Editor)                                                                                                                                                                                           | LuaIDE connects to the running Editor instance                                             |
+| LuaIDE launched from Lua script component and connects to Launched Editor | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Create a new Entity<br>5.  Attach a lua script component to the created Entity<br>6.  Click the open LuaIDE button on the script component<br>7.  Connect LuaIDE to Editor (Target > Editor)                                                                                                                                                          | LuaIDE connects to the running Editor instance                                             |
+| LuaIDE launched standalone with no Editor running                         | 1.  Remote Tools Connection gem is added and project built<br>2.  In file explorer, navigate to and launch LuaIDE.exe                                                                                                                                                                                                                                                                                                                                                                    | No errors occur when the LuaIDE is launched on its own with no Editor to connect to        |
+| Lua Debugging (step through, breakpoints)                                 | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Create a new Entity<br>5.  Attach a lua script component to the created Entity<br>6.  Create a new Lua Script (something basic with Debug.Log()s works fine)<br>7.  Assign the saved lua script<br>8.  Open the assigned lua script in the LuaIDE (if not already open)<br>9.  Put a breakpoint on one of the Debug.Log lines<br>10.  Enter game mode | Logs show the script stopped at the assigned breakpoint                                    |
+| Classes References Populate                                               | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Open the LuaIDE<br>5.  Connect LuaIDE to Editor (Target > Editor)<br>6.  Click on the Editor Window after to allow Debugging to attach<br>7.  Return to the Lua Editor                                                                                                                                                                                | The class references list populates with available classes, ebuses, etc.                   |
+| Editor performance when Lua Editor is Launched                            | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Observe performance in Editor<br>5.  Open the LuaIDE<br>6.  Observe performance in Editor                                                                                                                                                                                                                                                             | There should be no notable change in Editor performance with the LuaIDE open and connected |
+| Launcher connection                                                       | 1.  Remote Tools Connection gem is added and project built<br>2.  Launch a project's Game Launcher<br>3.  Launch the LuaIDE and select the game launcher as Target                                                                                                                                                                                                                                                                                                                       | The LuaIDE successfully connects to the game launcher                                      |
+| LuaIDE (Tools Menu) - No Connection Gem                                   | 1.  Remote Tools Connection gem is disabled and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Launch the LuaIDE from the tools menu (Tools > Lua Editor)                                                                                                                                                                                                                                                                                         | The LuaIDE and Editor remain stable                                                        |
+| LuaIDE (Script Component) - No Connection Gem                             | 1.  Remote Tools Connection gem is disabled and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Create a new Entity<br>5.  Attach a lua script component to the created Entity<br>6.  Click the open LuaIDE button on the script component                                                                                                                                                                                                         | The LuaIDE and Editor remain stable                                                        |
+| Script Canvas - No Connection Gem                                         | 1.  Remote Tools Connection gem is disabled and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Launch Script Canvas from the Tools Menu                                                                                                                                                                                                                                                                                                           | The Script Canvas tool launches and the Editor remains stable                              |
+| Script Canvas - With Connection Gem                                       | 1.  Remote Tools Connection gem is disabled and project built<br>2.  Launch the Editor<br>3.  Create or open an existing level<br>4.  Launch Script Canvas from the Tools Menu                                                                                                                                                                                                                                                                                                           | The Script Canvas tool launches and the Editor remains stable                              |