Merge pull request #39 from defold/extension-debugging

Extension debugging

docs/en/en.json

@@ -475,6 +475,18 @@
                     {
                         "path": "/manuals/extensions-build-variants",
                         "name": "Build Variants"
+                    },
+                    {
+                        "path": "/manuals/extensions-debugging",
+                        "name": "Debugging tips"
+                    },
+                    {
+                        "path": "/manuals/extensions-debugging-android",
+                        "name": "Debugging on Android"
+                    },
+                    {
+                        "path": "/manuals/extensions-debugging-ios",
+                        "name": "Debugging on iOS"
                     }
                 ]
             }

docs/en/manuals/extensions-debugging-android.md

@@ -0,0 +1,78 @@
+# Debugging on Android
+
+Here we list some ways to debug your executable running on an Android Device
+
+## Android Studio
+
+* Prepare the bundle by setting the `android.debuggable` option in `game.project`
+
+	![android.debuggable](images/extensions/debugging/android/game_project_debuggable.png)
+
+* Bundle the app in debug mode into a folder of choice.
+
+	![bundle_android](images/extensions/debugging/android/bundle_android.png)
+
+* Launch [Android Studio](https://developer.android.com/studio/)
+
+* Choose `Profile or debug APK`
+
+	![debug_apk](images/extensions/debugging/android/android_profile_or_debug.png)
+
+* Choose the apk bundle you just created
+
+	![select_apk](images/extensions/debugging/android/android_select_apk.png)
+
+* Select the main `.so` file, and make sure it has debug symbols
+
+	![select_so](images/extensions/debugging/android/android_missing_symbols.png)
+
+* If it doesn't, upload an unstripped `.so` file. (size is around 20mb)
+
+* Path mappings help you remap where the individual paths from where the executable was built (in the cloud) to an actual folder on your local drive.
+
+* Select the .so file, then add a mapping your local drive
+
+	![path_mapping1](images/extensions/debugging/android/path_mappings_android.png)
+
+	![path_mapping2](images/extensions/debugging/android/path_mappings_android2.png)
+
+* If you have access to the engine source, add a path mapping to that too
+
+		* make sure to checkout the version you are currently debugging
+
+			defold$ git checkout 1.2.148
+
+* Press `Apply changes`
+
+* You should now see the source mapped in your project
+
+	![source](images/extensions/debugging/android/source_mappings_android.png)
+
+* Add a breakpoint
+
+	![breakpoint](images/extensions/debugging/android/breakpoint_android.png)
+
+* Press `Run` -> `Debug "Appname"` and invoke the code you meant to break into
+
+	![breakpoint](images/extensions/debugging/android/callstack_variables_android.png)
+
+* You can now step in the callstack as well as inspect the variables
+
+
+## Notes
+
+### Native Extension job folder
+
+Currently, the workflow is a bit troublesome for development. This is because the job folder name
+is random for each build, making the path mapping invalid for each build.
+
+However, it works fine for a debugging session.
+
+The path mappings are stored in the <project>.iml file in the Android Studio project.
+
+It's possible to get the job folder from the executable
+
+	$ arm-linux-androideabi-readelf --string-dump=.debug_str build/armv7-android/libdmengine.so | grep /job
+
+The jobfolder is named like so `job1298751322870374150`, each time with a random number.
+

docs/en/manuals/extensions-debugging-ios.md

@@ -0,0 +1,131 @@
+# Debugging on iOS/macOS
+
+Here we describe how to debug a build using [XCode](https://developer.apple.com/xcode/), Apple's preferred IDE for developing for macOS and iOS.
+
+## Xcode
+
+* Bundle the app by using bob, with the `--with-symbols` option
+
+		$ cd myproject
+		$ wget http://d.defold.com/archive/<sha1>/bob/bob.jar
+		$ java -jar bob.jar --platform armv7-darwin build --with-symbols debug --archive bundle -bo build/ios -mp <app>.mobileprovision --identity "iPhone Developer: Your Name (ID)"
+
+* Install the app, either with `XCode`, `iTunes` or [ios-deploy](https://github.com/ios-control/ios-deploy)
+
+		$ ios-deploy -b <AppName>.ipa
+
+* Get the `.dSYM` folder (i.e the debug symbols)
+
+	* If it's not using Native Extensions, you can download the `.dSYM` file from [d.defold.com](http://d.defold.com)
+
+	* If you are using a native extension, then the `.dSYM` folder is generated when you build with [bob.jar](https://www.defold.com/manuals/bob/). Only building is required (no archive or bundling):
+
+			$ cd myproject
+			$ unzip .internal/cache/arm64-ios/build.zip
+			$ mv dmengine.dSYM <AppName>.dSYM
+			$ mv <AppName>.dSYM/Contents/Resources/DWARF/dmengine <AppName>.dSYM/Contents/Resources/DWARF/<AppName>
+
+
+### Create Project
+
+To properly debug, we need to have a project, and the source code mapped.
+We're not using this project to build things, only debug.
+
+* Create new XCode project, choose the `Game` template
+
+	![project_template](images/extensions/debugging/ios/project_template.png)
+
+* Choose a name (e.g. `debug`) and the default settings
+
+* Choose a folder to save the project into
+
+* Add your code to the app
+
+	![add_files](images/extensions/debugging/ios/add_files.png)
+
+* Make sure the "Copy items if needed" is unchecked.
+
+	![add_source](images/extensions/debugging/ios/add_source.png)
+
+* This is the end result
+
+	![added_source](images/extensions/debugging/ios/added_source.png)
+
+
+* Disable the `Build` step
+
+	![edit_scheme](images/extensions/debugging/ios/edit_scheme.png)
+
+	![disable_build](images/extensions/debugging/ios/disable_build.png)
+
+* Set the `Deployment target` version so it's now larger than your device iOS version
+
+	![deployment_version](images/extensions/debugging/ios/deployment_version.png)
+
+* Select the target device
+
+	![select_device](images/extensions/debugging/ios/select_device.png)
+
+
+### Launch the debugger
+
+You have a few options to debug an app
+
+1. Either choose `Debug` -> `Attach to process...` and select the app from there
+
+1. Or choose the `Attach to process by PID or Process name`
+
+	![select_device](images/extensions/debugging/ios/attach_to_process_name.png)
+
+	1. Start the app on the device
+
+1. In `Edit Scheme` add the <AppName>.app folder as the executable
+
+### Debug symbols
+
+**To use lldb, the execution must be paused**
+
+* Add the `.dSYM` path to lldb
+
+		(lldb) add-dsym <PathTo.dSYM>
+
+	![add_dsym](images/extensions/debugging/ios/add_dsym.png)
+
+* Verify that `lldb` read the symbols successfully
+
+		(lldb) image list <AppName>
+
+### Path mappings
+
+* Add the engine source (change accordingly for your need)
+
+		(lldb) settings set target.source-map /Users/builder/ci/builds/engine-ios-64-master/build /Users/mathiaswesterdahl/work/defold
+		(lldb) settings append target.source-map /private/var/folders/m5/bcw7ykhd6vq9lwjzq1mkp8j00000gn/T/job4836347589046353012/upload/videoplayer/src /Users/mathiaswesterdahl/work/projects/extension-videoplayer-native/videoplayer/src
+
+	* It's possible to get the job folder from the executable.
+	The jobfolder is named like so `job1298751322870374150`, each time with a random number.
+
+			$ dsymutil -dump-debug-map <executable> 2>&1 >/dev/null | grep /job
+
+* Verify the source mappings
+
+		(lldb) settings show target.source-map
+
+You can check what source file a symbol was originating from using
+
+	(lldb) image lookup -va <SymbolName>
+
+
+### Breakpoints
+
+* Open a file in the project view, and set a breakpoint
+
+	![breakpoint](images/extensions/debugging/ios/breakpoint.png)
+
+## Notes
+
+### Check UUID of binary
+
+In order for the debugger to accept the `.dSYM` folder, the UUID need to match the UUID of the executable being debugged. You can check the UUID like so:
+
+	$ dwarfdump -u <PathToBinary>

docs/en/manuals/extensions-debugging.md

@@ -0,0 +1,151 @@
+# Debugging native code
+
+Here we describe some ways to debug an application, and also symbolicate crash logs.
+
+## Debugger
+
+When you get a particularly troublesome issue in your code,
+there are several ways to find the root cause.
+
+The most common way is to run the code via a `debugger`.
+It let's you step through the code, set `breakpoints` and it will stop the execution if you get a crash.
+
+There are several debuggers around for each platform.
+
+* Visual studio - Windows
+* VSCode - Windows, macOS, Linux
+* Android Studio - Windows, macOS, Linux
+* XCode - macOS
+* WinDBG - Windows
+* lldb / gdb - macOS, Linux, (Windows)
+* ios-deploy - macOS
+
+And each tool can debug certain platforms:
+
+* Visual studio - Windows + platforms supporting gdbserver (E.g. Linux/Android)
+* VSCode - Windows, macOS (lldb), Linux (lldb/gdb) + platforms supporting gdbserver
+* XCode -  macOS, iOS
+* Android Studio - Android
+* WinDBG - Windows
+* lldb/gdb - macOS, Linux, (iOS)
+* ios-deploy - iOS (via lldb)
+
+
+## Print debugging
+
+In certain cases, one might want to add printf() statements to the code.
+Afterwards, you can get the logs from your device and analyze them.
+
+Note that Defold by default only prints using dmLog* functions in the debug build.
+
+### [Android](./extensions-debugging-android.md)
+
+On Android, the simplest way to get the log is to run `adb` in the terminal.
+You can also see the `console` inside Android Studio, which is the same thing.
+
+If you get hold of a stack trace from the Android logs, you might be able to symbolicate it using [ndk-stack](https://developer.android.com/ndk/guides/ndk-stack.html)
+
+### [iOS](./extensions-debugging-ios.md)
+
+On iOS, you need to open either iTunes or XCode to view the device logs.
+
+## Defold Crash Log
+
+The Defold engine saves a `_crash` file when it does a hard crash.
+It will contain information about the system as well as the crash.
+
+You can use the [crash module](https://www.defold.com/ref/crash/) to read this file in the subsequent session.
+
+You are adviced to read the file, gather information and send it to a server of choice to aggregate the data.
+
+### Getting the crash log from a device
+
+#### Android
+
+The adb output says where it is located (different location on different devices)
+
+If the app is [debuggable](https://www.defold.com/manuals/project-settings/#_android), you can get the crash log like so:
+
+	$ adb shell "run-as com.defold.adtest sh -c 'cat /data/data/com.defold.adtest/files/_crash'" > ./_crash
+
+#### iOS
+
+In iTunes, you can view/download an apps container.
+
+In the `XCode -> Devices` window, you can also select the crash logs
+
+
+## Symbolication
+
+If you get a callstack from either a `_crash` file or a log file, you can start symbolicate it.
+This means translating each address in the callstack into a filename and line number, which in turn helps
+when finding out the root cause.
+
+### Get correct engine
+
+It is important that you match the correct engine with the callstack.
+Otherwize it's very likely to send you debugging the incorrect things.
+
+Also, if you are building with native extensions, be sure to add the flag [--with-symbols](https://www.defold.com/manuals/bob/)
+so that you get all the needed data from the build server. For instance, in the `build.zip` you'll find the `dmengine.dSYM` folder for iOS/macOS builds.
+
+Android/Linux executables already contain the debug symbols.
+
+Also, you should keep an unstripped version of the engine.
+This allows for the best symbolication of the callstack.
+
+
+### Android
+
+1. Get it from your build folder
+
+	$ ls <project>/build/<platform>/[lib]dmengine[.exe|.so]
+
+1. Unzip to a folder:
+
+	$ unzip dmengine.apk -d dmengine_1_2_105
+
+1. Find the callstack address
+
+	E.g. in the non symbolicated callstack on Crash Analytics, it could look like this
+
+	#00 pc 00257224 libmy_game_name.so
+
+	Where *00257224* is the address
+
+1. Resolve the address
+
+    $ arm-linux-androideabi-addr2line -C -f -e dmengine_1_2_105/lib/armeabi-v7a/libdmengine.so _address_
+
+### iOS
+
+1. If you are using Native Extensions, the server can provide the symbols (.dSYM) for you (pass "--with-symbols" to bob.jar)
+
+	$ unzip <project>/build/arm64-darwin/build.zip
+	# it will produce a Contents/Resources/DWARF/dmengine
+
+1. If you're not using Native Extensions, download the vanilla symbols:
+
+	$ wget http://d.defold.com/archive/<sha1>/engine/arm64-darwin/dmengine.dSYM
+
+1. Symbolicate using load address
+
+	For some reason, simply putting the address from the callstack doesn't work (i.e. load address 0x0)
+
+		$ atos -arch arm64 -o Contents/Resources/DWARF/dmengine 0x1492c4
+
+	# Neither does specifying the load address directly
+
+		$ atos -arch arm64 -o MyApp.dSYM/Contents/Resources/DWARF/MyApp -l0x100000000 0x1492c4
+
+	Adding the load address to the address works:
+
+		$ atos -arch arm64 -o MyApp.dSYM/Contents/Resources/DWARF/MyApp 0x1001492c4
+		dmCrash::OnCrash(int) (in MyApp) (backtrace_execinfo.cpp:27)
+
+
+
+
+
+
+