more content for kemi tutorial

kamailio-kemi-framework/docs/acc.md

@@ -2,14 +2,24 @@
 
 The functions exported by `acc` module to KEMI are listed in the next sections.
 
+The documentation of `acc` module is available online at:
+
+  * [acc.html](https://kamailio.org/docs/modules/devel/modules/acc.html)
+
 ### KSR.acc.acc_log_request(...) ###
 
 `int KSR.acc.acc_log_request("comment")`
 
+Equivalent of native cfg function: `acc_log_request("comment")`.
+
 ### KSR.acc.acc_db_request(...) ###
 
 `int KSR.acc.acc_db_request("comment", "dbtable")`
 
+Equivalent of native cfg function: `acc_db_request("comment", "dbtable")`.
+
 ### KSR.acc.acc_request(...) ###
 
-`int KSR.acc.acc_request("comment", "dbtable")`
+`int KSR.acc.acc_request("comment", "dbtable")`
+
+Equivalent of native cfg function: `acc_request("comment", "dbtable")`.

kamailio-kemi-framework/docs/core.md

@@ -3,7 +3,8 @@
 The core of Kamailio exports via KEMI:
 
   * several functions directly to `KSR` module (like `KSR.function(params)`), which are mostly
-  the main function from the core and for writing log messages
+  the main function from the core and for writing log messages (some being part of xlog module for
+  native kamailio.cfg language)
   * the `KSR.hdr` submodule, which are the most used functions for managing SIP message headers
 
 Example of using KEMI functions exported to Lua interpreter:
@@ -18,22 +19,42 @@ Exported functions from core directly to KSR module or KSR.hdr submodule are lis
 
 `void KSR.dbg("msg")`
 
+Write a log message to DEBUG level.
+
 ### void KSR.err(...) ###
 
 `void KSR.err("msg")`
 
+Write a log message to ERROR level.
+
 ### void KSR.info(...) ###
 
 `void KSR.info("msg")`
 
+Write a log message to INFO level.
+
 ### oid KSR.log(...) ###
 
 `void KSR.log("level", "msg")`
 
+Write a log message specifying the level value. The level parameter can be:
+
+  * "dbg"
+  * "info"
+  * "warn"
+  * "crit"
+  * "err"
+
+If level value is not matched, then "err" log level is used.
+
 ### KSR.drop(...) ###
 
 `void KSR.drop()`
 
+Set the DROP flag, so at the end of KEMI script execution, the SIP request branch or the SIP response is not forwarded.
+
+Note: it doesn't not stop the execution of KEMI script, see KSR.x.drop() for an alternative with that purpose.
+
 ### KSR.is_myself(...) ###
 
 `bool KSR.is_myself("uri")`

kamailio-kemi-framework/docs/index.md

@@ -1,36 +1,85 @@
-# Kamailio Kemi Framework #
+# Kamailio KEMI Framework #
 
 ```
 Author: Daniel-Constantin Mierla ([email protected])
+
+Support: <[email protected]>
 ```
 
 ## Introduction ##
 
-Kamailio uses a scripting laguage for its configuration file (kamailio.cfg). This scripting language (referred also as
-native scripting language) was developed from scratch, with initial design going back to years 2001-2002.
+### Preamble ###
+
+`Kamailio` uses a scripting laguage for its configuration file (`kamailio.cfg`). This scripting language (referred also
+as native scripting language) was developed from scratch, with initial design going back to years 2001-2002.
 
 The configuration file is composed from several main statements:
 
   * global parameters
   * loading modules
   * module parameters
-  * routing blocks
+  * routing blocks (e.g., `request_route {...}`, `reply_route {...}`, `branch_route {...}`, ...)
 
-In terms of execution, kamailio.cfg interprets only once, at startup, the statements related to:
+In terms of execution, the `kamailio.cfg` interpreter runs only once (at startup) the statements related to:
 
   * global parameters
   * loading modules
   * module parameters
+  * initialization event routes (e.g., `event_route[htable:mod-init] {...}`)
+
+At runtime, the `kamailio.cfg` interpreter can run many times:
+
+  * routing blocks  (e.g., `request_route {...}`, `reply_route {...}`, `branch_route {...}`, ...)
 
 The routing blocks are similar to functions and can be executed many times, some of them for each received
 SIP message, some of them on various events fired by core or modules.
 
 While the native scripting language offers a large set of functions and is able to do many logical, arithmetic
-and string operations, its limits are met in many cases, especially when in need to integrate with external systems.
+and string operations, its limits are met in some cases, especially when in need to integrate with external systems.
 
-Another missing feature in the native scripting language is reloading without a restart.
+Another missing feature in the native scripting language is reloading without a restart. The interpreter was designed
+to precompile `kamailio.cfg` at startup, doing specific optimization for fast runtime operations. Adding support for
+reload will result in losing those optimizations and doing checks to see if the execution script needs to be parsed
+again.
 
-Kamailio Embedded Interface (KEMI) framework was added first in Kamailio v5.0.0 to allow using other scripting
+The solution to meet the demands of having a more flexible and feature-rich scripting language, with the possibility
+of reloading the routing script without restart, as well keeping the old native interpreter, was to develop a new
+framework for executing SIP routing scripts written in other programming languages. It was named
+`Kamailio Embedded Interface` (`KEMI`) framework.
+
+### KEMI Overview ###
+
+`Kamailio Embedded Interface` (`KEMI`) framework was added first in `Kamailio v5.0.0` to allow using other scripting
 languages to write SIP routing logic instead of using the native routing blocks. The interpreters for these scripting
-languages are embedded by Kamailio, initialized at startup, in this way being also as fast as poosible during runtime
-execution.
+languages are embedded by Kamailio, initialized at startup, in this way being also as fast as possible during runtime
+execution.
+
+With KEMI routing script, the `kamailio.cfg` keeps only the parts with:
+
+  * global parameters
+  * loading modules and
+  * modules settings
+
+All of these parts are evaluated only once at startup. Many of the global and module parameters can be changed at
+runtime via `RPC` commands.
+
+The KEMI comes in the picture by allowing that the equivalend of routing blocks to be written in a different scripting
+language. In other words, the routing blocks are now functions written in a KEMI supported scripting language.
+
+For each new KEMI supported scripting language a module has to be developed for Kamailio, this module linking
+Kamailio to the scripting language interpreter.
+
+At this moment there are couple of supported KEMI scripting languages, respectively `JavaScript`, `Lua`, `Python`,
+`Squirrel` -- see next sections for more details about these programming languages.
+
+Note: Kamailio has other modules that allow inline execution of scripts written in other programming languages, such
+as `Perl`, `.Net` (`C#`, etc.), `Java`, but they don't implement the KEMI yet, likely that some of them will get support in
+the near future.
+
+The main benefits of using KEMI framework:
+
+  * reload of SIP routing scripts without restart of Kamailio
+  * better documentation knowledge base for supported scripting languages
+  * more complete and flexible scripting languages
+  * larger set of libraries that can be used from the scripting languages
+  * external tools to troubleshoot or test the scripts

kamailio-kemi-framework/docs/kemi.md

@@ -30,6 +30,27 @@ modparam("app_jsdk", "load", "/path/to/script.js")
 cfgengine "jsdt"
 ```
 
+Inside the JavaScript script, following functions have a predefined role:
+
+  * `ksr_request_route()` - is executed by Kamailio core every time a SIP request is received. If this function is
+  not defined, then Kamailio will write error messages. This is equivalent of `request_route {}` from `kamailio.cfg`.
+  * `ksr_reply_route()` - is executed by Kamailio core every time a SIP Response (reply) is received. If this function
+  is not defined, then Kamailio will not write error messages. This is equivalent of `reply_route {}`
+  from `kamailio.cfg`.
+  * `ksr_onsend_route()` - is executed by Kamailio core every time a SIP request (and optionally for a response) is
+  sent out. If this function is not defined, then Kamailio will not write error messages. This is equivalent of
+  `onsend_route {}` from `kamailio.cfg`.
+  * `branch route callback` - the name of the JavaScript function to be executed instead of a branch route has to be
+  provided as parameter to `KSR.tm.t_on_branch(…)`
+  * `onreply route callback` - the name of the JavaScript function to be executed instead of an onreply route has to be
+  provided as parameter to `KSR.tm.t_on_reply(…)`
+  * `failure route callback` - the name of the JavaScript function to be executed instead of a failure route has to be
+  provided as parameter to `KSR.tm.t_on_failure(…)`
+  * `branch failure route callback` - the name of the JavaScript function to be executed instead of an `event route` for
+  branch failure has to be provided as parameter to `KSR.tm.t_on_branch_failure(…)`
+  * `event route callback` - the name of the JavaScript function to be exectued instead of module specific
+  `event_route` blocks is provided via `event_callback` parameter of that module
+
 A complete example of using JavaScript as KEMI languages is offered by the next two files:
 
   * [kamailio-basic-kemi.cfg](https://github.com/kamailio/kamailio/blob/master/misc/examples/kemi/kamailio-basic-kemi.cfg)
@@ -59,6 +80,29 @@ modparam("app_lua", "load", "/path/to/script.lua")
 cfgengine "lua"
 ```
 
+Inside the Lua script, following functions have a predefined role:
+
+  * `ksr_request_route()` - is executed by Kamailio core every time a SIP request is received. If this function is
+  not defined, then Kamailio will write error messages. This is equivalent of `request_route {}` from `kamailio.cfg`.
+  * `ksr_reply_route()` - is executed by Kamailio core every time a SIP Response (reply) is received. If this function
+  is not defined, then Kamailio will not write error messages. This is equivalent of `reply_route {}`
+  from `kamailio.cfg`.
+  * `ksr_onsend_route()` - is executed by Kamailio core every time a SIP request (and optionally for a response) is
+  sent out. If this function is not defined, then Kamailio will not write error messages. This is equivalent of
+  `onsend_route {}` from `kamailio.cfg`.
+  * `branch route callback` - the name of the Lua function to be executed instead of a branch route has to be
+  provided as parameter to `KSR.tm.t_on_branch(…)`
+  * `onreply route callback` - the name of the Lua function to be executed instead of an onreply route has to be
+  provided as parameter to `KSR.tm.t_on_reply(…)`
+  * `failure route callback` - the name of the Lua function to be executed instead of a failure route has to be
+  provided as parameter to `KSR.tm.t_on_failure(…)`
+  * `branch failure route callback` - the name of the Lua function to be executed instead of an `event route` for
+  branch failure has to be provided as parameter to `KSR.tm.t_on_branch_failure(…)`
+  * `event route callback` - the name of the Lua function to be exectued instead of module specific
+  `event_route` blocks is provided via `event_callback` parameter of that module
+
+Note: besides the new KEMI Lua KSR module, the old `sr` Lua module exported by `app_lua` is still available.
+
 A complete example of using Lua as KEMI languages is offered by the next two files:
 
   * [kamailio-basic-kemi.cfg](https://github.com/kamailio/kamailio/blob/master/misc/examples/kemi/kamailio-basic-kemi.cfg)
@@ -77,6 +121,119 @@ The documentation for `app_lua` is available at:
 
   * [app_lua.html](https://kamailio.org/docs/modules/devel/modules/app_lua.html)
 
+#### Basic KEMI Lua Scripting Example ####
+
+The file `kamailio.cfg` with global parameters and module settings:
+
+```
+#!KAMAILIO
+
+####### Global Parameters #########
+
+debug=3
+log_stderror=yes
+fork=yes
+children=2
+ 
+memdbg=5
+memlog=5
+ 
+auto_aliases=no
+ 
+listen=udp:127.0.0.1:5060
+ 
+mpath="modules"
+ 
+loadmodule "jsonrpcs.so"
+loadmodule "kex.so"
+loadmodule "tm.so"
+loadmodule "tmx.so"
+loadmodule "sl.so"
+loadmodule "pv.so"
+loadmodule "maxfwd.so"
+loadmodule "textops.so"
+loadmodule "xlog.so"
+loadmodule "ctl.so"
+loadmodule "debugger.so"
+loadmodule "app_lua.so"
+ 
+# ----------------- setting module-specific parameters ---------------
+ 
+# ----- jsonrpcs params -----
+modparam("jsonrpcs", "pretty_format", 1)
+ 
+# ----- tm params -----
+# auto-discard branches from previous serial forking leg
+modparam("tm", "failure_reply_mode", 3)
+# default retransmission timeout: 30sec
+modparam("tm", "fr_timer", 30000)
+# default invite retransmission timeout after 1xx: 120sec
+modparam("tm", "fr_inv_timer", 120000)
+ 
+# ----- debugger params -----
+modparam("debugger", "cfgtrace", 1)
+ 
+####### Routing Logic ########
+ 
+modparam("app_lua", "load", "/etc/kamailio/kamailio.lua")
+ 
+cfgengine "lua"
+```
+
+The file `/etc/kamailio/kamailio.lua` with the routing logic for runtime:
+
+```Lua
+-- Kamailio - equivalent of routing blocks in Lua
+-- KSR - the new dynamic object exporting Kamailio functions
+-- sr - the old static object exporting Kamailio functions
+--
+-- SIP request routing
+-- equivalent of request_route{}
+function ksr_request_route()
+	KSR.info("===== request - from kamailio lua script\n");
+ 
+	if KSR.maxfwd.process_maxfwd(10) < 0 then
+		KSR.sl.send_reply(483, "Too Many Hops");
+		return;
+	end
+ 
+	-- KSR.sl.sreply(200, "OK Lua");
+ 
+	KSR.pv.sets("$du", "sip:127.0.0.1:5080")
+	KSR.tm.t_on_branch("ksr_branch_route_one");
+	KSR.tm.t_on_reply("ksr_onreply_route_one");
+	KSR.tm.t_on_failure("ksr_failure_route_one");
+ 
+	if KSR.tm.t_relay() < 0 then
+		KSR.sl.send_reply(500, "Server error")
+	end
+end
+ 
+-- SIP response routing
+-- equivalent of reply_route{}
+function ksr_reply_route()
+	KSR.info("===== response - from kamailio lua script\n");
+end
+ 
+-- branch route callback
+-- equivalent of a branch_route{}
+function ksr_branch_route_one()
+	KSR.info("===== branch route - from kamailio lua script\n");
+end
+ 
+-- onreply route callback
+-- equivalent of an onreply_route{}
+function ksr_onreply_route_one()
+	KSR.info("===== onreply route - from kamailio lua script\n");
+end
+ 
+-- failure route callback
+-- equivalent of a failure_route{}
+function ksr_failure_route_one()
+	KSR.info("===== failure route - from kamailio lua script\n");
+end
+```
+
 ### Python KEMI Interpreter ###
 
 It is implemented by `app_python` module. The Python interpreter is linked from `libpython`, supported Python versions:
@@ -88,6 +245,34 @@ modparam("app_python", "script_name", "/path/to/script.py")
 cfgengine "python"
 ```
 
+In the Python script you have to declare the global `mod_init()` method where to instantiate an object of a class that
+implements the other callback methods (functions) to be executed by Kamailio.
+
+Inside the new class, the following methods are relevant:
+
+  * `ksr_request_route(self, msg)` - is executed by Kamailio core every time a SIP request is received. If this
+  function is not defined, then Kamailio will write error messages. This is equivalent of `request_route {}` from
+  `kamailio.cfg`.
+  * `ksr_reply_route(self, msg)` - is executed by Kamailio core every time a SIP Response (reply) is received. If this
+  function is not defined, then Kamailio will not write error messages. This is equivalent of `reply_route {}` from
+  `kamailio.cfg`.
+  * `ksr_onsend_route(self, msg)` - is executed by Kamailio core every time a SIP request (and optionally for a
+  response) is sent out. If this function is not defined, then Kamailio will not write error messages. This is
+  equivalent of `onsend_route {}` from `kamailio.cfg`.
+  * `branch route callback` - the name of the Python function to be executed instead of a branch route has to be
+  provided as parameter `to KSR.tm.t_on_branch(…)`
+  * `onreply route callback` - the name of the Python function to be executed instead of an onreply route has to be
+  provided as parameter to `KSR.tm.t_on_reply(…)`
+  * `failure route callback` - the name of the Python function to be executed instead of a failure route has to be
+  provided as parameter to `KSR.tm.t_on_failure(…)`
+  * `branch failure route callback` - the name of the Python function to be executed instead of an event route for
+  branch failure has to be provided as parameter to `KSR.tm.t_on_branch_failure(…)`
+  * `event route callback` - the name of the Python function to be exectued instead of module specific
+  `event_route` blocks is provided via `event_callback` parameter of that module
+
+Note: besides the new KEMI Python KSR module, the old `Router` Python module exported by `app_python` is still
+available.
+
 A complete example of using Python as KEMI languages is offered by the next two files:
 
   * [kamailio-basic-kemi.cfg](https://github.com/kamailio/kamailio/blob/master/misc/examples/kemi/kamailio-basic-kemi.cfg)
@@ -106,6 +291,120 @@ The documentation for `app_python` is available at:
 
   * [app_python.html](https://kamailio.org/docs/modules/devel/modules/app_python.html)
 
+#### Basic KEMI Python Scripting Example ####
+
+The file `kamailio.cfg` with the global parameters and modules settings:
+
+```
+#!KAMAILIO
+
+####### Global Parameters #########
+ 
+debug=4
+log_stderror=yes
+fork=yes
+children=2
+ 
+memdbg=5
+memlog=5
+ 
+auto_aliases=no
+ 
+listen=udp:127.0.0.1:5060
+ 
+mpath="modules"
+ 
+loadmodule "mi_fifo.so"
+loadmodule "kex.so"
+loadmodule "tm.so"
+loadmodule "tmx.so"
+loadmodule "sl.so"
+loadmodule "pv.so"
+loadmodule "maxfwd.so"
+loadmodule "textops.so"
+loadmodule "xlog.so"
+loadmodule "ctl.so"
+loadmodule "mi_rpc.so"
+loadmodule "debugger.so"
+loadmodule "app_python.so"
+ 
+# ----------------- setting module-specific parameters ---------------
+ 
+# ----- mi_fifo params -----
+modparam("mi_fifo", "fifo_name", "/tmp/kamailio_fifo")
+ 
+ 
+# ----- tm params -----
+# auto-discard branches from previous serial forking leg
+modparam("tm", "failure_reply_mode", 3)
+# default retransmission timeout: 30sec
+modparam("tm", "fr_timer", 30000)
+# default invite retransmission timeout after 1xx: 120sec
+modparam("tm", "fr_inv_timer", 120000)
+ 
+# ----- debugger params -----
+modparam("debugger", "cfgtrace", 1)
+ 
+####### Routing Logic ########
+ 
+modparam("app_python", "script_name", "/etc/kamailio/kamailio.py")
+ 
+cfgengine "python"
+```
+
+The file `/etc/kamailio/kamailio.py` with the routing logic for runtime:
+
+```Python
+import sys
+import Router.Logger as Logger
+import KSR as KSR
+ 
+def dumpObj(obj):
+    for attr in dir(obj):
+        # KSR.info("obj.%s = %s\n" % (attr, getattr(obj, attr)));
+        Logger.LM_INFO("obj.%s = %s\n" % (attr, getattr(obj, attr)));
+ 
+def mod_init():
+    KSR.info("===== from Python mod init\n");
+    # dumpObj(KSR);
+    return kamailio();
+ 
+class kamailio:
+    def __init__(self):
+        KSR.info('===== kamailio.__init__\n')
+ 
+    def child_init(self, rank):
+        KSR.info('===== kamailio.child_init(%d)\n' % rank)
+        return 0
+ 
+    def ksr_request_route(self, msg):
+        KSR.info("===== request - from kamailio python script\n");
+        msg.rewrite_ruri("sip:[email protected]:5080");
+        KSR.tm.t_on_branch("ksr_branch_route_one");
+        KSR.tm.t_on_reply("ksr_onreply_route_one");
+        KSR.tm.t_on_failure("ksr_failure_route_one");
+        KSR.sl.send_reply(100, "Trying")
+        if KSR.tm.t_relay() < 0 :
+            KSR.sl.send_reply(500, "Server error")
+        return 1;
+ 
+    def ksr_reply_route(self, msg):
+        KSR.info("===== response - from kamailio python script\n");
+        return 1;
+ 
+    def ksr_branch_route_one(self, msg):
+        KSR.info("===== branch route - from kamailio python script\n");
+        return 1;
+ 
+    def ksr_onreply_route_one(self, msg):
+        KSR.info("===== onreply route - from kamailio python script\n");
+        return 1;
+ 
+    def ksr_failure_route_one(self, msg):
+        KSR.info("===== failure route - from kamailio python script\n");
+        return 1;
+```
+
 ### Squirrel KEMI Interpreter ###
 
 It is implemented by `app_sqlang` module. The Squirrel language interpreter is imported inside the module from
@@ -178,4 +477,86 @@ kamctl rpc app_jsdt.api_list
 kamctl rpc app_lua.api_list
 kamctl rpc app_python.api_list
 kamctl rpc app_sqlang.api_list
-```
+```
+
+## Exporting C Function To KEMI Interpreters ##
+
+Because Kamailio needs to load modules in order to export useful functions to KEMI, statical wrappers to C functions
+implemented in other modules cannot be used, because they will introduce dependencies on each embedded interpreter
+for all modules.
+
+The implementation relies on defining a set of generic functions that are exported to each embedded interpreter, which
+are associated at startup with a Kamailio C functions. The lookup at runtime is by an integer index, therefore very
+fast.
+
+Currently the association table size is 1024 (it means that there can be maximum 1024 Kamailio C functions exported
+to the interpreter by a configuration file). The number can be increased, but it should be fairly enough as all
+`kamailio.cfg` functions are around 1000 and it is no real use case to load all the modules at the same time for use
+in production. Also, many functions may not be exported to an embedded language, as they have native alternative in
+the embedded language.
+
+Each existing component of Kamailio (e.g., module), can export new functions to KEMI in the following way:
+
+  * declare an array of type sr_kemi_t
+  * register it to KEMI in mod_register() function (or at startup for core components) using sr_kemi_modules_add()
+
+The structure `sr_kemi_t` is declared in Kamailio core, the file `kemi.h`:
+
+```C
+#define SR_KEMI_PARAMS_MAX	6
+ 
+typedef struct sr_kemi {
+	str mname; /* sub-module name */
+	str fname; /* function name */
+	int rtype; /* return type (supported SR_KEMIP_INT/BOOL) */
+	void *func; /* pointer to the C function to be executed */
+	int ptypes[SR_KEMI_PARAMS_MAX]; /* array with the type of parameters */
+} sr_kemi_t;
+```
+
+Next C code snippet shows how sl module exports two functions:
+
+  * C function `sl_send_reply_str(…)` is exported as `sl.sreply(…)`
+  * C function `send_reply(…)` is exported as `sl.freply(…)`
+
+```C
+static sr_kemi_t sl_kemi_exports[] = {
+	{ str_init("sl"), str_init("sreply"),
+		SR_KEMIP_INT, sl_send_reply_str,
+		{ SR_KEMIP_INT, SR_KEMIP_STR, SR_KEMIP_NONE,
+			SR_KEMIP_NONE, SR_KEMIP_NONE, SR_KEMIP_NONE }
+	},
+	{ str_init("sl"), str_init("freply"),
+		SR_KEMIP_INT, send_reply,
+		{ SR_KEMIP_INT, SR_KEMIP_STR, SR_KEMIP_NONE,
+			SR_KEMIP_NONE, SR_KEMIP_NONE, SR_KEMIP_NONE }
+	},
+ 
+	{ {0, 0}, {0, 0}, 0, NULL, { 0, 0, 0, 0, 0, 0 } }
+};
+ 
+int mod_register(char *path, int *dlflags, void *p1, void *p2)
+{
+	sr_kemi_modules_add(sl_kemi_exports);
+	return 0;
+}
+```
+
+Note that the exported array is ended by a sentinel of `0`/`NULL` values for all fields.
+
+Exported functions must take first parameter as `sip_msg_t*` type (which is the structure with the SIP message being
+processed), then followed by up to 6 int or str* parameters. When `SR_KEMIP_NONE` is given in the array with the types
+of parameters, it means there is no parameter from there on (some compilers may rise warning, so it is recommended
+to fill all 6 items in array).
+
+The functions exported by Kamailio core are listed inside the array `_sr_kemi_core` from the file `kemi.c`.
+
+Not all combinations of extra (after `sip_msg_t*`) parameters types are supported right now - currently the are:
+
+  * `1 param` - can be `int` of `str*`
+  * `2 params` - any combination of `int` or `str*`
+  * `3 params` - any combination of `int` or `str*`
+  * `4 params` - all have to be `str*` (other combinations to be added as needed)
+  * `5 params` - all have to be `str*` (other combinations to be added as needed)
+  * `6 params` - all have to be `str*` (other combinations to be added as needed)
+