more foobar added

doc/tmemo/README

@@ -1,7 +1,7 @@
 This directory contains short technical memos documenting 
-technical decisions made or planned to be made in ser or 
-accompanying applications. The memos serve as requests
-for comments in the literal sense (not to be confused
+best practices and technical decisions made or proposed
+for ser or accompanying applications. The memos serve as 
+requests for comments in the literal sense (not to be confused
 with IETF's RFCs).
 
 The documents here are drafts, for whose technical maturity
@@ -17,3 +17,7 @@ in plain ASCII. Their filenames consists of
 For example: tmemo-johndoe-backtobackua.txt
 No version numbers are used in filename -- these are displayed 
 in text and assigned by CVS server. 
+
+--
+TODO: reasonable topics which I would like to be addressed here
+include NAT traversal and multidomain management. -jiri

doc/tmemo/tmemo-jiri-b2bua.txt

@@ -1,7 +1,7 @@
 $Id$
 
-Building Prepaid Scenarios Using SIP/SER
-========================================
+Draft Design Options for Building Prepaid Scenarios Using SIP/SER
+==================================================================
 
 Jiri Kuthan, iptel.org, January 2003
 
@@ -67,13 +67,16 @@ end-devices.
 The behaviour of the session-timer-based construct is as follows:
 a caller intiates a call through a proxy server. The proxy server
 determines maximum acceptable call length and inidiacates it using
-the session timer mechanisms. The timer is then propagates to
+the session timer mechanisms. The timer is then propagated to
 the end-device using SIP. If it actually hits, the terminating 
 gateway will try to revitalize the session using a re-INVITE. 
 The proxy server then can recalculate available credit, and if too 
 low, deny the re-invitation. The end-device is then supposed 
 to terminate the call using a BYE.
 
+  Note: a similar technique can be used to keep firewall
+  pinholes alive.
+
 We have never experimented with the session-timer-based solution.
 We do not know if some session timer negotiation troubles can
 occur. We do not know how widely support of session timer is
@@ -82,13 +85,45 @@ effort for session-timer will result in some changes and when
 it will complete. Nevertheless, we think it is worth trying.
 Its appeal is it leaves call-termination, a call-stateful
 feature, down in the end-devices and does not pose too big
-burden on server developers and especially operators.
+burden on server developers and especially operators. The
+scenario is mentioned in [ccframework], section 7.2.12.
 
 WE THUS ENCOURAGE VOLUNTEERS TO EXPERIMENT WITH THIS OPTION.
 TAKE THE GATEWAY YOU HAVE, LOOK AT IT IF IT SUPPORTS ST,
 ADD ST TO SER PROXY AND CHECK IF THINGS WORK.
 
-
+If end-devices do not make the job of call cut-off well, someone
+else, a third-party, will have to. That is where B2BUA comes
+in. It is a singaling entity that terminates signaling dialog
+with both call parties and keeps dialog state during the whole
+conversation time. Acting as a UA allows it to generate SIP
+messages on its own (as opposed to just realying someone 
+else's messages, as proxy do). Because it appears as a simple
+UA to each party in the call, it works with any phone without
+need for support for any extensions. Problems with use of
+B2BUA are described in the next section.
+
+One could perhaps make the cut-off component easier to
+implement than a whole B2BUA. Well, as long as the implementation
+is expected to be 100% conform, there is no way around it.
+The BYE initiator needs to be aware of current dialog state,
+it will not be able to do its job otherwise. Obvisouly,
+it needs to remember initial constant information such as
+call-id. However, it needs to keep information that changes
+too. Particularly, it must know current CSeq -- Cseq gets
+updated with in-dialog transactions and failure to send
+a proper CSeq would result in denied BYE. An implementer
+wishing to make his life easier and avoid the B2BUA
+machinery could feel tempted just to "spoof" dialog state
+in a proxy, (i.e., not to translate one into another one) and
+if it decides to cut-off, send a BYE using the current
+status. That introduced race conditions, however, as
+another transaction may show up in middle of the cut-off
+process, increase remote party's CSeq and disqualify the
+delayed cut-off BYE.
+
+So design choices are either B2BUA, or a "dialog spoofer"
+with a risk of race conditions.
 
 2. What Are B2BUA limitations?
 ------------------------------
@@ -112,7 +147,9 @@ c) e2e security does not work -- implementations willing to
    SIP message bodies. B2BUA breaks the e2e security if
    it needs to change the body.
 d) economical aspects: it is simply yet another piece of
-   software you need to purchase or develop
+   non-trivial software you need to purchase or develop and
+   operate. that's maybe the strongest argument: it is
+   expensive.
 
 Lot of this conversation has taken place on IETF SIP
 and SIPPING mailing lists. Few messages from these
@@ -122,138 +159,24 @@ discussions are referred from
 3. How to Implement a B2BUA Using ser
 -------------------------------------
 
+One of the design decisions is whether to make B2BUA work
+as a module or an external application bound to ser via some
+of its interfaces. 
 
+You will have to keep dialog table keyed by callid and tags,
+update it on INVITE complition, remove on BYE, updated on
+e.g. INFO changing dialog state.
 
-At 10:00 AM 1/6/2003, chang hui wrote:
->Jiri,
->
->Thanks for your explanation, and let me know the architecture drawback of the B2BUA.
-
-
-I've already done so in my previous email. If something was not clear
-enough, let me know.
-
-
->Since we have no way to choose other means to implement pre-paid, we have to go along with B2BUA in a short term.
->Could you give me any advise how to implement B2BUA based on SER and estimate the work we should do?
->Could you give me a performance estimate?
-
-
-A hand-waving guestimate is performance degrades by 50%.
-(We currently achieve up to 3-5 kCPS on a PC -- fair capacity
- to slice off from.).
-
-
-a B2BUA does a lot of things:
-- first, it keeps dialog state -- it rememembers cseq,  callid, 
-  route set, etc. for the whole time of a call (i.e., it eats 
-  memory). All this information is needed when you later wish 
-  to initiate correct BYEs.
-- it translates UAC to UAS transactions and vice versa
-- you probably want to save the dialog state on some persistence
-  storage (mysql) -- signaling would not work on reboot otherwise
-
-
-That would take quite some development work. I think the amount
-of work can be somewhat lowered if normal (record-routed) proxy 
-processing is used, as opposed to a full B2BUA which terminats
-all UAS transactions and translates them to UAC transactions.
-You then still need to do the following:
-- keeping a dialog table (keyed by callid and local/remote tags)
-- updating the dialog table (new items on INVITE completion, remove 
-  dialogs on BYE, update dialog state, such as CSeq, on any other 
-  request).
-- starting a timer on beginning of a dialog that -- when expired,
-  subject to balance and charging plans --  sends BYEs to all call  
-  parties using dialog context.
-
-
-That could be implemented as a new ser module, which registers
-TM-module callbacks to be updated on transactions completions.
-One could also move the dialog maintenance out of ser to some
-shell scripts to make programming easier. That would however
-very likely degrade performance noticeably.
-
+The cut-off timer needs to be started too.
 
 Also note, that these scenarios are based only on signaling -- there
 are no PSTN-prepaid-style anouncement "you can call 5 minutes"
 and "your call will be cut off in 20 seconds". It is doable too,
-but it is probably more meaningful to start with the signaling
-part.
-
-
--Jiri
+but it is probably more meaningful to start with the signaling.
+See [tmemo_media] for more on that.
 
+Appendix: References
+--------------------
+[ccframework] http://www.iptel.org/ietf/callprocessing/#draft-ietf-sipping-cc-framework
 
 
->Best Regards and Thanks.
->
->
->Chang Hui
->-----Original Message-----
->From: Jiri Kuthan [mailto:[email protected]]
->Sent: Saturday, January 04, 2003 8:29 PM
->To: chang hui; [email protected]
->Subject: RE: [Serusers] About B2BUA
->
->Hello,
->
->I see -- prepaid scenarios are indeed difficult without a B2BUA.
->There has been a proposal few times to use session timer (a proxy
->looks at ballance and attaches a hint to SIP requests indicating
->when a call should terminate), but the work has not been pursued.
->
->You may find a discussion of B2BUA architectural drawbacks on the
->SIP mailing list, selected postings are at http://www.iptel.org/info/trends/#b2bua.
->imho, the most compelling issue is that of robustness and scalability.
->A b2bua needs to keep track of all current calls. A broken b2bua affects
->signaling for all existing calls.
->
->Basically, a B2BUA is simply two UAs glued together. It accepts
->transactions as a server, and initiates client transactions
->based on them. It keeps dialog state (callid, cseqs, etc.) and
->may initiate in-dialog transactions on its own (like the BYE
->transaction in which you are interested).
->
->It is doable to implement a B2BUA on top of ser, but it would
->cost quite some development effort. Particularly, it would take
->dialog maintenance (better with persistency so that signaling
->does not get broken on reboot). We  can provide guidanance to
->volunteers willing to go through this exercise.
->
->-Jiri
->
->At 02:28 AM 1/4/2003, chang hui wrote:
->>Jiri,
->>
->>Thanks for your prompt response.
->>We want to implement a pre-paid system in which once subscriber's balance is depleted, the dialog could be torn in time. However other Proxy or other elements could not take part in the call, they could not send a BYE to caller directly. It's the why we consider B2BUA.
->>We project to build a B2BUA to support voice/video/IM at first stage, and support other SIP based services as they emerged.
->>However, I just noticed the definition of B2BUA in 2543-bis04 in several sentences,  there has no other analysis on performance, reliability, limitations and how to implement it. So, I hope to get help from the society.
->>Thanks for your help again.
->>
->>Koalas
->>
->>-----Original Message-----
->>From: Jiri Kuthan [mailto:[email protected]]
->>Sent: Friday, January 03, 2003 11:06 PM
->>To: chang hui; [email protected]
->>Subject: Re: [Serusers] About B2BUA
->>
->>Hello,
->>
->>ser is not a B2BUA -- it can act as proxy, redirect, transactional UAS
->>or registrar. These modes make a vast majority of network scenarios
->>happy without a need to use a B2BUA. Which is good, because B2BUAs
->>inherently have certain scalability, reliability and security limitations.
->>
->>Is there a particular reason why you would like to use a B2BUA?
->>
->>-Jiri
->>
->>At 08:00 AM 1/3/2003, chang hui wrote:
->>>Hi All,
->>>
->>>I am newbie of this field, thanks everyone help me.
->>>I am interesting in B2BUA, however, except some brief defination in 3261, I could not find any further defination or how to implement about B2BUA, I noticed that SER could be implemented as a B2BUA, where can I find some implementation? or where can I get any description?
->>>

doc/tmemo/tmemo-jiri-media.txt

@@ -16,7 +16,8 @@ media server is to bind voice to SIP applications with optional
 support of other tools (SIP SUB/NOT, mysql, TTS, etc.) It has
 to be configurable in such a way it can act in different component
 roles: click-to-dial server, voicemail server, conferencing server, 
-text-to-speech anouncement server, etc.
+text-to-speech anouncement server, etc. (see [ccframework] for
+a longer listing of related applications)
 
 TOC
 ---
@@ -31,6 +32,11 @@ SIP components without a need for a B2BUA, a technology we consider
 suboptimal.  (This network architecture puts only very little addition
 requirements on the media server.)
 
+Note that this section tends to focus on specific design choices.
+A nice review of the whole spolution space for composing services
+is provided by Mahy at al in [ccframework]. This document is 
+referred to many times throughout this memo -- it is a really
+nice piece of work. 
 
 Section 2, Media Server Requirements, explains basic requirements
 a media server needs to fulful to make a good job in the component
@@ -40,8 +46,6 @@ script, are explained in section 3.
 Related work, references and example scripts are attached in
 appendices.
 
-
-
 1) Targeted Scenarios and Component Model
 --------------------------------------
 Many application scenarios can provide a pleasant experience to users 
@@ -71,15 +75,17 @@ On success, the controller can then hand-off the call to a PSTN gateway.
 This architecture is extremelly good in that it introduces distributed 
 components. Decomposition, an imporant design principle, is performed 
 in a fair, peer-2-peer manner that allows linking SIP devices in
-a very flexible way.
+a very flexible way. No additional mechanisms (except perhaps HTTP
+reporting) like CTI are needed -- SIP is the service glue.
 
 The biggest shortcoming of this architecture is imho its central piece, 
 the controller. It is simply too central. A B2BUA design  inherently causes 
 many concerns: security, scalability, and reliability ones. B2BUA solutions 
 proposed in 3pcc draft [3pcc] by Rosenberg have several signaling drawbacks 
 too: tricky media matching (flow III), backwards compatibility
-(flow IV), etc. There is also the economical aspect: a B2BUA
-costs money or development effort.
+(flow IV), etc. There is also the economical aspect, which is possibly
+the biggest one: a B2BUA costs money or development effort, and the
+aforementioned issues cost more operational overhead too.
 
 We believe it is beneficial to avoid such B2BUA constructs. The mechanism
 we are advocating is distributing the service state machine accross 
@@ -128,7 +134,8 @@ eliminates a need for the "HTTP reporting hack" in jdr's architecture.
 Session status is reported in SIP URIs. Cooperating components just 
 need to agree on a scheme for URI usage. That should be easy for SIP 
 servers as URI processing is a primary SIP ability.
-(A similar proposal for such use of URIs was stated in [msuri].)
+(A similar proposal for such use of URIs was stated in [msuri]
+ and [ccframework].)
 
 A simple application of this more distributed approach is REFER-based 
 "click-to-dial" service. In this scenario, a media component gets somehow 
@@ -141,10 +148,26 @@ The "pre-paid verification component" referred to in this section is another
 example use of this model. It establishes a call with caller, looks at 
 desired destination, processes PIN in media stream, and makes a decision 
 to hand-over to a gateway. It than disappears from the conversation.
+(See the [b2bua] tmemo for how to implement call cut-off, a feature
+ esential to operation of prepaid scenarios. the memo explains how
+ to achieve cut-off without a B2BUA.)
 
 Note that the application call-control framework [ccframework] by Mahy et al. 
 explicitely mentions a more peer-2-peer oriented approach based on REFER as 
-a good alternative to a centralized B2BUA approach. 
+a good alternative to a centralized B2BUA approach and gives some more
+details.
+
+  Quicknote: this multi-stage conversation model based on REFER use has
+  some appeals for charging too -- in case separate stages need to be
+  charged in a different manner (like initial anouncement "this
+  800 number is free only from US" for free, and then the real
+  IVR for whatever charges apply). It clearly separates these
+  stages in terms of calls and transactions. We think that is a clean
+  way of doing things as opposed to breaking transaction model
+  by some 18x provisional "half-talk"  constructs. (Which raise
+  unclarity like "is the call established or not", "how long does
+  a proxy server need to keep provisional transactional state",
+  "when should I charge for 18x", and whatsoever.)
 
 
 
@@ -181,6 +204,11 @@ speech software  [cmuspeech] (!!!) including Sphinx, festvox,
 openvxi are examples of other pieces of work worth intergrating
 with.
 
+Another important example of what should be achievable with
+the externsibility framework is updating SUB/NOT state. Whatever
+the udpate mechanism is, it must be doable to allow things such as 
+message waiting indication [mwi].
+
 3) On Scripting Language
 ---------------------
 
@@ -200,8 +228,9 @@ abstraction and simplicity are the key for application programming.
 
 The primary living space of the media server programming language
 should be calls. Scripts should be able to deal with calls:
-initiate, terminate and transfer them. ([ccframework] coins
-"replace", "join", "fork").
+initiate, terminate and transfer them. [ccframework] coins
+additional ones: "replace", "join", "fork". It also well explains
+how to compose some well-known services using these features.
 
 An important lower-level escape way should be the ability to initiate
 in-call (in-dialogue) transaction. That is what allows the server
@@ -223,29 +252,93 @@ The sources include but are not limited to SIP messsages, timers
 recording), external events from local apps (perhaps via FIFO), 
 media events (DTMF), SIP notifications.
 
-There was a proposal too, to introduce notion of SUB/NOT and presence
-to the language. Examples of use are "initiate a conference call when all 
-invited  users are on-line", "repeat a call when called party is
-no longer busy" [dialogpackage], "query participant list in a multi-party
-conversation", etc. We haven't discussed yet whether, and if so
-how such scenarios should be reflected in the language.
+The SIP notifications should be easy to map to the the event
+system. Scripts can subscribe to event, and when they occur,
+SIP NOTIFIes are translates to the script's event system.
+It can be used for implementing things such as "retry
+when a user is no longer busy" [dialogpackage], or keep
+updated on list of participants in a conversation.
+
+I think a great simplification of event processing is to guarantee their 
+processing in series. It avoids all sorts of nasty issues which pop up 
+when events related to a call are processed in parallel. The synchronization 
+issues will reduce to event queue maintenance and execution logic will be 
+easier to understand. I think we can happily trade it for some - probably 
+marginal - performance decrease.
+
+That would imply an event queue associated with each call. Its filled in from 
+all sort of event sources (Web, FIFO, media, timers, etc.). Events are picked 
+up from the queue in order of arrival. On call termination, the queue is 
+destroyed, empty or not.
+
+A question is how to deal with some long jobs -- such as recording or playing
+media. Waiting until they complete may take infinitely, and result in blocking
+such event like incoming BYE, which should actually stop a recording.
+
+I suggest that well-known long jobs, such as playing or recording
+are simply sent to background and then script processing continues.
+The question is what qualifies as "long job" -- I think quite few
+things which may potentially take infinite time. Recording and playing
+are important examples, call set-up (time until callee answers) or
+NOTIFY during call transfer other ones. When such background
+"infinite" jobs complete, user can be notified via the event system.
+
+  An alternative would be to introduce a background operator
+  like & in shells. However I suspect that unwise users could
+  forget to use it and cause infinite blocks. Defining in a 
+  case-by-case way that certain operations are blocking seems
+  safer to me.
+
+
+Most other jobs are transaction-related and are short enough,
+so that processing of other events can wait until they complete.
+BYE/REFERR, whateverver, takes in the worst case of an error
+time until final response timer hits -- delay which should be 
+tolerable to any other signaling. If I for example initate
+an INFO transaction and BYE arrives in the meantime, it is
+imho not so bad to let the BYE wait until INFO completes.
+
 
 requriement summary)
 
 So far, we have identified the following requirements:
-    - programming effectivity (easy and intutitive to use)
+    - programming effectivity (easy and intutitive to use); abstraction
+	  from protocol details, focusing on calls and primitives about
+	  them (initiate, terminate, transfer, perhaps join, fork
+	  and replace too); some simple in-dialog transaction processing
+	  should be an escape for other signaling things
     - parallelism (mutltiple scripts processed at the same time, 
       multiple calls refered from a single script)     
     - variables (refering to multiple calls)
-    - event processing
+    - event processing -- ability to map a variety of events
+	  to the event system (SIP NOTIFIes, FIFO requests, 
+	  call-related SIP requests such as INVITE/BYE, timers,
+	  media events, etc.); events processed in series
+    - services described in [ccframework] should be verified
+      to be doable with the language
     - ability to change script without rebooting the server
+	- uri processing is an absolute must to be able to implement
+	  the component model as described above
+	- textual processing -- one should be able to compose new
+	  transactions out of parts of existing requests (there should 
+	  be some automated request->var casting, e.g., $Request.from) and
+	  dialog state. 
+           ret=$call.new_transaction("INFO", 
+               "headerfield: value\n\hf2: ".$Request.from."\n", "two USD");
     - extensibility (i.e., the ability of the environment to link 
-      external binary libs and refer to them from scripts)
+      external binary libs and refer to them from scripts); particular
+      items of interest are mysql support, http support, tts,
+	  support for updating SUB/NOT status (such as for [mwi])
 
 Some design options mentioned so far (nice but not required)
     - have some casting from input to variables (e.g, $request.header.callid)
     - use OO -- there are many people for whom OO is easier
     - exceptions to group error processing
+	- variable scope and context -- it would be imho nice if
+	  all variables related to a call would be tightened to
+	  it, so that they are accessible whenever another
+	  call-related event occurs, and they are not accessible
+	  from anywhere else
 
 main-loop language)
 
@@ -263,16 +356,23 @@ the scripting languages with code libraries. Other cons are
 bigger image size and dependency on third-party software.
 However, risks of bugs and unability to tweak things are rather 
 low with well-established open-source software like python.
-Possibly, syntax of an own language might better capture
-semantics of the media server.
 
-As said, no determination has been made yet. Author of this
-memo is little a bit uncomfortable with current amount of
-development work put on ser team and hopes that use of an
-off-the-shelve language would save work cycles. (Hopefuly,
-this hope will not be broken by tremendous effort spent
-in integration with supporting libraries.)
+Also, a new language would have the benefit of making the
+syntax more visibly tied to the semantics model.
 
+As said, no determination has been made yet. Author of this
+memo changes his opinion on this issue in hourly intervals.
+The amount of work to be done with a new language is
+scaring and may become an overkill. On the other hand
+making sure that the language well expresses the
+nature of the server is appealing. Perhaps one could
+reuse some API for linking external libraries like 
+those used in PHP or python, so than getting access to
+the libraries would be easy.
+
+BEFORE ANY DECISION IS MADE, WE SHOULD BEST GO THROUGH
+FEW MORE EXAMPLES ([ccframework]) AND SHOW HOW THEY CAN
+BE ACHIEVED USING THE DESIGNED ENVIRONMENT.
 
 see more )
 
@@ -307,6 +407,7 @@ B) References
 [festival] http://www.cstr.ed.ac.uk/projects/festival
 [mscml] http://www.iptel.org/ietf/callprocessing/apps/#draft-vandyke-mscml
 [msuri] http://www.iptel.org/info/players/ietf/allsipdir/draft-burger-sipping-msuri-01.txt
+[mwi] http://www.iptel.org/info/players/ietf/callprocessing/#draft-ietf-sipping-mwi
 [kpml] http://www.iptel.org/ietf/callprocessing/apps/#draft-burger-sipping-kpml
 [nv] http://www.naturalvoices.att.com/
 [refer] http://www.iptel.org/ietf/callprocessing/refer/#draft-ietf-sip-refer