- support for multi-value filtering when generating records

- client side filtering via set_dbopt ("client_side_filtering"), to enable resonable filtering of multivalue fields and fields without ordering rules
- all comparision operations supported by filter (requires appropriate rules at LDAP server), otherwise client side filtering may be applied
- generalized time timezone and localtime, ZULU support
- generalized time fraction seconds support
- support for alias dereferencing in openldap, option in table definition
- support for referral dereferencing in openldap, option in table definition
- doc added

modules_s/ldap/README

@@ -0,0 +1,222 @@
+
+1. ldap module
+
+Jan Janak
+
+   Iptel.org
+
+   Copyright © 2008 Iptel.org GmBH
+   Revision History
+   Revision $Revision$ $Date$
+     _________________________________________________________________
+
+   1.1. Overview
+   1.2. Dependencies
+   1.3. Parameters
+
+        1.3.1. config (string)
+        1.3.2. reconnect_attempt (integer)
+
+   1.4. Functions
+
+1.1. Overview
+
+   The LDAP module is database driver, i.e. it implements DB API functions. The
+   goal is map database query defined by table, matching fields and result
+   fields to LDAP search in sub-tree defined by root, object class, attributes
+   and pass it to the OpenLDAP which communicates with the LDAP server.
+
+   This procedure is sometimes tricky because the LDAP does not support all
+   database features or supports them in different manner. Here we must express
+   especially filtering and multi-values. The multi-value is de facto array of
+   single values. If the LDAP module get a multi-value field then generates
+   record for every single value, respectively for every combination in case
+   the more fields contain multi-value.
+
+   The LDAP supports natively "AND", "OR", "NOT" logical operators and "equal",
+   "non-equal", "less-or-equal" and "greater-or-equal" comparison operators.
+   Therefore    "less"    and   "greater"   operators   are   mapped   as
+   "less/greater-or-equal-AND-not-equal". It's important realize it when the
+   attribute which will be used for filtering may contain multi-value. The LDAP
+   server evaluates comparison operator on multi-value so that the result for
+   record is true if the condition is satisfied for any single value. The
+   single values not satisfying condition are not truncated. It implies two
+   cases for positive comparison, e.g. "equal", the result contains values not
+   satisfying the condition, the case may be handled by additional filter in
+   the LDAP module, the negative comparison, e.g. "non-equal", does not return
+   record at all. Because the LDAP module cannot know if the LDAP attribute may
+   logically  contain  multi-value  so  there is introduced DB API option
+   client_side_filtering which forces filtering such fields in the LDAP module,
+   i.e.  the  LDAP server returns larger result set because the filtering
+   condition is not passed there.
+
+   The necessary condition of successful filtering of particular attribute at
+   the LDAP server is correct attribute definition. The "equal"/"non-equal"
+   operator requires equality matching rule, the "greater"/"less" operator
+   requires ordering matching rule. If required matching rule is missing the
+   LDAP server silently returns empty result set. In case of double filtering
+   both at the LDAP servar and the LDAP module, e.g. multi-value and equal
+   comparison, check the LDAP server matching rule satisfies your needs or use
+   client_side_filtering feature.
+
+   The LDAP server may be identified either complete specification of host,
+   user, password in URI or is specification reference to connection section of
+   config file. Note in the second case there is only one slash.
+
+   Example 1. URI example
+        ...
+        modparam("auth", "db_url", "ldap://admin:[email protected]");
+
+        modparam("auth", "db_url", "ldap:/ldap_server1");
+
+        ...
+
+   Features:
+     * simple, SASL authentication, TLS
+     * server and client side filtering
+     * read-only queries
+     * optional referral chasing by OpenLDAP
+     * optional reference chasing by OpenLDAP
+
+1.2. Dependencies
+
+   none
+
+1.3. Parameters
+
+1.3.1. config (string)
+
+   Default value is ldap.cfg.
+
+   The filename (relatively to ser config file) of mapping database to LDAP
+   definition. It is the main configuration file for the LDAP module in SER.
+   The  configuration  file maps database table names used in SER to LDAP
+   directory sub-trees to be searched. In addition to that the configuration
+   file also allows to configure the LDAP search filter and maps database field
+   names to LDAP attribute names and vice versa.
+
+   Example 2. Example config
+        ...
+        modparam("ldap", "config", "my-ldap.cfg");
+        ...
+
+   Example 3. Configuration file example
+# Supported Attribute Type Names:
+#  * GeneralizedTime
+#  * Integer
+#  * BitString
+#  * Boolean
+#  * String
+#  * Binary
+#  * Float
+#
+
+[connection:ldap_server1]
+host=127.0.0.1
+port=389
+username=ser
+password=heslo
+# LDAP or LDAP SASL authentication mechanism.
+# Allowed values: none (default), simple, digest-md5, external
+authtype=simple
+
+# tls encryption
+tls=off
+
+# Specifies the file that contains certificates for all of the Certificate
+# Authorities the ldap module will recognize.
+ca_list=/home/kg/work/openssl/demoCA/cacert.pem
+
+# Specifies what checks to perform on server certificates in a TLS session
+# allowed values are never/allow/try/demand
+# see the TLS_REQCERT tls option part of ldap.conf(8) man page for more details
+require_certificate=demand
+
+#
+# Table credentials contains SIP digest authentication credentials.
+#
+[table:credentials]
+
+# In our LDAP directory we store SIP digest credentials under
+# "Digest Credentials" organization unit so this is where searches for digest
+# credentials should start.
+base = "ou=Digest Credentials,dc=iptel,dc=org"
+
+# We search the whole subtree.
+scope = subtree
+
+# For digest credentials we are only interested in objects with objectClass
+# 'digestAuthCredentials', objects of all other types are ignored.
+filter = "(objectClass=digestAuthCredentials)"
+
+# Mapping of field names to LDAP attribute names and vice versa. Names are
+# delimited using ':', the first name is database field name as used in SER
+# modules, the second name (after :) is corresponding LDAP attribute name,
+# optionally preceeded with LDAP attribute syntax name in parentheses.
+field_map = password : (Binary) digestPassword
+field_map = realm : digestRealm
+field_map = auth_username : digestUsername
+field_map = uid : serUID
+field_map = flags : (BitString) serFlags
+
+# retrieve at most sizelimit entries for a search
+#sizelimit = 2147483647
+
+# wait at most timelimit seconds for a search to complete
+#timelimit = 120
+
+# chase references automatically by OpenLDAP. Default is "never"
+# chase_references = never | searching | finding | always
+
+# chase referrals automatically by OpenLDAP. Default is "no"
+# chase_referrals = yes | no
+
+#
+# Domain table stores information about virtual domains
+#
+[table:domain]
+
+# Objects mapping domain IDs to domain names and vice versa are stored
+# in the subtree with the following root:
+base = "ou=Domains,dc=iptel,dc=org"
+
+scope = subtree
+
+# We are only interested in serDomain objects when looking up information
+# about virtual domains.
+filter = "(objectClass=serDomain)"
+
+field_map = did : (String) serDID
+field_map = domain : (String) serDomain
+field_map = flags : (BitString) serFlags
+
+#
+# Table domain_attrs contains domain attributes, domain attributes store
+# extra information about virtual domains.
+#
+[table:domain_attrs]
+base = "ou=Domains, dc=iptel,dc=org"
+scope = subtree
+
+filter = "(objectClass=serDomainAttr)"
+
+field_map = did : serDID
+field_map = name : serAttrName
+field_map = type : (Integer) serAttrType
+field_map = value : serAttrValue
+field_map = flags : (BitString) serFlags
+
+1.3.2. reconnect_attempt (integer)
+
+   Default value is 3.
+
+   Number of reconnect attempts when connection to the LDAP server is lost.
+
+   Example 4. Example reconnect_attempt
+        ...
+        modparam("ldap", "reconnect_attempt", "5");
+        ...
+
+1.4. Functions
+
+   none

modules_s/ldap/doc/Makefile

@@ -0,0 +1,29 @@
+#
+# The list of documents to build (without extensions)
+#
+DOCUMENTS = ldap
+
+#
+# The root directory containing Makefile.doc
+#
+ROOT_DIR=../../..
+
+#
+# Validate docbook documents before generating output
+# (may be slow)
+#
+#VALIDATE=1
+
+#
+# You can override the stylesheet used to generate
+# xhtml documents here
+#
+#XHTML_XSL=$(ROOT_DIR)/doc/stylesheets/xhtml.xsl
+
+#
+# You can override the stylesheet used to generate
+# plain text documents here
+#
+#TXT_XSL=$(XHTML_XSL)
+
+include $(ROOT_DIR)/Makefile.doc

modules_s/ldap/doc/ldap.xml

@@ -0,0 +1,310 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE section PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 
+   "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd">
+
+<section id="ldap" xmlns:xi="http://www.w3.org/2001/XInclude">
+    <sectioninfo>
+	<authorgroup>
+	    <author>
+		<firstname>Jan</firstname>
+		<surname>Janak</surname>
+		<affiliation><orgname>Iptel.org</orgname></affiliation>
+		<address>
+		    <email>jan  at iptel dot org</email>
+		</address>
+	    </author>
+	</authorgroup>
+	<copyright>
+	    <year>2008</year>
+	    <holder>Iptel.org GmBH</holder>
+	</copyright>
+	<revhistory>
+	    <revision>
+		<revnumber>$Revision$</revnumber>
+		<date>$Date$</date>
+	    </revision>
+	</revhistory>
+
+    </sectioninfo>
+
+    <title>ldap module</title>
+
+    <section id="ldap.overview">
+		<title>Overview</title>
+		<para>
+		The LDAP module is database driver, i.e. it implements DB API functions.
+		The goal is map database query defined by table, matching fields and result fields
+		to LDAP search in sub-tree defined by root, object class, attributes and
+		pass it to the <emphasis>OpenLDAP</emphasis> which communicates with the LDAP server.
+		</para>
+		
+		<para>
+		This procedure is sometimes tricky because the LDAP does not support
+		all database features or supports them in different manner. Here we
+		must express especially <emphasis>filtering</emphasis> and <emphasis>
+		multi-values</emphasis>. The multi-value is de facto array of single
+		values. If the LDAP module get a multi-value field then generates
+		record for every single value, respectively for every combination
+		in case the more fields contain multi-value.
+		</para>
+
+		<para>		
+		The LDAP supports natively "AND", "OR", "NOT" logical operators and "equal", "non-equal",
+		"less-or-equal" and "greater-or-equal" comparison operators. Therefore
+		"less" and "greater" operators are mapped as "less/greater-or-equal-AND-not-equal".
+		It's important realize it when the attribute which will be used for
+		filtering may contain multi-value. 
+		The LDAP server evaluates comparison operator on multi-value so that
+		the result for record is true if the condition is satisfied for any single
+		value. The single values not satisfying condition are not truncated. It implies two cases
+		for positive comparison, e.g. "equal", the result contains values not satisfying the
+		condition, the case may be handled by additional filter in the LDAP module,
+		the negative comparison, e.g. "non-equal", does not return record at all.
+		Because the LDAP module cannot know if the LDAP attribute may
+		logically contain multi-value so there is introduced DB API option <emphasis>client_side_filtering</emphasis>
+		which forces filtering such fields in the LDAP module, i.e. the LDAP server returns
+		larger result set because the filtering condition is not passed there.
+		</para>
+
+		<para>
+		The necessary condition of successful filtering of particular
+		attribute at the LDAP server is correct attribute definition.
+		The "equal"/"non-equal" operator requires <emphasis>equality matching rule</emphasis>,
+		the "greater"/"less" operator requires <emphasis>ordering matching rule</emphasis>.
+		If required matching rule is missing the LDAP server silently returns
+		empty result set. In case of double filtering both at the LDAP servar and the LDAP
+		module, e.g. multi-value and equal comparison, check the LDAP server matching
+		rule satisfies your needs or use <emphasis>client_side_filtering</emphasis> feature. 
+		</para>
+
+		<para>
+		The LDAP server may be identified either 
+		complete specification of host, user, password in URI or
+		is specification reference to <varname>connection</varname> section
+		of <varname>config</varname> file. Note in the second case there is only
+		one slash.
+		</para>
+
+		<example>
+			<title>URI example</title>
+			<programlisting>
+	...
+	modparam("auth", "db_url", "ldap://admin:[email protected]");
+
+	modparam("auth", "db_url", "ldap:/ldap_server1");
+
+	...
+			</programlisting>
+		</example>
+
+		<para>
+			Features:
+			<itemizedlist>
+				<listitem>
+					<para>
+						simple, SASL authentication, TLS
+					</para>
+				</listitem>
+				<listitem>
+					<para>
+						server and client side filtering
+					</para>
+				</listitem>
+				<listitem>
+					<para>
+						read-only queries
+					</para>
+				</listitem>
+				<listitem>
+					<para>
+						optional referral chasing by OpenLDAP
+					</para>
+				</listitem>
+				<listitem>
+					<para>
+						optional reference chasing by OpenLDAP
+
+					</para>
+				</listitem>
+			</itemizedlist>
+		</para>
+    </section>
+
+	<section id="ldap.dep">
+		<title>Dependencies</title>
+				   
+		<para>
+		none
+		</para>
+	</section>
+
+	<section id="ldap.parameters">
+
+		<title>Parameters</title>
+
+		<section id="config">
+			<title><varname>config</varname> (string)</title>
+			<para>
+				Default value is <emphasis>ldap.cfg</emphasis>.
+			</para>
+			<para>
+			The filename (relatively to ser config file) of mapping database to LDAP definition.
+
+			It is the main configuration file for the LDAP module in SER.
+			The configuration file maps database table names used in SER to LDAP directory
+			sub-trees to be searched. In addition to that the configuration file also allows to
+			configure the LDAP search filter and maps database field names to
+			LDAP attribute names and vice versa. 
+			</para>
+			<example>
+				<title>Example <varname>config</varname></title>
+				<programlisting>
+	...
+	modparam("ldap", "config", "my-ldap.cfg");
+	...
+				</programlisting>
+			</example>
+
+
+			<example>
+				<title>Configuration file example</title>
+				<programlisting>
+# Supported Attribute Type Names:
+#  * GeneralizedTime
+#  * Integer
+#  * BitString
+#  * Boolean
+#  * String
+#  * Binary
+#  * Float
+#
+
+[connection:ldap_server1]
+host=127.0.0.1
+port=389
+username=ser
+password=heslo
+# LDAP or LDAP SASL authentication mechanism.
+# Allowed values: none (default), simple, digest-md5, external
+authtype=simple
+
+# tls encryption
+tls=off
+
+# Specifies the file that contains certificates for all of the Certificate
+# Authorities the ldap module will recognize.
+ca_list=/home/kg/work/openssl/demoCA/cacert.pem
+
+# Specifies what checks to perform on server certificates in a TLS session
+# allowed values are never/allow/try/demand
+# see the TLS_REQCERT tls option part of ldap.conf(8) man page for more details
+require_certificate=demand
+
+#
+# Table credentials contains SIP digest authentication credentials.
+#
+[table:credentials]
+
+# In our LDAP directory we store SIP digest credentials under 
+# "Digest Credentials" organization unit so this is where searches for digest 
+# credentials should start.
+base = "ou=Digest Credentials,dc=iptel,dc=org"
+
+# We search the whole subtree.
+scope = subtree
+
+# For digest credentials we are only interested in objects with objectClass 
+# 'digestAuthCredentials', objects of all other types are ignored.
+filter = "(objectClass=digestAuthCredentials)"
+
+# Mapping of field names to LDAP attribute names and vice versa. Names are
+# delimited using ':', the first name is database field name as used in SER
+# modules, the second name (after :) is corresponding LDAP attribute name,
+# optionally preceeded with LDAP attribute syntax name in parentheses.
+field_map = password : (Binary) digestPassword
+field_map = realm : digestRealm
+field_map = auth_username : digestUsername
+field_map = uid : serUID
+field_map = flags : (BitString) serFlags
+
+# retrieve at most sizelimit entries for a search
+#sizelimit = 2147483647
+
+# wait at most timelimit seconds for a search to complete
+#timelimit = 120
+
+# chase references automatically by OpenLDAP. Default is "never"
+# chase_references = never | searching | finding | always
+
+# chase referrals automatically by OpenLDAP. Default is "no"
+# chase_referrals = yes | no
+
+#
+# Domain table stores information about virtual domains
+#
+[table:domain]
+
+# Objects mapping domain IDs to domain names and vice versa are stored
+# in the subtree with the following root:
+base = "ou=Domains,dc=iptel,dc=org"
+
+scope = subtree
+
+# We are only interested in serDomain objects when looking up information
+# about virtual domains.
+filter = "(objectClass=serDomain)"
+
+field_map = did : (String) serDID
+field_map = domain : (String) serDomain
+field_map = flags : (BitString) serFlags
+
+#
+# Table domain_attrs contains domain attributes, domain attributes store
+# extra information about virtual domains.
+#
+[table:domain_attrs]
+base = "ou=Domains, dc=iptel,dc=org"
+scope = subtree
+
+filter = "(objectClass=serDomainAttr)"
+
+field_map = did : serDID
+field_map = name : serAttrName
+field_map = type : (Integer) serAttrType
+field_map = value : serAttrValue
+field_map = flags : (BitString) serFlags			
+				</programlisting>
+			</example>
+			
+		</section>
+
+		<section id="reconnect_attempt">
+			<title><varname>reconnect_attempt</varname> (integer)</title>
+			<para>
+			Default value is <emphasis>3</emphasis>.
+			</para>
+			<para>
+			Number of reconnect attempts when connection to the LDAP server is lost.
+			</para>
+
+			<example>
+				<title>Example <varname>reconnect_attempt</varname></title>
+				<programlisting>
+	...
+	modparam("ldap", "reconnect_attempt", "5");
+	...
+				</programlisting>
+			</example>
+		</section>
+
+	</section>
+
+	<section id="ldap.functions">
+		<title>Functions</title>
+		<para>
+		none
+		</para>
+	</section>
+
+</section>
+

modules_s/ldap/ld_cfg.c

@@ -249,6 +249,14 @@ static cfg_option_t scope_values[] = {
 };
 
 
+static cfg_option_t deref_values[] = {
+	{"never",     .val = LDAP_DEREF_NEVER },   /* default, 0x00 */
+	{"searching", .val = LDAP_DEREF_SEARCHING},
+	{"finding",   .val = LDAP_DEREF_FINDING },
+	{"always",    .val = LDAP_DEREF_ALWAYS },
+	{0}
+};
+
 static cfg_option_t ldap_tab_options[] = {
 	{"scope",     .param = scope_values, .f = cfg_parse_enum_opt},
 	{"field_map", .f = parse_field_map},
@@ -256,6 +264,8 @@ static cfg_option_t ldap_tab_options[] = {
 	{"base",      .f = cfg_parse_str_opt, .flags = CFG_STR_PKGMEM},
 	{"timelimit", .f = cfg_parse_int_opt},
 	{"sizelimit", .f = cfg_parse_int_opt},
+	{"chase_references",  .param = deref_values, .f = cfg_parse_enum_opt},
+	{"chase_referrals",   .f = cfg_parse_bool_opt},
 	{0}
 };
 
@@ -275,9 +285,9 @@ static cfg_option_t ldap_con_options[] = {
 	{"username", 		    .f = cfg_parse_str_opt, .flags = CFG_STR_PKGMEM},
 	{"password", 		    .f = cfg_parse_str_opt, .flags = CFG_STR_PKGMEM},
 	{"authtype", 		    .param = auth_values, .f = cfg_parse_enum_opt},
-	{"tls",			        .f = cfg_parse_bool_opt},
+	{"tls",			    .f = cfg_parse_bool_opt},
 	{"ca_list",  		    .f = cfg_parse_str_opt, .flags = CFG_STR_PKGMEM},
-	{"require_certificate", .f = cfg_parse_str_opt, .flags = CFG_STR_PKGMEM},
+	{"require_certificate",     .f = cfg_parse_str_opt, .flags = CFG_STR_PKGMEM},
 	{0}
 };
 
@@ -332,6 +342,10 @@ static int parse_section(void* param, cfg_parser_t* st, unsigned int flags)
 		}
 		ldap_tab_options[4].param = &cfg->timelimit;
 		ldap_tab_options[5].param = &cfg->sizelimit;
+		for(i = 0; deref_values[i].name; i++) {
+			deref_values[i].param = &cfg->chase_references;
+		}
+		ldap_tab_options[7].param = &cfg->chase_referrals;
 	} else if (type == LDAP_CON_SECTION) {
 		if ((cinfo = pkg_malloc(sizeof(*cinfo))) == NULL) {
 			ERR("ldap:%s:%d: Out of memory\n", st->file, st->line);

modules_s/ldap/ld_cfg.h

@@ -45,6 +45,8 @@ struct ld_cfg {
 	int n;          /**< Number of fields in the arrays */
 	int sizelimit; /**< retrieve at most sizelimit entries for a search */
 	int timelimit; /**< wait at most timelimit seconds for a search to complete */
+	int chase_references;  /**< dereference option for LDAP library */
+	int chase_referrals;   /**< follow referrals option for LDAP library */
 	struct ld_cfg* next; /**< The next table in the list */
 };
 

modules_s/ldap/ld_cmd.c

@@ -90,6 +90,8 @@ int ld_cmd(db_cmd_t* cmd)
 {
 	struct ld_cmd* lcmd;
 	struct ld_cfg* cfg;
+	struct ld_fld* lfld;
+	int i, j;
 
 	lcmd = (struct ld_cmd*)pkg_malloc(sizeof(struct ld_cmd));
 	if (lcmd == NULL) {
@@ -134,11 +136,37 @@ int ld_cmd(db_cmd_t* cmd)
 	if (cfg->filter.s) {
 		lcmd->filter = cfg->filter;
 	}
+	lcmd->chase_references = cfg->chase_references;
+	lcmd->chase_referrals = cfg->chase_referrals;
 
 	if (ld_resolve_fld(cmd->match, cfg) < 0) goto error;
 	if (ld_resolve_fld(cmd->result, cfg) < 0) goto error;
 
+	/* prepare filter for each result field */
+	for(i = 0; !DB_FLD_EMPTY(cmd->result) && !DB_FLD_LAST(cmd->result[i]); i++) {
+		int n;
+		lfld = DB_GET_PAYLOAD(cmd->result + i);
+		lfld->filter = NULL;
+	
+		for(j = 0, n = 0; !DB_FLD_EMPTY(cmd->match) && !DB_FLD_LAST(cmd->match[j]); j++) {
+			if (strcmp(cmd->result[i].name, cmd->match[j].name) == 0)
+				n++;	
+		}
+		
+		if (n > 0) {
+			lfld->filter = pkg_malloc((n+1)*sizeof(*(lfld->filter)));
+			if (!lfld->filter) return -1 /* E_OUT_OF_MEM*/;
+			for(j = 0, n = 0; !DB_FLD_EMPTY(cmd->match) && !DB_FLD_LAST(cmd->match[j]); j++) {
+				if (strcmp(cmd->result[i].name, cmd->match[j].name) == 0) {
+					lfld->filter[n] = cmd->match+j;
+					n++;
+				}
+			}
+			lfld->filter[n] = NULL;
+		}
+	}
 	if (build_result_array(&lcmd->result, cmd) < 0) goto error;
+
 	DB_SET_PAYLOAD(cmd, lcmd);
 	return 0;
 
@@ -162,13 +190,12 @@ int ld_cmd_exec(db_res_t* res, db_cmd_t* cmd)
 	char* filter, *err_desc;
 	int ret, err;
 	LDAPMessage *msg, *resmsg;
-	int reconn_cnt = glb_reconn_cnt;
+	int reconn_cnt;
 	int msgid;
 	char *oid;
 	struct berval *data;
 	struct timeval restimeout;
 
-
 	filter = NULL;
 	err_desc = NULL;
 	resmsg = NULL;
@@ -179,14 +206,21 @@ int ld_cmd_exec(db_res_t* res, db_cmd_t* cmd)
 	con = cmd->ctx->con[db_payload_idx];
 	lcmd = DB_GET_PAYLOAD(cmd);
 	lcon = DB_GET_PAYLOAD(con);
+	
+	reconn_cnt = ld_reconnect_attempt;
 
-	if (ld_fld2ldap(&filter, cmd->match, &lcmd->filter) < 0) {
+	if (ld_prepare_ldap_filter(&filter, cmd, &lcmd->filter) < 0) {
 		ERR("ldap: Error while building LDAP search filter\n");
 		goto error;
 	}
 
+	DBG("ldap: ldap_search(base:'%s', filter:'%s')\n", lcmd->base, filter);
 	do {
 		if (lcon->flags & LD_CONNECTED) {
+			ldap_set_option(lcon->con, LDAP_OPT_DEREF, ((void *)&lcmd->chase_references));
+			/* there is alternative method using LDAP_CONTROL_REFERRALS per request but is not well documented */
+			ldap_set_option(lcon->con, LDAP_OPT_REFERRALS, lcmd->chase_referrals?LDAP_OPT_ON:LDAP_OPT_OFF);
+		
 			ret = ldap_search_ext(lcon->con, lcmd->base, lcmd->scope, filter,
 								  lcmd->result, 0, NULL, NULL,
 								  lcmd->timelimit.tv_sec ? &lcmd->timelimit : NULL,
@@ -232,7 +266,6 @@ int ld_cmd_exec(db_res_t* res, db_cmd_t* cmd)
 		}
 	} while (ret <= 0);
 
-
 	/* looking for unsolicited messages */
 	for (msg = ldap_first_message(lcon->con, resmsg);
 		 msg != NULL;
@@ -302,7 +335,7 @@ static int search_entry(db_res_t* res, int init)
 	db_con_t* con;
 	struct ld_res* lres;
 	struct ld_con* lcon;
-
+	int r;
 	lres = DB_GET_PAYLOAD(res);
 	/* FIXME */
 	con = res->cmd->ctx->con[db_payload_idx];
@@ -314,19 +347,24 @@ static int search_entry(db_res_t* res, int init)
 	    /* there is no more value combination result left */
 	    || ld_incindex(res->cmd->result)) {
 
-		if (init)
-			lres->current = ldap_first_message(lcon->con, lres->msg);
-		else
-			lres->current = ldap_next_message(lcon->con, lres->current);
-
-		while(lres->current) {
-			if (ldap_msgtype(lres->current) == LDAP_RES_SEARCH_ENTRY) {
-				break;
+		do {
+			if (init) {
+				lres->current = ldap_first_message(lcon->con, lres->msg);
+				init = 0;
 			}
-			lres->current = ldap_next_message(lcon->con, lres->current);
-		}
-		if (lres->current == NULL) return 1;
-		if (ld_ldap2fldinit(res->cmd->result, lcon->con, lres->current) < 0) return -1;
+			else
+				lres->current = ldap_next_message(lcon->con, lres->current);
+			
+			while(lres->current) {
+				if (ldap_msgtype(lres->current) == LDAP_RES_SEARCH_ENTRY) {
+					break;
+				}
+				lres->current = ldap_next_message(lcon->con, lres->current);
+			}
+			if (lres->current == NULL) return 1;
+			r = ld_ldap2fldinit(res->cmd->result, lcon->con, lres->current);
+		} while (r > 0);
+		if (r < 0) return -1;
 	} else {
 		if (ld_ldap2fld(res->cmd->result, lcon->con, lres->current) < 0) return -1;
 	}
@@ -347,5 +385,35 @@ int ld_cmd_next(db_res_t* res)
 	return search_entry(res, 0);
 }
 
+#define is_space(c) ((c)==' '||(c)==','||(c)==';'||(c)=='\t'||(c)=='\n'||(c)=='\r'||(c)=='\0')
+
+int ld_cmd_setopt(db_cmd_t* cmd, char* optname, va_list ap)
+{
+	struct ld_fld* lfld;
+	char* val, *c;
+	int i;
+		
+	if (!strcasecmp("client_side_filtering", optname)) {
+		val = va_arg(ap, char*);
+
+		for(i = 0; !DB_FLD_EMPTY(cmd->result) && !DB_FLD_LAST(cmd->result[i]); i++) {
+			c = val;
+			do {
+				c = strstr(c, cmd->result[i].name);
+				if (c) {
+					if ((c == val || is_space(*(c-1))) && is_space(*(c+strlen(cmd->result[i].name)))) {
+						lfld = (struct ld_fld*)DB_GET_PAYLOAD(cmd->result + i);
+						lfld->client_side_filtering = 1;
+						break;
+					}
+					c += strlen(cmd->result[i].name);
+				}
+			} while (c != NULL);
+		}
+	}
+	else
+		return 1;
+        return 0;
+}
 
 /** @} */

modules_s/ldap/ld_cmd.h

@@ -55,6 +55,8 @@ struct ld_cmd {
 	char** result; /**< An array with result attribute names for ldap_search */
 	int sizelimit; /**< retrieve at most sizelimit entries for a search */
 	struct timeval timelimit; /**< wait at most timelimit seconds for a search to complete */
+	int chase_references;  /**< dereference option for LDAP library */
+	int chase_referrals;   /**< follow referrals option for LDAP library */
 };
 
 
@@ -86,6 +88,8 @@ int ld_cmd_first(db_res_t* res);
 
 int ld_cmd_next(db_res_t* res);
 
+int ld_cmd_setopt(db_cmd_t* cmd, char* optname, va_list ap);
+
 /** @} */
 
 #endif /* _LD_CMD_H */

modules_s/ldap/ld_con.c

@@ -30,8 +30,6 @@
  * Functions related to connections to LDAP servers.
  */
 
-#define LDAP_DEPRECATED 1
-
 #include "ld_con.h"
 #include "ld_uri.h"
 

modules_s/ldap/ld_fld.h

@@ -40,6 +40,7 @@ struct ld_cfg;
 
 #include "../../db/db_gen.h"
 #include "../../db/db_fld.h"
+#include "../../db/db_cmd.h"
 
 #include <ldap.h>
 
@@ -57,9 +58,11 @@ struct ld_fld {
 	db_drv_t gen;
 	str attr;               /**< Name of corresponding LDAP attribute */
 	enum ld_syntax syntax;  /**< LDAP attribute syntax */
-	struct berval** values; /**< Values retrieved from the LDAP result */
+	struct berval** values; /**< Values retrieved from the LDAP result */    	
 	unsigned int valuesnum;
 	unsigned int index;
+	db_fld_t** filter;	/**< filter applied on the field pointing to db_cmd_t.match[] */
+	int client_side_filtering;  /**< do not pass filter to LDAP server but filter result set */
 };
 
 
@@ -76,9 +79,7 @@ int ld_fld(db_fld_t* fld, char* table);
 
 int ld_resolve_fld(db_fld_t* fld, struct ld_cfg* cfg);
 
-int ld_ldap2fld(db_fld_t* fld, LDAP* ldap, LDAPMessage* msg);
-
-int ld_fld2ldap(char** filter, db_fld_t* fld, str* add);
+int ld_prepare_ldap_filter(char** filter, db_cmd_t* cmd, str* add);
 
 int ld_incindex(db_fld_t* fld);
 

modules_s/ldap/ld_mod.c

@@ -49,7 +49,7 @@
 #include <ldap.h>
 
 str ld_cfg_file = STR_STATIC_INIT("ldap.cfg");
-int glb_reconn_cnt = 3;
+int ld_reconnect_attempt = 3;
 
 static int ld_mod_init(void);
 static void ld_mod_destroy(void);
@@ -73,7 +73,7 @@ static cmd_export_t cmds[] = {
 	{"db_fld",    (cmd_function)ld_fld, 0, 0, 0},
 	{"db_first",  (cmd_function)ld_cmd_first, 0, 0, 0},
 	{"db_next",   (cmd_function)ld_cmd_next, 0, 0, 0},
-	{"db_setopt", (cmd_function)NULL, 0, 0, 0},
+	{"db_setopt", (cmd_function)ld_cmd_setopt, 0, 0, 0},
 	{"db_getopt", (cmd_function)NULL, 0, 0, 0},
 	{0, 0, 0, 0, 0}
 };
@@ -84,7 +84,7 @@ static cmd_export_t cmds[] = {
  */
 static param_export_t params[] = {
 	{"config", PARAM_STR, &ld_cfg_file},
-	{"reconnect_attempt", PARAM_INT, &glb_reconn_cnt},
+	{"reconnect_attempt", PARAM_INT, &ld_reconnect_attempt},
 	{0, 0, 0}
 };
 

modules_s/ldap/ld_mod.h

@@ -30,12 +30,11 @@
  */
 /** @{ */
 
-extern int glb_reconn_cnt;
-
 /** \file
  * LDAP module interface.
  */
 
+extern int ld_reconnect_attempt;
 /** @} */
 
 #endif /* _LD_MOD_H */

modules_s/ldap/ldap.cfg

@@ -69,6 +69,12 @@ field_map = flags : (BitString) serFlags
 # wait at most timelimit seconds for a search to complete
 #timelimit = 120
 
+# chase references automatically by OpenLDAP. Default is "never"
+# chase_references = never | searching | finding | always
+
+# chase referrals automatically by OpenLDAP. Default is "no"
+# chase_referrals = yes | no
+
 #
 # Domain table stores information about virtual domains
 #

modules_s/ldap/standards.txt

@@ -30,6 +30,9 @@ RFC2798: Definition of the inetOrgPerson LDAP Object Class
 
   This document has been updated by RFC 3698, RFC 4519 and RFC 4524.
 
+RFC4514: Lightweight Directory Access Protocol (LDAP): String Representation
+        of Distinguished Names
+
 RFC4515: Lightweight Directory Access Protocol (LDAP): String Representation
          of Search Filters
 

modules_s/ldap/todo.txt

@@ -66,11 +66,11 @@ X Escape Values of Parameters Injected Into the Search Filter
   internally as time_t, but the parses should be able to skip fractions of
   seconds gracefully.
 
-* Eliminate Use of All libldap Functions Marked as Deprecated (SER-401)
+X Eliminate Use of All libldap Functions Marked as Deprecated (SER-401)
 
   We should eliminate the use of all functions that are marked as deprecated
   in the header field and remove LDAP_DEPRECATED define from all files.
-
+  
 X Support for Bool Attribute Types
 
   The module should be able to convert boolean attributes from the LDAP result
@@ -83,11 +83,11 @@ X Support for Syntax Specification in the Config File
   attributes in the configuration file so that the module knows how to convert
   DB API fields into attribute values in command parameters.
 
-* Support DB_NEQ Operator in Integer Attributes (SER-402)
+X Support DB_NEQ Operator in Integer Attributes (SER-402)
  
   fld!=10 could be converted to the search filter as: (|(fld<=9)(fld>=11))
 
-* Indicate Underflow/Overflow in Integer Attributes (SER-403)
+X Indicate Underflow/Overflow in Integer Attributes (SER-403)
 
   The operators < and > cannot be used in LDAP search filter, RFC4515 only
   supports <= and >= so we create the search filter like this: fld<10 ->