From dazaba07 at gmail.com Tue Jul 15 00:57:20 2014 From: dazaba07 at gmail.com (David Zamora) Date: Mon, 14 Jul 2014 16:57:20 -0600 Subject: [python-ldap] Problem with sync_ldap_groups_to_svn_authz.py Message-ID: Hi all, Basically I want to run a POC using this tool. I am working on a Windows Server 2008. I just downloaded Python 2.7.8 version and Python LDAP-2.4.15 I opened the GUI console and tried to execute the script sync_ldap_groups_to_svn_authz.py. The problem that I have is: This .py file is not located at any Lib folder. Actually I was looking for in the entire hard drive. I am wondering if the newest version change the name of the file, I want to execute soething like this: python sync_ldap_groups_to_svn_authz.py -d "CN=XX,OU=Generic-Account,OU=Resources,DC=AAA,DC=corp,DC=BBB,DC=com" -l " soething.corp.com" -b "DC=corp,DC=CCC,DC=com" -i "sAMAccountName" > svn_auth.txt Thanks in advance for your help -- David Zamora -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at stroeder.com Tue Jul 15 10:02:16 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Tue, 15 Jul 2014 10:02:16 +0200 Subject: [python-ldap] Problem with sync_ldap_groups_to_svn_authz.py In-Reply-To: References: Message-ID: <53C4E008.7080502@stroeder.com> David Zamora wrote: > Basically I want to run a POC using this tool. I am working on a Windows > Server 2008. I've never used sync_ldap_groups_to_svn_authz.py myself. > I just downloaded Python 2.7.8 version and Python LDAP-2.4.15 And you run the installer I presume. > I opened the GUI console and tried to execute the script > sync_ldap_groups_to_svn_authz.py. On-topic on this list: At first you can simple type import ldap in the Python interpreter to check whether python-ldap was installed correctly. > The problem that I have is: This .py file is not located at any Lib folder. > Actually I was looking for in the entire hard drive. > > I am wondering if the newest version change the name of the file, I want > to execute soething like this: > > python sync_ldap_groups_to_svn_authz.py -d > "CN=XX,OU=Generic-Account,OU=Resources,DC=AAA,DC=corp,DC=BBB,DC=com" -l " > soething.corp.com" -b "DC=corp,DC=CCC,DC=com" -i "sAMAccountName" > > svn_auth.txt Rather off-topic on this list: Under Windows you should type the full path of the Python interpreter followed by the full path of the script. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Fri Jul 25 12:40:16 2014 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 25 Jul 2014 12:40:16 +0200 Subject: [python-ldap] [PATCH] New hook syncrepl_refreshdone() in ldap.syncrepl.SyncReplConsumer Message-ID: <53D23410.4070407@redhat.com> Hello, attached patches improve SyncReplConsumer a little bit: 0001-Fix-indentation-in-Demo-pyasn1-syncrepl.py.patch fixes syncrepl demo so it works again. 0002-Add-syncrepl_refreshdone-method-to-SyncReplConsumer.patch adds new hook syncrepl_refreshdone() to ldap.syncrepl.SyncReplConsumer. See docstring for details. The same patches are also available from https://github.com/spacekpe/python-ldap/commits/syncrepl_refreshdone Have a nice day! -- Petr Spacek @ Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Fix-indentation-in-Demo-pyasn1-syncrepl.py.patch Type: text/x-patch Size: 858 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: 0002-Add-syncrepl_refreshdone-method-to-SyncReplConsumer.patch Type: text/x-patch Size: 3572 bytes Desc: not available URL: From pspacek at redhat.com Mon Aug 11 15:10:46 2014 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 11 Aug 2014 15:10:46 +0200 Subject: [python-ldap] [PATCH] New hook syncrepl_refreshdone() in ldap.syncrepl.SyncReplConsumer In-Reply-To: <53D23410.4070407@redhat.com> References: <53D23410.4070407@redhat.com> Message-ID: <53E8C0D6.5040602@redhat.com> Hello, I can see in CVS that you have merged the indentation fix but not the syncrepl_refreshdone() hook. Should I improve the code? I will be glad to rework it as necessary to make is acceptable. Just tell me :-) Thank you for your time! Petr Spacek @ Red Hat On 25.7.2014 12:40, Petr Spacek wrote: > Hello, > > attached patches improve SyncReplConsumer a little bit: > > 0001-Fix-indentation-in-Demo-pyasn1-syncrepl.py.patch fixes syncrepl demo so > it works again. > > 0002-Add-syncrepl_refreshdone-method-to-SyncReplConsumer.patch adds new hook > syncrepl_refreshdone() to ldap.syncrepl.SyncReplConsumer. See docstring for > details. > > The same patches are also available from > https://github.com/spacekpe/python-ldap/commits/syncrepl_refreshdone > > Have a nice day! From JohnTheodore42 at gmail.com Mon Aug 18 18:44:10 2014 From: JohnTheodore42 at gmail.com (John Theodore) Date: Mon, 18 Aug 2014 09:44:10 -0700 Subject: [python-ldap] Read/Change public ssh key with ldap-python Message-ID: I'm trying to make a simple web app that can manage/change my public ssh keys on ldap using python-ldap and django. I'm trying to understand the ldap library better. I know I can query the ldap box and get a list of attributes (including public ssh keys) by doing this command: l=ldap.initialize("ldap://myldaphost.mydomain.com") l.simple_bind_s("CN=ldapuser,CN=Users,DC=mydomain, DC=com","secret") r = l.search_s( "CN=ANYCOMPUTER,CN=Computers,DC=mydomain,DC=co m", ldap.SCOPE_SUBTREE, # this is the default of ldapsearch "(objectClass=*)" ) . What I'm trying to figure out, is how can I delete or update an ssh key for a specific account? The command I would be doing on ldapmodify is this: ldapmodify -h localhost -D "uid=user,ou=People,dc=mgmt,dc=symcpe,dc=net" -w $passwrd -f /tmp/usertomod.ldif. How can I duplicate this command in python-ldap? -------------- next part -------------- An HTML attachment was scrubbed... URL: From raphael.barrois at m4x.org Mon Aug 18 20:15:15 2014 From: raphael.barrois at m4x.org (=?UTF-8?B?UmFwaGHDq2w=?= Barrois) Date: Mon, 18 Aug 2014 20:15:15 +0200 Subject: [python-ldap] Read/Change public ssh key with ldap-python In-Reply-To: References: Message-ID: <20140818201515.77b6e3f0@ithor> On Mon, 18 Aug 2014 09:44:10 -0700 John Theodore wrote: > I'm trying to make a simple web app that can manage/change my public > ssh keys on ldap using python-ldap and django. I'm trying to > understand the ldap library better. I know I can query the ldap box > and get a list of attributes (including public ssh keys) by doing > this command: > > l=ldap.initialize("ldap://myldaphost.mydomain.com") > l.simple_bind_s("CN=ldapuser,CN=Users,DC=mydomain, DC=com","secret") > r = l.search_s( > "CN=ANYCOMPUTER,CN=Computers,DC=mydomain,DC=co m", > ldap.SCOPE_SUBTREE, # this is the default of ldapsearch > "(objectClass=*)" > ) > > . > What I'm trying to figure out, is how can I delete or update an ssh > key for a specific account? The command I would be doing on > ldapmodify is this: > > ldapmodify -h localhost -D > "uid=user,ou=People,dc=mgmt,dc=symcpe,dc=net" -w $passwrd > -f /tmp/usertomod.ldif. > > How can I duplicate this command in python-ldap? Hi John, This would be done with modify() and delete(): http://www.python-ldap.org/doc/html/ldap.html#ldap.LDAPObject.modify http://www.python-ldap.org/doc/html/ldap.html#ldap.LDAPObject.delete If you're using Django, you might also be interested in django-ldapdb (https://github.com/jlaine/django-ldapdb), which provides Django-like models backed on a LDAP database, i.e with a higher level API. -- Rapha?l From mucahit at konya.edu.tr Sat Aug 23 02:12:06 2014 From: mucahit at konya.edu.tr (Mucahit BUYUKYILMAZ) Date: Sat, 23 Aug 2014 00:12:06 -0000 Subject: [python-ldap] ppolicy response message In-Reply-To: <1862544139.1126810.1408751403482.JavaMail.zimbra@konya.edu.tr> Message-ID: <83114109.1126819.1408751708410.JavaMail.zimbra@konya.edu.tr> Hello everybody, I am building sso with openldap for our university. I am using ppolicy. Everything is good but when i try to bind a locked user with python-ldap it gave me 49 INVALID_CREDENTIALS: {'desc': 'Invalid credentials'}. ldapwhoami gives "ldap_bind: Invalid credentials (49); Account locked" with "-e ppolicy" I am searching 2 days how to do that but couldnt find yet. I checked this one https://mail.python.org/pipermail/python-ldap/2012q4/003156.html is it implemented? or how can I handle this issue BR, M?cahit From JohnTheodore42 at gmail.com Mon Aug 25 20:32:08 2014 From: JohnTheodore42 at gmail.com (John Theodore) Date: Mon, 25 Aug 2014 11:32:08 -0700 Subject: [python-ldap] Read/Change public ssh key with ldap-python In-Reply-To: <20140818201515.77b6e3f0@ithor> References: <20140818201515.77b6e3f0@ithor> Message-ID: So I setup python-ldapdb and have a little api that is working. I can query and read from the ldap box using python-ldapdb and their models. My project is here: https://github.com/JohnTheodore/django-ldapdb The problem is I can't modify the ldap object (say change the sshPublicKey attribute) and .save() it. When I try to .save() I'm getting: NO_SUCH_OBJECT: {'matched': 'uid=john_theodore,ou=People,dc=mgmt,dc=mydomain,dc=net', 'desc': 'No such object'} My paste/gist which shows the log is here (I turned on debug mode): https://gist.github.com/JohnTheodore/6d317fe1b337bd402ee3 Any ideas why I can't modify an entry? basically I want to rewrite the sshkeys attribute and then save() it. On Mon, Aug 18, 2014 at 11:15 AM, Rapha?l Barrois wrote: > On Mon, 18 Aug 2014 09:44:10 -0700 > John Theodore wrote: > > > I'm trying to make a simple web app that can manage/change my public > > ssh keys on ldap using python-ldap and django. I'm trying to > > understand the ldap library better. I know I can query the ldap box > > and get a list of attributes (including public ssh keys) by doing > > this command: > > > > l=ldap.initialize("ldap://myldaphost.mydomain.com") > > l.simple_bind_s("CN=ldapuser,CN=Users,DC=mydomain, DC=com","secret") > > r = l.search_s( > > "CN=ANYCOMPUTER,CN=Computers,DC=mydomain,DC=co m", > > ldap.SCOPE_SUBTREE, # this is the default of ldapsearch > > "(objectClass=*)" > > ) > > > > . > > What I'm trying to figure out, is how can I delete or update an ssh > > key for a specific account? The command I would be doing on > > ldapmodify is this: > > > > ldapmodify -h localhost -D > > "uid=user,ou=People,dc=mgmt,dc=symcpe,dc=net" -w $passwrd > > -f /tmp/usertomod.ldif. > > > > How can I duplicate this command in python-ldap? > > Hi John, > > This would be done with modify() and delete(): > http://www.python-ldap.org/doc/html/ldap.html#ldap.LDAPObject.modify > http://www.python-ldap.org/doc/html/ldap.html#ldap.LDAPObject.delete > > If you're using Django, you might also be interested in django-ldapdb ( > https://github.com/jlaine/django-ldapdb), which provides Django-like > models backed on a LDAP database, i.e with a higher level API. > > > -- > Rapha?l > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pauli.sundberg at iki.fi Thu Aug 28 12:03:12 2014 From: pauli.sundberg at iki.fi (Pauli Sundberg) Date: Thu, 28 Aug 2014 13:03:12 +0300 Subject: [python-ldap] Setting option OPT_X_SASL_MECH gives ValueError "unknown option 24832" Message-ID: Hi all. Problem in short: I tried setting ldap.OPT_X_SASL_MECH with: > import ldap > ldap.set_option( ldap.OPT_X_SASL_MECH, "foo") and that gives me ValueError: "unknown option 24832". I tried this on Debian 7 (apt-get installed), with Ubuntu 14.04 with pip installed and apt-get installed python-ldap package. To be sure, i tried it also with > import ldap > l = ldap.initialize("ldap://localhost/") > l.set_option( ldap.OPT_X_SASL_MECH, "foo") and that yield same error. ldap.SASL_AVAIL gives me 1 -- so installation should be ok ? Any suggestions what am i doing wrong? I also tried using $HOME/.ldaprc (and checked that strace gives opened ok) for 'SASL_MECH' but to me it seems like its having no effect (and i get no error setting it to say 'foo') Background: I am trying to use LDAP for authenticating Django site. Without options the authentication goes plaintext, and that is not acceptable. So i am trying to use MD5 diggest, that gives some security. I am using the python-django-ldap [1] . I tried setting MD5 on with setting AUTH_LDAP_CONNECTION_OPTIONS described in [1] with "A dictionary of options to pass to each connection to the LDAP server via LDAPObject.set_option()". [1] (https://pythonhosted.org/django-auth-ldap/) Thanks, Pauli Sundberg -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark at proseconsulting.co.uk Tue Sep 2 00:25:22 2014 From: mark at proseconsulting.co.uk (Mark R Bannister) Date: Mon, 01 Sep 2014 23:25:22 +0100 Subject: [python-ldap] ldap.init_fd() patch for python-ldap-2.4.15 Message-ID: <5404F252.8030702@proseconsulting.co.uk> Hi, I needed ldap.init_fd(), so I wrote a patch. Please find attached. Hope this works for you too. I've only tested on OpenSuSE. This is an example of how to use it: ### import ldap import socket sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(("localhost", 389)) l = ldap.init_fd(sock.fileno(), ldap.PROTO_TCP, "ldap://localhost:389") ... ### I found I had to install openldap2-debugsource to get ldap_pvt.h. I added /usr/src/debug/openldap-2.4.33/include to include_dirs in setup.cfg to get it to compile. This header worked fine with openldap2-2.4.26. Best regards, Mark R. Bannister. Author of DBIS: http://technicalprose.blogspot.co.uk/2013/08/introducing-dbis.html -------------- next part -------------- diff -u -r a/python-ldap-2.4.15/Lib/ldap/functions.py b/python-ldap-2.4.15/Lib/ldap/functions.py --- a/python-ldap-2.4.15/Lib/ldap/functions.py 2011-11-25 12:22:02.000000000 +0000 +++ b/python-ldap-2.4.15/Lib/ldap/functions.py 2014-09-01 22:34:16.881369666 +0100 @@ -21,7 +21,7 @@ from ldap import __version__ __all__ = [ - 'open','initialize','init', + 'open','initialize','init_fd','init', 'explode_dn','explode_rdn', 'get_option','set_option', ] @@ -32,7 +32,7 @@ from ldap.dn import explode_dn,explode_rdn -from ldap.ldapobject import LDAPObject +from ldap.ldapobject import LDAPObject,LDAPObjectFromFD if __debug__: # Tracing is only supported in debugging mode @@ -91,6 +91,29 @@ return LDAPObject(uri,trace_level,trace_file,trace_stack_limit) +def init_fd(fd,proto,uri=None,trace_level=0,trace_file=sys.stdout,trace_stack_limit=None): + """ + Return LDAPObject instance by opening LDAP connection over + existing connection on the provided socket + + Parameters: + fd + File descriptor of an already-opened socket, e.g. socket.fileno() + proto + One of ldap.PROTO_TCP, ldap.PROTO_UDP or ldap.PROTO_IPC. + uri + LDAP URL containing at least connection scheme and hostport, + e.g. ldap://localhost:389 + This is optional and is provided for informational purposes. + trace_level + If non-zero a trace output of LDAP calls is generated. + trace_file + File object where to write the trace output to. + Default is to use stdout. + """ + return LDAPObjectFromFD(fd,proto,uri,trace_level,trace_file,trace_stack_limit) + + def open(host,port=389,trace_level=0,trace_file=sys.stdout,trace_stack_limit=None): """ Return LDAPObject instance by opening LDAP connection to diff -u -r a/python-ldap-2.4.15/Lib/ldap/__init__.py b/python-ldap-2.4.15/Lib/ldap/__init__.py --- a/python-ldap-2.4.15/Lib/ldap/__init__.py 2014-03-24 10:20:16.000000000 +0000 +++ b/python-ldap-2.4.15/Lib/ldap/__init__.py 2014-09-01 21:38:40.930387447 +0100 @@ -82,7 +82,7 @@ # Create module-wide lock for serializing all calls into underlying LDAP lib _ldap_module_lock = LDAPLock(desc='Module wide') -from functions import open,initialize,init,get_option,set_option +from functions import open,initialize,init_fd,init,get_option,set_option from ldap.dn import explode_dn,explode_rdn,str2dn,dn2str del str2dn diff -u -r a/python-ldap-2.4.15/Lib/ldap/ldapobject.py b/python-ldap-2.4.15/Lib/ldap/ldapobject.py --- a/python-ldap-2.4.15/Lib/ldap/ldapobject.py 2014-03-07 20:01:12.000000000 +0000 +++ b/python-ldap-2.4.15/Lib/ldap/ldapobject.py 2014-09-01 22:58:01.736461427 +0100 @@ -23,6 +23,8 @@ __all__ = [ 'LDAPObject', 'SimpleLDAPObject', + 'LDAPObjectFromFD', + 'SimpleLDAPObjectFromFD', 'NonblockingLDAPObject', 'ReconnectLDAPObject', ] @@ -879,6 +881,28 @@ return self._apply_method_s(SimpleLDAPObject.whoami_s,*args,**kwargs) +class SimpleLDAPObjectFromFD(SimpleLDAPObject): + """ + Identical to SimpleLDAPObject except that it uses ldap_init_fd() + """ + + def __init__( + self,fd,proto,uri=None, + trace_level=0,trace_file=None,trace_stack_limit=5 + ): + self._trace_level = trace_level + self._trace_file = trace_file or sys.stdout + self._trace_stack_limit = trace_stack_limit + self._fd = fd + self._proto = proto + self._uri = uri + self._ldap_object_lock = self._ldap_lock('opcall') + self._l = ldap.functions._ldap_function_call(ldap._ldap_module_lock,_ldap.init_fd,fd,proto,uri) + self.timeout = -1 + self.protocol_version = ldap.VERSION3 + + # The class called LDAPObject will be used as default for # ldap.open() and ldap.initialize() LDAPObject = SimpleLDAPObject +LDAPObjectFromFD = SimpleLDAPObjectFromFD diff -u -r a/python-ldap-2.4.15/Modules/constants.c b/python-ldap-2.4.15/Modules/constants.c --- a/python-ldap-2.4.15/Modules/constants.c 2014-03-24 10:20:16.000000000 +0000 +++ b/python-ldap-2.4.15/Modules/constants.c 2014-09-01 22:24:39.152943357 +0100 @@ -6,6 +6,7 @@ #include "constants.h" #include "lber.h" #include "ldap.h" +#include "ldap_pvt.h" static PyObject* reverse; static PyObject* forward; @@ -140,6 +141,10 @@ add_int(d,DEREF_ALWAYS); add_int(d,NO_LIMIT); + add_int(d,PROTO_TCP); + add_int(d,PROTO_UDP); + add_int(d,PROTO_IPC); + add_int(d,OPT_API_INFO); add_int(d,OPT_DEREF); add_int(d,OPT_SIZELIMIT); diff -u -r a/python-ldap-2.4.15/Modules/functions.c b/python-ldap-2.4.15/Modules/functions.c --- a/python-ldap-2.4.15/Modules/functions.c 2011-06-08 20:35:33.000000000 +0100 +++ b/python-ldap-2.4.15/Modules/functions.c 2014-09-01 22:43:23.434833804 +0100 @@ -7,6 +7,7 @@ #include "berval.h" #include "errors.h" #include "options.h" +#include "ldap_pvt.h" /* ldap_initialize */ @@ -29,6 +30,27 @@ } +/* ldap_init_fd */ + +static PyObject* +l_ldap_init_fd(PyObject* unused, PyObject *args) +{ + char *uri = NULL; + LDAP *ld = NULL; + int ret, fd, proto; + + if (!PyArg_ParseTuple(args, "ii|s", &fd, &proto, &uri)) + return NULL; + + Py_BEGIN_ALLOW_THREADS + ret = ldap_init_fd((ber_socket_t)fd, proto, uri, &ld); + Py_END_ALLOW_THREADS + if (ret != LDAP_SUCCESS) + return LDAPerror(ld, "ldap_init_fd"); + return (PyObject*)newLDAPObject(ld); +} + + /* ldap_str2dn */ static PyObject* @@ -137,6 +159,7 @@ static PyMethodDef methods[] = { { "initialize", (PyCFunction)l_ldap_initialize, METH_VARARGS }, + { "init_fd", (PyCFunction)l_ldap_init_fd, METH_VARARGS }, { "str2dn", (PyCFunction)l_ldap_str2dn, METH_VARARGS }, { "set_option", (PyCFunction)l_ldap_set_option, METH_VARARGS }, { "get_option", (PyCFunction)l_ldap_get_option, METH_VARARGS }, From hahn at univention.de Fri Sep 5 19:29:26 2014 From: hahn at univention.de (Philipp Hahn) Date: Fri, 05 Sep 2014 19:29:26 +0200 Subject: [python-ldap] [BUG, PATCH] Recursive locking problem when using TLS with reconnect Message-ID: <5409F2F6.2090908@univention.de> Hello, After updating python-ldap from 2.3.11 to 2.4.10 one of our tests testing the reconnect behavior in obscure cases starts failing: the test just hangs. The problem also exists with 2.4.13. I think this is related to the change in 2.4.9 "ldapobject.ReconnectLDAPObject.reconnect() now does kind of an internal locking to pause other threads while reconnecting is pending." Using "gdb -p $PID -ex 'thread apply all bt'" I was able to track down a recursive locking problem: > #0 sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:86 ... > #7 PyEval_EvalFrameEx (.../usr/lib/python2.7/dist-packages/ldap/ldapobject.py, line 772, in reconnect ... ... > #11 PyEval_EvalFrameEx (.../usr/lib/python2.7/dist-packages/ldap/ldapobject.py, line 823, in _apply_method_s ... ... > #16 PyEval_EvalFrameEx (.../usr/lib/python2.7/dist-packages/ldap/ldapobject.py, line 842, in start_tls_s ... ... > #20 PyEval_EvalFrameEx (.../usr/lib/python2.7/dist-packages/ldap/ldapobject.py, line 787, in reconnect ... ... > #24 PyEval_EvalFrameEx (.../usr/lib/python2.7/dist-packages/ldap/ldapobject.py, line 823, in _apply_method_s ... ... > #29 PyEval_EvalFrameEx (.../usr/lib/python2.7/dist-packages/ldap/ldapobject.py, line 876, in search_ext_s ... ... > #33 PyEval_EvalFrameEx (.../usr/lib/pymodules/python2.7/univention/uldap.py, line 339, in search ... ... > #37 PyEval_EvalFrameEx (.../usr/lib/pymodules/python2.7/univention/uldap.py, line 350, in searchDn ... ... As you can see its a TLS encrypted connection, which again fails and starts to re-connect, while it is already re-connecting. This then dead-locks when the self._reconnect_lock is acquired a 2nd time by the same thread. The following patches solves the hang for me: --- Lib/ldap/ldapobject.py.orig 2014-09-05 18:13:21.113628647 +0200 +++ Lib/ldap/ldapobject.py 2014-09-05 18:23:05.913107047 +0200 @@ -784,7 +784,7 @@ self._restore_options() # StartTLS extended operation in case this was called before if self._start_tls: - self.start_tls_s() + SimpleLDAPObject.start_tls_s(self) # Repeat last simple or SASL bind self._apply_last_bind() except (ldap.SERVER_DOWN,ldap.TIMEOUT),e: Sincerely Philipp Hahn PS: Is your CVS tree still the master or is there some official GIT/Subverion repository for easier cloning? PPS: -- Philipp Hahn Open Source Software Engineer Univention GmbH be open. Mary-Somerville-Str. 1 D-28359 Bremen Tel.: +49 421 22232-0 Fax : +49 421 22232-99 hahn at univention.de http://www.univention.de/ Gesch?ftsf?hrer: Peter H. Ganten HRB 20755 Amtsgericht Bremen Steuer-Nr.: 71-597-02876 From michael at stroeder.com Mon Sep 8 09:14:20 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Mon, 08 Sep 2014 09:14:20 +0200 Subject: [python-ldap] [BUG, PATCH] Recursive locking problem when using TLS with reconnect In-Reply-To: <5409F2F6.2090908@univention.de> References: <5409F2F6.2090908@univention.de> Message-ID: <540D574C.8040707@stroeder.com> Philipp Hahn wrote: > After updating python-ldap from 2.3.11 to 2.4.10 one of our tests > testing the reconnect behavior in obscure cases starts failing: the test > just hangs. > The problem also exists with 2.4.13. > > I think this is related to the change in 2.4.9 > "ldapobject.ReconnectLDAPObject.reconnect() now does kind of an internal > locking to pause other threads while reconnecting is pending." This was fixed in 2.4.14. Upgrading to 2.4.15 is a good idea. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From hahn at univention.de Mon Sep 8 10:38:42 2014 From: hahn at univention.de (Philipp Hahn) Date: Mon, 08 Sep 2014 10:38:42 +0200 Subject: [python-ldap] [BUG, PATCH] Recursive locking problem when using TLS with reconnect In-Reply-To: <540D574C.8040707@stroeder.com> References: <5409F2F6.2090908@univention.de> <540D574C.8040707@stroeder.com> Message-ID: <540D6B12.7040308@univention.de> Hello, On 08.09.2014 09:14, Michael Str?der wrote: > Philipp Hahn wrote: >> After updating python-ldap from 2.3.11 to 2.4.10 one of our tests >> testing the reconnect behavior in obscure cases starts failing: the test >> just hangs. >> The problem also exists with 2.4.13. >> >> I think this is related to the change in 2.4.9 >> "ldapobject.ReconnectLDAPObject.reconnect() now does kind of an internal >> locking to pause other threads while reconnecting is pending." > > This was fixed in 2.4.14. Upgrading to 2.4.15 is a good idea. For me it is NOT fixed in 2.4.15: curl -s 'https://pypi.python.org/packages/source/p/python-ldap/python-ldap-2.4.15.tar.gz#md5=f12183c87579631584c4bbe2d85ad0d9' | tar xfzO - python-ldap-2.4.15/Lib/ldap/ldapobject.py | grep -A3 -B3 -n -F '.start_tls_s(' 784- self._restore_options() 785- # StartTLS extended operation in case this was called before 786- if self._start_tls: 787: self.start_tls_s() 788- # Repeat last simple or SASL bind 789- self._apply_last_bind() 790- except (ldap.SERVER_DOWN,ldap.TIMEOUT),e: Philipp From michael at stroeder.com Mon Sep 8 14:33:16 2014 From: michael at stroeder.com (=?UTF-8?B?TWljaGFlbCBTdHLDtmRlcg==?=) Date: Mon, 08 Sep 2014 14:33:16 +0200 Subject: [python-ldap] ppolicy response message In-Reply-To: <83114109.1126819.1408751708410.JavaMail.zimbra@konya.edu.tr> References: <83114109.1126819.1408751708410.JavaMail.zimbra@konya.edu.tr> Message-ID: <540DA20C.5040401@stroeder.com> Mucahit BUYUKYILMAZ wrote: > I am building sso with openldap for our university. I am using ppolicy. Everything is good but when i try to bind a locked user with python-ldap it gave me 49 INVALID_CREDENTIALS: {'desc': 'Invalid credentials'}. > > ldapwhoami gives "ldap_bind: Invalid credentials (49); Account locked" with "-e ppolicy" > > I am searching 2 days how to do that but couldnt find yet. I checked this one https://mail.python.org/pipermail/python-ldap/2012q4/003156.html is it implemented? or how can I handle this issue Sorry, one of the deficiencies of python-ldap is that response control values are not extracted from the LDAP response in case of errors. See also: https://mail.python.org/pipermail/python-ldap/2012q4/003159.html This is an important point in TODO. So if anyone with enough C development skills has time to develop a patch I'd be happy to merge that. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Mon Sep 8 15:22:54 2014 From: pspacek at redhat.com (Petr Spacek) Date: Mon, 08 Sep 2014 15:22:54 +0200 Subject: [python-ldap] [PATCH] New hook syncrepl_refreshdone() in ldap.syncrepl.SyncReplConsumer In-Reply-To: <53E8C0D6.5040602@redhat.com> References: <53D23410.4070407@redhat.com> <53E8C0D6.5040602@redhat.com> Message-ID: <540DADAE.4050507@redhat.com> Hello, could I help somehow with the proposed patch? I would be happy to rework it as necessary. Thank you! Petr Spacek @ Red Hat On 11.8.2014 15:10, Petr Spacek wrote: > Hello, > > I can see in CVS that you have merged the indentation fix but not the > syncrepl_refreshdone() hook. > > Should I improve the code? I will be glad to rework it as necessary to make is > acceptable. Just tell me :-) > > Thank you for your time! > > Petr Spacek @ Red Hat > > On 25.7.2014 12:40, Petr Spacek wrote: >> Hello, >> >> attached patches improve SyncReplConsumer a little bit: >> >> 0001-Fix-indentation-in-Demo-pyasn1-syncrepl.py.patch fixes syncrepl demo so >> it works again. >> >> 0002-Add-syncrepl_refreshdone-method-to-SyncReplConsumer.patch adds new hook >> syncrepl_refreshdone() to ldap.syncrepl.SyncReplConsumer. See docstring for >> details. >> >> The same patches are also available from >> https://github.com/spacekpe/python-ldap/commits/syncrepl_refreshdone >> >> Have a nice day! From michael at stroeder.com Mon Sep 8 21:49:12 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Mon, 08 Sep 2014 21:49:12 +0200 Subject: [python-ldap] [BUG, PATCH] Recursive locking problem when using TLS with reconnect In-Reply-To: <540D6B12.7040308@univention.de> References: <5409F2F6.2090908@univention.de> <540D574C.8040707@stroeder.com> <540D6B12.7040308@univention.de> Message-ID: <540E0838.2050303@stroeder.com> Philipp Hahn wrote: > On 08.09.2014 09:14, Michael Str?der wrote: >> >> This was fixed in 2.4.14. Upgrading to 2.4.15 is a good idea. > > For me it is NOT fixed in 2.4.15: Sorry, overlooked the real issue. Committed fix to CVS. Please test and confirm. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From hahn at univention.de Tue Sep 9 11:26:19 2014 From: hahn at univention.de (Philipp Hahn) Date: Tue, 09 Sep 2014 11:26:19 +0200 Subject: [python-ldap] [BUG, PATCH] Recursive locking problem when using TLS with reconnect In-Reply-To: <540E0838.2050303@stroeder.com> References: <5409F2F6.2090908@univention.de> <540D574C.8040707@stroeder.com> <540D6B12.7040308@univention.de> <540E0838.2050303@stroeder.com> Message-ID: <540EC7BB.9020309@univention.de> Hello, On 08.09.2014 21:49, Michael Str?der wrote: > Sorry, overlooked the real issue. > > Committed fix to CVS. Please test and confirm. looks wrong: "self" is now missing as the first and only parameter: --- python-ldap/Lib/ldap/ldapobject.py 2014/07/25 17:08:56 +++ python-ldap/Lib/ldap/ldapobject.py 2014/09/08 19:48:11 @@ -826,7 +826,7 @@ self._restore_options() # StartTLS extended operation in case this was called before if self._start_tls: - SimpleLDAPObject.start_tls_s() + SimpleLDAPObject.start_tls_s(self) # Repeat last simple or SASL bind self._apply_last_bind() except (ldap.SERVER_DOWN,ldap.TIMEOUT),e: Sincerely Philipp From michael at stroeder.com Tue Sep 9 12:15:04 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Tue, 09 Sep 2014 12:15:04 +0200 Subject: [python-ldap] [BUG, PATCH] Recursive locking problem when using TLS with reconnect In-Reply-To: <540EC7BB.9020309@univention.de> References: <5409F2F6.2090908@univention.de> <540D574C.8040707@stroeder.com> <540D6B12.7040308@univention.de> <540E0838.2050303@stroeder.com> <540EC7BB.9020309@univention.de> Message-ID: <540ED328.5040505@stroeder.com> Philipp Hahn wrote: > Hello, > > On 08.09.2014 21:49, Michael Str?der wrote: >> Sorry, overlooked the real issue. >> >> Committed fix to CVS. Please test and confirm. > > > looks wrong: > > "self" is now missing as the first and only parameter: Hmmpf! Thanks. Committed. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Wed Sep 10 14:20:33 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Wed, 10 Sep 2014 14:20:33 +0200 Subject: [python-ldap] ANN: python-ldap 2.4.16 Message-ID: <54104211.6060400@stroeder.com> Find a new release of python-ldap: http://pypi.python.org/pypi/python-ldap/2.4.16 python-ldap provides an object-oriented API to access LDAP directory servers from Python programs. It mainly wraps the OpenLDAP 2.x libs for that purpose. Additionally it contains modules for other LDAP-related stuff (e.g. processing LDIF, LDAP URLs and LDAPv3 schema). Project's web site: http://www.python-ldap.org/ Ciao, Michael. ---------------------------------------------------------------- Released 2.4.16 2014-09-10 Changes since 2.4.15: Lib/ * New convenience function ldap.dn.is_dn() * New convenience function ldap.escape_str() * New convenience methods LDAPObject.read_s() and LDAPObject.find_unique_entry() * Fixed invoking start_tls_s() in ReconnectLDAPObject.reconnect() (thanks to Philipp Hahn) From martin.basti at gmail.com Wed Sep 10 14:24:29 2014 From: martin.basti at gmail.com (Martin Basti) Date: Wed, 10 Sep 2014 14:24:29 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed Message-ID: Hello, I need to detect if syncrepl refresh phase is completed during syncrepl_poll (in refresh&persist mode), is there any way how to do it? I found a patch https://github.com/spacekpe/python-ldap/commit/8b4f935a97759c692637fb9a81e7d353ace27f53 is it the right way? ?Martin? -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at stroeder.com Wed Sep 10 14:43:52 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Wed, 10 Sep 2014 14:43:52 +0200 Subject: [python-ldap] ldap.init_fd() patch for python-ldap-2.4.15 In-Reply-To: <5404F252.8030702@proseconsulting.co.uk> References: <5404F252.8030702@proseconsulting.co.uk> Message-ID: <54104788.9000806@stroeder.com> Mark R Bannister wrote: > I needed ldap.init_fd(), so I wrote a patch. Please find attached. Hope this > works for you too. I've only tested on OpenSuSE. Could you please elaborate on why you need that? Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Wed Sep 10 14:46:06 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Wed, 10 Sep 2014 14:46:06 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: References: Message-ID: <5410480E.70708@stroeder.com> Martin Basti wrote: > I need to detect if syncrepl refresh phase is completed during > syncrepl_poll (in refresh&persist mode), is there any way how to do it? > > I found a patch > https://github.com/spacekpe/python-ldap/commit/8b4f935a97759c692637fb9a81e7d353ace27f53 > > is it the right way? Petr sent his patches to the mailing list in July. Currently they are still pending for review. I might have a chance to review and commit them during the next two weeks. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From mark at proseconsulting.co.uk Wed Sep 10 18:54:25 2014 From: mark at proseconsulting.co.uk (Mark R Bannister) Date: Wed, 10 Sep 2014 17:54:25 +0100 Subject: [python-ldap] ldap.init_fd() patch for python-ldap-2.4.15 In-Reply-To: <54104788.9000806@stroeder.com> References: <5404F252.8030702@proseconsulting.co.uk> <54104788.9000806@stroeder.com> Message-ID: <54108241.50106@proseconsulting.co.uk> On 10/09/2014 13:43, Michael Str?der wrote: > Mark R Bannister wrote: >> I needed ldap.init_fd(), so I wrote a patch. Please find attached. Hope this >> works for you too. I've only tested on OpenSuSE. > Could you please elaborate on why you need that? > > Ciao, Michael. > > Hi Michael, Yes of course. I have a multi-threaded server that maintains a pool of worker threads. Each thread can manage multiple client requests, and each client request may be processing a different LDAP connection. The main worker loop needs to select() on a list of file descriptors to see if there is any pending I/O to process. This might be new incoming sockets, new commands received from existing clients, or new data received from an asynchronous LDAP operation. I therefore need ldap.init_fd() so that I have a file descriptor that I can pass to select(). Best regards, Mark. From michael at stroeder.com Fri Sep 12 14:08:48 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Fri, 12 Sep 2014 14:08:48 +0200 Subject: [python-ldap] ldap.init_fd() patch for python-ldap-2.4.15 In-Reply-To: <54108241.50106@proseconsulting.co.uk> References: <5404F252.8030702@proseconsulting.co.uk> <54104788.9000806@stroeder.com> <54108241.50106@proseconsulting.co.uk> Message-ID: <5412E250.3010205@stroeder.com> Mark R Bannister wrote: > On 10/09/2014 13:43, Michael Str?der wrote: >> Mark R Bannister wrote: >>> I needed ldap.init_fd(), so I wrote a patch. Please find attached. Hope this >>> works for you too. I've only tested on OpenSuSE. >> Could you please elaborate on why you need that? > > Yes of course. I have a multi-threaded server that maintains a pool of worker > threads. Each thread can manage multiple client requests, and each client > request may be processing a different LDAP connection. The main worker loop > needs to select() on a list of file descriptors to see if there is any pending > I/O to process. This might be new incoming sockets, new commands received from > existing clients, or new data received from an asynchronous LDAP operation. > > I therefore need ldap.init_fd() so that I have a file descriptor that I can > pass to select(). I'm a bit concerned about new LDAPObject classes not fitting into normal class hierarchy. Therefore I've added support to CVS HEAD for getting the file descriptor of an existing connection via LDAPObject.get_option(ldap.OPT_DESC). Could you please test whether that would also be a viable solution for use-case? In fact I thought many times whether LDAPObject class should open connection itself, especially to get more control with error handling (various TLS errors/checks etc.). So instead of passing in the fd the LDAPObject class itself would use socket module and use _ldap.init_fd(). But I'm not sure whether I overlook all implications by such an approach for existing code. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From lcantu at austin.utexas.edu Fri Sep 12 18:59:17 2014 From: lcantu at austin.utexas.edu (Cantu, Liza J) Date: Fri, 12 Sep 2014 16:59:17 +0000 Subject: [python-ldap] Assertion Failed Error Message-ID: Hello, I have a very tricky issue to debug. I am working on a project on my local server and I am receiving the following error when making a call using ldap: python: references.c:33: ldap_first_reference: Assertion `( (ld)->ld_options.ldo_valid == 0x2 )' failed. Aborted My project uses Python 2.6, Django 1.4.14, and python ldap 2.3.12. The bind appears to be successful but then the connection is "closed (connection lost)". I have verified that my credentials are correct. The issue mostly happens when working locally on our VM, but it also has affected our code in production. To my knowledge, there is no way in Python to catch this error - it completely breaks our code and causes a server error. I've been using pdb to step through the relevant code, which is essentially: try: l = ldap.initialize("ldaps://" + host) l.simple_bind_s(user_name, user_pass) results = l.search_s(base_dn, ldap.SCOPE_SUBTREE, query, srch_attrs) The calls to initialize and simple_bind_s both complete successfully; it is when we start drilling into search_s that we eventually hit the error in the _ldap_call method of the SimpleLDAPObject class: def _ldap_call(self,func,*args,**kwargs): """ Wrapper method mainly for serializing calls into OpenLDAP libs and trace logs """ self._ldap_object_lock.acquire() if __debug__: if self._trace_level>=1:# and func.__name__!='result': self._trace_file.write('*** %s - %s (%s,%s)\n' % ( self._uri, self.__class__.__name__+'.'+func.__name__, repr(args),repr(kwargs) )) if self._trace_level>=3: traceback.print_stack(limit=self._trace_stack_limit,file=self._trace_file) try: try: result = func(*args,**kwargs) It bombs as soon as it tries assigning a value to result; the relevant arguments at that point are: self = func = args = (1, 1, -1) kwargs = {} Has anyone seen this issue? Or can anyone shed some light on what I am doing wrong? Thanks, Liza -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at stroeder.com Fri Sep 12 19:45:38 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Fri, 12 Sep 2014 19:45:38 +0200 Subject: [python-ldap] Assertion Failed Error In-Reply-To: References: Message-ID: <54133142.3030104@stroeder.com> Cantu, Liza J wrote: > python: references.c:33: ldap_first_reference: Assertion `( (ld)->ld_options.ldo_valid == 0x2 )' failed. Aborted > > My project uses Python 2.6, Django 1.4.14, and python ldap 2.3.12. Which libldap (OpenLDAP client libs) is this? And the OS? Note that python-ldap 2.3.12 was released four years ago and that some strange things can happen on various platforms because e.g. Debian and RedHat don't link libldap against OpenSSL. They are using GnuTLS (Debian) or libnss (RHEL 6.x) instead. So I'd recommend to try without TLS to rule out those issues. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From lcantu at austin.utexas.edu Fri Sep 12 23:46:31 2014 From: lcantu at austin.utexas.edu (Cantu, Liza J) Date: Fri, 12 Sep 2014 21:46:31 +0000 Subject: [python-ldap] Assertion Failed Error Message-ID: Michael, I am not to certain about the technical aspects of how ldap works, but I figured out we are actually using openldap version 2.4.23 and Red Hat Enterprise Linux Server release 6.5 (Santiago). We have also tried using the following command and get the same assertion failed error: ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER) Please reply if you have any other suggestions or need more info. I appreciate any help since this is affecting our local VM and deployed versions. Thank you, Liza ________________________________ From: Michael Str?der Sent: ?9/?12/?2014 12:47 PM To: Cantu, Liza J; python-ldap at python.org Subject: Re: [python-ldap] Assertion Failed Error Cantu, Liza J wrote: > python: references.c:33: ldap_first_reference: Assertion `( (ld)->ld_options.ldo_valid == 0x2 )' failed. Aborted > > My project uses Python 2.6, Django 1.4.14, and python ldap 2.3.12. Which libldap (OpenLDAP client libs) is this? And the OS? Note that python-ldap 2.3.12 was released four years ago and that some strange things can happen on various platforms because e.g. Debian and RedHat don't link libldap against OpenSSL. They are using GnuTLS (Debian) or libnss (RHEL 6.x) instead. So I'd recommend to try without TLS to rule out those issues. Ciao, Michael. -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at stroeder.com Tue Sep 16 00:32:20 2014 From: michael at stroeder.com (=?UTF-8?B?TWljaGFlbCBTdHLDtmRlcg==?=) Date: Tue, 16 Sep 2014 00:32:20 +0200 Subject: [python-ldap] Assertion Failed Error In-Reply-To: References: Message-ID: <541768F4.5030601@stroeder.com> Cantu, Liza J wrote: > I am not to certain about the technical aspects of how ldap works, but I > figured out we are actually using openldap version 2.4.23 and Red Hat > Enterprise Linux Server release 6.5 (Santiago). > We have also tried using the following command and get the same assertion failed error: > ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER) In RHEL 6.x libldap is linked to libnss. => Most likely this is caused by RHEL-specific build. Please ask Red Hat's support what makes their OpenLDAP build crash. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From kpnemesis at gmail.com Fri Sep 19 16:44:09 2014 From: kpnemesis at gmail.com (Kernel Panic) Date: Fri, 19 Sep 2014 15:44:09 +0100 Subject: [python-ldap] AD objectGUID conversion to string Message-ID: Hello everyone, when I extract the objectGUID value from an AD account I believe it is extracted in binary format, can anyone show me a way to convert that to a string similar to what you would get from Microsoft's csvde.exe tool? Thankyou for any help. -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglist0 at skurfer.com Fri Sep 19 20:53:27 2014 From: mailinglist0 at skurfer.com (Rob McBroom) Date: Fri, 19 Sep 2014 14:53:27 -0400 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: References: Message-ID: On 19 Sep 2014, at 10:44, Kernel Panic wrote: > Hello everyone, when I extract the objectGUID value from an AD account > I > believe it is extracted in binary format, can anyone show me a way to > convert that to a string similar to what you would get from > Microsoft's > csvde.exe tool? No access to an AD server any more, but is it possible that the value is base64 encoded? If `ldapsearch` shows two colons instead of one after that attribute, that?s probably the case. If so, you just need to decode it. There are a million ways to do it. In Python, it?s as easy as import base64 base64.decodestring(whatever) -- Rob McBroom http://www.skurfer.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at stroeder.com Fri Sep 19 21:27:42 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Fri, 19 Sep 2014 21:27:42 +0200 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: References: Message-ID: <541C83AE.5050404@stroeder.com> Kernel Panic wrote: > Hello everyone, when I extract the objectGUID value from an AD account I > believe it is extracted in binary format, can anyone show me a way to > convert that to a string similar to what you would get from Microsoft's > csvde.exe tool? In web2ldap I have code similar to this: import uuid u = uuid.UUID(bytes=ldap_entry['objectGUID'][0]) Not sure whether csvde.exe really displays objectGUID value as UUID string representation though. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Fri Sep 19 21:29:25 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Fri, 19 Sep 2014 21:29:25 +0200 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: References: Message-ID: <541C8415.2070108@stroeder.com> Rob McBroom wrote: > On 19 Sep 2014, at 10:44, Kernel Panic wrote: > >> Hello everyone, when I extract the objectGUID value from an AD account I >> believe it is extracted in binary format, can anyone show me a way to >> convert that to a string similar to what you would get from Microsoft's >> csvde.exe tool? > > No access to an AD server any more, but is it possible that the value is > base64 encoded? It is - in LDIF representation. But that's another thing and not relevant when retrieving the value directly via LDAP. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From kpnemesis at gmail.com Tue Sep 23 17:02:43 2014 From: kpnemesis at gmail.com (Kernel Panic) Date: Tue, 23 Sep 2014 16:02:43 +0100 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: <541C83AE.5050404@stroeder.com> References: <541C83AE.5050404@stroeder.com> Message-ID: Hello, thanks for the replies, here is an example of the output of the objectGUID query: '\xd3\x18W|T\xc4!A\x90\xe3\xa7\xf7\xe8\xe1\xd9\xf1' Can anyone confirm what this format is? Thnaks. On 19 September 2014 20:27, Michael Str?der wrote: > Kernel Panic wrote: > > Hello everyone, when I extract the objectGUID value from an AD account I > > believe it is extracted in binary format, can anyone show me a way to > > convert that to a string similar to what you would get from Microsoft's > > csvde.exe tool? > > In web2ldap I have code similar to this: > > import uuid > u = uuid.UUID(bytes=ldap_entry['objectGUID'][0]) > > Not sure whether csvde.exe really displays objectGUID value as UUID string > representation though. > > Ciao, Michael. > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jcasale at activenetwerx.com Tue Sep 23 20:33:43 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Tue, 23 Sep 2014 18:33:43 +0000 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: References: <541C83AE.5050404@stroeder.com> Message-ID: <0355e02342a146279991d48ec624630c@exch.activenetwerx.com> > Hello, thanks for the replies, here is an example of the output of the > objectGUID query:?? '\xd3\x18W|T\xc4!A\x90\xe3\xa7\xf7\xe8\xe1\xd9\xf1' > Can anyone confirm what this format is? > Thnaks. As Michaels code illustrated, the value is byte encoded. Data in AD is typically utf-16 encoded. The "\xnn" notation is hexadecimal values, see string literals in the Python docs. In [1]: import uuid In [2]: s = b'\xd3\x18W|T\xc4!A\x90\xe3\xa7\xf7\xe8\xe1\xd9\xf1' In [3]: uuid.UUID(bytes_le=s) Out[3]: UUID('7c5718d3-c454-4121-90e3-a7f7e8e1d9f1') hth, jlc From michael at stroeder.com Tue Sep 23 21:57:56 2014 From: michael at stroeder.com (=?UTF-8?B?TWljaGFlbCBTdHLDtmRlcg==?=) Date: Tue, 23 Sep 2014 21:57:56 +0200 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: <0355e02342a146279991d48ec624630c@exch.activenetwerx.com> References: <541C83AE.5050404@stroeder.com> <0355e02342a146279991d48ec624630c@exch.activenetwerx.com> Message-ID: <5421D0C4.6080207@stroeder.com> Joseph L. Casale wrote: >> here is an example of the output of the >> objectGUID query: '\xd3\x18W|T\xc4!A\x90\xe3\xa7\xf7\xe8\xe1\xd9\xf1' >> Can anyone confirm what this format is? > > As Michaels code illustrated, the value is byte encoded. Yes, the LDAP syntax of 'objectGUID' is simply OctetString. > Data in AD is typically utf-16 encoded. No! The LDAP interface of MS AD is LDAPv3 compliant and therefore uses UTF-8 for LDAP syntax DirectoryString. > The "\xnn" notation is hexadecimal values, see string literals in the Python docs. > > In [1]: import uuid > > In [2]: s = b'\xd3\x18W|T\xc4!A\x90\xe3\xa7\xf7\xe8\xe1\xd9\xf1' > > In [3]: uuid.UUID(bytes_le=s) > Out[3]: UUID('7c5718d3-c454-4121-90e3-a7f7e8e1d9f1') Yupp! Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From jcasale at activenetwerx.com Wed Sep 24 04:37:17 2014 From: jcasale at activenetwerx.com (Joseph L. Casale) Date: Wed, 24 Sep 2014 02:37:17 +0000 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: <5421D0C4.6080207@stroeder.com> References: <541C83AE.5050404@stroeder.com> <0355e02342a146279991d48ec624630c@exch.activenetwerx.com> <5421D0C4.6080207@stroeder.com> Message-ID: > > Data in AD is typically utf-16 encoded. > > No! The LDAP interface of MS AD is LDAPv3 compliant and therefore uses UTF-8 > for LDAP syntax DirectoryString. Well, to be specific I referred to data _in_ AD, not data _on the wire_. To be honest it wasn't even relevant in this case but more for cases like setting unicodePwd. But we digress... jlc From robert.meany at gmail.com Wed Sep 24 05:17:03 2014 From: robert.meany at gmail.com (Robert Meany) Date: Tue, 23 Sep 2014 23:17:03 -0400 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: References: <541C83AE.5050404@stroeder.com> <0355e02342a146279991d48ec624630c@exch.activenetwerx.com> <5421D0C4.6080207@stroeder.com> Message-ID: Woops, I replied earlier, but I didnt reply-all... See following pasted code I use for converting what's returned from AD via the python-ldap query to readable format: http://pastebin.com/1FQfFUhc On Tue, Sep 23, 2014 at 10:37 PM, Joseph L. Casale < jcasale at activenetwerx.com> wrote: > > > Data in AD is typically utf-16 encoded. > > > > No! The LDAP interface of MS AD is LDAPv3 compliant and therefore uses > UTF-8 > > for LDAP syntax DirectoryString. > > Well, to be specific I referred to data _in_ AD, not data _on the wire_. > To be honest > it wasn't even relevant in this case but more for cases like setting > unicodePwd. > > But we digress... > > jlc > _______________________________________________ > python-ldap mailing list > python-ldap at python.org > https://mail.python.org/mailman/listinfo/python-ldap > -------------- next part -------------- An HTML attachment was scrubbed... URL: From michael at stroeder.com Wed Sep 24 10:20:26 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Wed, 24 Sep 2014 10:20:26 +0200 Subject: [python-ldap] AD objectGUID conversion to string In-Reply-To: References: <541C83AE.5050404@stroeder.com> <0355e02342a146279991d48ec624630c@exch.activenetwerx.com> <5421D0C4.6080207@stroeder.com> Message-ID: <54227ECA.3090504@stroeder.com> Joseph L. Casale wrote: >>> Data in AD is typically utf-16 encoded. >> >> No! The LDAP interface of MS AD is LDAPv3 compliant and therefore uses UTF-8 >> for LDAP syntax DirectoryString. > > Well, to be specific I referred to data _in_ AD, not data _on the wire_. To be honest > it wasn't even relevant in this case but more for cases like setting unicodePwd. Yes, 'unicodePwd' is very badly designed. Double quotes as single-byte ASCII chars enclosing the UTF-16 low-endian password value. Yuck! But all other DirectoryString values are simply UTF-8. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2398 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Wed Sep 24 21:07:32 2014 From: pspacek at redhat.com (Petr Spacek) Date: Wed, 24 Sep 2014 21:07:32 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: <5410480E.70708@stroeder.com> References: <5410480E.70708@stroeder.com> Message-ID: <54231674.8000900@redhat.com> Hello Michael and list, On 10.9.2014 14:46, Michael Str?der wrote: > Martin Basti wrote: >> I need to detect if syncrepl refresh phase is completed during >> syncrepl_poll (in refresh&persist mode), is there any way how to do it? >> >> I found a patch >> https://github.com/spacekpe/python-ldap/commit/8b4f935a97759c692637fb9a81e7d353ace27f53 >> >> is it the right way? > > Petr sent his patches to the mailing list in July. Currently they are still > pending for review. I might have a chance to review and commit them during the > next two weeks. I'm very sorry for bothering you. I would like to push latest version of python-ldap package to Fedora so I would like to know if the patch can be included or not. (Obviously Fedora don't want to provide non-standard API.) The patch or some other alternative for end-of-refresh-phase-detection is required for further development in the FreeIPA project... I would not ask for it if it was unnecessary. Thank you very much for your time! -- Petr Spacek @ Red Hat From pspacek at redhat.com Thu Sep 25 12:22:36 2014 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 25 Sep 2014 12:22:36 +0200 Subject: [python-ldap] Reference leaks in error-handling paths Message-ID: <5423ECEC.8050706@redhat.com> Hello, static analysis tool gcc-python-plugin detected some bugs within the C Python extension module. Full report from analyzer is at: https://fedorapeople.org/~dmalcolm/gcc-python-plugin/2012-04-04/python-ldap-2.4.6-2.fc17/ Within the category "Reference leaks" the 6 issues reported are all memory leaks in error-handling paths. 5 out of them relate to Py_BuildValue: http://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html#reference-leak-in-py-buildvalue The other, in File: Modules/LDAPObject.c Function: interaction Error: ob_refcnt of '*result' is 1 too high is a missing Py_DECREF(). Detailed discussion about some other potential bugs is at: https://bugzilla.redhat.com/show_bug.cgi?id=809943 Credit for uncovering these problems belongs to Dave Malcolm . I'm not able to send you patch now so I'm sending the bug report to let you know about the problem. Have a nice day! -- Petr Spacek @ Red Hat From michael at stroeder.com Thu Sep 25 16:27:06 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Thu, 25 Sep 2014 16:27:06 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: <54231674.8000900@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> Message-ID: <5424263A.4070807@stroeder.com> Petr Spacek wrote: > On 10.9.2014 14:46, Michael Str?der wrote: >> Martin Basti wrote: >>> I need to detect if syncrepl refresh phase is completed during >>> syncrepl_poll (in refresh&persist mode), is there any way how to do it? >>> >>> I found a patch >>> https://github.com/spacekpe/python-ldap/commit/8b4f935a97759c692637fb9a81e7d353ace27f53 >>> >>> is it the right way? >> >> Petr sent his patches to the mailing list in July. Currently they are still >> pending for review. I might have a chance to review and commit them during the >> next two weeks. > > I'm very sorry for bothering you. I would like to push latest version of > python-ldap package to Fedora so I would like to know if the patch can be > included or not. (Obviously Fedora don't want to provide non-standard API.) > > The patch or some other alternative for end-of-refresh-phase-detection is > required for further development in the FreeIPA project... Personally I generally dislike callbacks. A better approach should IMO specify just a generic handler method an application can override in a sub-class. Can you develop a patch like this? Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Thu Sep 25 16:41:47 2014 From: pspacek at redhat.com (Petr Spacek) Date: Thu, 25 Sep 2014 16:41:47 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: <5424263A.4070807@stroeder.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <5424263A.4070807@stroeder.com> Message-ID: <542429AB.5050102@redhat.com> On 25.9.2014 16:27, Michael Str?der wrote: > Petr Spacek wrote: >> On 10.9.2014 14:46, Michael Str?der wrote: >>> Martin Basti wrote: >>>> I need to detect if syncrepl refresh phase is completed during >>>> syncrepl_poll (in refresh&persist mode), is there any way how to do it? >>>> >>>> I found a patch >>>> https://github.com/spacekpe/python-ldap/commit/8b4f935a97759c692637fb9a81e7d353ace27f53 >>>> >>>> is it the right way? >>> >>> Petr sent his patches to the mailing list in July. Currently they are still >>> pending for review. I might have a chance to review and commit them during the >>> next two weeks. >> >> I'm very sorry for bothering you. I would like to push latest version of >> python-ldap package to Fedora so I would like to know if the patch can be >> included or not. (Obviously Fedora don't want to provide non-standard API.) >> >> The patch or some other alternative for end-of-refresh-phase-detection is >> required for further development in the FreeIPA project... > > Personally I generally dislike callbacks. A better approach should IMO specify > just a generic handler method an application can override in a sub-class. > > Can you develop a patch like this? The patch adds method SyncreplConsumer.syncrepl_refreshdone() and this method should be overridden in sub-class in the same way as existing method syncrepl_entry() and the like. I agree that variable name 'callback' is not the best but it actually is not a callback, it works in the same way as similar methods in SyncreplConsumer class. I hope I didn't miss something obvious, please correct me if I'm wrong. -- Petr Spacek @ Red Hat From michael at stroeder.com Thu Sep 25 18:32:56 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Thu, 25 Sep 2014 18:32:56 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: <542429AB.5050102@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <5424263A.4070807@stroeder.com> <542429AB.5050102@redhat.com> Message-ID: <542443B8.5020305@stroeder.com> Petr Spacek wrote: > On 25.9.2014 16:27, Michael Str?der wrote: >> Petr Spacek wrote: >>> On 10.9.2014 14:46, Michael Str?der wrote: >>>> Martin Basti wrote: >>>>> I need to detect if syncrepl refresh phase is completed during >>>>> syncrepl_poll (in refresh&persist mode), is there any way how to do it? >>>>> >>>>> I found a patch >>>>> https://github.com/spacekpe/python-ldap/commit/8b4f935a97759c692637fb9a81e7d353ace27f53 >>>>> >>>>> >>>>> is it the right way? >>>> >>>> Petr sent his patches to the mailing list in July. Currently they are still >>>> pending for review. I might have a chance to review and commit them during >>>> the >>>> next two weeks. >>> >>> I'm very sorry for bothering you. I would like to push latest version of >>> python-ldap package to Fedora so I would like to know if the patch can be >>> included or not. (Obviously Fedora don't want to provide non-standard API.) >>> >>> The patch or some other alternative for end-of-refresh-phase-detection is >>> required for further development in the FreeIPA project... >> >> Personally I generally dislike callbacks. A better approach should IMO specify >> just a generic handler method an application can override in a sub-class. >> >> Can you develop a patch like this? > > The patch adds method SyncreplConsumer.syncrepl_refreshdone() and this method > should be overridden in sub-class in the same way as existing method > syncrepl_entry() and the like. > > I agree that variable name 'callback' is not the best but it actually is not a > callback, it works in the same way as similar methods in SyncreplConsumer class. I see. I've committed to CVS. Please test. Could you please provide better var names instead of 'callback' and 'newvalue'? And a __doc__ string for _syncrepl_update_refreshdone() would also be appreciated. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Thu Sep 25 18:37:50 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Thu, 25 Sep 2014 18:37:50 +0200 Subject: [python-ldap] syncrepl and FreeIPA (was: Need to detect if syncrepl refresh phase is completed) In-Reply-To: <54231674.8000900@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> Message-ID: <542444DE.6000400@stroeder.com> Petr Spacek wrote: > The patch or some other alternative for end-of-refresh-phase-detection is > required for further development in the FreeIPA project... I'm curious: What's the use-case for syncrepl in FreeIPA? Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From cmikk at qwest.net Thu Sep 25 20:33:59 2014 From: cmikk at qwest.net (Chris Mikkelson) Date: Thu, 25 Sep 2014 13:33:59 -0500 Subject: [python-ldap] syncrepl and FreeIPA (was: Need to detect if syncrepl refresh phase is completed) In-Reply-To: <542444DE.6000400@stroeder.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> Message-ID: <20140925183359.GA3775@tig.oss.uswest.net> On Thu, Sep 25, 2014 at 06:37:50PM +0200, Michael Str?der wrote: > Petr Spacek wrote: > > The patch or some other alternative for end-of-refresh-phase-detection is > > required for further development in the FreeIPA project... > > I'm curious: > What's the use-case for syncrepl in FreeIPA? I'm also curious about this. FWIW, I've detected the end of refresh by watching for the first syncrepl_set_cookie() call. I do not know if this heuristic relies on OpenLDAP's implementation, but in order for this not to work, the server would have to make the rather odd choice to send a cookie along with an entry in the middle of the refresh phase. A separate method is a cleaner approach, though. It should suffice (and address the style nits) to replace the two current occurrences of: self.__refreshDone = sim.refreshDelete['refreshDone'] with: if sim.refreshDelete['refreshDone']: self.__refreshDone = True self.syncrepl_refreshdone() Patch attached. -- Chris Mikkelson | Quidquid latine dictum sit, altum viditur cmikk at qwest.net | -------------- next part -------------- A non-text attachment was scrubbed... Name: syncrepl-refreshdone.patch Type: text/x-diff Size: 2108 bytes Desc: not available URL: From michael at stroeder.com Thu Sep 25 22:27:26 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Thu, 25 Sep 2014 22:27:26 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: <20140925183359.GA3775@tig.oss.uswest.net> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> Message-ID: <54247AAE.9000208@stroeder.com> (restored former subject) Chris Mikkelson wrote: > FWIW, I've detected the end of refresh by watching for the first > syncrepl_set_cookie() call. I do not know if this heuristic relies > on OpenLDAP's implementation, but in order for this not to work, the > server would have to make the rather odd choice to send a cookie > along with an entry in the middle of the refresh phase. > > A separate method is a cleaner approach, though. > > It should suffice (and address the style nits) to replace > the two current occurrences of: > > self.__refreshDone = sim.refreshDelete['refreshDone'] > > with: > if sim.refreshDelete['refreshDone']: > self.__refreshDone = True > self.syncrepl_refreshdone() > > Patch attached. This would have been also my naive approach. But this make me wonder about an limitation of the current implementation: In theory a syncrepl client could initiate several syncrepl searches with different search parameters on the same LDAP connection. AFAICS this would not work with current class SyncReplConsumer. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Fri Sep 26 09:05:00 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Fri, 26 Sep 2014 09:05:00 +0200 Subject: [python-ldap] Need to detect if syncrepl refresh phase is completed In-Reply-To: <20140926002602.GA65251@uerige.oss.uswest.net> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54247AAE.9000208@stroeder.com> <20140926002602.GA65251@uerige.oss.uswest.net> Message-ID: <5425101C.6000306@stroeder.com> Chris Mikkelson wrote: > On Thu, Sep 25, 2014 at 10:27:26PM +0200, Michael Strder wrote: >> (restored former subject) >> >> Chris Mikkelson wrote: >>> It should suffice (and address the style nits) to replace >>> the two current occurrences of: >>> >>> self.__refreshDone = sim.refreshDelete['refreshDone'] >>> >>> with: >>> if sim.refreshDelete['refreshDone']: >>> self.__refreshDone = True >>> self.syncrepl_refreshdone() >>> >>> Patch attached. >> >> This would have been also my naive approach. >> >> But this make me wonder about an limitation of the current implementation: >> In theory a syncrepl client could initiate several syncrepl searches with >> different search parameters on the same LDAP connection. AFAICS this would not >> work with current class SyncReplConsumer. > > Indeed, but for more than just this reason. [..] Yes. > Fixing this would not just require tracking state on > a per-msgid basis internally, but also exposing the > msgid in syncrepl_setcookie() and syncrepl_getcookie(). > I'm not sure this would be worth the complication for > most use cases. > > I think even OpenLDAP manages multiple connections in this > case. At least it should be clearly documented somewhere. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Fri Sep 26 12:17:17 2014 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 26 Sep 2014 12:17:17 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <542444DE.6000400@stroeder.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> Message-ID: <54253D2D.9010808@redhat.com> On 25.9.2014 18:37, Michael Str?der wrote: > Petr Spacek wrote: >> The patch or some other alternative for end-of-refresh-phase-detection is >> required for further development in the FreeIPA project... > > I'm curious: > What's the use-case for syncrepl in FreeIPA? In this particular case it is used for LDAP<->OpenDNSSEC integration: LDAP is an authoritative source of DNS data but OpenDNSSEC has own (local) SQL database. We have a little daemon which uses syncrepl to get list of all DNS zones with required attributes (dnssec = enabled and so on). This daemon reconfigures OpenDNSSEC at run-time accordingly. For this use-case we need to reconstruct complete list of zones first in LDAP (i.e. detect end of refresh phase) and do modifications in OpenDNSSEC configuration only if the zone was removed or added. Obviously, this can be solved by polling if latency or higher server load are not a problem. IPA wants to stay effective & low-latency at the same time. I will send reply to the alternative approaches to respective threads. -- Petr Spacek @ Red Hat From pspacek at redhat.com Fri Sep 26 14:04:41 2014 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 26 Sep 2014 14:04:41 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <20140925183359.GA3775@tig.oss.uswest.net> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> Message-ID: <54255659.8090601@redhat.com> On 25.9.2014 20:33, Chris Mikkelson wrote: > On Thu, Sep 25, 2014 at 06:37:50PM +0200, Michael Str?der wrote: >> Petr Spacek wrote: >>> The patch or some other alternative for end-of-refresh-phase-detection is >>> required for further development in the FreeIPA project... >> >> I'm curious: >> What's the use-case for syncrepl in FreeIPA? > > I'm also curious about this. > > FWIW, I've detected the end of refresh by watching for the first > syncrepl_set_cookie() call. I do not know if this heuristic relies > on OpenLDAP's implementation, but in order for this not to work, the > server would have to make the rather odd choice to send a cookie > along with an entry in the middle of the refresh phase. > > A separate method is a cleaner approach, though. > > It should suffice (and address the style nits) to replace > the two current occurrences of: > > self.__refreshDone = sim.refreshDelete['refreshDone'] > > with: > if sim.refreshDelete['refreshDone']: > self.__refreshDone = True > self.syncrepl_refreshdone() > > Patch attached. I agree that Chris's approach is better. I mis-interpreted RFC 4533 and thought that Sync Info Message could potentially contain *both* refreshDelete[refreshDone] = TRUE and refreshPresent[refreshDone] = TRUE. This assumption is clearly wrong (see RFC 4533 section 2.5: "syncInfoValue ::= CHOICE" ...) so we should use Chris's patch. It is smaller and nicer :-) I'm attaching fixed Chris's patch which is applicable on top of latest CVS checkout. It effectively revers my patch and replaces it with Chris's code. I corrected two mistakes in the proposed patch: - the original patch read value refreshDone value from sim.refreshPresnt even if there was only sim.refreshDelete present - typo "refreshPresnt" The API stays same. Have a nice day! -- Petr Spacek @ Red Hat -------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Reworked-syncrepl_refreshdone-hook-as-proposed-by-Ch.patch Type: text/x-patch Size: 2334 bytes Desc: not available URL: From michael at stroeder.com Fri Sep 26 14:18:43 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Fri, 26 Sep 2014 14:18:43 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <54255659.8090601@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> Message-ID: <542559A3.5060604@stroeder.com> Petr Spacek wrote: > I'm attaching fixed Chris's patch which is applicable on top of latest CVS > checkout. It effectively revers my patch and replaces it with Chris's code. > > I corrected two mistakes in the proposed patch: > - the original patch read value refreshDone value from sim.refreshPresnt even > if there was only sim.refreshDelete present > - typo "refreshPresnt" > > The API stays same. Committed to CVS HEAD. Please test. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Fri Sep 26 14:43:23 2014 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 26 Sep 2014 14:43:23 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <542559A3.5060604@stroeder.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> <542559A3.5060604@stroeder.com> Message-ID: <54255F6B.5000508@redhat.com> On 26.9.2014 14:18, Michael Str?der wrote: > Petr Spacek wrote: >> >I'm attaching fixed Chris's patch which is applicable on top of latest CVS >> >checkout. It effectively revers my patch and replaces it with Chris's code. >> > >> >I corrected two mistakes in the proposed patch: >> >- the original patch read value refreshDone value from sim.refreshPresnt even >> >if there was only sim.refreshDelete present >> >- typo "refreshPresnt" >> > >> >The API stays same. > Committed to CVS HEAD. Please test. I confirm that it works for me (as checked out from CVS). I'm going to put this patch to Fedora 20+. Thank you very much and have a nice day! -- Petr Spacek @ Red Hat From michael at stroeder.com Fri Sep 26 16:58:49 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Fri, 26 Sep 2014 16:58:49 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <54255F6B.5000508@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> <542559A3.5060604@stroeder.com> <54255F6B.5000508@redhat.com> Message-ID: <54257F29.3030804@stroeder.com> Petr Spacek wrote: > I confirm that it works for me (as checked out from CVS). I'm going to put > this patch to Fedora 20+. I'd prefer to cut a new release 2.4.17 to avoid distribution patches. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From pspacek at redhat.com Fri Sep 26 17:02:43 2014 From: pspacek at redhat.com (Petr Spacek) Date: Fri, 26 Sep 2014 17:02:43 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <54257F29.3030804@stroeder.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> <542559A3.5060604@stroeder.com> <54255F6B.5000508@redhat.com> <54257F29.3030804@stroeder.com> Message-ID: <54258013.808@redhat.com> On 26.9.2014 16:58, Michael Str?der wrote: > Petr Spacek wrote: >> I confirm that it works for me (as checked out from CVS). I'm going to put >> this patch to Fedora 20+. > > I'd prefer to cut a new release 2.4.17 to avoid distribution patches. As you wish. I didn't have time to build the package today so I will try it again on Tuesday or so. -- Petr Spacek @ Red Hat From michael at stroeder.com Fri Sep 26 17:08:43 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Fri, 26 Sep 2014 17:08:43 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <54258013.808@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> <542559A3.5060604@stroeder.com> <54255F6B.5000508@redhat.com> <54257F29.3030804@stroeder.com> <54258013.808@redhat.com> Message-ID: <5425817B.6070803@stroeder.com> Petr Spacek wrote: > On 26.9.2014 16:58, Michael Str?der wrote: >> Petr Spacek wrote: >>> I confirm that it works for me (as checked out from CVS). I'm going to put >>> this patch to Fedora 20+. >> >> I'd prefer to cut a new release 2.4.17 to avoid distribution patches. > > As you wish. I didn't have time to build the package today so I will try it > again on Tuesday or so. It does not take much time to cut the release. Any other important things missing for a 2.4.17 which can be done within a couple of hours? Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Fri Sep 26 18:12:02 2014 From: michael at stroeder.com (Michael Ströder) Date: Fri, 26 Sep 2014 18:12:02 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <20140926160354.GA67674@uerige.oss.uswest.net> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> <542559A3.5060604@stroeder.com> <54255F6B.5000508@redhat.com> <54257F29.3030804@stroeder.com> <54258013.808@redhat.com> <5425817B.6070803@stroeder.com> <20140926160354.GA67674@uerige.oss.uswest.net> Message-ID: <54259052.8040306@stroeder.com> Chris Mikkelson wrote: > How about the following quick doc fix, per your suggestion upthread: Committed (with slightly different wording) Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Sat Sep 27 12:06:16 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Sat, 27 Sep 2014 12:06:16 +0200 Subject: [python-ldap] ANN: python-ldap 2.4.17 Message-ID: <54268C18.3060407@stroeder.com> Find a new release of python-ldap: http://pypi.python.org/pypi/python-ldap/2.4.17 python-ldap provides an object-oriented API to access LDAP directory servers from Python programs. It mainly wraps the OpenLDAP 2.x libs for that purpose. Additionally it contains modules for other LDAP-related stuff (e.g. processing LDIF, LDAP URLs and LDAPv3 schema). Project's web site: http://www.python-ldap.org/ Ciao, Michael. ---------------------------------------------------------------- Released 2.4.17 2014-09-27 Changes since 2.4.16: Lib/ * New hook syncrepl_refreshdone() in ldap.syncrepl.SyncReplConsumer (thanks to Petr Spacek and Chris Mikkelson) Modules/ * Added support for getting file descriptor of connection with ldap.OPT_DESC From michael at stroeder.com Sat Sep 27 12:14:35 2014 From: michael at stroeder.com (=?ISO-8859-1?Q?Michael_Str=F6der?=) Date: Sat, 27 Sep 2014 12:14:35 +0200 Subject: [python-ldap] ldap.init_fd() patch for python-ldap-2.4.15 In-Reply-To: <5412E250.3010205@stroeder.com> References: <5404F252.8030702@proseconsulting.co.uk> <54104788.9000806@stroeder.com> <54108241.50106@proseconsulting.co.uk> <5412E250.3010205@stroeder.com> Message-ID: <54268E0B.3090503@stroeder.com> Michael Str?der wrote: > Therefore I've added support to CVS HEAD for getting the file descriptor of an > existing connection via LDAPObject.get_option(ldap.OPT_DESC). This was now released with 2.4.17. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: From michael at stroeder.com Sat Sep 27 12:15:18 2014 From: michael at stroeder.com (=?windows-1252?Q?Michael_Str=F6der?=) Date: Sat, 27 Sep 2014 12:15:18 +0200 Subject: [python-ldap] syncrepl and FreeIPA In-Reply-To: <54258013.808@redhat.com> References: <5410480E.70708@stroeder.com> <54231674.8000900@redhat.com> <542444DE.6000400@stroeder.com> <20140925183359.GA3775@tig.oss.uswest.net> <54255659.8090601@redhat.com> <542559A3.5060604@stroeder.com> <54255F6B.5000508@redhat.com> <54257F29.3030804@stroeder.com> <54258013.808@redhat.com> Message-ID: <54268E36.2090105@stroeder.com> Petr Spacek wrote: > On 26.9.2014 16:58, Michael Str?der wrote: >> Petr Spacek wrote: >>> I confirm that it works for me (as checked out from CVS). I'm going to put >>> this patch to Fedora 20+. >> >> I'd prefer to cut a new release 2.4.17 to avoid distribution patches. > > As you wish. I didn't have time to build the package today so I will try it > again on Tuesday or so. This was now released with 2.4.17. Ciao, Michael. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4252 bytes Desc: S/MIME Cryptographic Signature URL: