From ondra at mistotebe.net Tue Apr 30 07:56:15 2019 From: ondra at mistotebe.net (=?utf-8?B?T25kxZllaiBLdXpuw61r?=) Date: Tue, 30 Apr 2019 13:56:15 +0200 Subject: [python-ldap] LDAPError exceptions Message-ID: <20190430115615.eh2l3c4yqhlilivl@mistotebe.net> Hi, I've opened the issue #278 at https://github.com/python-ldap/python-ldap/issues/278 and would like to discuss how/whether to go about doing this. I've found some discussions on the list about this[0] but nothing seems to have been done in the end? The way I see it, the major issue is coming from the fact that each exception seems to be a single singleton raised directly from the C module. I wonder whether for operation responses, it's not better to return the actual message and raise the exception in the Python code (with the view of making it optional) which might make it easier to actually attach the correct information (msgid, controls, description) as well. This would really help me progress with an asyncio wrapper I've started working on (which might eventually help making the syncrepl consumer a better citizen as well). Any thoughts? Thanks, Ondrej [0]. https://mail.python.org/pipermail/python-ldap/2015q3/003587.html From ondra at mistotebe.net Thu May 2 12:38:14 2019 From: ondra at mistotebe.net (=?utf-8?B?T25kxZllaiBLdXpuw61r?=) Date: Thu, 2 May 2019 18:38:14 +0200 Subject: [python-ldap] LDAPError exceptions In-Reply-To: <20190430115615.eh2l3c4yqhlilivl@mistotebe.net> References: <20190430115615.eh2l3c4yqhlilivl@mistotebe.net> Message-ID: <20190502163814.s2r3rgh5x6haeme4@mistotebe.net> On Tue, Apr 30, 2019 at 01:56:15PM +0200, Ond?ej Kuzn?k wrote: > Hi, > I've opened the issue #278 at > https://github.com/python-ldap/python-ldap/issues/278 > and would like to discuss how/whether to go about doing this. > > I've found some discussions on the list about this[0] but nothing seems > to have been done in the end? > > The way I see it, the major issue is coming from the fact that each > exception seems to be a single singleton raised directly from the C > module. I wonder whether for operation responses, it's not better to > return the actual message and raise the exception in the Python code > (with the view of making it optional) which might make it easier to > actually attach the correct information (msgid, controls, description) > as well. > > This would really help me progress with an asyncio wrapper I've started > working on (which might eventually help making the syncrepl consumer a > better citizen as well). > > Any thoughts? I've now gone ahead and created a preliminary series of patches to support this: https://github.com/mistotebe/python-ldap/tree/exceptions Most of the work should be uncontroversial but the last commit, where I've made it possible to avoid exceptions if the user wants to deal with the requests on their own. The main problem there is twofold: 1. result4 has no way to pass the result code back to the caller 2. The module doesn't currently provide names for the result codes, as they are used by the exceptions. For the latter, a fix might either be to expose the return codes in a separate module or just let the exception types also have numeric values associated with them. The former is a bit tricky and I'd like to avoid things like result5(), since at some point we might need result6() and so on. One option is to make a result-like function that returns a dictionary/namedtuple or something else that's easy to extend in the future? Regards, -- Ond?ej Kuzn?k Senior Software Engineer Symas Corporation https://www.symas.com/ Packaged, certified, and supported LDAP solutions powered by OpenLDAP From muzzol at gmail.com Wed May 29 11:43:55 2019 From: muzzol at gmail.com (muzzol) Date: Wed, 29 May 2019 17:43:55 +0200 Subject: [python-ldap] syncrepl_entry returns all entries Message-ID: hi! I'm playing with ldap.syncrepl and I would like to know if is there a way to get only modified atributes. with syncrepl_entry I get the DN and all attributes, so I'm not sure if it's up to the client to compare old and new attributes or if I'm missing something. thanks in advance, mussol -- ======================== ^ ^ O O (_ _) muzzol(a)muzzol.com ======================== No atribueixis qualitats humanes als ordinadors. No els hi agrada. ======================== "El gobierno espa?ol s?lo habla con terroristas, homosexuales y catalanes, a ver cuando se decide a hablar con gente normal" Jim?nez Losantos ======================== -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondra at mistotebe.net Thu May 30 05:24:35 2019 From: ondra at mistotebe.net (=?utf-8?B?T25kxZllaiBLdXpuw61r?=) Date: Thu, 30 May 2019 11:24:35 +0200 Subject: [python-ldap] syncrepl_entry returns all entries In-Reply-To: References: Message-ID: <20190530092435.wvktfjkt3gdsw43r@mistotebe.net> On Wed, May 29, 2019 at 05:43:55PM +0200, muzzol wrote: > hi! > > I'm playing with ldap.syncrepl and I would like to know if is there a way > to get only modified atributes. > > with syncrepl_entry I get the DN and all attributes, so I'm not sure if > it's up to the client to compare old and new attributes or if I'm missing > something. Syncrepl just notifies the client about the new version of the entry as modified, the client is then expected to locate the entry with the given entryUUID and replace it with the new version. If you are replicating from OpenLDAP, you can set up the accesslog overlay and replicate its database (after you have run the initial sync on the original DB), you can send the same cookie when transitioning from one to the other. Entries there describe changes performed to the database and this is exactly what OpenLDAP uses to implement delta-sync replication. For slightly more information see https://openldap.org/doc/admin24/replication.html Regards, -- Ond?ej Kuzn?k Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP From muzzol at gmail.com Thu May 30 06:03:09 2019 From: muzzol at gmail.com (muzzol) Date: Thu, 30 May 2019 12:03:09 +0200 Subject: [python-ldap] syncrepl_entry returns all entries In-Reply-To: <20190530092435.wvktfjkt3gdsw43r@mistotebe.net> References: <20190530092435.wvktfjkt3gdsw43r@mistotebe.net> Message-ID: > Syncrepl just notifies the client about the new version of the entry as > modified, the client is then expected to locate the entry with the given > entryUUID and replace it with the new version. > > ok, it would be nice if modified attributes were notified too. > If you are replicating from OpenLDAP, you can set up the accesslog > overlay and replicate its database (after you have run the initial sync > on the original DB), you can send the same cookie when transitioning > from one to the other. Entries there describe changes performed to the > database and this is exactly what OpenLDAP uses to implement delta-sync > replication. > > but that's provider-dependent and I would like to implement something more agnostic and relying only in protocol. I'm using RH/389 DS mostly in my installations btw. do you know if is there any plans to implement this? reading RFC -- ======================== ^ ^ O O (_ _) muzzol(a)muzzol.com ======================== No atribueixis qualitats humanes als ordinadors. No els hi agrada. ======================== "El gobierno espa?ol s?lo habla con terroristas, homosexuales y catalanes, a ver cuando se decide a hablar con gente normal" Jim?nez Losantos ======================== -------------- next part -------------- An HTML attachment was scrubbed... URL: From muzzol at gmail.com Thu May 30 06:05:26 2019 From: muzzol at gmail.com (muzzol) Date: Thu, 30 May 2019 12:05:26 +0200 Subject: [python-ldap] syncrepl_entry returns all entries In-Reply-To: References: <20190530092435.wvktfjkt3gdsw43r@mistotebe.net> Message-ID: Syncrepl just notifies the client about the new version of the entry as > modified, the client is then expected to locate the entry with the given >> entryUUID and replace it with the new version. >> >> > ok, it would be nice if modified attributes were notified too. > > > >> If you are replicating from OpenLDAP, you can set up the accesslog >> overlay and replicate its database (after you have run the initial sync >> on the original DB), you can send the same cookie when transitioning >> from one to the other. Entries there describe changes performed to the >> database and this is exactly what OpenLDAP uses to implement delta-sync >> replication. >> >> > but that's provider-dependent and I would like to implement something more > agnostic and relying only in protocol. > I'm using RH/389 DS mostly in my installations btw. > > do you know if is there any plans to implement this? > reading RFC > > sorry, hit send by mistake. do you know if is there any plans to implement this? reading https://tools.ietf.org/html/rfc4533 it seems that server could send modifications: "the server sends copies of entries that have changed with a Sync State Control indicating state add" but I'm not sure if I'm interpreting that right. best regards, mussol -- ======================== ^ ^ O O (_ _) muzzol(a)muzzol.com ======================== No atribueixis qualitats humanes als ordinadors. No els hi agrada. ======================== "El gobierno espa?ol s?lo habla con terroristas, homosexuales y catalanes, a ver cuando se decide a hablar con gente normal" Jim?nez Losantos ======================== -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondra at mistotebe.net Thu May 30 06:41:54 2019 From: ondra at mistotebe.net (=?utf-8?B?T25kxZllaiBLdXpuw61r?=) Date: Thu, 30 May 2019 12:41:54 +0200 Subject: [python-ldap] syncrepl_entry returns all entries In-Reply-To: References: <20190530092435.wvktfjkt3gdsw43r@mistotebe.net> Message-ID: <20190530104154.gbhgcs55anbhcnli@mistotebe.net> On Thu, May 30, 2019 at 12:05:26PM +0200, muzzol wrote: > Syncrepl just notifies the client about the new version of the entry as > >> modified, the client is then expected to locate the entry with the given >>> entryUUID and replace it with the new version. >>> >>> >> ok, it would be nice if modified attributes were notified too. All attributes you request are sent, just like in a regular search. >>> If you are replicating from OpenLDAP, you can set up the accesslog >>> overlay and replicate its database (after you have run the initial sync >>> on the original DB), you can send the same cookie when transitioning >>> from one to the other. Entries there describe changes performed to the >>> database and this is exactly what OpenLDAP uses to implement delta-sync >>> replication. >>> >>> >> but that's provider-dependent and I would like to implement something more >> agnostic and relying only in protocol. Sure, but there is no standardised protocol for this at the moment. There was an effort to publish the OpenLDAP behaviour as an RFC, but that would need to be revived. >> I'm using RH/389 DS mostly in my installations btw. >> >> do you know if is there any plans to implement this? >> reading RFC > > sorry, hit send by mistake. > > do you know if is there any plans to implement this? > reading https://tools.ietf.org/html/rfc4533 The protocol doesn't allow what you ask for. And dealing with certain kinds of changes (increment/partial delete) would need special handling if you wanted to transmit changes. That's why implementations that support change based replication either implement their own protocols (many proprietary LDAP servers) or explicitly replicate a log database. No idea whether 389 supports change based replication and how. > it seems that server could send modifications: > "the server sends copies of entries that have changed with a Sync State > Control indicating state add" It does just that already, yes. > but I'm not sure if I'm interpreting that right. Syncrepl (RFC4533) is just a more efficient search (or persistent search if using refreshAndPersist), there's nothing in the design to support transmitting changes only, as compared to the entries that result of those changes. Regards, -- Ond?ej Kuzn?k Senior Software Engineer Symas Corporation http://www.symas.com Packaged, certified, and supported LDAP solutions powered by OpenLDAP