[python-ldap] Performance improvement for schema.tokenizer.split_tokens

[Michael Ströder]

Christian Heimes wrote:
> On 2017-02-18 18:39, Michael Ströder wrote:
>> Michael Ströder wrote:
>>> I see you're coming close to my own (non-published) attempts of working around some
>>> schema bugs. But is it still that much faster then? ;-)
>>
>> Overall schema parsing done in ldap.schema.SubSchema.__init__() is twice as fast with
>> your recent implementation. That's great!
> 
> here is a patch against latest CVS head including some additional test
> cases.

I've slightly modified your patch for backward compability to older Python versions.

> I changed the regular expression to handle backslash quoting of
> single quotes within single quotes. A co-worker also noticed that my
> implementation did not check for closing parenthesis before opening
> parenthesis. I added a check for the special case, too (thx Standa).

Thanks. This revealed an error in my Æ-DIR suppl. schema for web2ldap.

Maybe I did something wrong but one bad case does not raise ValueError as expected:

$ PYTHONPATH=Lib python Tests/t_ldap_schema_tokenizer.py
.Fxx.
======================================================================
FAIL: test_broken (__main__.TestSplitTokens)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "Tests/t_ldap_schema_tokenizer.py", line 115, in test_broken
    self._run_failure_tests(TESTCASES_BROKEN)
  File "Tests/t_ldap_schema_tokenizer.py", line 84, in _run_failure_tests
    should_have_failed,
AssertionError: 1 value(s) should have raised ValueError: ['( BLUB )) DA (']

----------------------------------------------------------------------
Ran 5 tests in 0.001s

FAILED (failures=1, expected failures=2)

Ciao, Michael.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3829 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://mail.python.org/pipermail/python-ldap/attachments/20170222/888aca0f/attachment.bin>