From andy47 at halfcooked.com  Sun Jan  3 22:26:48 2016
From: andy47 at halfcooked.com (Andy Todd)
Date: Mon, 4 Jan 2016 14:26:48 +1100
Subject: [DB-SIG] SQL CONNECTION
In-Reply-To: <CAB0HjaHsoq5wpe3Bw=DMmoAr4ZsKXrf4cSK8oHOvTf=M=wxShQ@mail.gmail.com>
References: <CAB0HjaHsoq5wpe3Bw=DMmoAr4ZsKXrf4cSK8oHOvTf=M=wxShQ@mail.gmail.com>
Message-ID: <5689E678.8050405@halfcooked.com>



On 31/12/2015 1:31 AM, Dale Wheeler wrote:
> Hi-
> Can you help?
> I am trying to connect to SQL Anywhere (v9) with iPython Ananconda.
> I am on a Windows 7 PC and both Python and SQL are installed locally.
>
> Is this at all possible or do i need to upgrade my version of SQL?
>
> Many thanks
>
> Dale
>
>
> _______________________________________________
> DB-SIG maillist  -  DB-SIG at python.org
> https://mail.python.org/mailman/listinfo/db-sig
>

Dale,

You need to install a Python module that allows you to talk to SQL 
Anywhere. A quick search shows this - 
https://pypi.python.org/pypi/sqlanydb/1.0.5

I've not used SQL Anywhere so can't testify if this is the best solution.

As you are using Anaconda it *should* be fairly straightforward to 
install this Python module. Run the "Anaconda Command Prompt" from your 
start menu and just type

c:\> conda install sqlanydb

Then after you have installed the package start up a Python prompt and 
follow the instructions on the package page link I included above.

Good luck,
Andy
-- 
-----------------------------------------------------------------------------
 From the desk of Andrew J Todd esq - http://www.halfcooked.com/
"It's still magic even if you know how it's done." - Terry Pratchett

From gtobias at tobcm.com  Sun Jan 10 14:25:45 2016
From: gtobias at tobcm.com (greg tobias)
Date: Sun, 10 Jan 2016 14:25:45 -0500
Subject: [DB-SIG] PEP 249--connection problems
Message-ID: <000c01d14bdc$b407e680$1c17b380$@tobcm.com>

Hi. I am a windows user trying to establish a 'connect' using 'pymysql' in
py3. I am looking for documentation on what information I need to put into
the 'input fields' ie..

 

pymysql.connect(host='127.0.0.1',unix_socket='/var/run/mysqld/mysqld.sock',u
ser = 'root', passwd = '423094424209484',db='mysql')

Thanks a lot for your help. Regards, Greg.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/db-sig/attachments/20160110/5ba58f3a/attachment.html>

From songofacandy at gmail.com  Sun Jan 10 18:32:47 2016
From: songofacandy at gmail.com (INADA Naoki)
Date: Mon, 11 Jan 2016 08:32:47 +0900
Subject: [DB-SIG] PEP 249--connection problems
In-Reply-To: <000c01d14bdc$b407e680$1c17b380$@tobcm.com>
References: <000c01d14bdc$b407e680$1c17b380$@tobcm.com>
Message-ID: <CAEfz+TwK90LLczENkwXzukZb+4Q2zT2Z9uu8Ntbzjo6bK+Q-5g@mail.gmail.com>

Hi, greg.

First of all, there is PyMySQL users Google group.
I found you luckily.  But the group is better place to ask a question.
https://groups.google.com/forum/#!forum/pymysql-users



On Mon, Jan 11, 2016 at 4:25 AM, greg tobias <gtobias at tobcm.com> wrote:

> Hi. I am a windows user trying to establish a 'connect' using 'pymysql' in
> py3. I am looking for documentation on what information I need to put into
> the 'input fields' ie?.
>

The docstring.


>
>
> pymysql.connect(host='127.0.0.1',unix_socket='/var/run/mysqld/mysqld.sock',user
> = 'root', passwd = '423094424209484',db='mysql')
>
> Thanks a lot for your help. Regards, Greg.
>
> _______________________________________________
> DB-SIG maillist  -  DB-SIG at python.org
> https://mail.python.org/mailman/listinfo/db-sig
>
>


-- 
INADA Naoki  <songofacandy at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/db-sig/attachments/20160111/22191903/attachment.html>

From songofacandy at gmail.com  Tue Jan 12 07:59:08 2016
From: songofacandy at gmail.com (INADA Naoki)
Date: Tue, 12 Jan 2016 21:59:08 +0900
Subject: [DB-SIG] Should Binary accept unicode string?
Message-ID: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>

Hi, all.

I found DB-API 2.0 defines Binary() as Binary(string).
https://www.python.org/dev/peps/pep-0249/#binary

What the string means?
On Python 2, should Binary accept unicode?
On Python 3, should Binary accept str?

-- 
INADA Naoki  <songofacandy at gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/db-sig/attachments/20160112/830934f6/attachment.html>

From mike_mp at zzzcomputing.com  Fri Jan 15 09:39:24 2016
From: mike_mp at zzzcomputing.com (Mike Bayer)
Date: Fri, 15 Jan 2016 09:39:24 -0500
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
Message-ID: <5699049C.9090807@zzzcomputing.com>



On 01/12/2016 07:59 AM, INADA Naoki wrote:
> Hi, all.
> 
> I found DB-API 2.0 defines Binary() as Binary(string).
> https://www.python.org/dev/peps/pep-0249/#binary
> 
> What the string means?
> On Python 2, should Binary accept unicode?
> On Python 3, should Binary accept str?


IMO, Py2K's "binary" is str, that's all you should accept, and Py3K's
"binary" is "bytes", that's all you should accept.   A Unicode object is
not a "binary" any more than a Python dictionary is.  If your users are
looking for the "convenience" of assuming unicodes become binaries by
using the nearest available character set to encode it, they need to be
reminded of "explicit is better than implicit" :).




> 
> -- 
> INADA Naoki  <songofacandy at gmail.com <mailto:songofacandy at gmail.com>>
> 
> 
> _______________________________________________
> DB-SIG maillist  -  DB-SIG at python.org
> https://mail.python.org/mailman/listinfo/db-sig
> 

From mal at egenix.com  Fri Jan 15 09:47:07 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 15 Jan 2016 15:47:07 +0100
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
Message-ID: <5699066B.4080004@egenix.com>

On 12.01.2016 13:59, INADA Naoki wrote:
> Hi, all.
> 
> I found DB-API 2.0 defines Binary() as Binary(string).
> https://www.python.org/dev/peps/pep-0249/#binary
> 
> What the string means?
> On Python 2, should Binary accept unicode?
> On Python 3, should Binary accept str?

The Binary() wrapper is intended to provide extra information
for the database module and marks the intent of the user to have
the input parameter be bound to the binding parameter as
binary rather than text (e.g. VARBINARY rather than VARCHAR).

For Python 2, you'd probably use something like Binary=buffer.
On Python 3, Binary=bytes or Binary=bytearray seem like a natural
choices.

The choice of possible input parameters for Binary() is
really up to the database module author.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 15 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From mike_mp at zzzcomputing.com  Fri Jan 15 10:52:19 2016
From: mike_mp at zzzcomputing.com (Mike Bayer)
Date: Fri, 15 Jan 2016 10:52:19 -0500
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <5699066B.4080004@egenix.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com>
Message-ID: <569915B3.2000609@zzzcomputing.com>



On 01/15/2016 09:47 AM, M.-A. Lemburg wrote:
> On 12.01.2016 13:59, INADA Naoki wrote:
>> Hi, all.
>>
>> I found DB-API 2.0 defines Binary() as Binary(string).
>> https://www.python.org/dev/peps/pep-0249/#binary
>>
>> What the string means?
>> On Python 2, should Binary accept unicode?
>> On Python 3, should Binary accept str?
> 
> The Binary() wrapper is intended to provide extra information
> for the database module and marks the intent of the user to have
> the input parameter be bound to the binding parameter as
> binary rather than text (e.g. VARBINARY rather than VARCHAR).
> 
> For Python 2, you'd probably use something like Binary=buffer.
> On Python 3, Binary=bytes or Binary=bytearray seem like a natural
> choices.
> 
> The choice of possible input parameters for Binary() is
> really up to the database module author.

I still don't understand this philosophy of pep-249.   Allowing DBAPIs
to arbitrarily decide how strict / loose they want to be for
user-defined data passed to even very well known datatypes has a
negative impact on portability.   It means that code I write for one
DBAPI will fail on another.   Is it your view that databases and DBAPIs
are so fundamentally different, even for basic things like
unicodes/bytes, that attempting to provide for portability is hopeless?
 Why even have a pep-249 if I should expect that I have to rewrite my
whole application when switching DBAPIs anyway?

Obviosly, full portability between DBAPIs and databases is never going
to be possible.  But for easy things where a pro-portability decision is
clearly very feasible, like, "do / don't accept a unicode object for a
bytes type", why can't a decision be made?





> 

From mal at egenix.com  Fri Jan 15 11:15:52 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 15 Jan 2016 17:15:52 +0100
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <569915B3.2000609@zzzcomputing.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com> <569915B3.2000609@zzzcomputing.com>
Message-ID: <56991B38.1090607@egenix.com>

On 15.01.2016 16:52, Mike Bayer wrote:
> 
> 
> On 01/15/2016 09:47 AM, M.-A. Lemburg wrote:
>> On 12.01.2016 13:59, INADA Naoki wrote:
>>> Hi, all.
>>>
>>> I found DB-API 2.0 defines Binary() as Binary(string).
>>> https://www.python.org/dev/peps/pep-0249/#binary
>>>
>>> What the string means?
>>> On Python 2, should Binary accept unicode?
>>> On Python 3, should Binary accept str?
>>
>> The Binary() wrapper is intended to provide extra information
>> for the database module and marks the intent of the user to have
>> the input parameter be bound to the binding parameter as
>> binary rather than text (e.g. VARBINARY rather than VARCHAR).
>>
>> For Python 2, you'd probably use something like Binary=buffer.
>> On Python 3, Binary=bytes or Binary=bytearray seem like a natural
>> choices.
>>
>> The choice of possible input parameters for Binary() is
>> really up to the database module author.
> 
> I still don't understand this philosophy of pep-249.   Allowing DBAPIs
> to arbitrarily decide how strict / loose they want to be for
> user-defined data passed to even very well known datatypes has a
> negative impact on portability.   It means that code I write for one
> DBAPI will fail on another.   Is it your view that databases and DBAPIs
> are so fundamentally different, even for basic things like
> unicodes/bytes, that attempting to provide for portability is hopeless?
>  Why even have a pep-249 if I should expect that I have to rewrite my
> whole application when switching DBAPIs anyway?
>
> Obviosly, full portability between DBAPIs and databases is never going
> to be possible.  But for easy things where a pro-portability decision is
> clearly very feasible, like, "do / don't accept a unicode object for a
> bytes type", why can't a decision be made?

I think you are misunderstanding the purpose of the Binary() helper:
This was added as a portable way to tell the database interface
to bind data as binary to the parameter, nothing more.

Since some database modules rely on the Python type of the
input parameters to tell whether to bind as binary,
character, numeric, etc., but did have the distinction between
binary and text data in Python 2, as we now do in Python 3,
the Binary() wrapper was added to make the distinction clear.

The types Binary() allows as input are not part of the DB-API,
just like we don't make any comments about the allowed input
types for any other parameter type the database interface
may support.

This adds flexibility and makes it possible to create
interface modules which support a great deal more than
just a few standard Python data types.

Back to the choices I mentioned for Binary():

In Python 2, buffer() does allow unicode objects
on input, and what you as result corresponds to the binary
representation of the unicode object as used by Python.

In Python 3, bytes() requires to be more specific and you
need to provide an encoding. The result is an encoded binary
version of the text input.

Both are reasonable choices for a Binary() wrapper and
the result is easy to detect as "bind me as binary data"
for the database module.

It is not uncommon to convert text data to
binary data for storage, esp. when dealing with larger
blobs you just want to manage and not work on, or when
you want to preserve it in exactly the same form you pass
it to the database (without any implicit normalizations,
surrogate conversions, warnings, etc.).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 15 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/


From mike_mp at zzzcomputing.com  Fri Jan 15 12:21:33 2016
From: mike_mp at zzzcomputing.com (Mike Bayer)
Date: Fri, 15 Jan 2016 12:21:33 -0500
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <56991B38.1090607@egenix.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com> <569915B3.2000609@zzzcomputing.com>
 <56991B38.1090607@egenix.com>
Message-ID: <56992A9D.6080509@zzzcomputing.com>



On 01/15/2016 11:15 AM, M.-A. Lemburg wrote:
> On 15.01.2016 16:52, Mike Bayer wrote:
>>
>>
>> On 01/15/2016 09:47 AM, M.-A. Lemburg wrote:
>>> On 12.01.2016 13:59, INADA Naoki wrote:
>>>> Hi, all.
>>>>
>>>> I found DB-API 2.0 defines Binary() as Binary(string).
>>>> https://www.python.org/dev/peps/pep-0249/#binary
>>>>
>>>> What the string means?
>>>> On Python 2, should Binary accept unicode?
>>>> On Python 3, should Binary accept str?
>>>
>>> The Binary() wrapper is intended to provide extra information
>>> for the database module and marks the intent of the user to have
>>> the input parameter be bound to the binding parameter as
>>> binary rather than text (e.g. VARBINARY rather than VARCHAR).
>>>
>>> For Python 2, you'd probably use something like Binary=buffer.
>>> On Python 3, Binary=bytes or Binary=bytearray seem like a natural
>>> choices.
>>>
>>> The choice of possible input parameters for Binary() is
>>> really up to the database module author.
>>
>> I still don't understand this philosophy of pep-249.   Allowing DBAPIs
>> to arbitrarily decide how strict / loose they want to be for
>> user-defined data passed to even very well known datatypes has a
>> negative impact on portability.   It means that code I write for one
>> DBAPI will fail on another.   Is it your view that databases and DBAPIs
>> are so fundamentally different, even for basic things like
>> unicodes/bytes, that attempting to provide for portability is hopeless?
>>  Why even have a pep-249 if I should expect that I have to rewrite my
>> whole application when switching DBAPIs anyway?
>>
>> Obviosly, full portability between DBAPIs and databases is never going
>> to be possible.  But for easy things where a pro-portability decision is
>> clearly very feasible, like, "do / don't accept a unicode object for a
>> bytes type", why can't a decision be made?
> 
> I think you are misunderstanding the purpose of the Binary() helper:
> This was added as a portable way to tell the database interface
> to bind data as binary to the parameter, nothing more.

I fully understand its purpose.  I'm referring to the scope of what we
mean by "bind data".

> 
> Since some database modules rely on the Python type of the
> input parameters to tell whether to bind as binary,
> character, numeric, etc., but did have the distinction between
> binary and text data in Python 2, as we now do in Python 3,
> the Binary() wrapper was added to make the distinction clear.

> 
> The types Binary() allows as input are not part of the DB-API,
> just like we don't make any comments about the allowed input
> types for any other parameter type the database interface
> may support.

Yes, and it is this philosophy I am commenting on - when it is very
obvious that the Python data in question does not align to the database
type being referred to without a conversion taking place, where that
conversion is non-obvious (e.g. a "guess" must be made).

This is not the same as when we have say a Python datetime.datetime()
object being mapped to a TIMESTAMP field - there's a natural conversion
that can take place here, assuming the types line up as far as presence
or not of fields like microseconds or timezone.   The fields of datetime
objects can line up exactly against a backend database without the need
for implicit decisionmaking about format.

Whereas if we tried to map an integer value to a TIMESTAMP field, this
is not a natural conversion - the integer can mean all kinds of things
like the epoch in seconds, days, etc.   If this data is not accepted by
the database directly and the driver makes an arbitrary choice to make
this the epoch in days since 1970, that's pretty arbitrary.  It would be
surprising behavior.


> 
> This adds flexibility and makes it possible to create
> interface modules which support a great deal more than
> just a few standard Python data types.

that is absolutely true, however, I don't see how you can deny that
portability is negatively impacted when on one DBAPI I can say:

cursor.execute(
    "insert into table (q) values (?)", [Binary(u'some unicode')]
)

and on another, it fails, and I instead have to type:

cursor.execute(
     "insert into table (q) values (?)", [Binary(u'some
unicode'.encode('utf-8'))]
)


The latter bit of code in fact can work on *both* systems.  The first
bit of code, cannot.  By disallowing the first style, the developer is
encouraged to write portable code.  By allowing it, developers are led
into writing code that is not portable.

This is portability in a nutshell, and I'm sure you understand this.
The emphasis pep-249 puts on "flexibility" is often at the cost of
encouraging "portability", even in very obvious cases like this one, is
the philosophy I find questionable, being that this is Python and not Perl.




> 
> Back to the choices I mentioned for Binary():
> 
> In Python 2, buffer() does allow unicode objects
> on input, and what you as result corresponds to the binary
> representation of the unicode object as used by Python.


The buffer() type produces Python's internal representation of the
unicode data; it does not, for example, try to first encode the unicode
object based on the platform charset.   For the use case people expect
when converting unicode objects to bytes, the behavior of buffer() is
completely surprising:

>>> list(buffer('f'))
['f']
>>> list(buffer(u'f'))
['f', '\x00', '\x00', '\x00']

The latter is the utf-32 representation that my Python interpreter is
using.  Other Python interpreters, like the one on my mac, return a
utf-16 representation:

>>> list(buffer(u'f'))
['f', '\x00']

neither the utf-16 or utf-32 formats are commonly used with database
applications as a transport format; utf-8 is vastly more common.  It
should not be controversial that exposing the internal Python Unicode
format, which isn't even portable across individual Python builds, is
entirely inappropriate for any kind of data exchange between systems.

That the buffer() construct was removed in Python 3 and replaced with
memoryview() which does not accept strings at all is part of the bigger
story that Python 2's unicode story, including the behavior of buffer(),
was considered to be a design mistake.

> 
> In Python 3, bytes() requires to be more specific and you
> need to provide an encoding. The result is an encoded binary
> version of the text input.
> 
> Both are reasonable choices for a Binary() wrapper and
> the result is easy to detect as "bind me as binary data"
> for the database module.
> 
> It is not uncommon to convert text data to
> binary data for storage, esp. when dealing with larger
> blobs you just want to manage and not work on,

that's not controversial.  But having the driver make the following guess:

1. unicode passed, use buffer() to return the Python interpreter's
internal representation of it (or utf-32 always, or utf-16 always, or.. ?)

2. unicode passed, try to encode with utf-8 because that's "probably"
want someone wants

3. unicode passed, try to encode with the particular charset that's
configured with this driver (if the driver supports configured charsets
at all)

I would argue this scheme works against portability, because it is
impossible to avoid choosing an arbitrary conversion scheme.   This
arbitrary-ness is a design mistake that Python 3 repaired.


 or when
> you want to preserve it in exactly the same form you pass
> it to the database (without any implicit normalizations,
> surrogate conversions, warnings, etc.).

The "form" of Python Unicode data outside of the interpreter is
essentially undefined.  This would mean one is storing data in their
database that not only can't be read portably between major versions of
Python, it can't be read portably between *builds* of the same Python
version.   Developers would vastly prefer their application raise an
error rather than implicitly pass the internal Unicode format of data to
their database, since this is not a use case anyone has.

From vernondcole at gmail.com  Fri Jan 15 17:14:59 2016
From: vernondcole at gmail.com (Vernon D. Cole)
Date: Fri, 15 Jan 2016 15:14:59 -0700
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <56992A9D.6080509@zzzcomputing.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com> <569915B3.2000609@zzzcomputing.com>
 <56991B38.1090607@egenix.com> <56992A9D.6080509@zzzcomputing.com>
Message-ID: <CAH-ZgAch3V_Ouj2k+Zj+msK6ZwEwAFtrCigDjHFG1xqy=t2hGg@mail.gmail.com>

Mike:

  Thank you for your long explanation. I got lost somewhere in the
middle there, though.

If you are suggesting that better documentation be added to PEP-249,
then perhaps you could include a suggestion as to a (brief) note which
could be appended.

If you are suggesting that the PEP be expanded to provide a service
not now generally available, then perhaps you ought to start the
long-talked-about-but-never-tried task of writing a DBAPI level 3 PEP.
The present PEP (written long before Python 3) is lacking (and in at
least one case, actually wrong) any specific Python 3 implementation
details. Several enhancements and improvements to the PEP have been
talked about, most of which -- like the one you seem to be making --
would significantly enhance a programmer's experience. There are also
several "standard extensions" and frequently added extras which could
be included in the body of a new PEP.

From mike_mp at zzzcomputing.com  Sat Jan 16 10:27:58 2016
From: mike_mp at zzzcomputing.com (Mike Bayer)
Date: Sat, 16 Jan 2016 10:27:58 -0500
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <CAH-ZgAch3V_Ouj2k+Zj+msK6ZwEwAFtrCigDjHFG1xqy=t2hGg@mail.gmail.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com> <569915B3.2000609@zzzcomputing.com>
 <56991B38.1090607@egenix.com> <56992A9D.6080509@zzzcomputing.com>
 <CAH-ZgAch3V_Ouj2k+Zj+msK6ZwEwAFtrCigDjHFG1xqy=t2hGg@mail.gmail.com>
Message-ID: <569A617E.2000708@zzzcomputing.com>



On 01/15/2016 05:14 PM, Vernon D. Cole wrote:
> Mike:
> 
>   Thank you for your long explanation. I got lost somewhere in the
> middle there, though.
> 
> If you are suggesting that better documentation be added to PEP-249,
> then perhaps you could include a suggestion as to a (brief) note which
> could be appended.


Well, I wasn't even going that far.  I'm trying to get a handle on what
pep-249's position is as far as portability of datatypes.   It has
always struck me as very weak.



> 
> If you are suggesting that the PEP be expanded to provide a service
> not now generally available, then perhaps you ought to start the
> long-talked-about-but-never-tried task of writing a DBAPI level 3 PEP.

Well if a DBAPI driver would like to accept a Python unicode object to a
Binary() and produce bytes, some conversion is needed, and there are
many possible conversions that could take place - there is every
possible encoding, and at typically at least four potential candidates
among those available.

It's my position that the Binary() type should *not* offer to
automatically choose such a conversion and should only accept Python
types (or 3rd party extension types, sure) that are explicitly 1-1
mappable to a stream of bytes without a "conversion decision" being
made.  The type of conversion should not be guessed among a choice of
several / hundreds within the Binary type.

So definitely, not proposing any new service other than "disallow
ambiguous input".


> The present PEP (written long before Python 3) is lacking (and in at
> least one case, actually wrong) any specific Python 3 implementation
> details. Several enhancements and improvements to the PEP have been
> talked about, most of which -- like the one you seem to be making --
> would significantly enhance a programmer's experience. There are also
> several "standard extensions" and frequently added extras which could
> be included in the body of a new PEP.

I'd love to write the pep for DBAPI3.   That task doesn't daunt me, but
the task of evaluating reactions and changes to it and shepherding it to
acceptance certainly does.   I seem to generally be on the other side of
most issues on this list and the task would be more surmountable if I
had some allies to assist during the long process of defense.

that said if we could at least produce / point me to a document that
lists out the general things people are looking for, for me I know its
fewer bindparam formats and explicitness about how unicode / strings
should be handled, also there's the prepared statement thing.  Oh, and
the tuple/list thing for result rows that I think I'm the only person
who cares about that.





> 

From mal at egenix.com  Sat Jan 16 14:30:56 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 16 Jan 2016 20:30:56 +0100
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <569A617E.2000708@zzzcomputing.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com> <569915B3.2000609@zzzcomputing.com>
 <56991B38.1090607@egenix.com> <56992A9D.6080509@zzzcomputing.com>
 <CAH-ZgAch3V_Ouj2k+Zj+msK6ZwEwAFtrCigDjHFG1xqy=t2hGg@mail.gmail.com>
 <569A617E.2000708@zzzcomputing.com>
Message-ID: <569A9A70.8010608@egenix.com>

On 16.01.2016 16:27, Mike Bayer wrote:
> 
> 
> On 01/15/2016 05:14 PM, Vernon D. Cole wrote:
>> Mike:
>>
>>   Thank you for your long explanation. I got lost somewhere in the
>> middle there, though.
>>
>> If you are suggesting that better documentation be added to PEP-249,
>> then perhaps you could include a suggestion as to a (brief) note which
>> could be appended.
> 
> 
> Well, I wasn't even going that far.  I'm trying to get a handle on what
> pep-249's position is as far as portability of datatypes.   It has
> always struck me as very weak.
> 
> 
> 
>>
>> If you are suggesting that the PEP be expanded to provide a service
>> not now generally available, then perhaps you ought to start the
>> long-talked-about-but-never-tried task of writing a DBAPI level 3 PEP.
> 
> Well if a DBAPI driver would like to accept a Python unicode object to a
> Binary() and produce bytes, some conversion is needed, and there are
> many possible conversions that could take place - there is every
> possible encoding, and at typically at least four potential candidates
> among those available.
> 
> It's my position that the Binary() type should *not* offer to
> automatically choose such a conversion and should only accept Python
> types (or 3rd party extension types, sure) that are explicitly 1-1
> mappable to a stream of bytes without a "conversion decision" being
> made.  The type of conversion should not be guessed among a choice of
> several / hundreds within the Binary type.
> 
> So definitely, not proposing any new service other than "disallow
> ambiguous input".

I still don't understand why you want to restrict Binary()
to perform automatic conversions on the input types.

Unicode is just one example of where you can implement such
conversion, e.g. a database module may want to automatically
convert Unicode to UTF-8. For database backends which don't
provide Unicode support, this is usually also being done
for string parameter types.

mxODBC, for example, allows setting a per connection .encoding
attribute to define which encoding to use in such cases.

But again, Unicode is just one example. Binary() may also
apply automatic conversions for other types, such as images,
numeric arrays, etc.

The DB-API standard cannot define which types to autoconvert
and which not. This is a conscious decision left to the database
module authors.

They have to make similar choices for all other parameter
types as well, e.g. whether to convert datetime values to
strings, ticks or whether to reject them.

In many cases, the database backends don't provide parameter
type information, so the database module has to decide what
to do. In other cases, the database module may get type information
from the database and then has to decide what to do with the
input parameters passed to it from Python.

Back to Binary(): What we could do is recommend to use e.g.
buffer() for Python 2 as default implementation and bytes()
for Python 3.

The fact that buffer() does accept Unicode objects in Python 2
is due to the way the buffer interface works in Python 2 (in 2000
we thought it would be a good idea to allow access to
the UCS-2 data; later on, when we added UCS-4 support, we
could not easily remove this feature anymore).

For Python 3, bytes() won't accept Unicode because the buffer
interface was changed in Python 3 to no longer expose the
binary buffer interface.

Would that make you happy ? :-)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mike_mp at zzzcomputing.com  Sat Jan 16 22:51:54 2016
From: mike_mp at zzzcomputing.com (Mike Bayer)
Date: Sat, 16 Jan 2016 22:51:54 -0500
Subject: [DB-SIG] Should Binary accept unicode string?
In-Reply-To: <569A9A70.8010608@egenix.com>
References: <CAEfz+TwbFzqyGdEQH1v+RE5tx6YXxdVaTM40n5EL-8WAp363Sw@mail.gmail.com>
 <5699066B.4080004@egenix.com> <569915B3.2000609@zzzcomputing.com>
 <56991B38.1090607@egenix.com> <56992A9D.6080509@zzzcomputing.com>
 <CAH-ZgAch3V_Ouj2k+Zj+msK6ZwEwAFtrCigDjHFG1xqy=t2hGg@mail.gmail.com>
 <569A617E.2000708@zzzcomputing.com> <569A9A70.8010608@egenix.com>
Message-ID: <569B0FDA.8090006@zzzcomputing.com>



On 01/16/2016 02:30 PM, M.-A. Lemburg wrote:
> On 16.01.2016 16:27, Mike Bayer wrote:
>>
>>
>> On 01/15/2016 05:14 PM, Vernon D. Cole wrote:
>>> Mike:
>>>
>>>   Thank you for your long explanation. I got lost somewhere in the
>>> middle there, though.
>>>
>>> If you are suggesting that better documentation be added to PEP-249,
>>> then perhaps you could include a suggestion as to a (brief) note which
>>> could be appended.
>>
>>
>> Well, I wasn't even going that far.  I'm trying to get a handle on what
>> pep-249's position is as far as portability of datatypes.   It has
>> always struck me as very weak.
>>
>>
>>
>>>
>>> If you are suggesting that the PEP be expanded to provide a service
>>> not now generally available, then perhaps you ought to start the
>>> long-talked-about-but-never-tried task of writing a DBAPI level 3 PEP.
>>
>> Well if a DBAPI driver would like to accept a Python unicode object to a
>> Binary() and produce bytes, some conversion is needed, and there are
>> many possible conversions that could take place - there is every
>> possible encoding, and at typically at least four potential candidates
>> among those available.
>>
>> It's my position that the Binary() type should *not* offer to
>> automatically choose such a conversion and should only accept Python
>> types (or 3rd party extension types, sure) that are explicitly 1-1
>> mappable to a stream of bytes without a "conversion decision" being
>> made.  The type of conversion should not be guessed among a choice of
>> several / hundreds within the Binary type.
>>
>> So definitely, not proposing any new service other than "disallow
>> ambiguous input".
> 
> I still don't understand why you want to restrict Binary()
> to perform automatic conversions on the input types.

it would be consistent with the philosophy of Python 3 itself that
unicode and "bytes" are two different things without an implicit
conversion.   It's also a place that without clear guidance in the pep,
some DBAPIs are going to do, and others not, leading to non-portable code.

Basically, if pep-249 said, "The Binary() object should accept Python
unicode objects and should encode them to bytes using an encoding
indicated by the .encoding attribute on Connection", that would be
better than the current situation of that it says nothing at all.

It's the "it says nothing at all" part here that's more troubling to me,
rather than whether or not a Unicode should be encoded when passed to a
Binary().   If the spec had a recommendation for where this encoding
should come from, then that allows portable code to be written.

> 
> Unicode is just one example of where you can implement such
> conversion, e.g. a database module may want to automatically
> convert Unicode to UTF-8. For database backends which don't
> provide Unicode support, this is usually also being done
> for string parameter types.

it is true that even in Python 3, DBAPIs are obviously taking on the
task of figuring out an appropriate encoding to use for strings, which
are unicode objects in Python 3.   It strongly suggests that pep-249 or
its successor would be served by referring to an "encoding" setting in a
standard way.


> 
> mxODBC, for example, allows setting a per connection .encoding
> attribute to define which encoding to use in such cases.
> 
> But again, Unicode is just one example. Binary() may also
> apply automatic conversions for other types, such as images,
> numeric arrays, etc.
> 
> The DB-API standard cannot define which types to autoconvert
> and which not. This is a conscious decision left to the database
> module authors.
> 
> They have to make similar choices for all other parameter
> types as well, e.g. whether to convert datetime values to
> strings, ticks or whether to reject them.

> In many cases, the database backends don't provide parameter
> type information, so the database module has to decide what
> to do. In other cases, the database module may get type information
> from the database and then has to decide what to do with the
> input parameters passed to it from Python.
> 
> Back to Binary(): What we could do is recommend to use e.g.
> buffer() for Python 2 as default implementation and bytes()
> for Python 3.

I think Binary should certainly accept str in Py2K, since that's the
normal place we get "bytes" from in Py2K.  If it wants to accept a
buffer() also, that's fine (because no guess needs to be made), I doubt
anyone will use it though as its a deprecated type and they can just use
str.

> 
> The fact that buffer() does accept Unicode objects in Python 2
> is due to the way the buffer interface works in Python 2 (in 2000
> we thought it would be a good idea to allow access to
> the UCS-2 data; later on, when we added UCS-4 support, we
> could not easily remove this feature anymore).

yeah UCS-2 / UCS-4, not too useful in the outside world these days :)

> 
> For Python 3, bytes() won't accept Unicode because the buffer
> interface was changed in Python 3 to no longer expose the
> binary buffer interface.
> 
> Would that make you happy ? :-)

it's all great!  I only seek to understand the thinking behind DBAPI's
philosophy in areas like these.


> 

From madhusudhan.achar at oracle.com  Fri Jan 22 03:45:03 2016
From: madhusudhan.achar at oracle.com (madhusudhan achar)
Date: Fri, 22 Jan 2016 14:15:03 +0530
Subject: [DB-SIG] Need clarifications/help on preserving revisions of
 files/permissions using Python
Message-ID: <56A1EC0F.2000409@oracle.com>

Hello,

My current problem is :

I have scripts written currently in python, wherein we make lot of 
changes to files (system), permission changes to
files and directories, changes to kernel parameters. We would like to 
maintain the revisions of these changes and should
be able tor rollback the changes at any point in time.

The initial thought is to use MySQLdb.

My questions are:
1. Provide me sample references to such problems if they exist.
2. How do we preserve changes to files? we need to save the entire files.
3. We need to make permission changes of files/directories (owner/group 
changes)
4. How do we store the changes/files? Should they stored as tables?
5. How do we restore the stored tables and restore files and file/dir 
permissions..?

I shall look forward to your help/suggestions...

thanks
Madhu

From mal at egenix.com  Mon Jan 25 06:16:46 2016
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 25 Jan 2016 12:16:46 +0100
Subject: [DB-SIG] Need clarifications/help on preserving revisions of
 files/permissions using Python
In-Reply-To: <56A1EC0F.2000409@oracle.com>
References: <56A1EC0F.2000409@oracle.com>
Message-ID: <56A6041E.4080800@egenix.com>

Hello Madhu,

I am not sure how your posting is relevant to this mailing
list. Most of your questions seem to have to do with OS
permissions and storing/restoring files.

You can store the necessary details in a database, but the
actual logic for managing the files themselves will have
to be done using e.g. the os module.

You might also want to look at a configuration system such as
Ansible or SaltStack for managing your systems.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Jan 25 2016)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...           http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...           http://zope.egenix.com/
________________________________________________________________________

::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/
                      http://www.malemburg.com/



On 22.01.2016 09:45, madhusudhan achar wrote:
> Hello,
> 
> My current problem is :
> 
> I have scripts written currently in python, wherein we make lot of changes to files (system),
> permission changes to
> files and directories, changes to kernel parameters. We would like to maintain the revisions of
> these changes and should
> be able tor rollback the changes at any point in time.
> 
> The initial thought is to use MySQLdb.
> 
> My questions are:
> 1. Provide me sample references to such problems if they exist.
> 2. How do we preserve changes to files? we need to save the entire files.
> 3. We need to make permission changes of files/directories (owner/group changes)
> 4. How do we store the changes/files? Should they stored as tables?
> 5. How do we restore the stored tables and restore files and file/dir permissions..?
> 
> I shall look forward to your help/suggestions...
> 
> thanks
> Madhu
> _______________________________________________
> DB-SIG maillist  -  DB-SIG at python.org
> https://mail.python.org/mailman/listinfo/db-sig