[pypy-dev] [Python-ideas] Re: Explicitly defining a string buffer object (aka StringIO += operator)

[Paul Sokolovsky]

Hello,

On Mon, 30 Mar 2020 23:20:28 -0400
David Mertz <mertz at gnosis.cx> wrote:

> I have myself been "guilty" of using the problem style for N < 10.  In
> fact, I had forgotten about the optimization even, since my uses are
> negligible time.

I personally don't think it's a "problem style" per se. I'm using it all
the time. I'm going to keep using it. We all love Python for being a
language which you can easily prototype in, and don't have to
concern yourself with implementation details unless you want. My
concern for Python to be a language which can be progressively and
easily optimized when you need it.

Going over the code and changing many lines along the lines of diff:

- buf += piece
+ buf.append(piece)

isn't my idea of "seamless" optimization, especially that I know
it guaranteedly will grow my memory usage.

Sadly I came to conclusion that even

- buf += piece
+ buf.write(piece)

patching hurts my aesthetic feelings. At least, that's the only
explanation I have for why in one of my modules
https://github.com/pfalcon/pycopy-lib/blob/master/utokenize/utokenize.py#L44
I converted one loop from "str +=" to "StringIO.write()", but not
another. I then tried to think what could help with that and having +=
on StringIO seemed to do the trick.

> 
> For stuff like this, it's fast no matter what:
> 
> for clause in query_clauses:
>     sql += clause
> 
> Maybe I have a WHERE or two.  Maybe an ORDER BY.  Etc.  But if I'm
> sure there won't be more than 6 such clauses to the query I'm
> building, so what? Or probably likewise with bits of a file path, or
> a URL with optional parameters, and a few other things.
> 
> On Mon, Mar 30, 2020 at 11:15 PM David Mertz <mertz at gnosis.cx> wrote:
> 
> > Does anyone know if any linters find and warn about the `string +=
> > word` in a loop pattern? It feels like a linter would be the place
> > to do that.  I don't think we could possibly make it an actual
> > interpreter warning given borderline OK uses (or possibly even
> > preferred ones).  But a little nagging in tooling could draw
> > attention.

[]

-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com