[Python-Dev] Subtle difference between f-strings and str.format()

[Tim Peters]

[Steven D'Aprano <steve at pearwood.info>]
> ...
> Is there a down-side to 2b? It sounds like something you might end up
> doing at a later date regardless of what you do now.

There are always downsides ;-)

As Serhiy noted later, the idea that "it's faster" is an educated
guess - you can't know before it's implemented.  Changes to the very
complicated eval loop often have not only surprising speed
consequences on one platform, but even consequences in opposite
directions across platforms.  Not necessarily in the part you directly
changed, either.  Optimizing C compilers just can't reliably guess
what's most important in such a massive pile of test-and-branch laden
code.  Indeed, which paths through the eval loop _are_ most important
depend on the Python program you're running at the time (which is,
e.g., why "profile-guided optimization" was invented).

So there's an ocean of potential complications there, and wading
through those has opportunity cost too:  Serhiy is a very productive
contributor, but time he spends on this is time he won't be spending
on other things of potentially greater value.  That's all up to him,
though.

I'm not keen on changing the behavior of f-strings regardless (2a or
2b).  While their implementation details aren't documented, they were
intentional, and follow the pattern increasingly large parts of the
language and std library adopted after the iterator protocol was
introduced:  compute intermediate results as they're needed, not all
in advance.  That's proved to have many advantages.

It's certainly possible to write custom purely functional (no side
effects) __format__ methods such that memory use in an f-string
remains bounded under the current implementation, but can grow without
bound if all __format__ arguments need to be evaluated before any
formatting begins.  It's akin to the difference between iterating over
range() and xrange() in Python 2.

I don't know that there's any real f-string code out there _relying_
on that - but don't know that there isn't either.  It's more plausible
to me than that there are non-functional real __format__ methods.

I'd be happiest if no behaviors changed in anything.  Then the only
downsides to optimizing are code bloat, code churn, new bugs, subtler
corner cases, less predictable behavior for end users, and increased
implementation complexity forever after ;-)