[Python-Dev] Subtle difference between f-strings and str.format()

[Eric V. Smith]

On 3/29/2018 6:17 AM, Jeff Allen wrote:
> My credentials for this are that I re-worked str.format in Jython quite 
> extensively, and I followed the design of f-strings a bit when they were 
> introduced, but I haven't used them to write anything.

Thanks for your work on Jython. And hop on the f-string bandwagon!

> The difference Serhiy identifies emerges (I think) because in the 
> conventional interpretation of a format call, the arguments of format 
> are evaluated left-to right (all of them) and then formatted in the 
> order references are encountered to these values in a tuple or 
> dictionary. In an f-string expressions are evaluated as they are 
> encountered. A more testing example is therefore perhaps:
> 
>      '{1} {0}'.format(a(), b()) # E1
> 
>      f'{b()}{a()}'              # E2
> 
> 
> I think I would be very surprised to find b called before a in E1 
> because of the general contract on the meaning of method calls. I'm 
> assuming that's what an AST-based optimisation would do? There's no 
> reason in E2 to call them in any other order than b then a and the 
> documentation tells me they are.
> 
> But do I expect a() to be called before the results of b() are 
> formatted? In E1 I definitely expect that. In E2 I don't think I'd be 
> surprised either way. Forced to guess, I would guess that b() would be 
> formatted and in the output buffer before a() was called, since it gives 
> the implementation fewer things to remember. Then I hope I would not 
> depend on this guesswork. Strictly-speaking the documentation doesn't 
> say when the result is formatted in relation to the evaluation of other 
> expressions, so there is permission for Serhiy's idea #2.

I don't think we should restrict f-strings to having to evaluate all of 
the expressions before formatting. But, if we do restrict it, we should 
document whatever the order is in 3.6 and add tests to ensure the 
behavior doesn't change.

> I think the (internal) AST change implied in Serhiy's idea #1 is the 
> price one has to pay *if* one insists on optimising str.format().
> 
> str.format just a method like any other. The reasons would have to be 
> very strong to give it special-case semantics. I agree that the cases 
> are rare in which one would notice a difference. (Mostly I think it 
> would be a surprise during debugging.) But I think users should be able 
> to rely on the semantics of call. Easier optimisation doesn't seem to me 
> a strong enough argument.
> 
> This leaves me at:
> 1: +1
> 2a, 2b: +0
> 3: -1

#1 seems so complex as to not be worth it, given the likely small 
overall impact of the optimization to a large program. If the speedup 
really is sufficiently important for a particular piece of code, I'd 
suggest just rewriting the code to use f-strings, and the author could 
then determine if the transformation breaks anything. Maybe write a 2to3 
like tool that would identify places where str.format or %-formatting 
could be replaced by f-strings? I know I'd run it on my code, if it 
existed. Because the optimization can only work code with literals, I 
think manually modifying the source code is an acceptable solution if 
the possible change in semantics implied by #3 are unacceptable.

Eric.