[Python-ideas] Extending language syntax

[Andrew Barnert]

On Nov 11, 2013, at 21:03, Gregory Salvan <apieum at gmail.com> wrote:

> Thank you.
> 
> You're almost right, but there is misunderstanding.
> 
> I suggested generating value object by freezing object states and to have object identities defined on their values instead of their memory allocation.
> so :
> pi1 = keyword(3.14)
> pi2 = keyword(3.14)
> assert pi1 is pi2  # is true

So effectively you want to add interning for arbitrary types.

So... why? 

You won't get any direct performance benefits from being able to use is instead of == here--comparing two floats is as fast as comparing two pointers on most platforms. And why would you want users to compare with is instead of ==? It means everyone who uses pi has to know that it's a keyword, even though he gets no real benefit from doing so.

There might be some indirect benefits in that we could skip refcounting on objects that are guaranteed to live forever. But you could get the same benefits with a less drastic change--basically, just a way to declare an object as "permanent", without changing its semantics in any other way. That sub-idea might be worth exploring.

> and:
> you = Person('Andrew Barnert')
> andrew = keyword(you)
> andrew.name = 'Dr. Sam Beckett'  # will raise an error.

That's already easy today. For example, you can declare name as a @property with no setter, or inherit from namedtuple.


> It is value object not singleton because pi1 and pi2 are not of the same instance.

Why not? In fact, that seems to lose most of the benefits of interning. Effectively you've just created normal objects that can override the is operator, which I think is a bad idea.

> This matter when doing concurrency but this is not the point as you can cheat with "repr" or "hash".
> 
> I would focus mainly on how to extend language syntax and permit writing code like:
> 'abc' should_equal 'abc'

I'm not sure how this fits in with the rest of your idea at all. That requires changing the parser at runtime, which has nothing to do with objects being permanent or their is operators being overridable to mean equality or anything else.

Meanwhile, you should take a look at how parsing and compiling works in Python (the dev guide has a great section on it). Modules are parsed and compiled, and then the bytecode is run at import time. If that code can change the grammar, how can we use compiled bytecode at all? Without an implementation that compiles and interprets statement by statement on the fly, it seems like this would be very difficult.

You may want to look at MacroPy, which allows making (less dramatic) changes to the language syntax via import hooks.

> Value object seems to me necessary to avoid side effects.

I don't think I understand why, if by "value object" you mean "object with a custom is operator". If you just mean "immutable object", that sounds more reasonable (although I'm still not sure i get it), but again, we already have those. A float, for example, is already immutable today.

> I miss your point about marshal/pickle and will dig it to understand.
> I hope in future I can be more relevant.
> 
> Sorry for the inconvenience, thanks for taking the time to answer,
> good continuation.
> 
> 
> 
> 
> 2013/11/11 Andrew Barnert <abarnert at yahoo.com>
>> I could go through this point by point, but I think I can head it all off by explaining what I think is a fundamental misconception that's misleading you.
>> 
>> In Python, variables aren't "memory locations" where values live, they're just names that are bound to values that live somewhere on their own. Rebinding a variable to a different value doesn't mutate anything. You can't make variables refer to other variables, and making two variables refer to the same value doesn't have the same referential transparency, thread safety, etc. issues as making a variable refer to another variable. For example, in the following code:
>> 
>>     me = Person('Andrew Barnert')
>>     andrew = me
>>     me = Person('Dr. Sam Beckett')
>> 
>> ... nothing has been mutated. In particular, the andrew variable is unchanged; it's still referring to the same Person('Andrew Barnert') object as before, not the new one. 
>> 
>> If I make the Person type mutable, and call a mutating method on it (or do so implicitly, by setting an attribute or a keyed or indexed member), then of course I can change the value, which will be visible to all variables referring to that value. But assignment does not do that.
>> 
>> So, you already have almost everything you want. You can create new immutable types and global singleton values of those types. Or create Enums. Or just create singleton objects whose type doesn't matter, which are immutable and compared by identity, just by calling object(). All of this is trivial.
>> 
>> Taking one of your examples:
>> 
>>     Pi = 3.14159
>> 
>> ... gives you everything you wanted, except for the fact that you can accidentally or maliciously rebind it to a new value. It's already an immutable, thread-safe constant.
>> 
>> You can even make a variable name un-rebindable with a module import hook that produces a custom globals, if you really want to, although I can't imagine that ever being worth doing.
>> 
>> The only additional thing a keyword gives you is that the compiler can prevent you from rebinding the name to a different value, at compile time rather than run time. That's it.
>> 
>> Meanwhile, the disadvantage of allowing new keywords to be defined for immutable constants is huge. You could no longer compile, or even parse, any code without checking each token against the current runtime environment. That makes the parser slower and more complicated, introduces a dependency that makes it hard to keep the components separate, makes pyc files and marshal/pickle and so on useless, makes it much harder for code tools to use the ast module, etc. All for a very tiny benefit.
>> 
>> Sent from a random iPhone
>> 
>> On Nov 11, 2013, at 3:20, Gregory Salvan <apieum at gmail.com> wrote:
>> 
>>> Hi all,
>>> I've just joined you, I hope I'll not make you loose your time with questions already covered.
>>> Sorry, if my suggest represents lot of work (certainly too much) and about my lack of knowledge about internal implementation, but I hope opening a discussion can help.
>>> 
>>> I would suggest to introduce a new statement to provide a way to extend language syntax.
>>> 
>>> Abstract:
>>> While reading this article about None history I realised I was not alone to wonder about some aspect of the language.
>>> 
>>> Some of these are:
>>> - constant and immutability
>>> - null object and "None is a singleton" 
>>> - readability of code
>>> 
>>> Constant:
>>> It's a convention, no matter with that. 
>>> My question is more if there is no memory overuse to have fixed values that behave as variable.
>>> 
>>> Null Object:
>>> NoneType is not overridable.
>>> Having null objects instance of NoneType wouldn't make sense ?
>>> Can't assign keywords so "None is a singleton".
>>> Wouldn't it be more simple to have "keyword" type and be able to assign a keyword to a variable ?
>>> With an object hierarchy like: keyword <- ReservedKeyword
>>> I find it can make clearer why some keywords are reserved without changing actual behaviour.
>>> 
>>> Readability of code:
>>> I have shouldDsl use case in mind.
>>> The introduction illustrate my thought:
>>> "The goal of Should-DSL is to write should expectations in Python as clear and readable as possible, using an “almost” natural language (limited by some Python language’s constraints)."
>>> 
>>> I find ShouldDsl perturbing, as the aim is laudable but the implementation stays a hack of the language.
>>> I see the influence of functional paradigm in this syntax and agreed with the fact it becomes a "must have".
>>> 
>>> I love Python syntax and find, it has very few limits, but maybe it can be improved by reducing some constraints.
>>> 
>>> One concerns in extending syntax is that the language can evolve on users usage.
>>> I see it like an open laboratory where developpers experiments new keywords like "should" implemented in their favorite language and when this keyword is largely accepted by the community it can finally integrate the core.
>>> 
>>> A solution:
>>> I was first thinking of a statement like "Instance" to declare an immutable object with a single instance (like None), but I then consider how it would be great to be able to extend the syntax.
>>> I thought the keyword "Keyword", which exists in clojure for example, can do the job.
>>> 
>>> A first implementation can be:
>>> 
>>> Keyword <name>(<keyword, literal or object>):
>>>   field = <something>  # fixed value: self.field = "something" will raise an error
>>>   def behaviour(self, *args, **kwargs):
>>>     # do something
>>>   def __get__(self, left, right, block_content):
>>>     # what to do when accessing keyword
>>> 
>>> Inline implementation:
>>> Keyword <name>(<keyword, literal or object>)
>>> 
>>> 
>>> Examples of use:
>>> 
>>> Constant:
>>>   Keyword Pi(3.14159)
>>> 
>>> Null Object and immutable objects:
>>>   Class Person(object):
>>>     def __init__(self, name):
>>>       self.name = name
>>> 
>>>   JohnDoe = Person(name="anonymous")
>>> 
>>>   Keyword Anonymous(None, JohnDoe)
>>> 
>>>   Anonymous.name = "JohnDoe"  # raise an error
>>>   assert isinstance(Anonymous, type(None))  # is true
>>>   
>>> 
>>> As Anonymous is immutable , it is also thread safe.
>>> The mecanism which consist of freezing an object can be applied in other situation.
>>> 
>>> An interesting approach can be to define keywords identity throught a hash of their value, like value object.
>>> 
>>> Should keyword:
>>> 
>>> Keyword ShouldEqual(keyword):
>>>   def __get__(self, left, right, block):
>>>     if left != right:
>>>       raise AssertError()
>>> 
>>> ### weird examples:
>>> 
>>> Multiline Lambdas:
>>> Keyword MultilineLambda(keyword):
>>>   def __get__(self, left, right, block_content):
>>>     def multiline_lambda(*args, **kwargs):
>>>       self.assert_args_valid(right, args, kwargs)  # compare "right" (tuple) with args and raise exception in case not valid
>>>       return self.run(block_content, args, kwargs)  # "run" is a shortcut because I don't know which type is the most appropriate for block content
>>>     return multiline_lambda
>>> 
>>>   divide = MultilineLambda a, b:
>>>     if b == 0:
>>>        raise MyDivideByZeroError()
>>>     return a/b
>>> 
>>> Reimplementing else:
>>> Keyword otherwise(keyword):
>>>   def __get__(self, left, right, block):
>>>     if left: return  # left is the result of "if" block
>>>     self.run(block)
>>> 
>>> if something:
>>>   # do something
>>> otherwise:
>>>   # other
>>> 
>>> Whereas this example is weird, it seems to me that we can better figure out what language do.
>>> 
>>> 
>>> Thanks all to have kept python open and give us opportunities to freely discuss about its future implementation.
>>> 
>>> Have a nice day,
>>> Grégory
>>> 
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
> 
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20131112/690c743c/attachment-0001.html>