[code-quality] RedBaron, a bottom-up refactoring lib/tool for python

[Dave Halter]

2014-11-17 21:00 GMT+01:00 Laurent Peuch <cortex at worlddomination.be>:

> Hello everyone and thanks for your answers :)
>
> > > > I'm currently working on the integration of the lib2to3 parser into
> Jedi.
> > > > This would make refactoring really easy (I'm about 50% done with the
> > > > parser). It's also well tested and offers a few other advantages.
> > > >
> > > > In a perfect world, we could now combine our projects :-) I will
> look in
> > > > detail at Red Baron on Monday.
> > >
> > > David, we've been talking about this during the latest EuroPython, and
> I've
> > > talked with Laurent yesterday at the Capitole du Libre in Toulouse:
> IMO we
> > > could
> > > start by extracting from lib2to3 "the" parser that could be used by
> every
> > > tools
> > > like ours (refactoring, completion, static analysis...). It would be:
> > > * loss-less (comments, indents...)
> > > * accurate (eg from/to line numbers)
> > > * fast
> > > * version agnostic within a reasonable frame (eg 2.7 -> 3.4?)
> > >
> >
> > Yes, that's what I'm trying to create now (based on lib2to3). However my
> > biggest problem is that I have to rewrite my evaluation engine as well,
> > because it was depending on the old parser. I have two additional
> > constraints:
> >
> > - decent error recovery
> > - memory efficient
> >
> > The "fast" part is something I'm very eager to implement. I have done
> this
> > before with my old parser. My approach is to re-parse only the parts of
> the
> > file that have changed.
> > My idea of "version agnostic" is to have the parser and the evaluation
> > engine try to adhere to one version. My goal is to give Jedi a Python
> > version number and Jedi would work according to that. This would make
> > linting really cool, because you can run the linter for different Python
> > versions.
> >
> > I would recommend you to wait until I have finished my parser (1-2
> months)
> > or can at least report back. You can then either take the parser and fork
> > it or take the evaluation engine as well.
>
> Well, wouldn't it be a good idea if instead of us waiting 1-2 months
> that you finish your parser then seeing if it also fits our needs we
> discuss a bit of what each one of us needs and see if we could find an
> agreement on what could be a common ST for all of us?
>
> I can totally understand that discussing this right now might not be
> the most appealing idea but maybe think a bit about it of the benefit
> that we could get from a common AST for all of us:
>
> * only one code base to maintain instead of everyone doing his
>   own parser on his side
> * we could share our efforts on making it as good as possible
> * more time spend on doing the actual tools than the backend
> * a de facto reference for every python developer that wants to join
>   the field of tooling: more people -> more tools -> better python in
>   general
>
> I really think that we should at least try to see if it's possible,
> having this kind of tool would really benefit us and the python
> community in general.
>
> What do you think?
>

I totally agree. However the Jedi's parser right now (parser branch) is in
a very confusing state. There's still a lot of old code lingering around
that doesn't help a lot in understanding it. I'm going to clean it until
the end of this week and then we can talk about it.

The full 1-2 months would include Jedi being fast again + passing all
tests. So I don't think we need to wait for that. I'm working full time on
this project so I'm progressing quite quickly.

~ Dave

>
> --
>
> Laurent Peuch -- Bram
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141118/7f916ed1/attachment.html>