[code-quality] RedBaron, a bottom-up refactoring lib/tool for python

[Dave Halter]

Hi Laurent

Great to see somebody finally tackling refactoring.

I'm answering, because I think we're working on the same issue. But we have
finished two different parts: You have finished a refactoring
implementation and I have finished the static analysis part. I'm the author
of Jedi. https://github.com/davidhalter/jedi/

I'm currently working on the integration of the lib2to3 parser into Jedi.
This would make refactoring really easy (I'm about 50% done with the
parser). It's also well tested and offers a few other advantages.

In a perfect world, we could now combine our projects :-) I will look in
detail at Red Baron on Monday.

~ Dave

PS: If you want to use a tool for static analysis, please use either Jedi
or astroid, I don't think rope still makes sense, because the project is
inactive.

2014-11-14 13:05 GMT+01:00 Laurent Peuch <cortex at worlddomination.be>:

> Hello everyone,
>
> Someone has suggested me to talk about the project I'm working on
> right now on this mailing list because this has a lot of chances to
> interest you.
>
> This tool is an answer to a frustration that I've had while trying to
> build tools for python and projects I was working on. While there is
> already good capacities in python to analyse code (ast.py, astroid
> (while it wasn't out at that time)), the "write code that modify
> source code" was really missing (in my opinion and my knowledge of the
> existing tools).
>
> I wanted a very intuitive and easy to use library that allows me to
> query and modify my source code only in the place I wanted to modify
> it without touching the rest of the code. So I've built what can be
> describe as "the BeautifulSoup of python source code".
>
> To do so, I've built what can be called "a lossless AST" for python
> (designed to be used by humans), an AST that satisfy this equation:
>
>     source_code == ast_to_source(source_to_ast(source_code))
>
> It produces json-serializable python data structures (because data
> structures are easier to use and don't hide anything from you).
>
> And now the part that should interest you more: on top on that AST,
> I've built an high level "query and modification" library that wraps
> those AST nodes into objects. I've put a lot of efforts in making this
> library intuitive and very easy to use while removing you the burden
> of having to deal with low level details. This "BeautifulSoup of the
> python source code" is called Redbaron.
>
> It looks like this:
>
>     from redbaron import RedBaron
>
>     # simple API
>
>     # pass string
>     red = RedBaron("some_value = 42")
>
>     # get string back
>     red.dumps()
>
> Queries are like BeautifulSoup:
>
>     red.find("int", value=4)
>     red.find_all("def", name="stuff")
>
> (You can pass lambda/regex/special syntaxe for globs/regex etc... to
> queries, they should be powerful enough for the vast majorities of
> your needs).
>
> Nodes modification is very simple: just pass source code stored in
> string and "voilà":
>
>     red = RedBaron("some_value = 42")
>     red[0].value = "1 + 1"  # some_value = 1 + 1
>
>     red = RedBaron("def stuff():\n    plop")
>     red[0].value = "some_code"  # def stuff():\n    some_code
>
>     # notice that the input is correctly formatting, indented and it
>     # also takes care of not breaking the next node indentation
>     # works too with decorators and where you expect it to works
>
> (It is possible to pass it ast datastructure or RedBaron objects
> to).
>
> And I've made an abstraction on top of "list of things" so you don't
> have to take care about when you need to put a separator (for eg: a
> "," in a list):
>
>     red = RedBaron("[1, 2, 3]")
>     red[0].append("plop")  # [1, 2, 3, plop]
>
>     # want to add a new django app to INSTALLED_APPS? just do:
>     red.find("assignment", target=lambda x: x.dumps() ==
> "INSTALLED_APPLICATIONS").value.append("'another_app'")
>     # notice that the formatting of the list is detected
>
>     # want to add "@profile" to every function of the root level for
>     # line_profiler?
>     red('def', recursive=False).map(lambda x: x.decorators.insert(0,
> '@profile'))
>
>     # and remove them
>     red("decorator", lambda x: x.dumps() == "@decorator").map(lambda x:
> x.parent.parent.decorators.remove(x))
>
>     # convert every "print a" to "logger.debug(a)
>     red('print', value=lambda x: len(x) == 1).map(lambda x:
> x.replace('logger.debug(%s)' % x.value.dumps())
>
>     # and print a, b, c to logger.debug("%s %s %s" % (a, b, c))
>     red('print', value=lambda x: len(x) == 1).map(lambda x:
> x.replace('logger.debug("%s" % (%s))' % (" ".join('%s' * len(x.value)))
>
> Both library and fully tested (more than 2000 tests in total), fully
> *documented* (with lots of examples) and under freesoftware licences.
> I consider RedBaron to be in alpha stage, it is already very stable
> but a significant number of edge cases are probably not handled yet.
>
> Important point: RedBaron is not and will not do static analysis,
> I'm probably going to integrate (or integrate RedBaron into) a tool
> that already do that like astroid or rope.
>
> Links:
>
> * RedBaron tutorial: https://redbaron.readthedocs.org/en/latest/tuto.html
> * RedBaron documentation: https://redbaron.readthedocs.org
> * RedBaron source code: https://github.com/psycojoker/redbaron
>
> * Baron (the AST) source code: https://github.com/psycojoker/baron
> * Baron documentation: https://baron.readthedocs.org
>
> I hope that I have trigger your interest and I'm very
> interested by your feedback,
>
> Have a nice day and thanks for your time,
>
> PS: I've only been aware of the capacities of lib2to3 since 2 months
> and was very unhappy to discover it so late (I've spent months or
> googling before deciding to start this project), I'll probably swap my
> parser with lib2to3 one in the future.
>
> --
>
> Laurent Peuch -- Bram
> _______________________________________________
> code-quality mailing list
> code-quality at python.org
> https://mail.python.org/mailman/listinfo/code-quality
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/code-quality/attachments/20141115/a2543479/attachment.html>