[Mailman-Developers] GSOC - Seeking insight on Authenticated RESTful APIs

[Stephen J. Turnbull]

Before I get started on the content, note that I'm going to be rather
critical of most posts of all students for a while, mostly negative.
**This is not a reflection of the quality of the candidate or of the
proposals involved.** Instead, it reflects the fact that we need to
learn to communicate efficiently with each other.  Some of that can
come from mentor adjustment, but (for reasons I'll go into some other
time if asked) mostly adjustment is going to need to be on the student
side.

It follows that there are two kinds of criticism here.  One is of the
content of the project.  We'll try to be constructive there, but
sometimes we can only ask questions which often sounds pretty negative
(especially to students with no good answers yet!)  I'm sorry about
that, but it's necessary.  The other is of the presentation,
especially of content that is inappropriate and should not be
included.  That necessarily is going to be mostly negative.

Rahul Gaur writes:

 > Hi all ,
 > 
 > I have applied for the project mentioned on the ideas page *Authenticated
 > REST-API in Postorius/Django - *[0]
 > So as per the Project idea, I have been doing some research on my end
 > figuring out what would be the best way to achieve this.
 > Since Postorius is based on Django , I have been trying Django specific
 > solutions to serve the functionality currently offered by the Postorius
 > web interface in the form of RESTful API's.

The above is a good introduction.

 > As of now , I have been working on two different Django based frameworks to
 > build RESTful APIs.
 > 
 > 1. Django-REST-Framework
 > 
 > 2. Django-Tastypie
 > 
 > Initially , I was familiar with tastypie and I was strongly considering to
 > use Tastypie only for this project. However after some advice from Richard
 > , I am trying  to be more flexible and I am trying to evaluate both the
 > frameworks separately.

With all due respect to Richard (if that's what he advised), "being
flexible" here is not a good reason to look at multiple frameworks.  I
would recommend going with the framework you know.

 > I am yet to figure out how to use [django-tasty-pie] with Non-ORM
 > data resources precisely how to integrate it into postorius so I
 > can serve the APIs .

This, on the other hand, is a reason to consider changing, if it's
really unclear how to use Tasty Pie to do the job.

But why are you specifying "non-ORM"?  Django already provides the
ORM, and Postorius is (eventually) going to provide access to all
"user-servicable parts" (where in this context "user" includes site
admin, list admin, moderator, and subscriber for sure, and possibly
other roles as they are defined).  For Postorius's UI face, that is
going to pretty much have to go through the ORM.  If you need access
to something that's not in the ORM, I would suggest fixing the models.

 > I would be glad , if I could get some more help on this.

You need to say what you're trying to access for help on this.  Your
description of working with the django-rest-framework below is much
easier to provide help for.  If this is the best you can do for Tasty
Pie I'd say you're pretty much blocked on this route, and you should
just switch to django-rest-framework, unless you find a clear reason
to prefer Tasty Pie.

In any case, I think first you should think about the APIs you want to
serve, rather than writing code that serves no clear purpose.

1.  Who does the API serve (site admin, list admin, moderator,
    subscriber, ...)?

2.  What do they want to accomplish?

3.  What needs to be done in the Mailman core to have that effect?

4.  What facility does Postorius provide for dealing with Mailman?
    (In this particular case, why isn't it accessible via the Django
    ORM and models used by Postorius?)

5.  How would you express this in a REST API?

On to django-rest-framework:

 > I don't know if this is the right approach to the project , but let me try
 > to explain :
 > Since Django-rest-framework has following concept :
 > It provides serializers to convert complex data into python native data
 > type and which could be easily rendered into json.

Wacky likes the HAL+JSON approach, I think.
http://stateless.co/hal_specification.html
http://tools.ietf.org/html/draft-kelly-json-hal-05

I think that's a good place to start for defining structures.  It
seems very flexible, and there may already be Python modules for
implementing it.

 >     mail_host = serializers.CharField(max_length=200)
 >     url_host = serializers.CharField(max_length=200)
 >     contact_add = serializers.CharField(max_length=200)
 >     description = serializers.CharField(max_length=200)

"Which of these things is not like the others?"  Ie, why "contact_add"
and not "contact_list" or "contacts"?

 >     def __init__(self, field ,field2 , field3, field4):
 >         self.mail_host = field
 >         self.url_host = field2
 >         self.contact_add = field3
 >         self.description = field4

Give the parameters descriptive names:

    # n.b. Fixed the commas.
    def __init__(self, mail_host, url_host, contact_add, description):
        self.mail_host = mail_host
        self.url_host = url_host
        self.contact_add = contact_add
        self.description = description

Maybe it looks a little wasteful (and violates DRY), but this is the
best way I've found.

 > And my Views is something like this :

 >     def all(self, request, *args, **kwargs):
 >         dict = []

This is a list, not a dict.

 >         dictb  = []
 >         dictc = []
 >         dictd = []

Why a list per attribute?  This isn't an very extensible or
maintainable API, because you'll have to change all methods like this
one every time you add or remove or rename an attribute.

 > Please let me know If I am working in the right direction

Well, I assume you actually ran the program, so you know it works.
But I don't see what it's *for*, so I can't say if you're going in the
right direction or not.

 > Now about the authentication part , what would be the best possible
 > solution for providing authentication ?

At this point, no authentication at all.  Only allow connections from
localhost, and let them do anything the interface permits.  Delegate
authentication to the client (which has to run on the same host, and
presumably will be a web service so the site admin can use HTTPS +
HTTP Basic Auth in the webserver).

Authorization is another matter.  Wacky and I discussed ACLs a little
bit, and I *think* (but haven't asked him directly) he would support
an "Authorizable" mixin class (or decorator, maybe) that would be
applied to every "piece of information" that your API serves.

BTW: It should be obviously that I'm not real clear on what
"Authorizable" actually does, or what "pieces of information" are.
That's your problem, in some sense, although we can discuss it here.

 > Also there has been a discussion on "Architecture for extra profile
 > info" , wouldn't this feature would be helpful in providing
 > specific set of permission for what part of data can be accessed by
 > authenticated API users ?

Well, no; the point of that thread was to try to define the API for
extra profile info.  The thread got sidetracked into authentication,
but there's a serious cart before horse problem here.  Some
information is so sensitive that it shouldn't be available on the net
at all.  Does Mailman store anything like that?  Are you sure?  (Hint:
the extra profile information might actually come from an enterprise
database.)  If you can't answer questions like that, designing authn
or authz systems is not possible.

I think you should ignore the auth* problem, and concentrate on what
information your API should provide, what packages it should come in,
and the RESTful API to accomplish that.  Once you have some direction
on that, where to add the auth* APIs, and what they're going to look
like should be much more clear.

 > Would it be more along the lines that any authenticated restful api user
 > would be able to access all the resources offered by public facing RESTful
 > API ?

Surely not.  For example, subscribers aren't allowed to create lists
in Mailman 2, so I would assume any services provided by Mailman 3
should implement the same restriction.

 > Another thing I have been considering , what would be the end
 > deliverables of this project , what I mean to say is will the
 > Public facing RESTful API be offering functionality currently
 > possible with postorius via RESTful APIs or it should also include
 > more features ?

Well, Postorius is "currently" pretty incomplete, so we'd better do more
than "currently possible".

 > if this is the actual requirement of the project rather than
 > relaying the core api via postorius into a new api.

Since the "consumers" of this API are really Terri and Florian, you'd
better talk to them.  But since they're interested in "extra" profile
information, I think you need to be prepared for extensive additional
requirements beyond a mere relay of the core.

This is the important question at this stage.  I think you're asking
too many questions, in fact.  You need more focus.  I suggest some
questions about getting your feet wet with the frameworks are OK, plus
discussion of what information your REST API needs to be prepared to
provide are what you need now.