Puffin Automation Framework (PAF) query

[Ben McGinnes]

Alan Kennedy(alanmk at hotmail.com)@Fri, Nov 21, 2003 at 11:19:57AM +0000:
> 
> I tried it out on a testing job about a year ago. The following are
> points of note
> 
> 1. It is poorly documented.

Ah, yes, I've discovered that.  ;)

> Many important technical details can only
> be obtained by reading the source and knowing the behaviour of the
> python standard library.

That's a bit of a pain, but I've dived into a little Python already,
which is what gave Puffin a good mark on the list.

> 2. Its logging eats memory. Puffin builds an XML DOM to store all of
> the results. Which means it eats memory *real* fast.

That doesn't sound good, but at the moment time is more of an issue than
memory issues.

> 3. IIRC, It only supports HTTP 1.0, meaning that it does not represent
> the actual client->server network activity you would see with real
> world apps.

If that's the case it may be a problem, I'll keep my eyes peeled.

> I would only consider using Puffin if you're requirements are very
> basic, i.e.
> 
> 1. You have a fixed set of URL requests that you want to automate.

The sort of testing being tried thus far is just a series of simple
simulations (e.g. Joe User logs in and logs out immediately, or logs in
to view a piece of data and logs out).  We're not (yet) trying to
actively break the application with invalid data and the like.

> 2. You want to munge data in transmitted and returned resources.

That may not be much of a concern if we can see a report indicating that
a particular series of actions succeeded or failed.

> 3. You only need to run one of them at a time.

Given that Puffin can be called from the command line and set with
different "workspaces" (i.e. config files) then it's possible to work
around this.  Since the plan was to automate it via cron or similar
anyway this is not a great hassle.

> Dunno about contacting Keyton, but feel free to ask questions. I may
> be able to answer.

Ah well, it was worth a shot.

The major issue I've encountered at the moment is a conflict between the
plan.xml and action.xml files.  From what I can tell the plan.xml file
is supposed to specifically call the actions specified in action.xml in
order to run them.  Yet at the moment if puffin detects objects of the
same name in the two files, it breaks with a bunch of Traceback errors.

Ultimately they claim the XML was not well formed, but that XML has been
independently verified as legitimate.  The only conflict we can narrow
it down too is the similar names and, indeed, when I change the action
names in plan.xml it appears to work (except that the actions specified
in action.xml are not called, which they need to be).

> There are some java tools, scriptable in jython, that are much better
> all round testing tools, that are more robust. 

I'll certainly have a look, if only for general reference in the future,
but, unfortunately, time still is a primary factor.

> Check this archive message for details
>
> http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&selm=3F93BBA8.CCFBAFA1%40hotmail.com

Amusingly this thread was initiated by one of the other members of the
project, prior to my joining it.  Still, the URLs suggested by yourself
and others in the group will be reviewed once more.


Regards,
Ben