Controlling the passing of data

[Rustom Mody]

On Thursday, April 28, 2016 at 8:11:26 PM UTC+5:30, Dan Strohl wrote:
> In addition to Peter's points, 
> - I would suggest breaking out the list comprehensions into standard for loops and/or functions.  That makes it easier to read and troubleshoot.  (you can always re-optimize It if needed.)
> - Peter's point about making things into functions will also help troubleshooting.  You can better test the data going into and out of the function.  In my code, I will often have the file access and data processing as separate functions, then the main routine just calls the function to get data, passes that data to a function that manipulates it, and then pass the results to a function that writes it out.  This allows for much easier testing and troubleshooting of the individual functions.  (sometimes I will reassemble them into one function when I am done, depending on my needs)
> 
> More importantly though in terms of your getting help for your problem, the post is unclear (to me at least) in terms of what you are trying to achieve and what isn't working, 
> 
> Try considering the following suggestions:
> 
> - It is unclear what the problem is that you are having, you say you are trying to do x, and you are having problems in a part after a comment, but none of the comments say "this is breaking".  I assume you are talking about the comment that starts "Here I need to traverse", but don't know for sure, and even if it is, you don't specify what the problem actually is;  are you receiving an exception message (please let us know what it is), is it running, but not doing anything?  is it returning incorrect data? or???
> 
> - When posting, rather than commenting out code that you aren't using right now, and code that is working and not related to the problem, I recommend just deleting them so that people don't have to try to work through it.... for example, just remove the GetArgs function and just say fileList = "/xml_dir", and the section at the end that is all commented out, just remove it.  You should just have the minimum needed to replicate the problem.  
> 
> This will also help in troubleshooting, when I have a problem like this, and I can't figure out what is going on, I will copy the code to a new file and make a program that will handle a specific set of data, and try to do the one thing that is breaking, removing all the rest of the stuff, and test the results.. so, for example, copy a sample of the xml with a couple of data items into a string var, and have a program that processes that and checks to see if at the end you end up with a list of the right values by printing a list rather than muddying the waters with file access, and writing out csv's.  (by the way, this is a great time to start working with unit testing if you aren't already, it is simple to create this as a test case and you will find that if you start doing testing along the way, the time it takes to troubleshoot errors along the way will go down dramatically.)
> 
> - Be clear at the end what you expect to get, especially if it is not what you are getting... so, either in the code as a comment, or in a descriptive paragraph, have a section that said something like:  "At the end of the snippet, meetdata, racedata, clubdata, and raceid should be a list of dictionaries with the data from the xml" (or whatever... possibly with an example of what you would expect).  This is even more important if the problem you are having is that the code is not returning correct data.  This may not be as needed if the code is simply blowing up at line xx, though it would still help people understand your goal.
> 
> - For the example at least (you may choose to do differently in your live code), use nice explanatory variable names and don't rename imports, so it would be clearer to say "import pandas", then "frames = pandas.DataFrame[])".  That way the reader doesn't have to keep referring to the imports to figure out what is going on.
> 
> Remember, you are asking a large number of people for help, most of which are pretty busy already, so the more you can do to simplify and show the exact problem, the more (and more useful) help you are likely to receive.  To this lists credit, even if you are completely unclear in your question, you will likely get *something* back, (as you saw with Peters response), but what you get back is more likely to be a general suggestion rather than a specific fix for your problem.
> 
> Dan Strohl
> 
> 
> 
> 
> > -----Original Message-----
> > 
> > Sayth Renshaw wrote:
> > 
> > > In my file here I needed to traverse and modify the XML file I don't
> > > want to restore it or put it in a new variable or other format I just
> > > want to alter it and let it flow onto the list comprehensions as they were.
> > 
> > That looks like an arbitrary limitation to me. It's a bit like "I want to repair my
> > car with this big hammer".
> > 
> > > In particular here I am taking the id from race and putting it into
> > > the children of each race called nomination.
> > >
> > > I have put a comment above the new code which is causing the difficulty.
> > 
> > Your actual problem is drowned in too much source code. Can you restate it
> > in English, optionally with a few small snippets of Python?
> > 
> > It is not even clear what the code you provide should accomplish once it's
> > running as desired.
> > 
> > To give at least one code-related advice: You have a few repetitions of the
> > following structure
> > 
> > > meetattrs = ('id', 'venue', 'date', 'rail', 'weather',
> > > 'trackcondition')
> > 
> > >     meet = d('meeting')
> > 
> > >     meetdata = [[meet.eq(i).attr(x)
> > >                  for x in meetattrs] for i in range(len(meet))]
> > 
> > You should move the pieces into a function that works for meetings, clubs,
> > races, and so on. Finally (If I am repeating myself so be it): the occurence of
> > range(len(something)) in your code is a strong indication that you are not
> > using Python the way Guido intended it. Iterate over the `something` directly
> > whenever possible.
> > 
> > --
> > https://mail.python.org/mailman/listinfo/python-list

To add to that:
It is right to have a dislike for bad code and a desire for good code.
But this desire odes not translate into an effective communication. In particular:  I dont think anyone quite gets exactly you are after in:

> I don't want to restore it or put it in a new variable or other format I 
> just want to alter it and let it flow onto the list comprehensions as they were. 

So please cut down your example to 1/4 the size:
- Tiny inline triple-quoted xml
- Your attempt so far
- What you would like to do different