From LogiplexSoftware at earthlink.net Wed Mar 12 20:06:12 2003 From: LogiplexSoftware at earthlink.net (Cliff Wells) Date: 12 Mar 2003 11:06:12 -0800 Subject: [Csv] Initial checkin of csv.util Message-ID: <1047495972.1646.42.camel@software1.logiplex.internal> It's quiet in here. Anyway, I went ahead and committed what I've got for the sniffer. Sorry it's taken so long. Please note that my changes now make csv into a package so it must be imported with from csv import csv rather than import csv Regards, -- Cliff Wells, Software Engineer Logiplex Corporation (www.logiplex.net) (503) 978-6726 x308 (800) 735-0555 x308 From skip at pobox.com Thu Mar 13 02:14:29 2003 From: skip at pobox.com (Skip Montanaro) Date: Wed, 12 Mar 2003 19:14:29 -0600 Subject: [Csv] How to install csv now? Message-ID: <15983.56181.777087.422503@montanaro.dyndns.org> Now that Cliff has checked in util/sniffer.py how do we go about installing it? That is, how does csv/setup.py have to change? I think it would work if it was structured like sandbox csv setup.py csv __init__.py util __init__.py sniffer.py csv.py and setup.py declared that "csv" was a package. Unfortunately, I've always been leary of rearranging the CVS structure, especially if the SF folks are involved. So, is there a way to modify setup.py but retain the current structure? Skip From andrewm at object-craft.com.au Thu Mar 13 03:10:40 2003 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 13 Mar 2003 13:10:40 +1100 Subject: [Csv] How to install csv now? In-Reply-To: Message from Skip Montanaro <15983.56181.777087.422503@montanaro.dyndns.org> References: <15983.56181.777087.422503@montanaro.dyndns.org> Message-ID: <20030313021040.C250C3CC5F@coffee.object-craft.com.au> >Now that Cliff has checked in util/sniffer.py how do we go about installing >it? That is, how does csv/setup.py have to change? I think it would work >if it was structured like > > sandbox > csv > setup.py > csv > __init__.py > util > __init__.py > sniffer.py > csv.py > >and setup.py declared that "csv" was a package. I agree. I'd also be inclined to make __init__.py do 'from csv.csv import *', so that users can do "import csv" like normal, or 'import csv.util', etc. >Unfortunately, I've always been leary of rearranging the CVS structure, >especially if the SF folks are involved. > >So, is there a way to modify setup.py but retain the current structure? I've been trying to think of a way for a while, and nothing has occured - I think we just need to bite the bullet. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From LogiplexSoftware at earthlink.net Thu Mar 13 20:24:39 2003 From: LogiplexSoftware at earthlink.net (Cliff Wells) Date: 13 Mar 2003 11:24:39 -0800 Subject: [Csv] How to install csv now? In-Reply-To: <20030313021040.C250C3CC5F@coffee.object-craft.com.au> References: <15983.56181.777087.422503@montanaro.dyndns.org> <20030313021040.C250C3CC5F@coffee.object-craft.com.au> Message-ID: <1047583479.1644.94.camel@software1.logiplex.internal> On Wed, 2003-03-12 at 18:10, Andrew McNamara wrote: > >Now that Cliff has checked in util/sniffer.py how do we go about installing > >it? That is, how does csv/setup.py have to change? I think it would work > >if it was structured like > > > > sandbox > > csv > > setup.py > > csv > > __init__.py > > util > > __init__.py > > sniffer.py > > csv.py > > > >and setup.py declared that "csv" was a package. > > I agree. > > I'd also be inclined to make __init__.py do 'from csv.csv import *', so > that users can do "import csv" like normal, or 'import csv.util', etc. > > >Unfortunately, I've always been leary of rearranging the CVS structure, > >especially if the SF folks are involved. > > > >So, is there a way to modify setup.py but retain the current structure? > > I've been trying to think of a way for a while, and nothing has occured - > I think we just need to bite the bullet. I'm glad to see that I'm not the only one who was worried about this ;-) Part of the reason I hesitated checking my stuff in was due to packaging/installation issues... Anyway, I'd also like to hear some feedback on sniffer.py if anyone gets some time to play with it. I suspect a couple warts/bugs that I'm too brain-dead at the moment to try and look for. Also, what other parameters besides delim, quotechar, skipinitialspace do we think we should try to sniff? And finally, there is at least one case "all" "work" "and" "no" "play" where there is no delimiter. Right now sniffer returns '' (None already meaning we couldn't sniff it). How will this affect the csv parser proper? Is '' a valid argument for the delimiter? -- Cliff Wells, Software Engineer Logiplex Corporation (www.logiplex.net) (503) 978-6726 x308 (800) 735-0555 x308 From skip at pobox.com Wed Mar 19 00:09:46 2003 From: skip at pobox.com (Skip Montanaro) Date: Tue, 18 Mar 2003 17:09:46 -0600 Subject: [Csv] csv package ready for prime-time? Message-ID: <15991.42810.167975.876841@montanaro.dyndns.org> I'm ready to move the csv package out of the sandbox into the main CVS trunk. Since my last post there have been a few changes and comments: * Cliff Wells contributed his csv file parameter sniffing code * The installation is now a package instead of a single module (not sure if the docs have caught up with this change yet) * On the mailing list, the following threads of significance are found: - John Machin pointed out a few bugs and raised issues with my decision to ignore blank lines in my DictReader class. I don't believe we ever reached a concensus we were both happy with. (That is, John may still be slightly unhappy with the current results. I didn't change the behavior as a result of the thread.) - Andrew Dalke reported some problems using a space character as the delimiter which appear to be resolved. Is there a formal process for "dusting off" software which has been playing in the sandbox? What about getting PEP 305 stamped with the BDFL seal of approval? (I realize Guido's busy in the run-up to PyCon.) As a gentle reminder, the relevant URLs are http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/nondist/sandbox/csv/ http://www.python.org/peps/pep-0305.html You can browse the mailing list archives at http://manatee.mojam.com/pipermail/csv You can check out the code and play with it fairly easily. From your sandbox directory execute cvs up -dP . cd csv python setup.py install Feedback to the csv mailing list please (Reply-To: adjusted accordingly). Skip From guido at python.org Wed Mar 19 02:22:28 2003 From: guido at python.org (Guido van Rossum) Date: Tue, 18 Mar 2003 20:22:28 -0500 Subject: [Csv] Re: [Python-Dev] csv package ready for prime-time? In-Reply-To: "Your message of Tue, 18 Mar 2003 17:09:46 CST." <15991.42810.167975.876841@montanaro.dyndns.org> References: <15991.42810.167975.876841@montanaro.dyndns.org> Message-ID: <200303190122.h2J1MSI22680@pcp02138704pcs.reston01.va.comcast.net> > Is there a formal process for "dusting off" software which has been > playing in the sandbox? What about getting PEP 305 stamped with the > BDFL seal of approval? (I realize Guido's busy in the run-up to > PyCon.) I have no bandwidth for looking at this until after I'm back from the UK Python conference (April 7), but I think there's no reason why you should wait for me with moving the csv package to the dist/src tree. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Thu Mar 20 23:48:45 2003 From: skip at pobox.com (Skip Montanaro) Date: Thu, 20 Mar 2003 16:48:45 -0600 Subject: [Csv] Freeze on sandbox/csv Message-ID: <15994.17741.336204.807638@montanaro.dyndns.org> Folks, I'm about to migrate the csv package to the mainstream CVS. Please hold off on changes until I sound the all-clear. Skip From skip at pobox.com Fri Mar 21 00:38:54 2003 From: skip at pobox.com (Skip Montanaro) Date: Thu, 20 Mar 2003 17:38:54 -0600 Subject: [Csv] csv package stitched into CVS hierarchy Message-ID: <15994.20750.356162.465058@montanaro.dyndns.org> The csv package is now in the main branch of the CVS hierarchy. I will leave the structure in the sandbox for a few days before "cvs remove"ing it in case I missed something. Skip From neal at metaslash.com Sun Mar 23 03:35:39 2003 From: neal at metaslash.com (Neal Norwitz) Date: Sat, 22 Mar 2003 21:35:39 -0500 Subject: [Csv] code review Message-ID: <20030323023538.GI25722@epoch.metaslash.com> Here are notes about the version of CSV checked in to the Python 2.3 tree. * remove TODO comment at top of file--it's empty * is CSV going to be maintained outside the python tree? If not, remove the 2.2 compatibility macros for: PyDoc_STR, PyDoc_STRVAR, PyMODINIT_FUNC, etc. * inline the following functions since they are used only in one place get_string, set_string, get_nullchar_as_None, set_nullchar_as_None, join_reset (maybe) * rather than use PyErr_BadArgument, should you use assert? (first example, Dialect_set_quoting, line 218) * is it necessary to have Dialect_methods, can you use 0 for tp_methods? * remove commented out code (PyMem_DEL) on line 261 Have you used valgrind on the test to find memory overwrites/leaks? * PyString_AsString()[0] on line 331 could return NULL in which case you are dereferencing a NULL pointer * note sure why there are casts on 0 pointers lines 383-393, 733-743, 1144-1154, 1164-1165 * Reader_getiter() can be removed and use PyObject_SelfIter() * I think you need PyErr_NoMemory() before returning on line 768, 1178 * is PyString_AsString(self->dialect->lineterminator) on line 994 guaranteed not to return NULL? If not, it could crash by passing to memmove. * PyString_AsString() can return NULL on line 1048 and 1063, the result is passed to join_append() * iteratable should be iterable? (line 1088) * why doesn't csv_writerows() have a docstring? csv_writerow does * any PyUnicode_* methods should be protected with #ifdef Py_USING_UNICODE * csv_unregister_dialect, csv_get_dialect could use METH_O so you don't need to use PyArg_ParseTuple * in init_csv, recommend using PyModule_AddIntConstant and PyModule_AddStringConstant where appropriate Neal From skip at pobox.com Sun Mar 23 15:27:45 2003 From: skip at pobox.com (Skip Montanaro) Date: Sun, 23 Mar 2003 08:27:45 -0600 Subject: [Csv] code review In-Reply-To: <20030323023538.GI25722@epoch.metaslash.com> References: <20030323023538.GI25722@epoch.metaslash.com> Message-ID: <15997.50273.66532.119688@montanaro.dyndns.org> Neal, Thanks for the feedback. We'll work on the items you indicated. I can answer one item right now: * is CSV going to be maintained outside the python tree? If not, remove the 2.2 compatibility macros for: PyDoc_STR, PyDoc_STRVAR, PyMODINIT_FUNC, etc. We don't intend to maintain it outside the Python tree, however compatibility with 2.2 is still desirable, so I think we'll leave those macros in place. I'll add a comment to the top of the code about that. Skip