From jorisvandenbossche at gmail.com Tue Jun 12 10:22:34 2018 From: jorisvandenbossche at gmail.com (Joris Van den Bossche) Date: Tue, 12 Jun 2018 16:22:34 +0200 Subject: [Pandas-dev] Pandas dev sprint July 5-8 (pre-Scipy Conf) In-Reply-To: References: Message-ID: Dear all, As a follow-up on the mail below, the sprints will take place from July 5th afternoon until July 8th, probably in the offices of Anaconda. For now we will be with around 9 people, but if you still want to participate, see this mail as a last call and please respond to me! Also, if you have ideas of what should be discussed / worked on, remember this mail I sent earlier to the list: https://mail.python.org/pipermail/pandas-dev/2018-May/000752.html (and google doc: https://docs.google.com/document/d/1CPkc4avxb9XiEpI7Zz9Wg8FUkC_oXQqmdZzD-alCa7o/edit ) Best, Joris 2018-04-20 18:24 GMT+02:00 Joris Van den Bossche < jorisvandenbossche at gmail.com>: Dear all, > > We are planning to do a pandas sprint just before the Scipy Conference > (which is in Austin, Texas, July 9-13). The sprint would then be from July > 5 or 6 to July 8, also in Austin. > > We will meet there with some of the core devs, but we also want to invite > other pandas (or related libraries) contributors. > Note: the sprint is not meant to jump-start newcomers to contributing to > open source (for that, there is a sprint at the end of the Scipy). > > The exact planning of the sprint still needs to be discussed (will open a > separate thread for that), but topics will probably include: general issue > clean-up, discussing project management, roadmap to pandas 1.0 (what do we > still want to get in / change before that) and evolution of pandas > afterwards, ... and of course actual coding. > > *So if you are interested, certainly let something know! * > > We probably want to keep it somewhat limited in number, but don't hesitate > to state your interest. > If you would like to participate, but not sure if you would fit at such a > sprint, don't hesitate to mail me personally. If you would like to > participate, but would need financial support, you can also mail me (we > might have some funds, but not fully sure yet). > > Best, > Joris > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tom.augspurger88 at gmail.com Tue Jun 12 22:33:03 2018 From: tom.augspurger88 at gmail.com (Tom Augspurger) Date: Tue, 12 Jun 2018 21:33:03 -0500 Subject: [Pandas-dev] ANN: Pandas 0.23.1 Released Message-ID: Hi all, I'm happy to announce pandas that pandas 0.23.1 has been released. This is a minor bug-fix release in the 0.23.x series and includes some regression fixes, bug fixes, and performance improvements. We recommend that all users upgrade to this version. See the full whatsnew for a list of all the changes. The release can be installed with conda from the default channel and conda-forge:: conda install pandas Or via PyPI: python -m pip install --upgrade pandas A total of 30 people contributed patches to this release. People with a ?+? by their names contributed a patch for the first time. - Adam J. Stewart - Adam Kim + - Aly Sivji - Chalmer Lowe + - Damini Satya + - Dr. Irv - Gabe Fernando + - Giftlin Rajaiah - Jeff Reback - Jeremy Schendel + - Joris Van den Bossche - Kalyan Gokhale + - Kevin Sheppard - Matthew Roeschke - Max Kanter + - Ming Li - Pyry Kovanen + - Stefano Cianciulli - Tom Augspurger - Uddeshya Singh + - Wenhuan - William Ayd - chris-b1 - gfyoung - h-vetinari - nprad + - ssikdar1 + - tmnhat2001 - topper-123 - zertrin + -------------- next part -------------- An HTML attachment was scrubbed... URL: From arthurobdfv at gmail.com Thu Jun 14 13:47:48 2018 From: arthurobdfv at gmail.com (Arthuro Verissimo) Date: Thu, 14 Jun 2018 14:47:48 -0300 Subject: [Pandas-dev] How does Pandas from python reads the timestamp to datetime64? (HDF5) Message-ID: i'm creating an application on C# that generates a hdf5 document with one row of the compound data being a timestamp (using unix timestamp), and i wish i could load this hdf5 in pandas and it load the timestamp column as a datetime64. I've tried to create the file from pandas and i saw it creates the dataset with a lot of attributes that i assume pandas read and do some logic to convert it while reading the file (the datetime inside the file created from pandas is a timestamp too), i can't figure it out... Here is my dataset creation code and some screenshots of what i had investigated public void CriaHDF5Customizado(PackingConfigFile pf) { // PopulaOPC(pf); H5FileId arquivoHDF5 = H5F.create("C:/Users/arthuro/Desktop/timestampteste.h5", H5F.CreateMode.ACC_TRUNC); H5GroupId datasetGroup = H5G.create(arquivoHDF5, "Datasets"); H5GroupId infosGroup = H5G.create(arquivoHDF5, "Informations"); H5G.close(infosGroup); opcSt opHelper = new opcSt(); opHelper.dt = (Int64)DateTime.Parse("1996-11-9").Subtract(new DateTime(1970, 1, 1,12,00,00, DateTimeKind.Utc)).TotalSeconds; opHelper.qlt = (Int64)DateTime.Now.Subtract(new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc)).TotalSeconds; opHelper.vl = 123456.7f; int structsize = Marshal.SizeOf(opHelper); H5DataTypeId myOPCType = H5T.create(H5T.CreateClass.COMPOUND, structsize); //H5T.insert(myOPCType, "TimeStamp", 0, new H5DataTypeId(H5T.H5Type.NATIVE_LONG)); H5T.insert(myOPCType, "Quality", 8, new H5DataTypeId(H5T.H5Type.NATIVE_LONG)); H5T.insert(myOPCType, "Value", 16, new H5DataTypeId(H5T.H5Type.NATIVE_FLOAT)); long[] ds = new long[1]; ds[0] = 1; H5DataTypeId dtDatetime = H5T.create_array(new H5DataTypeId(H5T.H5Type.NATIVE_LLONG), ds); H5T.insert(myOPCType, "TimeStamp", 0, dtDatetime); foreach (BasicVariable bv in pf.basicVariableList.bvList) { bv.PopulateOPCUA(); var bvnow = bv; var bvNro = pf.basicVariableList.bvList.IndexOf(bv); long[] dims = new long[1]; dims[0] = bv.bvData.Count; H5DataSpaceId myDataSpace = H5S.create_simple(1, dims); H5DataSetId bvDset = H5D.create(datasetGroup, bv.bvTag, myOPCType, myDataSpace); var arrayaux = new List(); foreach (OPC_UA opc in bv.bvData) { var aux = new opcSt(opc.timeStamp, opc.quality, (float)opc.data); arrayaux.Add(aux); } H5D.write(bvDset, myOPCType, new H5Array(arrayaux.ToArray())); string[] stringteste = { "datetime64" }; H5DataTypeId attrDt = H5T.copy(H5T.H5Type.C_S1); H5T.setSize(attrDt, stringteste[0].Length); var longsz = new long[] { 1 }; var enc = new System.Text.ASCIIEncoding(); var array1 = enc.GetBytes(stringteste[0]); var charArray = new byte[stringteste[0].Length + 1]; array1.CopyTo(charArray, 0); charArray[stringteste[0].Length] = 0; H5DataSpaceId atribDS = H5S.create_simple(1, longsz); H5AttributeId Attribs = H5A.create(bvDset, "TimeStamp_dtype", attrDt, atribDS); H5A.write(Attribs, attrDt, new H5Array(charArray)); stringteste[0] = "(lp0L0La."; H5T.setSize(attrDt, stringteste[0].Length); longsz = new long[] { 1 }; enc = new System.Text.ASCIIEncoding(); array1 = enc.GetBytes(stringteste[0]); charArray = new byte[stringteste[0].Length + 1]; array1.CopyTo(charArray, 0); charArray[stringteste[0].Length] = 0; atribDS = H5S.create_simple(1, longsz); H5AttributeId Attribs2 = H5A.create(bvDset, "TimeStamp_kind", attrDt, atribDS); H5A.write(Attribs2, attrDt, new H5Array(charArray)); stringteste[0] = "N."; H5T.setSize(attrDt, stringteste[0].Length); longsz = new long[] { 1 }; enc = new System.Text.ASCIIEncoding(); array1 = enc.GetBytes(stringteste[0]); charArray = new byte[stringteste[0].Length + 1]; array1.CopyTo(charArray, 0); charArray[stringteste[0].Length] = 0; atribDS = H5S.create_simple(1, longsz); H5AttributeId Attribs3 = H5A.create(bvDset, "TimeStamp_meta", attrDt, atribDS); H5A.write(Attribs3, attrDt, new H5Array(charArray)); H5S.close(atribDS); H5A.close(Attribs); H5A.close(Attribs2); H5A.close(Attribs3); H5D.close(bvDset); H5S.close(myDataSpace); bv.bvData = new List(); } H5G.close(datasetGroup); H5F.close(arquivoHDF5); } Pandas reading (timestamp and datetime): being 1- The value read inside jupyter notebook (processed by pandas) 2- The real value inside the HDF5 File https://i.stack.imgur.com/ZZJ3A.png Ahd these are the attributes of the table created by pandas (that i've tried to recreate in my code) https://i.stack.imgur.com/U1Zom.png -------------- next part -------------- An HTML attachment was scrubbed... URL: From jorisvandenbossche at gmail.com Tue Jun 19 09:04:56 2018 From: jorisvandenbossche at gmail.com (Joris Van den Bossche) Date: Tue, 19 Jun 2018 15:04:56 +0200 Subject: [Pandas-dev] Pandas development hangout - Thursday June 21 at 13:00 UTC Message-ID: Hi all, We're having a dev chat this Friday (May 25) at 9:00 Eastern / 14:00 UTC+1 / 15:00 CEST (Europe). All are welcome to attend. We will mainly discuss 0.23.2 release and preparation for the dev sprint before Scipy Conf. Hangout: https://hangouts.google.com/hangouts/_/calendar/am9yaXN2YW5kZW5ib3NzY2hlQGdtYWlsLmNvbQ.7ft88t77r0mf11ad2kaqnm0ru6?authuser=0 Calendar invite: https://calendar.google.com/event?action=TEMPLATE&tmeid=N2Z0ODh0NzdyMG1mMTFhZDJrYXFubTBydTYgam9yaXN2YW5kZW5ib3NzY2hlQG0&tmsrc=jorisvandenbossche%40gmail.com Minutes: https://docs.google.com/document/d/1tGbTiYORHiSPgVMXawiweGJl Bw5dOkVJLY-licoBmBU/edit?usp=sharing Joris -------------- next part -------------- An HTML attachment was scrubbed... URL: From jorisvandenbossche at gmail.com Tue Jun 19 11:52:27 2018 From: jorisvandenbossche at gmail.com (Joris Van den Bossche) Date: Tue, 19 Jun 2018 17:52:27 +0200 Subject: [Pandas-dev] Pandas development hangout - Thursday June 21 at 13:00 UTC In-Reply-To: References: Message-ID: And of course, the header was right but the text in the mail not: the meeting is Thursday June 21, not in May .. 2018-06-19 15:04 GMT+02:00 Joris Van den Bossche < jorisvandenbossche at gmail.com>: > Hi all, > > We're having a dev chat this Friday (May 25) at 9:00 Eastern / 14:00 UTC+1 > / 15:00 CEST (Europe). > All are welcome to attend. > > We will mainly discuss 0.23.2 release and preparation for the dev sprint > before Scipy Conf. > > Hangout: https://hangouts.google.com/hangouts/_/calendar/ > am9yaXN2YW5kZW5ib3NzY2hlQGdtYWlsLmNvbQ.7ft88t77r0mf11ad2kaqnm0ru6? > authuser=0 > > Calendar invite: https://calendar.google.com/event?action=TEMPLATE&tmeid= > N2Z0ODh0NzdyMG1mMTFhZDJrYXFubTBydTYgam9yaXN2YW5kZW5ib3NzY2hl > QG0&tmsrc=jorisvandenbossche%40gmail.com > > Minutes: https://docs.google.com/document/d/1tGbTiYORHiSPgVMXawiweGJl > Bw5dOkVJLY-licoBmBU/edit?usp=sharing > > Joris > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at pietrobattiston.it Fri Jun 29 09:01:53 2018 From: me at pietrobattiston.it (Pietro Battiston) Date: Fri, 29 Jun 2018 15:01:53 +0200 Subject: [Pandas-dev] Temptative guidelines for indexing code Message-ID: <1530277313.29685.162.camel@pietrobattiston.it> Hi all, maybe they need discussion, or vice-versa they are quite obvious, but just to have a clear map of how we can improve indexing code, and "interpret" new issues/PRs/discussions regarding it, I put down some guidelines in the wiki: https://github.com/pandas-dev/pandas/wiki/(Temptative)-rules-for-restru cturing-indexing-code Joris suggested we can move it to the repo itself, as a sort of "design document", if at any time we want to ease a line-by-line review; for the moment, any comment is welcome. Pietro From tom.augspurger88 at gmail.com Fri Jun 29 09:15:13 2018 From: tom.augspurger88 at gmail.com (Tom Augspurger) Date: Fri, 29 Jun 2018 08:15:13 -0500 Subject: [Pandas-dev] Temptative guidelines for indexing code In-Reply-To: <1530277313.29685.162.camel@pietrobattiston.it> References: <1530277313.29685.162.camel@pietrobattiston.it> Message-ID: Looks like the link was mangled: https://github.com/pandas-dev/pandas/wiki/(Temptative)-rules-for-restructuring-indexing-code Tom On Fri, Jun 29, 2018 at 8:01 AM, Pietro Battiston wrote: > Hi all, > > maybe they need discussion, or vice-versa they are quite obvious, but > just to have a clear map of how we can improve indexing code, and > "interpret" new issues/PRs/discussions regarding it, I put down some > guidelines in the wiki: > > https://github.com/pandas-dev/pandas/wiki/(Temptative)-rules-for-restru > cturing-indexing-code > > Joris suggested we can move it to the repo itself, as a sort of "design > document", if at any time we want to ease a line-by-line review; for > the moment, any comment is welcome. > > Pietro > _______________________________________________ > Pandas-dev mailing list > Pandas-dev at python.org > https://mail.python.org/mailman/listinfo/pandas-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: