From garcia.marc at gmail.com  Thu Jan 12 07:58:55 2023
From: garcia.marc at gmail.com (Marc Garcia)
Date: Thu, 12 Jan 2023 19:58:55 +0700
Subject: [Pandas-dev] pandas new infrastructure (OVH donation)
In-Reply-To: <CAEk5N5tUEjX9ithe07RULjHU8LZf8ygKWEQjiejOzyuzYq6dCg@mail.gmail.com>
References: <CAEk5N5tJyqocthcrqiDMoQwLbu5jxcP-n7ys4D8kyTA7gVFy8A@mail.gmail.com>
 <CALQtMBbGq8rshiH+3cX9sowfWkzGBr_yekfmS_dHn5k4FCFB5w@mail.gmail.com>
 <CAEk5N5uUbU6vupcwQ3JnmMrXehNtyewR4sj_HpOzqV7ufqg9mw@mail.gmail.com>
 <CAJ=56RKHo5rTkD4MYJRCYQ_v8UqQa58gg3p0Ra9yQL3bKgZ5wQ@mail.gmail.com>
 <CAEk5N5soXVk=WmdaEZth6+8WLoGSCe5vOxKqFb3wQB8VBT27dA@mail.gmail.com>
 <CAEk5N5tUEjX9ithe07RULjHU8LZf8ygKWEQjiejOzyuzYq6dCg@mail.gmail.com>
Message-ID: <CAEk5N5uZLnCmxjhiMaQW4teMjgdVzNLDCsaNaVYPE1ew17Uk8g@mail.gmail.com>

Hi all, an update on this.

OVH added dedicated hardware to their public cloud, and we can now create
metal instances on demand. I created one, I isolated a CPU, and I'm running
our benchmark suite on one it for the last 50 commits of the project. Each
run takes 2 hours in the cheapest dedicated server, which costs 170 EUR per
month. I don't think more expensive instances will make a difference, since
they seem to have more cores, but not sure if a single core (what we're
using) would be faster. If everything looks good when the current run
finishes, I'll set things up so benchmarks start running automatically in
that machine, and results are published.

In parallel, I'm doing some research and some tests to see if we can detect
performance regressions in PRs. I don't think it makes much sense to do it
with asv, but I think it's feasible to build something specific. I'll send
an update with the results of the research and a proposal when I'm ready.

Any feedback or whatever just let me know.

On Tue, Nov 15, 2022 at 3:05 PM Marc Garcia <garcia.marc at gmail.com> wrote:

> Quick update about the new infrastructure.
>
> - New hosting for the website seems to be working just fine, no issues
> detected. I just stopped nginx in the old server, in case there is anything
> there still being used we hopefully realize. But if there are no issues and
> no objections, I'll be switching off the server in few days.
>
> - We should be able to start using dedicated hardware for the benchmarks
> from our OVH cloud account in December. It'll work as regular cloud
> instances, but with dedicated servers. We'll be doing some tests to try to
> get more stability in the benchmarks, and hopefully we can get something
> even better than until now when the OVH hardware is ready.
>
> On Thu, Nov 10, 2022 at 9:31 PM Marc Garcia <garcia.marc at gmail.com> wrote:
>
>> Oh, I forgot we were not using the rendered asv website from the old
>> server. We're using nginx, so I can easily make pandas.pydata.org/speed
>> show the content from that url. But I guess we can also check them directly
>> in the github pages url, not sure if it makes a difference.
>>
>> Let me know if it's useful, and I'll set it up. Thanks for the info!
>>
>> On Thu, Nov 10, 2022, 21:18 Richard Shadrach <rhshadrach at gmail.com>
>> wrote:
>>
>>> > Besides the open PR, the only missing thing are the benchmarks at (
>>> pandas.pydata.org/speed). The link is not working now, since I didn't
>>> move the benchmarks yet. But before moving this, we should also make the
>>> changes in the benchmarks repo, so benchmark results start to synchronize
>>> with the new server. Can someone with access to the server take care of it
>>> please (DM for the new server info).
>>>
>>> The link https://asv-runner.github.io/asv-collection/pandas/ is being
>>> automatically updated. Can we point to this URL for now, given that we may
>>> be changing how the benchmarks are run? If it's desirable to have the
>>> benchmarks results on the docs server and our current solution is deemed to
>>> be the long term one, I can work on the synchronization. However I'm
>>> resistant to putting in that work if it's just going to go away given the
>>> easier solution.
>>>
>>> Best,
>>> Richard
>>>
>>>
>>> On Wed, Nov 9, 2022 at 11:50 PM Marc Garcia <garcia.marc at gmail.com>
>>> wrote:
>>>
>>>> Some updates (the ones shared in yesterday's call, and some new ones.
>>>>
>>>> The cloud (bucket) storage didn't seem convenient for different
>>>> reasons, so I moved forward with a regular Ubuntu instance (the cheapest, 2
>>>> cores, 7Gb ram, 24 EUR/month). I moved now all the traffic to the new
>>>> instance, and since we've just got static file serving, the instance seems
>>>> to be more than enough to handle our traffic (I didn't see CPU or RAM
>>>> exceed 4% usage in the time I've been monitoring the resources). I've got a
>>>> PR open (#49614) to start syncing our web/docs with the new server. In few
>>>> hours I'll stop the nginx in the old server (I confirmed there is no
>>>> traffic already, since we use cloudflare our dns changes are immediate).
>>>> And in few days I'll switch off the instance in rackspace.
>>>>
>>>> Besides the open PR, the only missing thing are the benchmarks at (
>>>> pandas.pydata.org/speed). The link is not working now, since I didn't
>>>> move the benchmarks yet. But before moving this, we should also make the
>>>> changes in the benchmarks repo, so benchmark results start to synchronize
>>>> with the new server. Can someone with access to the server take care of it
>>>> please (DM for the new server info).
>>>>
>>>> On running the benchmarks in OVH, the VM instances don't seem to be
>>>> stable enough to keep track of performance over time, as it was likely.
>>>> Full results of the tests I did are in this repo:
>>>> https://gitlab.com/datapythonista/pandas_ovh_benchmarks . OVH is
>>>> checking the best way to give us access to dedicated hardware, will
>>>> continue with that once we've got it. In parallel to that, I'm planning to
>>>> do some tests to see if it could be feasible to use valgrind's cachegrind
>>>> (or equivalent) to instead of monitor time, we monitor CPU cycles. That
>>>> should make benchmarking much easier and faster, as any hardware would
>>>> work, and benchmarks could be run in parallel. With a dedicated server
>>>> we're likely to only be able to use a single core to have stable results,
>>>> which means that we can only run one benchmark suite per server every 3
>>>> hours. But implementing it can be tricky.
>>>>
>>>> About CIrun, as you say Joris, it's like a middle man between our
>>>> hardware (the OVH openstack API to create/delete instances) and GitHub
>>>> actions. We need to add an extra yaml file with the CIrun configuration,
>>>> and other than that we should be able to use OVH hardware directly from our
>>>> current CI jobs without changes (except one entry to say what instance we
>>>> want to use for the jobs running in OVH I assume).
>>>>
>>>> Please let me know of any feedback. In particular if you see any
>>>> problem with our website that could be caused by the migration.
>>>>
>>>> Cheers,
>>>>
>>>> On Thu, Nov 10, 2022 at 12:43 AM Joris Van den Bossche <
>>>> jorisvandenbossche at gmail.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Sat, 5 Nov 2022 at 15:24, Marc Garcia <garcia.marc at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi all,
>>>>>>
>>>>>> pandas has received a donation from OVHcloud
>>>>>> <https://www.ovhcloud.com/> to support the project infrastructure,
>>>>>> with OVHcloud public cloud credits (an initial amount of 10,000 EUR for a
>>>>>> period of one year). OVH is open to sponsor longer term and also other
>>>>>> projects of the ecosystem (or NumFOCUS as a whole), but we started with
>>>>>> this to have feedback at a smaller scale first.
>>>>>>
>>>>>> The credits will be used initially for:
>>>>>> - Hosting of the pandas website
>>>>>> - Running the pandas benchmarks
>>>>>> - Speeding up the project CI
>>>>>>
>>>>>> I detail next what I have in mind to set up for each. If anyone is
>>>>>> interested in getting involved, or has ideas, comments... please let me
>>>>>> know. I'll publish updates here as there is progress on this.
>>>>>>
>>>>>>
>>>>>> Website: I'm planning to experiment on splitting the website in two
>>>>>> (it'll be transparent for users). The website and the stable docs which
>>>>>> receive most of the traffic can probably be stored in Cloudflare pages.
>>>>>> We're already using Cloudflare as a CDN, so instead of using it as a cache,
>>>>>> we can publish the documents there. The rest of the docs (old versions and
>>>>>> the dev version) can be hosted in bucket storage of the OVHcloud. Response
>>>>>> times may be a bit slower, but our website is bigger than the Cloudflare
>>>>>> quota, and having old docs rarely accessed in a CDN seems unnecessary
>>>>>> anyway.
>>>>>>
>>>>>
>>>>> Splitting like that makes sense! (_if_ it is within quota, we could
>>>>> maybe consider keeping the dev docs, and only move old docs to bucket
>>>>> storage?)
>>>>>
>>>>>
>>>>>>
>>>>>> - Benchmarks: OVHcloud instances have guaranteed hardware, and we'll
>>>>>> be checking if this is enough for the results of the benchmarks to be
>>>>>> consistent over runs, or if there is too much variability and we need
>>>>>> dedicated hardware. If consistency is good enough that would be great,
>>>>>> since our benchmarks mostly use one core, and using dedicated hardware is
>>>>>> likely to be a decent waste of resources, since most servers will likely
>>>>>> have 16 cores or more. We'll discuss with OVH if dedicated hardware is
>>>>>> needed, as at the moment their public cloud doesn't offer it (there is an
>>>>>> alpha for providing dedicated instances, but we need to check with them).
>>>>>>
>>>>>> - Faster CI: Our GitHub runners are small, and most builds take
>>>>>> around one hour or more to finish. We should be able to use bigger OVH
>>>>>> instances for our existing CI pretty easily, via their OpenStack API and
>>>>>> CIrun.
>>>>>>
>>>>>
>>>>> I am not familiar with CIrun, but quickly checking it, that would
>>>>> basically be using our current github actions but through their
>>>>> "self-hosted" runner feature?
>>>>>
>>>>>
>>>>>> _______________________________________________
>>>>>> Pandas-dev mailing list
>>>>>> Pandas-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>
>>>>> _______________________________________________
>>>> Pandas-dev mailing list
>>>> Pandas-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230112/ca7f312d/attachment-0001.html>

From m.e.gorelli at gmail.com  Wed Jan 18 05:18:35 2023
From: m.e.gorelli at gmail.com (Marco Gorelli)
Date: Wed, 18 Jan 2023 10:18:35 +0000
Subject: [Pandas-dev] Is it time for a bi-weekly pandas call?
In-Reply-To: <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
References: <CA+gwkEAkMaRJK5X8BiaEyeDeOT8UWGH22pEo42mSrnvJ+P+VvQ@mail.gmail.com>
 <CAEk5N5sQ0CAis5wqNhUz65BHsHZ137RvetLX8pUtsH-C3vW+rw@mail.gmail.com>
 <CACvdwiNY+UoiK8zxuHD7HKnWY21s0uL-esLvGk+6XyPJBeFNtw@mail.gmail.com>
 <CAJ=56R+hG16HE65FLOKjZeCKq2xctd_FOxwTqTKNJqRhURWmMA@mail.gmail.com>
 <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
Message-ID: <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>

Just following up on this - shall we add another call to the calendar for
the 4th week of the month (and therefore have the next call next week)?

On Fri, Nov 11, 2022 at 3:08 PM Marco Gorelli <m.e.gorelli at gmail.com> wrote:

> Cool stuff
>
> Unless there's any objections,, then, let's add a call for the 4th week of
> each month (same day of week and time as the current one)
>
> On Wed, Oct 19, 2022 at 1:46 PM Richard Shadrach <rhshadrach at gmail.com>
> wrote:
>
>> I would also like bi-weekly.
>>
>> Best,
>> Richard
>>
>> On Tue, Oct 18, 2022, 12:34 Matthew Roeschke <emailformattr at gmail.com>
>> wrote:
>>
>>> I would be interested in meeting twice a month.
>>>
>>> On Sun, Oct 16, 2022 at 9:35 PM Marc Garcia <garcia.marc at gmail.com>
>>> wrote:
>>>
>>>> I'm personally happy with both frequencies, no preference.
>>>>
>>>> Couple of related things:
>>>> - Since this call is where a lot of the decision making happens, would
>>>> it make sense to discuss this as part of the governance discussions?
>>>> - Maybe worth also discussing the time of the call? I think the current
>>>> time is quite reasonable for many time zones, and it necessarily need to be
>>>> night time in some places during the call. But I wonder if it'd make a
>>>> difference if we have the call a bit later to contributors or potential
>>>> contributors in India, China... if we move the call one or two hours
>>>> earlier, and how this affects people in California or Hawaii. This shows
>>>> the current time in different time zones:
>>>> https://www.timeanddate.com/worldclock/converter.html?iso=20221107T180000&p1=103&p2=224&p3=179&p4=233&p5=1440&p6=125&p7=166&p8=776&p9=176&p10=28&p11=237&p12=240
>>>>
>>>> On Fri, Oct 14, 2022 at 5:13 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>> wrote:
>>>>
>>>>> Currently, there's a monthly pandas call, which has been in place for
>>>>> several years. Seeing as there's now more people working on pandas as part
>>>>> of their jobs, might it be time to increase the frequency? E.g. to meet
>>>>> every 2 weeks, instead of once a month?
>>>>> _______________________________________________
>>>>> Pandas-dev mailing list
>>>>> Pandas-dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>
>>>> _______________________________________________
>>>> Pandas-dev mailing list
>>>> Pandas-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>
>>>
>>>
>>> --
>>> Matthew Roeschke
>>> _______________________________________________
>>> Pandas-dev mailing list
>>> Pandas-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230118/36dfd89e/attachment.html>

From garcia.marc at gmail.com  Wed Jan 18 05:26:17 2023
From: garcia.marc at gmail.com (Marc Garcia)
Date: Wed, 18 Jan 2023 17:26:17 +0700
Subject: [Pandas-dev] Is it time for a bi-weekly pandas call?
In-Reply-To: <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>
References: <CA+gwkEAkMaRJK5X8BiaEyeDeOT8UWGH22pEo42mSrnvJ+P+VvQ@mail.gmail.com>
 <CAEk5N5sQ0CAis5wqNhUz65BHsHZ137RvetLX8pUtsH-C3vW+rw@mail.gmail.com>
 <CACvdwiNY+UoiK8zxuHD7HKnWY21s0uL-esLvGk+6XyPJBeFNtw@mail.gmail.com>
 <CAJ=56R+hG16HE65FLOKjZeCKq2xctd_FOxwTqTKNJqRhURWmMA@mail.gmail.com>
 <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
 <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>
Message-ID: <CAEk5N5viE4Wa8i3pp=eTQfzae4YFah0+JMdK3ths3cOyz42eVw@mail.gmail.com>

I think most people was in favor of this. Do you have permissions to edit
the calendar invite to make it bi-weekly? I think Tom or Joris created the
original invite, you can probably check with them if you don't.

On Wed, Jan 18, 2023 at 5:18 PM Marco Gorelli <m.e.gorelli at gmail.com> wrote:

> Just following up on this - shall we add another call to the calendar for
> the 4th week of the month (and therefore have the next call next week)?
>
> On Fri, Nov 11, 2022 at 3:08 PM Marco Gorelli <m.e.gorelli at gmail.com>
> wrote:
>
>> Cool stuff
>>
>> Unless there's any objections,, then, let's add a call for the 4th week
>> of each month (same day of week and time as the current one)
>>
>> On Wed, Oct 19, 2022 at 1:46 PM Richard Shadrach <rhshadrach at gmail.com>
>> wrote:
>>
>>> I would also like bi-weekly.
>>>
>>> Best,
>>> Richard
>>>
>>> On Tue, Oct 18, 2022, 12:34 Matthew Roeschke <emailformattr at gmail.com>
>>> wrote:
>>>
>>>> I would be interested in meeting twice a month.
>>>>
>>>> On Sun, Oct 16, 2022 at 9:35 PM Marc Garcia <garcia.marc at gmail.com>
>>>> wrote:
>>>>
>>>>> I'm personally happy with both frequencies, no preference.
>>>>>
>>>>> Couple of related things:
>>>>> - Since this call is where a lot of the decision making happens, would
>>>>> it make sense to discuss this as part of the governance discussions?
>>>>> - Maybe worth also discussing the time of the call? I think the
>>>>> current time is quite reasonable for many time zones, and it necessarily
>>>>> need to be night time in some places during the call. But I wonder if it'd
>>>>> make a difference if we have the call a bit later to contributors or
>>>>> potential contributors in India, China... if we move the call one or two
>>>>> hours earlier, and how this affects people in California or Hawaii. This
>>>>> shows the current time in different time zones:
>>>>> https://www.timeanddate.com/worldclock/converter.html?iso=20221107T180000&p1=103&p2=224&p3=179&p4=233&p5=1440&p6=125&p7=166&p8=776&p9=176&p10=28&p11=237&p12=240
>>>>>
>>>>> On Fri, Oct 14, 2022 at 5:13 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Currently, there's a monthly pandas call, which has been in place for
>>>>>> several years. Seeing as there's now more people working on pandas as part
>>>>>> of their jobs, might it be time to increase the frequency? E.g. to meet
>>>>>> every 2 weeks, instead of once a month?
>>>>>> _______________________________________________
>>>>>> Pandas-dev mailing list
>>>>>> Pandas-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>
>>>>> _______________________________________________
>>>>> Pandas-dev mailing list
>>>>> Pandas-dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>
>>>>
>>>>
>>>> --
>>>> Matthew Roeschke
>>>> _______________________________________________
>>>> Pandas-dev mailing list
>>>> Pandas-dev at python.org
>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230118/a0e35ce4/attachment-0001.html>

From m.e.gorelli at gmail.com  Wed Jan 18 12:53:54 2023
From: m.e.gorelli at gmail.com (Marco Gorelli)
Date: Wed, 18 Jan 2023 17:53:54 +0000
Subject: [Pandas-dev] Is it time for a bi-weekly pandas call?
In-Reply-To: <CAEk5N5viE4Wa8i3pp=eTQfzae4YFah0+JMdK3ths3cOyz42eVw@mail.gmail.com>
References: <CA+gwkEAkMaRJK5X8BiaEyeDeOT8UWGH22pEo42mSrnvJ+P+VvQ@mail.gmail.com>
 <CAEk5N5sQ0CAis5wqNhUz65BHsHZ137RvetLX8pUtsH-C3vW+rw@mail.gmail.com>
 <CACvdwiNY+UoiK8zxuHD7HKnWY21s0uL-esLvGk+6XyPJBeFNtw@mail.gmail.com>
 <CAJ=56R+hG16HE65FLOKjZeCKq2xctd_FOxwTqTKNJqRhURWmMA@mail.gmail.com>
 <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
 <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>
 <CAEk5N5viE4Wa8i3pp=eTQfzae4YFah0+JMdK3ths3cOyz42eVw@mail.gmail.com>
Message-ID: <CA+gwkEBON893UiF3seU9Vc1NAxDs+6rxF-zT1gc=isi1v+XVeg@mail.gmail.com>

Cool, update, thanks!

Speak to anyone who can make it next week

On Wed, Jan 18, 2023 at 10:26 AM Marc Garcia <garcia.marc at gmail.com> wrote:

> I think most people was in favor of this. Do you have permissions to edit
> the calendar invite to make it bi-weekly? I think Tom or Joris created the
> original invite, you can probably check with them if you don't.
>
> On Wed, Jan 18, 2023 at 5:18 PM Marco Gorelli <m.e.gorelli at gmail.com>
> wrote:
>
>> Just following up on this - shall we add another call to the calendar for
>> the 4th week of the month (and therefore have the next call next week)?
>>
>> On Fri, Nov 11, 2022 at 3:08 PM Marco Gorelli <m.e.gorelli at gmail.com>
>> wrote:
>>
>>> Cool stuff
>>>
>>> Unless there's any objections,, then, let's add a call for the 4th week
>>> of each month (same day of week and time as the current one)
>>>
>>> On Wed, Oct 19, 2022 at 1:46 PM Richard Shadrach <rhshadrach at gmail.com>
>>> wrote:
>>>
>>>> I would also like bi-weekly.
>>>>
>>>> Best,
>>>> Richard
>>>>
>>>> On Tue, Oct 18, 2022, 12:34 Matthew Roeschke <emailformattr at gmail.com>
>>>> wrote:
>>>>
>>>>> I would be interested in meeting twice a month.
>>>>>
>>>>> On Sun, Oct 16, 2022 at 9:35 PM Marc Garcia <garcia.marc at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I'm personally happy with both frequencies, no preference.
>>>>>>
>>>>>> Couple of related things:
>>>>>> - Since this call is where a lot of the decision making happens,
>>>>>> would it make sense to discuss this as part of the governance discussions?
>>>>>> - Maybe worth also discussing the time of the call? I think the
>>>>>> current time is quite reasonable for many time zones, and it necessarily
>>>>>> need to be night time in some places during the call. But I wonder if it'd
>>>>>> make a difference if we have the call a bit later to contributors or
>>>>>> potential contributors in India, China... if we move the call one or two
>>>>>> hours earlier, and how this affects people in California or Hawaii. This
>>>>>> shows the current time in different time zones:
>>>>>> https://www.timeanddate.com/worldclock/converter.html?iso=20221107T180000&p1=103&p2=224&p3=179&p4=233&p5=1440&p6=125&p7=166&p8=776&p9=176&p10=28&p11=237&p12=240
>>>>>>
>>>>>> On Fri, Oct 14, 2022 at 5:13 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Currently, there's a monthly pandas call, which has been in place
>>>>>>> for several years. Seeing as there's now more people working on pandas as
>>>>>>> part of their jobs, might it be time to increase the frequency? E.g. to
>>>>>>> meet every 2 weeks, instead of once a month?
>>>>>>> _______________________________________________
>>>>>>> Pandas-dev mailing list
>>>>>>> Pandas-dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Pandas-dev mailing list
>>>>>> Pandas-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Matthew Roeschke
>>>>> _______________________________________________
>>>>> Pandas-dev mailing list
>>>>> Pandas-dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230118/444a567f/attachment.html>

From tom.augspurger88 at gmail.com  Wed Jan 18 15:31:41 2023
From: tom.augspurger88 at gmail.com (Tom Augspurger)
Date: Wed, 18 Jan 2023 14:31:41 -0600
Subject: [Pandas-dev] Is it time for a bi-weekly pandas call?
In-Reply-To: <CA+gwkEBON893UiF3seU9Vc1NAxDs+6rxF-zT1gc=isi1v+XVeg@mail.gmail.com>
References: <CA+gwkEAkMaRJK5X8BiaEyeDeOT8UWGH22pEo42mSrnvJ+P+VvQ@mail.gmail.com>
 <CAEk5N5sQ0CAis5wqNhUz65BHsHZ137RvetLX8pUtsH-C3vW+rw@mail.gmail.com>
 <CACvdwiNY+UoiK8zxuHD7HKnWY21s0uL-esLvGk+6XyPJBeFNtw@mail.gmail.com>
 <CAJ=56R+hG16HE65FLOKjZeCKq2xctd_FOxwTqTKNJqRhURWmMA@mail.gmail.com>
 <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
 <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>
 <CAEk5N5viE4Wa8i3pp=eTQfzae4YFah0+JMdK3ths3cOyz42eVw@mail.gmail.com>
 <CA+gwkEBON893UiF3seU9Vc1NAxDs+6rxF-zT1gc=isi1v+XVeg@mail.gmail.com>
Message-ID: <CAE1aY-m_LBE+PXM6ucXF4V3ZQCLu73HwP9M-AT4SRv2Mqpue9w@mail.gmail.com>

I can update the calendar.

I think it's currently set for the second Wednesday of each month. Would
you like the 2nd & 4th Wednesday, or every other Wednesday?

Tom

On Wed, Jan 18, 2023 at 11:54 AM Marco Gorelli <m.e.gorelli at gmail.com>
wrote:

> Cool, update, thanks!
>
> Speak to anyone who can make it next week
>
> On Wed, Jan 18, 2023 at 10:26 AM Marc Garcia <garcia.marc at gmail.com>
> wrote:
>
>> I think most people was in favor of this. Do you have permissions to edit
>> the calendar invite to make it bi-weekly? I think Tom or Joris created the
>> original invite, you can probably check with them if you don't.
>>
>> On Wed, Jan 18, 2023 at 5:18 PM Marco Gorelli <m.e.gorelli at gmail.com>
>> wrote:
>>
>>> Just following up on this - shall we add another call to the calendar
>>> for the 4th week of the month (and therefore have the next call next week)?
>>>
>>> On Fri, Nov 11, 2022 at 3:08 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>> wrote:
>>>
>>>> Cool stuff
>>>>
>>>> Unless there's any objections,, then, let's add a call for the 4th week
>>>> of each month (same day of week and time as the current one)
>>>>
>>>> On Wed, Oct 19, 2022 at 1:46 PM Richard Shadrach <rhshadrach at gmail.com>
>>>> wrote:
>>>>
>>>>> I would also like bi-weekly.
>>>>>
>>>>> Best,
>>>>> Richard
>>>>>
>>>>> On Tue, Oct 18, 2022, 12:34 Matthew Roeschke <emailformattr at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I would be interested in meeting twice a month.
>>>>>>
>>>>>> On Sun, Oct 16, 2022 at 9:35 PM Marc Garcia <garcia.marc at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I'm personally happy with both frequencies, no preference.
>>>>>>>
>>>>>>> Couple of related things:
>>>>>>> - Since this call is where a lot of the decision making happens,
>>>>>>> would it make sense to discuss this as part of the governance discussions?
>>>>>>> - Maybe worth also discussing the time of the call? I think the
>>>>>>> current time is quite reasonable for many time zones, and it necessarily
>>>>>>> need to be night time in some places during the call. But I wonder if it'd
>>>>>>> make a difference if we have the call a bit later to contributors or
>>>>>>> potential contributors in India, China... if we move the call one or two
>>>>>>> hours earlier, and how this affects people in California or Hawaii. This
>>>>>>> shows the current time in different time zones:
>>>>>>> https://www.timeanddate.com/worldclock/converter.html?iso=20221107T180000&p1=103&p2=224&p3=179&p4=233&p5=1440&p6=125&p7=166&p8=776&p9=176&p10=28&p11=237&p12=240
>>>>>>>
>>>>>>> On Fri, Oct 14, 2022 at 5:13 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Currently, there's a monthly pandas call, which has been in place
>>>>>>>> for several years. Seeing as there's now more people working on pandas as
>>>>>>>> part of their jobs, might it be time to increase the frequency? E.g. to
>>>>>>>> meet every 2 weeks, instead of once a month?
>>>>>>>> _______________________________________________
>>>>>>>> Pandas-dev mailing list
>>>>>>>> Pandas-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Pandas-dev mailing list
>>>>>>> Pandas-dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Matthew Roeschke
>>>>>> _______________________________________________
>>>>>> Pandas-dev mailing list
>>>>>> Pandas-dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>
>>>>> _______________________________________________
> Pandas-dev mailing list
> Pandas-dev at python.org
> https://mail.python.org/mailman/listinfo/pandas-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230118/980d40c4/attachment-0001.html>

From tom.augspurger88 at gmail.com  Wed Jan 18 16:21:07 2023
From: tom.augspurger88 at gmail.com (Tom Augspurger)
Date: Wed, 18 Jan 2023 15:21:07 -0600
Subject: [Pandas-dev] Is it time for a bi-weekly pandas call?
In-Reply-To: <CA+gwkEC-bhAVjiD-VGUf1UpS0e9QKRmJPhiv_u4bJbjHDFaySA@mail.gmail.com>
References: <CA+gwkEAkMaRJK5X8BiaEyeDeOT8UWGH22pEo42mSrnvJ+P+VvQ@mail.gmail.com>
 <CAEk5N5sQ0CAis5wqNhUz65BHsHZ137RvetLX8pUtsH-C3vW+rw@mail.gmail.com>
 <CACvdwiNY+UoiK8zxuHD7HKnWY21s0uL-esLvGk+6XyPJBeFNtw@mail.gmail.com>
 <CAJ=56R+hG16HE65FLOKjZeCKq2xctd_FOxwTqTKNJqRhURWmMA@mail.gmail.com>
 <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
 <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>
 <CAEk5N5viE4Wa8i3pp=eTQfzae4YFah0+JMdK3ths3cOyz42eVw@mail.gmail.com>
 <CA+gwkEBON893UiF3seU9Vc1NAxDs+6rxF-zT1gc=isi1v+XVeg@mail.gmail.com>
 <CAE1aY-m_LBE+PXM6ucXF4V3ZQCLu73HwP9M-AT4SRv2Mqpue9w@mail.gmail.com>
 <CA+gwkEC-bhAVjiD-VGUf1UpS0e9QKRmJPhiv_u4bJbjHDFaySA@mail.gmail.com>
Message-ID: <CAE1aY-mWDzRdDZKFtQ7exWTRz3H7HEuNGnb_50suyP8jSvagdw@mail.gmail.com>

Ah, perfect.

On Wed, Jan 18, 2023 at 3:02 PM Marco Gorelli <m.e.gorelli at gmail.com> wrote:

> Sorry I meant to write "updated", it's now showing both the 2nd and 4th
> Wednesdays (thanks Noa!)
>
> On Wed, Jan 18, 2023 at 8:31 PM Tom Augspurger <tom.augspurger88 at gmail.com>
> wrote:
>
>> I can update the calendar.
>>
>> I think it's currently set for the second Wednesday of each month. Would
>> you like the 2nd & 4th Wednesday, or every other Wednesday?
>>
>> Tom
>>
>> On Wed, Jan 18, 2023 at 11:54 AM Marco Gorelli <m.e.gorelli at gmail.com>
>> wrote:
>>
>>> Cool, update, thanks!
>>>
>>> Speak to anyone who can make it next week
>>>
>>> On Wed, Jan 18, 2023 at 10:26 AM Marc Garcia <garcia.marc at gmail.com>
>>> wrote:
>>>
>>>> I think most people was in favor of this. Do you have permissions to
>>>> edit the calendar invite to make it bi-weekly? I think Tom or Joris created
>>>> the original invite, you can probably check with them if you don't.
>>>>
>>>> On Wed, Jan 18, 2023 at 5:18 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>> wrote:
>>>>
>>>>> Just following up on this - shall we add another call to the calendar
>>>>> for the 4th week of the month (and therefore have the next call next week)?
>>>>>
>>>>> On Fri, Nov 11, 2022 at 3:08 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Cool stuff
>>>>>>
>>>>>> Unless there's any objections,, then, let's add a call for the 4th
>>>>>> week of each month (same day of week and time as the current one)
>>>>>>
>>>>>> On Wed, Oct 19, 2022 at 1:46 PM Richard Shadrach <
>>>>>> rhshadrach at gmail.com> wrote:
>>>>>>
>>>>>>> I would also like bi-weekly.
>>>>>>>
>>>>>>> Best,
>>>>>>> Richard
>>>>>>>
>>>>>>> On Tue, Oct 18, 2022, 12:34 Matthew Roeschke <
>>>>>>> emailformattr at gmail.com> wrote:
>>>>>>>
>>>>>>>> I would be interested in meeting twice a month.
>>>>>>>>
>>>>>>>> On Sun, Oct 16, 2022 at 9:35 PM Marc Garcia <garcia.marc at gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I'm personally happy with both frequencies, no preference.
>>>>>>>>>
>>>>>>>>> Couple of related things:
>>>>>>>>> - Since this call is where a lot of the decision making happens,
>>>>>>>>> would it make sense to discuss this as part of the governance discussions?
>>>>>>>>> - Maybe worth also discussing the time of the call? I think the
>>>>>>>>> current time is quite reasonable for many time zones, and it necessarily
>>>>>>>>> need to be night time in some places during the call. But I wonder if it'd
>>>>>>>>> make a difference if we have the call a bit later to contributors or
>>>>>>>>> potential contributors in India, China... if we move the call one or two
>>>>>>>>> hours earlier, and how this affects people in California or Hawaii. This
>>>>>>>>> shows the current time in different time zones:
>>>>>>>>> https://www.timeanddate.com/worldclock/converter.html?iso=20221107T180000&p1=103&p2=224&p3=179&p4=233&p5=1440&p6=125&p7=166&p8=776&p9=176&p10=28&p11=237&p12=240
>>>>>>>>>
>>>>>>>>> On Fri, Oct 14, 2022 at 5:13 PM Marco Gorelli <
>>>>>>>>> m.e.gorelli at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Currently, there's a monthly pandas call, which has been in place
>>>>>>>>>> for several years. Seeing as there's now more people working on pandas as
>>>>>>>>>> part of their jobs, might it be time to increase the frequency? E.g. to
>>>>>>>>>> meet every 2 weeks, instead of once a month?
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Pandas-dev mailing list
>>>>>>>>>> Pandas-dev at python.org
>>>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Pandas-dev mailing list
>>>>>>>>> Pandas-dev at python.org
>>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Matthew Roeschke
>>>>>>>> _______________________________________________
>>>>>>>> Pandas-dev mailing list
>>>>>>>> Pandas-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>
>>>>>>> _______________________________________________
>>> Pandas-dev mailing list
>>> Pandas-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230118/678afa09/attachment.html>

From m.e.gorelli at gmail.com  Wed Jan 18 16:01:56 2023
From: m.e.gorelli at gmail.com (Marco Gorelli)
Date: Wed, 18 Jan 2023 21:01:56 +0000
Subject: [Pandas-dev] Is it time for a bi-weekly pandas call?
In-Reply-To: <CAE1aY-m_LBE+PXM6ucXF4V3ZQCLu73HwP9M-AT4SRv2Mqpue9w@mail.gmail.com>
References: <CA+gwkEAkMaRJK5X8BiaEyeDeOT8UWGH22pEo42mSrnvJ+P+VvQ@mail.gmail.com>
 <CAEk5N5sQ0CAis5wqNhUz65BHsHZ137RvetLX8pUtsH-C3vW+rw@mail.gmail.com>
 <CACvdwiNY+UoiK8zxuHD7HKnWY21s0uL-esLvGk+6XyPJBeFNtw@mail.gmail.com>
 <CAJ=56R+hG16HE65FLOKjZeCKq2xctd_FOxwTqTKNJqRhURWmMA@mail.gmail.com>
 <CA+gwkECSZwXcYK8MxKODu_ReaxcqTun=9t6_h5ttC5wwQa7Vig@mail.gmail.com>
 <CA+gwkEBoUEMzHyzKZOpo_uZ-nirVRr4xJUN66pAGUAFLmGTMLA@mail.gmail.com>
 <CAEk5N5viE4Wa8i3pp=eTQfzae4YFah0+JMdK3ths3cOyz42eVw@mail.gmail.com>
 <CA+gwkEBON893UiF3seU9Vc1NAxDs+6rxF-zT1gc=isi1v+XVeg@mail.gmail.com>
 <CAE1aY-m_LBE+PXM6ucXF4V3ZQCLu73HwP9M-AT4SRv2Mqpue9w@mail.gmail.com>
Message-ID: <CA+gwkEC-bhAVjiD-VGUf1UpS0e9QKRmJPhiv_u4bJbjHDFaySA@mail.gmail.com>

Sorry I meant to write "updated", it's now showing both the 2nd and 4th
Wednesdays (thanks Noa!)

On Wed, Jan 18, 2023 at 8:31 PM Tom Augspurger <tom.augspurger88 at gmail.com>
wrote:

> I can update the calendar.
>
> I think it's currently set for the second Wednesday of each month. Would
> you like the 2nd & 4th Wednesday, or every other Wednesday?
>
> Tom
>
> On Wed, Jan 18, 2023 at 11:54 AM Marco Gorelli <m.e.gorelli at gmail.com>
> wrote:
>
>> Cool, update, thanks!
>>
>> Speak to anyone who can make it next week
>>
>> On Wed, Jan 18, 2023 at 10:26 AM Marc Garcia <garcia.marc at gmail.com>
>> wrote:
>>
>>> I think most people was in favor of this. Do you have permissions to
>>> edit the calendar invite to make it bi-weekly? I think Tom or Joris created
>>> the original invite, you can probably check with them if you don't.
>>>
>>> On Wed, Jan 18, 2023 at 5:18 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>> wrote:
>>>
>>>> Just following up on this - shall we add another call to the calendar
>>>> for the 4th week of the month (and therefore have the next call next week)?
>>>>
>>>> On Fri, Nov 11, 2022 at 3:08 PM Marco Gorelli <m.e.gorelli at gmail.com>
>>>> wrote:
>>>>
>>>>> Cool stuff
>>>>>
>>>>> Unless there's any objections,, then, let's add a call for the 4th
>>>>> week of each month (same day of week and time as the current one)
>>>>>
>>>>> On Wed, Oct 19, 2022 at 1:46 PM Richard Shadrach <rhshadrach at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I would also like bi-weekly.
>>>>>>
>>>>>> Best,
>>>>>> Richard
>>>>>>
>>>>>> On Tue, Oct 18, 2022, 12:34 Matthew Roeschke <emailformattr at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> I would be interested in meeting twice a month.
>>>>>>>
>>>>>>> On Sun, Oct 16, 2022 at 9:35 PM Marc Garcia <garcia.marc at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I'm personally happy with both frequencies, no preference.
>>>>>>>>
>>>>>>>> Couple of related things:
>>>>>>>> - Since this call is where a lot of the decision making happens,
>>>>>>>> would it make sense to discuss this as part of the governance discussions?
>>>>>>>> - Maybe worth also discussing the time of the call? I think the
>>>>>>>> current time is quite reasonable for many time zones, and it necessarily
>>>>>>>> need to be night time in some places during the call. But I wonder if it'd
>>>>>>>> make a difference if we have the call a bit later to contributors or
>>>>>>>> potential contributors in India, China... if we move the call one or two
>>>>>>>> hours earlier, and how this affects people in California or Hawaii. This
>>>>>>>> shows the current time in different time zones:
>>>>>>>> https://www.timeanddate.com/worldclock/converter.html?iso=20221107T180000&p1=103&p2=224&p3=179&p4=233&p5=1440&p6=125&p7=166&p8=776&p9=176&p10=28&p11=237&p12=240
>>>>>>>>
>>>>>>>> On Fri, Oct 14, 2022 at 5:13 PM Marco Gorelli <
>>>>>>>> m.e.gorelli at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Currently, there's a monthly pandas call, which has been in place
>>>>>>>>> for several years. Seeing as there's now more people working on pandas as
>>>>>>>>> part of their jobs, might it be time to increase the frequency? E.g. to
>>>>>>>>> meet every 2 weeks, instead of once a month?
>>>>>>>>> _______________________________________________
>>>>>>>>> Pandas-dev mailing list
>>>>>>>>> Pandas-dev at python.org
>>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Pandas-dev mailing list
>>>>>>>> Pandas-dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Matthew Roeschke
>>>>>>> _______________________________________________
>>>>>>> Pandas-dev mailing list
>>>>>>> Pandas-dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/pandas-dev
>>>>>>>
>>>>>> _______________________________________________
>> Pandas-dev mailing list
>> Pandas-dev at python.org
>> https://mail.python.org/mailman/listinfo/pandas-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230118/fb7a295d/attachment-0001.html>

From garcia.marc at gmail.com  Thu Jan 19 21:38:41 2023
From: garcia.marc at gmail.com (Marc Garcia)
Date: Fri, 20 Jan 2023 09:38:41 +0700
Subject: [Pandas-dev] ANN: pandas v1.5.3
Message-ID: <CAEk5N5v07CwVz4ENGLG4NPXd+S80F4K9uYhT2D_M5e4mEKVs8w@mail.gmail.com>

We are pleased to announce the release of pandas v1.5.3.

This is a patch release in the 1.5.x series and includes some regression
fixes and bug fixes. We recommend that all users in the 1.5.x series
upgrade to this version.

See the release notes <https://pandas.pydata.org/docs/whatsnew/v1.5.3.html> for
a list of all the changes.

The release can be installed from PyPI

    python -m pip install --upgrade pandas==1.5.3

Or from conda-forge

    mamba install -c conda-forge pandas==1.5.3

Please report any issues with the release on the pandas issue tracker
<https://github.com/pandas-dev/pandas/issues/new/choose>.

Thanks to all the contributors
<https://pandas.pydata.org/docs/whatsnew/v1.5.3.html#contributors> who made
this release possible.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230120/645e249d/attachment.html>

From jbrockmendel at gmail.com  Thu Jan 26 17:29:54 2023
From: jbrockmendel at gmail.com (Brock Mendel)
Date: Thu, 26 Jan 2023 14:29:54 -0800
Subject: [Pandas-dev] EA Naming Conventions
Message-ID: <CAKf8g9Q+=M8mM2C4Gyc10rhc=7FZnQ578mMm9YQ0Wy2xRWskbQ@mail.gmail.com>

For historical reasons we've built up an EA namespace without much internal
logic in terms of what is public/private.  While this isn't _that_ big of a
deal, it'd be nice to make this more coherent.  I see two useful options:

1) Use the traditional "an underscore means this should only be called from
within self".  Very few methods on the base class satisfy that
characteristic, including the constructor _from_sequence.  One benefit of
moving to this is it would make "official" that we shouldn't be using
_values_for_foo from outside EA methods.

2) Use underscores to signal to 3rd party authors whether or not there
exists a working (not necessarily performant) implementation on the base
class.  In this scenario authors would _have_ to implement private methods,
while implementing public methods would be optional.

Thoughts?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/pandas-dev/attachments/20230126/5dc571e8/attachment.html>