[Pandas-dev] A case for a simplified (non-consolidating) BlockManager with 1D blocks

[Joris Van den Bossche]

On Mon, 1 Jun 2020 at 20:07, Brock Mendel <jbrockmendel at gmail.com> wrote:

> Joris and I accidentally took part of the discussion off-thread.  My
> suggestion boils down to: Let's
>
> 1) Identify pieces of this that we want to do regardless of whether we do
> the rest of it (e.g. consolidate only in internals, view-only indexing on
> columns).
>

Personally I am not sure it is worth trying to change consolidation
policies (moving to internals is certainly fine of course, but I mean eg
delaying) or copy/view semantics for the *current*, consolidated
BlockManager.

But there are certainly pieces in the internals that can be changed which
are useful regardless. I opened
https://github.com/pandas-dev/pandas/issues/34669 to have a more concrete
discussion about this on github.


> 2) Beef up the asvs to be give closer-to-full measure of the tradeoffs
> when an eventual proof of concept/PR is made.
>
>
We probably won't have a "one big PR" that is going to implement a
simplified block manager, so it's not really clear to me how ASV will help
with making a decision on this?
(it will for sure be very useful *along the way* to keep track of where we
need to optimize things to preserve performance)

Joris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pandas-dev/attachments/20200609/bba4b341/attachment.html>