numpy.frombuffer != unpack() ??

Sat May 17 20:18:39 EDT 2008

Marlin Rowley wrote:
> Actually in my traversal of the never-ending maze of understanding 
> arrays in python, I stumbled that the data in the wrong sequence.
>
> Let's get back to this transpose() deal, say we have these values as 
> integers (representing rgba again):
>
> [[[0,1,2,3]
> [4,5,6,7]
> [8,9,10,11]
> [12,13,14,15]]
>
> [[16,17,18,19]
> [20,21,22,23]
> [24,25,26,27]
> [28,29,30,31]]]
>
> Now if I do this: transpose((2,0,1)), I get this:
>
> [[[0,4,8,12] [16,20,24,28]]
> [[1,5,9,13] [17,21,25,29]]
> [[2,6,10,14][18,22,26,30]]
> [[3,7,11,15][19,23,27,31]]]
>
> This is NOT what I want. I want the new array to be:
>
> [0,4,8,12][1,5,9,13]
> [2,6,10,14][3,7,11,15]
> [16,20,24,28][17,21,25,29]
> [18,22,26,30][19,23,27,31]
>
> How do I do this?

That's s little ambiguous, but one of the following two 
transpose-reshape-print's might be what you want.

Gary Herron

import numpy

a = numpy.array([[[0,1,2,3],
[4,5,6,7],
[8,9,10,11],
[12,13,14,15]],

[[16,17,18,19],
[20,21,22,23],
[24,25,26,27],
[28,29,30,31]]])

print numpy.reshape(a.transpose(0,2,1), (8,4))
print numpy.reshape(a.transpose(0,2,1), (4,2,4))

output is:

[[ 0 4 8 12]
[ 1 5 9 13]
[ 2 6 10 14]
[ 3 7 11 15]
[16 20 24 28]
[17 21 25 29]
[18 22 26 30]
[19 23 27 31]]

and

[[[ 0 4 8 12]
[ 1 5 9 13]]

[[ 2 6 10 14]
[ 3 7 11 15]]

[[16 20 24 28]
[17 21 25 29]]

[[18 22 26 30]
[19 23 27 31]]]

>
> -M
>
>
>
>
>
> ------------------------------------------------------------------------
>
> > Date: Sat, 17 May 2008 08:58:08 -0700
> > From: gherron at islandtraining.com
> > To: marlin_rowley at hotmail.com
> > CC: python-list at python.org
> > Subject: Re: numpy.frombuffer != unpack() ??
> >
> > Marlin Rowley wrote:
> > >
> > > Very cool.
> > >
> > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > > a represents a tile with height of 2 and width of 4 with 4 bits/pixel
> > > for each color.
> > >
> > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')
> > > this seperates the stream into individual values - Check
> > >
> > > > >>> b.shape=(2,4,4)
> > >
> > > This reshapes the array so that b.shape=(height,width,#bits/pixel) 
> - Check
> > >
> > > >>> c = b.transpose((2,0,1))
> > >
> > > What does the (2,0,1) represent in terms of width and height and
> > > number of bytes?
> >
> > The (2,0,1) tells how to exchange the axes. For instance in a 2D array,
> > a normal transpose exchanges rows and columns. It will change a (a by
> > b) sized array into a (b by a) sized array. This would be equivalent to
> > the more saying interchange axes 0,1 to the new order of 1,0.
> >
> > In numpy with higher dimension arrays, the default transpose just
> > exchanges the first two axes, and the full transpose allows you to
> > specify exactly the new ordering of the exes.
> >
> > So transpose((2,0,1)) means take axes (0,1,2) to the new order
> > (2,1,0). In terms of sizes, an (a by b by c) sized array will end being
> > of size (c by a by b) in size.
> >
> > In terms of implementation, there may not be *any* data re-arrangement
> > in a transpose. The only thing that needs changing is how the indices
> > are converted to an actual machine address of an indexed item. The
> > documentation notes this by saying transpose returns a "new view" of 
> the
> > array. This explains why I copied the array before extracting bytes
> > out of it -- you really do need the elements in the new order for the
> > next operation.
> >
> > Gary Herron
> >
> > >
> > >
> > >
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > >
> > > > Date: Fri, 16 May 2008 17:08:20 -0700
> > > > From: gherron at islandtraining.com
> > > > To: marlin_rowley at hotmail.com; python-list at python.org
> > > > Subject: Re: numpy.frombuffer != unpack() ??
> > > >
> > > > Marlin Rowley wrote:
> > > > > All:
> > > > >
> > > > > Say I have an array:
> > > > >
> > > > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > > > >
> > > > > How do I make it so that I now have:
> > > > >
> > > > > starting with first element (a[0])
> > > > > new_arr[0] = 'r'
> > > > > new_arr[1] = 'g'
> > > > > new_arr[2] = 'b'
> > > > > new_arr[3] = 'a'
> > > > > new_arr[4] = 'r'
> > > > > .....
> > > > >
> > > > > continuing "through" a[1] with the same new_arr
> > > > > new_arr[N] = 'r'
> > > > > new_arr[N+1] = 'g'
> > > > > ....
> > > > >
> > > > > -M
> > > >
> > > > Numpy can do this for you. First, do you really mean the array to
> > > > contain lists of one string each? If so:
> > > >
> > > > >>> import numpy
> > > > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > > > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a
> > > > kludge here
> > > > >>> b
> > > > array(['r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 
> 'b', 'a',
> > > > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',
> > > > 'b', 'b', 'a', 'a', 'a', 'a'],
> > > > dtype='|S1')
> > > > >>> b.shape=(2,4,4)
> > > > >>> b
> > > > array([[['r', 'r', 'r', 'r'],
> > > > ['g', 'g', 'g', 'g'],
> > > > ['b', 'b', 'b', 'b'],
> > > > ['a', 'a', 'a', 'a']],
> > > >
> > > > [['r', 'r', 'r', 'r'],
> > > > ['g', 'g', 'g', 'g'],
> > > > ['b', 'b', 'b', 'b'],
> > > > ['a', 'a', 'a', 'a']]],
> > > > dtype='|S1')
> > > > >>> c = b.transpose((2,0,1))
> > > > >>> c
> > > > array([[['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']],
> > > >
> > > > [['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']],
> > > >
> > > > [['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']],
> > > >
> > > > [['r', 'g', 'b', 'a'],
> > > > ['r', 'g', 'b', 'a']]],
> > > > dtype='|S1')
> > > > >>> d=c.copy() # To make it contiguous
> > > > >>> d.shape = (32,)
> > > > >>> d
> > > > array(['r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 
> 'a', 'r',
> > > > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',
> > > > 'b', 'a', 'r', 'g', 'b', 'a'],
> > > > dtype='|S1')
> > > >
> > > > Done. Cool no?
> > > >
> > > > Gary Herron
> > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > 
> ------------------------------------------------------------------------
> > > > > From: marlin_rowley at hotmail.com
> > > > > To: robert.kern at gmail.com; python-list at python.org
> > > > > Subject: RE: numpy.frombuffer != unpack() ??
> > > > > Date: Fri, 16 May 2008 17:31:30 -0500
> > > > >
> > > > > Thank you! That solved it!
> > > > >
> > > > > -M
> > > > >
> > > > >
> > > > >
> > > 
> ------------------------------------------------------------------------
> > > > >
> > > > > > To: python-list at python.org
> > > > > > From: robert.kern at gmail.com
> > > > > > Subject: Re: numpy.frombuffer != unpack() ??
> > > > > > Date: Fri, 16 May 2008 17:25:00 -0500
> > > > > >
> > > > > > Marlin Rowley wrote:
> > > > > > > All:
> > > > > > >
> > > > > > > I'm getting different floating point values when I use numpy
> > > > > vs. unpack().
> > > > > > >
> > > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
> > > > > > > buffer = unpack("!f", byte)
> > > > > > >
> > > > > > > frgba[0] != buffer[0]
> > > > > > >
> > > > > > > why? This is forcing me use the unpack() function since it's
> > > > > giving me
> > > > > > > the correct values. What am I doing wrong?
> > > > > >
> > > > > > Endianness, perhaps? '!' specifies big-endian data (an alias for
> > > > > '>'). Most
> > > > > > likely, you are on a little-endian platform. All of the dtypes
> > > > > in numpy default
> > > > > > to the native-endianness unless specified. If you want to read
> > > > > big-endian data
> > > > > > using numpy, do this:
> > > > > >
> > > > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')
> > > > > >
> > > > > > If you have any more problems with numpy, please join us on the
> > > > > numpy mailing
> > > > > > list. When reporting problems, please try to provide a small but
> > > > > complete
> > > > > > snippet of self-contained code, the output that you got, and
> > > > > explain the output
> > > > > > that you expected to get. Thank you.
> > > > > >
> > > > > > http://www.scipy.org/Mailing_Lists
> > > > > >
> > > > > > --
> > > > > > Robert Kern
> > > > > >
> > > > > > "I have come to believe that the whole world is an enigma, a
> > > > > harmless enigma
> > > > > > that is made terrible by our own mad attempt to interpret it as
> > > > > though it had
> > > > > > an underlying truth."
> > > > > > -- Umberto Eco
> > > > > >
> > > > > > --
> > > > > > http://mail.python.org/mailman/listinfo/python-list
> > > > >
> > > > >
> > > > >
> > > 
> ------------------------------------------------------------------------
> > > > > E-mail for the greater good. Join the i’m Initiative from
> > > > > Microsoft.
> > > > >
> > > 
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
> > > > >
> > > > >
> > > > >
> > > > >
> > > 
> ------------------------------------------------------------------------
> > > > > E-mail for the greater good. Join the i’m Initiative from 
> Microsoft.
> > > > >
> > > 
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood> 
>
> > >
> > > > >
> > > > >
> > > 
> ------------------------------------------------------------------------
> > > > >
> > > > > --
> > > > > http://mail.python.org/mailman/listinfo/python-list
> > > >
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > > Keep your kids safer online with Windows Live Family Safety. Help
> > > protect your kids.
> > > 
> <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008> 
>
> > >
> > > 
> ------------------------------------------------------------------------
> > >
> > > --
> > > http://mail.python.org/mailman/listinfo/python-list
> >
>
>
> ------------------------------------------------------------------------
> Keep your kids safer online with Windows Live Family Safety. Help 
> protect your kids. 
> <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008> 
>
> ------------------------------------------------------------------------
>
> --
> http://mail.python.org/mailman/listinfo/python-list