numpy.frombuffer != unpack() ??

Sat May 17 11:58:08 EDT 2008

Marlin Rowley wrote:
>  
> Very cool.
>  
> > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> a represents a tile with height of 2 and width of 4 with 4 bits/pixel 
> for each color.
>  
> > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1')
> this seperates the stream into individual values - Check
>  
> > >>> b.shape=(2,4,4)
>
> This reshapes the array so that b.shape=(height,width,#bits/pixel) - Check
>  
>  >>> c = b.transpose((2,0,1))
>  
> What does the (2,0,1) represent in terms of width and height and 
> number of bytes?

The (2,0,1) tells how to exchange the axes.  For instance in a 2D array, 
a normal transpose exchanges rows and columns.  It will change a (a by 
b) sized array into a (b by a) sized array.  This would be equivalent to 
the more saying interchange axes 0,1 to the new order of 1,0. 

In numpy with higher dimension arrays, the default transpose just 
exchanges the first two axes, and the full transpose allows you to 
specify exactly the new ordering of the exes.  

So transpose((2,0,1))  means take axes (0,1,2) to the new order 
(2,1,0).  In terms of sizes, an (a by b by c) sized array will end being 
of size (c by a by b) in size. 

In terms of implementation, there may not be *any* data re-arrangement 
in a transpose.   The only thing that needs changing is how the indices 
are converted to an actual machine address of an indexed item.   The 
documentation notes this by saying transpose returns a "new view" of the 
array.   This explains why I copied the array before extracting bytes 
out of it -- you really do need the elements in the new order for the 
next operation.

Gary Herron

>
>
>
>
>  
> ------------------------------------------------------------------------
>
> > Date: Fri, 16 May 2008 17:08:20 -0700
> > From: gherron at islandtraining.com
> > To: marlin_rowley at hotmail.com; python-list at python.org
> > Subject: Re: numpy.frombuffer != unpack() ??
> >
> > Marlin Rowley wrote:
> > > All:
> > >
> > > Say I have an array:
> > >
> > > a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > >
> > > How do I make it so that I now have:
> > >
> > > starting with first element (a[0])
> > > new_arr[0] = 'r'
> > > new_arr[1] = 'g'
> > > new_arr[2] = 'b'
> > > new_arr[3] = 'a'
> > > new_arr[4] = 'r'
> > > .....
> > >
> > > continuing "through" a[1] with the same new_arr
> > > new_arr[N] = 'r'
> > > new_arr[N+1] = 'g'
> > > ....
> > >
> > > -M
> >
> > Numpy can do this for you. First, do you really mean the array to
> > contain lists of one string each? If so:
> >
> > >>> import numpy
> > >>> a = (['rrrrggggbbbbaaaa'],['rrrrggggbbbbaaaa'])
> > >>> b = numpy.frombuffer(''.join(sum(a,[])),dtype='S1') # Kind of a
> > kludge here
> > >>> b
> > array(['r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 'b', 'a',
> > 'a', 'a', 'a', 'r', 'r', 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b',
> > 'b', 'b', 'a', 'a', 'a', 'a'],
> > dtype='|S1')
> > >>> b.shape=(2,4,4)
> > >>> b
> > array([[['r', 'r', 'r', 'r'],
> > ['g', 'g', 'g', 'g'],
> > ['b', 'b', 'b', 'b'],
> > ['a', 'a', 'a', 'a']],
> >
> > [['r', 'r', 'r', 'r'],
> > ['g', 'g', 'g', 'g'],
> > ['b', 'b', 'b', 'b'],
> > ['a', 'a', 'a', 'a']]],
> > dtype='|S1')
> > >>> c = b.transpose((2,0,1))
> > >>> c
> > array([[['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']],
> >
> > [['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']],
> >
> > [['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']],
> >
> > [['r', 'g', 'b', 'a'],
> > ['r', 'g', 'b', 'a']]],
> > dtype='|S1')
> > >>> d=c.copy() # To make it contiguous
> > >>> d.shape = (32,)
> > >>> d
> > array(['r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r',
> > 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g', 'b', 'a', 'r', 'g',
> > 'b', 'a', 'r', 'g', 'b', 'a'],
> > dtype='|S1')
> >
> > Done. Cool no?
> >
> > Gary Herron
> >
> > >
> > >
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > > From: marlin_rowley at hotmail.com
> > > To: robert.kern at gmail.com; python-list at python.org
> > > Subject: RE: numpy.frombuffer != unpack() ??
> > > Date: Fri, 16 May 2008 17:31:30 -0500
> > >
> > > Thank you! That solved it!
> > >
> > > -M
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > >
> > > > To: python-list at python.org
> > > > From: robert.kern at gmail.com
> > > > Subject: Re: numpy.frombuffer != unpack() ??
> > > > Date: Fri, 16 May 2008 17:25:00 -0500
> > > >
> > > > Marlin Rowley wrote:
> > > > > All:
> > > > >
> > > > > I'm getting different floating point values when I use numpy
> > > vs. unpack().
> > > > >
> > > > > frgba = numpy.frombuffer(<string of bytes>, dtype=float32)
> > > > > buffer = unpack("!f", byte)
> > > > >
> > > > > frgba[0] != buffer[0]
> > > > >
> > > > > why? This is forcing me use the unpack() function since it's
> > > giving me
> > > > > the correct values. What am I doing wrong?
> > > >
> > > > Endianness, perhaps? '!' specifies big-endian data (an alias for
> > > '>'). Most
> > > > likely, you are on a little-endian platform. All of the dtypes
> > > in numpy default
> > > > to the native-endianness unless specified. If you want to read
> > > big-endian data
> > > > using numpy, do this:
> > > >
> > > > frgba = numpy.frombuffer(<string of bytes>, dtype='>f')
> > > >
> > > > If you have any more problems with numpy, please join us on the
> > > numpy mailing
> > > > list. When reporting problems, please try to provide a small but
> > > complete
> > > > snippet of self-contained code, the output that you got, and
> > > explain the output
> > > > that you expected to get. Thank you.
> > > >
> > > > http://www.scipy.org/Mailing_Lists
> > > >
> > > > --
> > > > Robert Kern
> > > >
> > > > "I have come to believe that the whole world is an enigma, a
> > > harmless enigma
> > > > that is made terrible by our own mad attempt to interpret it as
> > > though it had
> > > > an underlying truth."
> > > > -- Umberto Eco
> > > >
> > > > --
> > > > http://mail.python.org/mailman/listinfo/python-list
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > > E-mail for the greater good. Join the i’m Initiative from
> > > Microsoft.
> > > 
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood>
> > >
> > >
> > >
> > > 
> ------------------------------------------------------------------------
> > > E-mail for the greater good. Join the i’m Initiative from Microsoft.
> > > 
> <http://im.live.com/Messenger/IM/Join/Default.aspx?source=EML_WL_%20GreaterGood> 
>
> > >
> > > 
> ------------------------------------------------------------------------
> > >
> > > --
> > > http://mail.python.org/mailman/listinfo/python-list
> >
>
>
> ------------------------------------------------------------------------
> Keep your kids safer online with Windows Live Family Safety. Help 
> protect your kids. 
> <http://www.windowslive.com/family_safety/overview.html?ocid=TXT_TAGLM_WL_Refresh_family_safety_052008> 
>
> ------------------------------------------------------------------------
>
> --
> http://mail.python.org/mailman/listinfo/python-list