io module and pdf question

[Dave Angel]

On 06/25/2013 12:15 PM, jyoung79 at kc.rr.com wrote:
> Thank you Rusi and Christian!
>

Something I don't think was mentioned was that reading a text file in 
Python 3, and specifying latin-1, will work simply because every 
possible 8-bit byte is a character in Latin-1  That doesn't mean that 
those characters you get have any connection with the real meaning of 
the file.

> So it sounds like I should read the pdf data in as binary:
>
> --------------------
> import os
>
> pdfPath = '~/Desktop/test.pdf'
>
> colorlistData = ''
>
> with open(os.path.expanduser(pdfPath), 'rb') as f:
>      for i in f:
>          if 'XYZ:colorList' in i:
>              colorlistData = i.split('XYZ:colorList')[1]
>              break
>
> print(colorlistData)
> --------------------
>
> This gives me the error:
> TypeError: Type str doesn't support the buffer API

That's just a tiny piece of the error.  Post the full traceback, which 
shows the line that fails, and what called it, and so on.  In this case 
I'd guess that the line:
     for i in f:

is failing since that mechanism is for reading lines in a text file. 
For reading streams of bytes, you have the read() method, where you 
supply your own count.

>
> I admit I know nothing about binary, except it's ones and zeroes.  Is there a way to read it in as binary, convert it to ascii/unicode,

That makes no sense without knowing what the binary data represents.  It 
MIGHT be that pieces of it will actually be valid ascii, or valid 
unicode (encoded with some encoding).  But you would have to ask the 
author, or look up the spec for that particular binary file format.

I'm not familiar at all with how PDF's are encoded, so I don't know what 
the possibilities.

One hacky approach is to use the strings utility (standard on most 
versions of Unix/Linux) to basically throw out most of the file, keeping 
only those portions of it that happen to look like reasonable ASCII.  By 
default it captures each consecutive sequence of at least 4 ASCII 
printable characters, and puts a newline to represent one or more 
unprintable or non-ASCII characters between them.

If you cannot find strings (or string) for your OS, you can write the 
filter yourself.

But much better would be to use some library that understood the PDF 
format rules.

> and then somehow split it by newline characters so that I can pull the appropriate metadata lines out?  For example, XYZ:colorList="DarkBlue,Yellow"
>
> Thanks!
>
> Jay
>
> --
>
>> Most of the PDF objects are therefore not encoded. It is, however,
>> possible to include a PDF into another PDF and to encode it, but that's
>> a rare case. Therefore the metadata can usually be read in text mode.
>> However, to correctly find all objects, the xref-table indexes offsets
>> into the PDF. It must be treated binary in any case, and that's the
>> funny reason for the first 3 characters of the PDF - they must include
>> characters with the 8th bit set, such that FTP applications treat it as
>> binary.
>
>> 	Christian


-- 
DaveA