Characters aren't displayed correctly

[J. Clifford Dyer]

On Mon, 2009-03-02 at 06:16 -0800, Hussein B wrote:
> On Mar 2, 4:03 pm, "J. Clifford Dyer" <j... at sdf.lonestar.org> wrote:
> > On Mon, 2009-03-02 at 00:33 -0800, Hussein B wrote:
> > > On Mar 1, 11:27 pm, "J. Clifford Dyer" <j... at sdf.lonestar.org> wrote:
> > > > On Sun, 2009-03-01 at 09:51 -0500, Philip Semanchuk wrote:
> > > > > On Mar 1, 2009, at 8:31 AM, Hussein B wrote:
> >
> > > > > > Hey,
> > > > > > I'm retrieving records from MySQL database that contains non english
> > > > > > characters.
> > > > > > Then I create a String that contains HTML markup and column values
> > > > > > from the previous result set.
> > > > > > +++++
> > > > > > markup = u'''<table>.....'''
> > > > > > for row in rows:
> > > > > >     markup = markup + '<tr><td>' + row['id']
> > > > > > markup = markup + '</table>
> > > > > > +++++
> > > > > > Then I'm sending the email according to this tip:
> > > > > >http://code.activestate.com/recipes/473810/
> > > > > > Well, the email contains ????? characters for each non english ones.
> > > > > > Any ideas?
> >
> > > > > There's so many places where this could go wrong and you haven't  
> > > > > narrowed down the problem.
> >
> > > > > Are the characters stored in the database correctly?
> >
> > > > > Are they stored consistently (i.e. all using the same encoding, not  
> > > > > some using utf-8 and others using iso-8859-1)?
> >
> > > > > What are you getting out of the database? Is it being converted to  
> > > > > Unicode correctly, or at all?
> >
> > > > > Are you sure that the program you're using to view the email  
> > > > > understands the encoding?
> >
> > > > > Isolate those questions one at a time. Add some debugging breakpoints.  
> > > > > Ensure that you have what you think you have. You might not fix your  
> > > > > problem, but you will make it much smaller and more specific.
> >
> > > > > Good luck
> > > > > Philip
> >
> > > > Let me add to that checklist:
> >
> > > > Are you sure the email you are creating has the encoding declared
> > > > properly in the headers?
> >
> > > > Cheers,
> > > > Cliff
> >
> > > My HTML markup contains only table tags (you know, table, tr and td)
> >
> > Ah.  The issue is not with the HTML markup, but the email headers.  For
> > example, the email you sent me has a header that says:
> >
> > Content-type: text/plain; charset="iso-8859-1"
> >
> > Guessing from the recipe you linked to, you probably need something
> > like:
> >
> > msgRoot['Content-type'] = 'text/plain; charset="utf-16"'
> >
> > replacing utf-16 with whatever encoding you have encoded your email
> > with.
> >
> > Or it may be that the header has to be attached to the individual mime
> > parts.  I'm not as familiar with MIME.
> >
> > Cheers,
> > Cliff
> 
> Hey Cliff,
> I tried your tip and I still get the same thing (?????)
> I added print statement to print each value of the result set into the
> console, which also prints ???? characters instead of the real
> characters values.
> Maybe a conversion is happened upon getting the data from the
> database?
> (the values are stored correctly in the database)

Sorry for responding so late.

Getting ? characters when you print to the console does not
*necessarily* mean python is doing anything wrong.  It may be outputting
the correct characters, but your console lacks the proper font for
displaying those characters.  Try doing for c in email_body: print
hex(ord(c)), and see if the results correspond to what you're trying to
get.

> --
> http://mail.python.org/mailman/listinfo/python-list
> 

Cheers,
Cliff