Characters aren't displayed correctly

[J. Clifford Dyer]

On Mon, 2009-03-02 at 00:33 -0800, Hussein B wrote:
> On Mar 1, 11:27 pm, "J. Clifford Dyer" <j... at sdf.lonestar.org> wrote:
> > On Sun, 2009-03-01 at 09:51 -0500, Philip Semanchuk wrote:
> > > On Mar 1, 2009, at 8:31 AM, Hussein B wrote:
> >
> > > > Hey,
> > > > I'm retrieving records from MySQL database that contains non english
> > > > characters.
> > > > Then I create a String that contains HTML markup and column values
> > > > from the previous result set.
> > > > +++++
> > > > markup = u'''<table>.....'''
> > > > for row in rows:
> > > >     markup = markup + '<tr><td>' + row['id']
> > > > markup = markup + '</table>
> > > > +++++
> > > > Then I'm sending the email according to this tip:
> > > >http://code.activestate.com/recipes/473810/
> > > > Well, the email contains ????? characters for each non english ones.
> > > > Any ideas?
> >
> > > There's so many places where this could go wrong and you haven't  
> > > narrowed down the problem.
> >
> > > Are the characters stored in the database correctly?
> >
> > > Are they stored consistently (i.e. all using the same encoding, not  
> > > some using utf-8 and others using iso-8859-1)?
> >
> > > What are you getting out of the database? Is it being converted to  
> > > Unicode correctly, or at all?
> >
> > > Are you sure that the program you're using to view the email  
> > > understands the encoding?
> >
> > > Isolate those questions one at a time. Add some debugging breakpoints.  
> > > Ensure that you have what you think you have. You might not fix your  
> > > problem, but you will make it much smaller and more specific.
> >
> > > Good luck
> > > Philip
> >
> > Let me add to that checklist:
> >
> > Are you sure the email you are creating has the encoding declared
> > properly in the headers?
> >
> >
> >
> >
> > Cheers,
> > Cliff
> 
> My HTML markup contains only table tags (you know, table, tr and td)

Ah.  The issue is not with the HTML markup, but the email headers.  For
example, the email you sent me has a header that says: 

Content-type: text/plain; charset="iso-8859-1"

Guessing from the recipe you linked to, you probably need something
like:

msgRoot['Content-type'] = 'text/plain; charset="utf-16"'

replacing utf-16 with whatever encoding you have encoded your email
with.

Or it may be that the header has to be attached to the individual mime
parts.  I'm not as familiar with MIME.


Cheers,
Cliff