Changing filenames from Greeklish => Greek (subprocess complain)

[MRAB]

On 06/06/2013 04:43, Νικόλαος Κούρας wrote:
> Τη Τετάρτη, 5 Ιουνίου 2013 9:43:18 μ.μ. UTC+3, ο χρήστης Νικόλαος Κούρας έγραψε:
> > Τη Τετάρτη, 5 Ιουνίου 2013 9:32:15 μ.μ. UTC+3, ο χρήστης MRAB έγραψε:
> >
> > > On 05/06/2013 18:43, οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½ οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½ wrote:
> >
> > >
> >
> > > > οΏ½οΏ½ οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½, 5 οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½ 2013 8:56:36 οΏ½.οΏ½. UTC+3, οΏ½ οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½ Steven D'Aprano οΏ½οΏ½οΏ½οΏ½οΏ½οΏ½:
> >
> > >
> >
> > > >
> >
> > >
> >
> > > > Somehow, I don't know how because I didn't see it happen, you have one or
> >
> > >
> >
> > > > more files in that directory where the file name as bytes is invalid when
> >
> > >
> >
> > > > decoded as UTF-8, but your system is set to use UTF-8. So to fix this you
> >
> > >
> >
> > > > need to rename the file using some tool that doesn't care quite so much
> >
> > >
> >
> > > > about encodings. Use the bash command line to rename each file in turn
> >
> > >
> >
> > > > until the problem goes away.
> >
> > >
> >
> > > >
> >
> > >
> >
> > ' leade to that unknown encoding of this bytestream '\305\365\367\336\ \364\357\365\ \311\347\363\357\375.mp3'
> >
> > >
> >
> > > >
> >
> > >
> >
> > > > But please tell me Steven what linux tool you think it can encode the weird filename to proper 'οΏ½οΏ½οΏ½οΏ½ οΏ½οΏ½οΏ½ οΏ½οΏ½οΏ½οΏ½οΏ½.mp3' utf-8?
> >
> > >
> >
> > > >
> >
> > >
> >
> > > > or we cna write a script as i suggested to decode back the bytestream using all sorts of available decode charsets boiling down to the original greek letters.
> >
> > >
> >
> > > >
> >
> > >
> >
> >
> >
> >
> >
> > Actually you were correct i was typing greek and is aw the fileneme here in gogole groups as:
> >
> >
> >
> > > > But renaming ia hsell access like 'mv 'Euxi tou Ihsou.mp3' 'οΏ½οΏ½οΏ½οΏ½ οΏ½οΏ½οΏ½ οΏ½οΏ½οΏ½οΏ½οΏ½.mp3
> >
> >
> >
> > so maybe the filenames have to be decoded to greek-iso but then agian the contain both greek letters but their extension are in english chars like '.mp3'
> >
> >
> >
> >
> >
> > > Using Python, I think you could get the filenames using os.listdir,
> >
> > > passing the directory name as a bytestring so that it'll return the
> >
> > > names as bytestrings.
> >
> >
> >
> >
> >
> > > Then, for each name, you could decode from its current encoding and
> >
> > > encode to UTF-8 and rename the file, passing the old and new paths to
> >
> > > os.rename as bytestrings.
> >
> >
> >
> > Iam not sure i follow:
> >
> >
> >
> > Change this:
> >
> >
> >
> > # Compute a set of current fullpaths
> >
> > fullpaths = set()
> >
> > path = "/home/nikos/public_html/data/apps/"
> >
> >
> >
> > for root, dirs, files in os.walk(path):
> >
> > 	for fullpath in files:
> >
> > 		fullpaths.add( os.path.join(root, fullpath) )
> >
> >
> >
> >
> >
> > to what to make the full url readable by files.py?
>
> MRAB can you please explain in more clarity your idea of solution?
I was suggesting a way to rename the files so that their names are 
encoded in UTF-8 (they appear to be encoded in ISO-8859-7).

You MUST TEST IT thoroughly first, of course, before trying it on the 
actual files.

It could go something like this:


import os

# Give the path as a bytestring so that we'll get the names as bytestrings.
root_folder = b"/home/nikos/public_html/data/apps/"

# Setting TESTING to True will make it print out what renamings it will 
do, but
# not actually do them.
TESTING = True

# Walk through the files.
for root, dirs, files in os.walk(root_folder):
     for name in files:
         try:
             # Is this name encoded in UTF-8?
             name.decode("utf-8")
         except UnicodeDecodeError:
             # Decoding from UTF- failed, which means that the name is 
not valid
             # UTF-8.

             # It appears (from elsewhere) that the names are encoded in
             # ISO-8859-7, so decode from that and re-encode to UTF-8.
             new_name = name.decode("iso-8859-7").encode("utf-8")

             old_path = os.path.join(root, name)
             new_path = os.path.join(root, new_name)
             if TESTING:
                 print("Will rename {!r} to {!r}".format(old_path, 
new_path))
             else:
                 print("Renaming {!r} to {!r}".format(old_path, new_path))
                 os.rename(old_path, new_path)