High ascii bytes in folder names can break folder handling

[menno]

Fixed in encoding-bug branch but I'm not sure this is the correct way to solve the problem. Revisit.

[menno]

Ensure that this is no longer an issue:

In [8]: i.delete_folder('abcd\xff')
---------------------------------------------------------------------------
UnicodeDecodeError                        Traceback (most recent call last)

/home/msmits/<ipython console>

/usr/local/lib64/python2.4/site-packages/IMAPClient-0.5-py2.4.egg/imapclient/imapclient.pyc 
in delete_folder(self, folder)
     283         @return: Server response.
     284         '''
--> 285         typ, data = 
self._imap.delete(self._encode_folder_name(folder))
     286         self._checkok('delete', typ, data)
     287         return data[0]

/usr/local/lib64/python2.4/site-packages/IMAPClient-0.5-py2.4.egg/imapclient/imapclient.pyc 
in _encode_folder_name(self, name)
     570     def _encode_folder_name(self, name):
     571         if self.folder_encode:
--> 572             return imap_utf7.encode(name)
     573         return name
     574

/usr/local/lib64/python2.4/site-packages/IMAPClient-0.5-py2.4.egg/imapclient/imap_utf7.pyc 
in encode(s)
      41             _in.append(c)
      42     if _in:
---> 43         r.extend(['&', modified_base64(''.join(_in)), '-'])
      44     return ''.join(r)
      45

/usr/local/lib64/python2.4/site-packages/IMAPClient-0.5-py2.4.egg/imapclient/imap_utf7.pyc 
in modified_base64(s)
      67
      68 def modified_base64(s):
---> 69     s_utf7 = s.encode('utf-7')
      70     return s_utf7[1:-1].replace('/', ',')
      71

UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: 
ordinal not in range(128)

[menno]

What I don't like about the current fix is that you can create a folder passing a name like 'Left\xffRight' and then get back a folder name from list_folders() that isn't the same. IMAPClient is performing a little too much magic for my liking.

Instead of the transparent unicode conversion magic using latin-1, just raise a friendly ValueError? to indicate that a unicode object should be passed if out of range characters are passed (see section 5.1.3 of the IMAP RFC).

Since unicode objects can sometimes come back now, also change folder encoding to always return unicode objects. This is an API change and should be noted as such.

This is also good practice for easing Python 3 conversion.

Ensure there are unit and live tests that exercise internationalised folder names.

[menno]

Fixed in r147 and r148.