(societegenerale or tools.application.formatter.csv) csv formatter in boobank can't handle Unicode
Related to Bug #4014 on old tracker
Using societegenerale module from boobank (1.3), I get a Unicode error fetching one of my accounts :
In the following log I skipped a few lines since I don't think they're interesting and would take a while to anon. The command I used is the command Skrooge is using to fetch data for accounts, with added -d flag and without -q flag ; and it works perfectly fine on other accounts at the same bank.
I think the problem is not really with boobank or societegenerale module, but more with how csv formatter handles non-ASCII characters. Is there an option to use UTF-8 for csv formatter that we should know of ?
$ weboob-config update
=== [ 0%] Getting https://updates.weboob.org/1.3/main/
=== [100%] All modules are up-to-date.
$ boobank -f csv history "ACCOUNT_ID@societegenerale" -s rdate,type,raw,label,amount --condition "rdate>2000-01-01 OR date>2000-01-01 OR rdate>2000-01-01 00:00:00 OR date>2000-01-01 00:00:00" -n 99999 -d
...
rdate;type;raw;label;amount
2017-08-27 15:36:15,324:INFO:backend.societegenerale.browser:1.3:browsers.py:839:dump_state Stored cookies into storage
Traceback (most recent call last):
File "/home/gagbo/.local/bin/boobank", line 27, in <module>
Boobank.run()
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/console.py", line 219, in run
super(ConsoleApplication, klass).run(args)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/base.py", line 482, in run
sys.exit(app.main(args))
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/applications/boobank/boobank.py", line 721, in main
return super(Boobank, self).main(argv)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/repl.py", line 312, in main
ret = self.onecmd(cmd)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/repl.py", line 437, in onecmd
return super(ReplApplication, self).onecmd(line)
File "/usr/lib64/python2.7/cmd.py", line 221, in onecmd
return func(arg)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/repl.py", line 83, in inner
return f(self, *args, **kwargs)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/applications/boobank/boobank.py", line 504, in do_history
return self.show_history('iter_history', line)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/applications/boobank/boobank.py", line 485, in show_history
self.format(transaction)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/repl.py", line 1213, in format
self.formatter.format(obj=result, selected_fields=fields, alias=alias)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/formatters/iformatter.py", line 186, in format
formatted = self.format_obj(obj, alias)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/formatters/iformatter.py", line 220, in format_obj
return self.format_dict(obj.to_dict())
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/formatters/csv.py", line 42, in format_dict
return self.write_dict(item, self.outfile)
File "/home/gagbo/.local/lib/python2.7/site-packages/weboob/tools/application/formatters/csv.py", line 53, in write_dict
writer.writerow([unicode(v) for v in item.values()])
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 38: ordinal not in range(128)