- What weboob versions are supported?
- I'm using git/devel version and I encounter bugs after weboob-config update (like ImportError, AttributeError)!
- Is Python 3 the only supported version?
- Common problems
What weboob versions are supported?
TL;DR: weboob has a rolling release system for weboob apps + an internal update system for sites modules updates.
Weboob has 2 supported versions, stable and devel. Older versions do not receive modules updates.
Stable (current: 2.0)
The stable
version is commonly installed from pip or from distro packages. This is the recommended version for end-users.
It is considered stable because the core scraping library and its applications do not change. Stable releases typically occur on a yearly basis.
Scraping is volatile and websites often change, so even on the stable version, the weboob modules do receive updates frequently. The modules updates are cryptographically signed and published using weboob's internal update system (stored at https://updates.weboob.org/stable/). On the stable version, modules are updated by running weboob update
command (or in Qt apps, in the modules configuration).
Devel (current: 2.1)
This is the development version, commonly installed directly from the git
repository. This is recommended only for developers.
The core library and the applications are under development on this version and are often updated. When a weboob release is made, a snapshot of the devel
version state becomes the new stable
version.
While the devel version can receive modules updates through weboob's internal update system, it's recommended to fetch code directly from the git repository (using git-pull
). The weboob update
command only updates modules, not the core library, but on the devel version, modules may require new features from the core library, unlike on the stable version. To do so, one should edit the config file ~/.config/weboob/sources.list
to use the git
repository dir, and disable updates.weboob.org
.
weboob-config update
(like ImportError
, AttributeError
)!
I'm using git/devel version and I encounter bugs after TL;DR: if you don't do development or don't need command updates, use the stable
version, it's simpler.
In the weboob master
branch, the modules are frequently updated, and so is the weboob core library. weboob-config update
only fetches modules updates, not core library updates. However, some module updates require core updates to work properly.
So if you cloned the weboob git repository to use the master branch and installed it from there, weboob-config update
will not perform core updates, you should also run git-pull
frequently to have an up-to-date core.
But that's not enough! The commands and library are not installed in the git directory. The git repository might be something like ~/dev/weboob
, but when you typically ran pip install .
at first (or pipsi
or whatever), it copied a snapshot of the library and commands to (usually) ~/.local/bin
and ~/.local/lib/python3.X
, which are not updated when you simply run git-pull
...
After git-pull
, you should also run pip install .
again (or whatever command you used to install it) to have an up-to-date copy in ~/.local/lib/python3.X
. pip install -e .
doesn't have this drawback, as it merely points to the git repository dir, instead of snapshoting it.
If you installed through git repository/master branch, don't forget to git-pull
and pip install .
again along with weboob-config update
.
The stable version doesn't have this kind of issue, since the core updates are embedded in the modules, so updating the stable version is only a matter of weboob-config update
.
Is Python 3 the only supported version?
Python 3 is the main supported version. Python 2 may still be supported but will not be supported anymore in 2021.
Common problems
How to solve CAPTCHAs?
Though CAPTCHAs bring only fake protection and no value, websites use them. Weboob allows automated CAPTCHA solving using CapCaptchaSolver
. For now, this is limited to paid (cheap) services AntiCaptcha and DeathByCaptcha.
From a user point-of-view, some apps like boobank
can leverage this capability. In ~/.config/weboob/backends
, add something like
[anticaptcha]
_module = anticaptcha
api_key = <your api key>
When a command requires solving a CAPTCHA, it will be solved asynchronously and when it's done, the command should be restarted.
From a module developer point-a-view, raise the appropriate weboob.exceptions.CaptchaQuestion
subclass and a solver (if configured by the user) will solve it.
Weboob doesn't support yet user solving a CAPTCHA.
DH_KEY_TOO_SMALL
error?
What if I encounter This happens because sites (especially banks) have bad security and by default OpenSSL rejects those TLS configurations. It's not specific to weboob, even with cURL, you would get the same error. There are 2 solutions:
- either change the OpenSSL configuration, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907788#14
- or use NSS instead of OpenSSL, by passing
--nss
toboobank
command for example, it will require additional dependencies to be installed
boobank
) in a script, or when redirecting its stdout to a file or pipe, why do I get an UnicodeDecodeError
or UnicodeEncodeError
?
When running a weboob command (like If this does not happen in interactive mode, this is because Python considers stdout to be in ascii
mode when stdout is not a tty
.
This may happen often with Python 2, rarely with Python 3. Try setting an env variable before running boobank
or any other weboob command: export PYTHONIOENCODING=utf-8
.