Skip to content

weboob
weboob

The new woob repository is here: https://gitlab.com/woob/woob. This gitlab will be removed soon.

faq

Last edited by ntome
Page history

What weboob versions are supported?

TL;DR: weboob has a rolling release system for weboob apps + an internal update system for sites modules updates.

Weboob has 2 supported versions, stable and devel. Older versions do not receive modules updates.

Stable (current: 2.0)

The stable version is commonly installed from pip or from distro packages. This is the recommended version for end-users.

It is considered stable because the core scraping library and its applications do not change. Stable releases typically occur on a yearly basis.

Scraping is volatile and websites often change, so even on the stable version, the weboob modules do receive updates frequently. The modules updates are cryptographically signed and published using weboob's internal update system (stored at https://updates.weboob.org/stable/). On the stable version, modules are updated by running weboob update command (or in Qt apps, in the modules configuration).

Devel (current: 2.1)

This is the development version, commonly installed directly from the git repository. This is recommended only for developers.

The core library and the applications are under development on this version and are often updated. When a weboob release is made, a snapshot of the devel version state becomes the new stable version.

While the devel version can receive modules updates through weboob's internal update system, it's recommended to fetch code directly from the git repository (using git-pull). The weboob update command only updates modules, not the core library, but on the devel version, modules may require new features from the core library, unlike on the stable version. To do so, one should edit the config file ~/.config/weboob/sources.list to use the git repository dir, and disable updates.weboob.org.

I'm using git/devel version and I encounter bugs after weboob-config update (like ImportError, AttributeError)!

TL;DR: if you don't do development or don't need command updates, use the stable version, it's simpler.

In the weboob master branch, the modules are frequently updated, and so is the weboob core library. weboob-config update only fetches modules updates, not core library updates. However, some module updates require core updates to work properly.

So if you cloned the weboob git repository to use the master branch and installed it from there, weboob-config update will not perform core updates, you should also run git-pull frequently to have an up-to-date core.

But that's not enough! The commands and library are not installed in the git directory. The git repository might be something like ~/dev/weboob, but when you typically ran pip install . at first (or pipsi or whatever), it copied a snapshot of the library and commands to (usually) ~/.local/bin and ~/.local/lib/python3.X, which are not updated when you simply run git-pull...

After git-pull, you should also run pip install . again (or whatever command you used to install it) to have an up-to-date copy in ~/.local/lib/python3.X. pip install -e . doesn't have this drawback, as it merely points to the git repository dir, instead of snapshoting it.

If you installed through git repository/master branch, don't forget to git-pull and pip install . again along with weboob-config update.

The stable version doesn't have this kind of issue, since the core updates are embedded in the modules, so updating the stable version is only a matter of weboob-config update.

Is Python 3 the only supported version?

Python 3 is the main supported version. Python 2 may still be supported but will not be supported anymore in 2021.

Common problems

How to solve CAPTCHAs?

Though CAPTCHAs bring only fake protection and no value, websites use them. Weboob allows automated CAPTCHA solving using CapCaptchaSolver. For now, this is limited to paid (cheap) services AntiCaptcha and DeathByCaptcha.

From a user point-of-view, some apps like boobank can leverage this capability. In ~/.config/weboob/backends, add something like

[anticaptcha]
_module = anticaptcha
api_key = <your api key>

When a command requires solving a CAPTCHA, it will be solved asynchronously and when it's done, the command should be restarted.

From a module developer point-a-view, raise the appropriate weboob.exceptions.CaptchaQuestion subclass and a solver (if configured by the user) will solve it.

Weboob doesn't support yet user solving a CAPTCHA.

What if I encounter DH_KEY_TOO_SMALL error?

This happens because sites (especially banks) have bad security and by default OpenSSL rejects those TLS configurations. It's not specific to weboob, even with cURL, you would get the same error. There are 2 solutions:

When running a weboob command (like boobank) in a script, or when redirecting its stdout to a file or pipe, why do I get an UnicodeDecodeError or UnicodeEncodeError?

If this does not happen in interactive mode, this is because Python considers stdout to be in ascii mode when stdout is not a tty.

This may happen often with Python 2, rarely with Python 3. Try setting an env variable before running boobank or any other weboob command: export PYTHONIOENCODING=utf-8.