Having "class obj_x(ItemElement)" to do nested object parsing is
possible.
Support "class obj_x(ListElement)" to parse a list of sub-objects, and
coerce to list instead of an iterator.

Baptiste Delpey authored Dec 21, 2017 and Romain Bignon committed May 05, 2018

don't crash handling non-mandatory fields
instead we fill them with the new FetchError EmptyField

ntome authored Jul 26, 2017 and Romain Bignon committed Aug 30, 2017

The string was incorrectly formatted

We sometimes raises a SkipItem exception in attributes parsing, instead
to use filters like "condition" of "validate" methods. Even it was not
designed for that before, it's sometimes useful and easier to do so.

Since we accept it as a valid behavior (upper levels handle this
exception), we should not flood our users with a warning

The syntax for using a metaclass is different in python2 and python3,
and there are existing helpers.

Add six as a dependency in setup.py.

TODO this is less efficient on python2

With: futurize -w -1 weboob

Vincent Ardisson authored Feb 10, 2017 and Romain Bignon committed Feb 18, 2017

Highlighting can use significant disk space. Don't enable it with "-a"
command-line flag anymore.
To enable highlighting for a particular backend, set "_highlight_el = 1"
in the backend config file.

hydrargyrum authored Jan 13, 2017 and Romain Bignon committed Feb 09, 2017

Browser2 HTML selectors tweak attributes of involved elements, then the
ItemElement writes an HTML file with the modified attributes and
restores the attributes.
This makes it easier to debug by showing visually for each object which
HTML elements were selected.

hydrargyrum authored Jan 24, 2017 and Romain Bignon committed Jan 25, 2017

Since Python 2.7 is the minimum version supported,
collections.OrderedDict is guaranteed to exist and there is no need to
reimplement it.

use new DictElement in modules

flake8 --select W391,E302,E301,E304

autopep8 can't fix W391 even though it claims it can.
Fixed using a simple custom script.

autopep8 -a -r -i --select E711,E712,W601,W602,W603,W604,W690,E304,E401,E502 .
Also includes some manual reindentations (many are left after the print()
changes).
Manually checked, some modernizations not commited here.

This is a cleaner approach that requires less configuration
in other applications.
This also easily allows us to have another color.

Many checks were made on being exactly at the DEBUG level, they were
fixed to also check on being below DEBUG (i.e. DEBUG_FILTERS).

weboob.tools.browser -> weboob.deprecated.browser
weboob.tools.parsers -> weboob.deprecated.browser.parsers
weboob.tools.mech -> weboob.deprecated.mech
weboob.browser2 -> weboob.browser
weboob.core.exceptions -> weboob.exceptions

Also, the new tree for browser2 is:

weboob.browser: import weboob.browser.browsers.* and weboob.browser.url.*
weboob.browser.browsers: all browsers (including PagesBrowser and LoginBrowser)
weboob.browser.url: the URL class
weboob.browser.profiles: all Profile classes
weboob.browser.sessions: WeboobSession and FuturesSession
weboob.browser.cookies: that's a cookies thing
weboob.browser.pages: all Page and derivated classes, and Form class
weboob.browser.exceptions: specific browser exceptions
weboob.browser.elements: AbstractElement classes, and 'method' decorator
weboob.browser.filters.*: all filters