ntome authored Aug 27, 2020 and hydrargyrum committed Sep 09, 2020

When switching to regular imports, ItemElementFromAbstractPage will become
obsolete. However, they will not be removed immediately, but they will be
reimplemented to be compatible with the new system. But the reimplementation
works at import-time, not at exec-time, and it's not possible to access params
defined in outer scopes at import-time. So, we make sure the attributes are
explicitly defined, not implicitly.

ntome authored Jan 31, 2020 and hydrargyrum committed Feb 12, 2020

A logger named only "iter_accounts" was not explicit enough anyway.

Lucas Ficheux authored Oct 08, 2019 and hydrargyrum committed Jan 08, 2020

Added fill_env method to AbstractElement so that it can be
overriden in pages methods.

It is useful to chose how Env behaves

ntome authored Aug 21, 2019 and hydrargyrum committed Aug 28, 2019

@Method wraps a class into a function, but then we cannot access the
class anymore. For introspection, we expose it in a "klass" attribute.

Quentin Defenouillere authored Jul 04, 2019 and hydrargyrum committed Jul 18, 2019

The empty_xpath attribute is used to fetch an explicit message on a page
when there is no account, no transaction or no investment for example.
The goal of this attribute is to avoid silent bugs when an item page has
completely changed and the item_xpath does not match any element
anymore: in these cases we don't scrape any element but we don't crash
either ; whereas if empty_xpath is defined at least we will have a
warning that the page may have changed.

Having "class obj_x(ItemElement)" to do nested object parsing is
possible.
Support "class obj_x(ListElement)" to parse a list of sub-objects, and
coerce to list instead of an iterator.

Baptiste Delpey authored Dec 21, 2017 and Romain Bignon committed May 05, 2018

don't crash handling non-mandatory fields
instead we fill them with the new FetchError EmptyField

ntome authored Jul 26, 2017 and Romain Bignon committed Aug 30, 2017

The string was incorrectly formatted

We sometimes raises a SkipItem exception in attributes parsing, instead
to use filters like "condition" of "validate" methods. Even it was not
designed for that before, it's sometimes useful and easier to do so.

Since we accept it as a valid behavior (upper levels handle this
exception), we should not flood our users with a warning

The syntax for using a metaclass is different in python2 and python3,
and there are existing helpers.

Add six as a dependency in setup.py.

TODO this is less efficient on python2

With: futurize -w -1 weboob

Vincent Ardisson authored Feb 10, 2017 and Romain Bignon committed Feb 18, 2017

Highlighting can use significant disk space. Don't enable it with "-a"
command-line flag anymore.
To enable highlighting for a particular backend, set "_highlight_el = 1"
in the backend config file.

hydrargyrum authored Jan 13, 2017 and Romain Bignon committed Feb 09, 2017

Browser2 HTML selectors tweak attributes of involved elements, then the
ItemElement writes an HTML file with the modified attributes and
restores the attributes.
This makes it easier to debug by showing visually for each object which
HTML elements were selected.

hydrargyrum authored Jan 24, 2017 and Romain Bignon committed Jan 25, 2017

Since Python 2.7 is the minimum version supported,
collections.OrderedDict is guaranteed to exist and there is no need to
reimplement it.

use new DictElement in modules

flake8 --select W391,E302,E301,E304

autopep8 can't fix W391 even though it claims it can.
Fixed using a simple custom script.

autopep8 -a -r -i --select E711,E712,W601,W602,W603,W604,W690,E304,E401,E502 .
Also includes some manual reindentations (many are left after the print()
changes).
Manually checked, some modernizations not commited here.

This is a cleaner approach that requires less configuration
in other applications.
This also easily allows us to have another color.

Many checks were made on being exactly at the DEBUG level, they were
fixed to also check on being below DEBUG (i.e. DEBUG_FILTERS).

weboob.tools.browser -> weboob.deprecated.browser
weboob.tools.parsers -> weboob.deprecated.browser.parsers
weboob.tools.mech -> weboob.deprecated.mech
weboob.browser2 -> weboob.browser
weboob.core.exceptions -> weboob.exceptions

Also, the new tree for browser2 is:

weboob.browser: import weboob.browser.browsers.* and weboob.browser.url.*
weboob.browser.browsers: all browsers (including PagesBrowser and LoginBrowser)
weboob.browser.url: the URL class
weboob.browser.profiles: all Profile classes
weboob.browser.sessions: WeboobSession and FuturesSession
weboob.browser.cookies: that's a cookies thing
weboob.browser.pages: all Page and derivated classes, and Form class
weboob.browser.exceptions: specific browser exceptions
weboob.browser.elements: AbstractElement classes, and 'method' decorator
weboob.browser.filters.*: all filters