woob:f1079c783892360dedace28571143b271d23dc62 commitshttps://gitlab.com/woob/woob/-/commits/f1079c783892360dedace28571143b271d23dc622020-10-08T13:06:04+02:00https://gitlab.com/woob/woob/-/commit/f1079c783892360dedace28571143b271d23dc62[pagesjaunes] fix broken backend2020-10-08T13:06:04+02:00Bezleputhcarton_ben@yahoo.frhttps://gitlab.com/woob/woob/-/commit/06930540a69f7b527a148c2506235104e838fc0clxml: replace deprecated etree getiterator/getchildren2020-10-07T18:39:27+02:00Vincent Ardissonvardisson@budget-insight.com
These methods were deprecated in stdlib xml.etree and also in lxml, and
are ultimately removed in python 3.9.https://gitlab.com/woob/woob/-/commit/c19a0f02114e961a738bd8e1f144337fe4a6273f[bp] Handle new Certicode error message2020-10-07T18:39:27+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/de6e967ae62c7785c12f27e51f831d2acc4378d0[lcl] Add transfer_check_account_iban2020-10-07T18:39:27+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.com
Some accounts' ibans cannot be found anymore on the website. But since we
kept the iban stored on our side, the 'old' transfer.account_iban is not
empty when making a transfer. When we do not find the account based on its iban,
we search it based on its id. So the account is valid, the iban is just different.
This check allows to not have an assertion error when making a transfer from
an account in this situation.https://gitlab.com/woob/woob/-/commit/bed707b943bf19b829366bd2bc77b8b5f53cc760[ing] Use old URLs from boursedirect parent2020-10-07T18:39:26+02:00Christophe Francoischristophe.francois@budget-insight.com
Some URLs have been updated on Bourse Direct but not yet on ING. For now
we need to specify the old URLs in the abstract browser.https://gitlab.com/woob/woob/-/commit/908141f052c3401bf24f07f0568590cac98d8a06[boursedirect] New URLs & use TableElement for iter_history2020-10-07T18:39:26+02:00Christophe Francoischristophe.francois@budget-insight.com
URLs for market accounts and history have been updated.
The xpath for the history table has also changed, I took the time to
make iter_history use TableElement.https://gitlab.com/woob/woob/-/commit/5cab9c18e87b6214d2e636e0484126517d4dd1ed[creditmutuel] Fix: Handle SafeTrans page2020-10-07T18:39:26+02:00Damien Matdamien.mat@budget-insight.com
This page may appear just after login.
It is a 2FA method not handled yet.
The message will inform the user to switch to 'Confirmation Mobile',
which is an AppValidation method.https://gitlab.com/woob/woob/-/commit/8dfba9da16fbef45986db46d0082db94f633e9a6[caissedepargne] Handle CloudCard for transfers2020-10-07T18:39:26+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.com
CloudCard (aka Secur'Pass) is a method that asks for an AppValidation
when adding new recipients or making transfer when not on the Caisse
d'Epargne application.https://gitlab.com/woob/woob/-/commit/82a15a2be92b6544fe402cb6dfa1c563dfb9a83a[boursedirect] Handle new url after login2020-10-07T18:39:26+02:00Christophe Francoischristophe.francois@budget-insight.com
After login, we are now on the "inventory" page, which was not handled.
There is nothing to do on the page, the rest of the browsing looks
unchanged.https://gitlab.com/woob/woob/-/commit/1eb8659e98eeacb60dbae0ef900b6bd9dfe183f1[s2e] Handle case where BNpInvestmentsPage is not accessible2020-10-07T18:39:26+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
For those investments, trying the next request would lead to a 401
error. Also handle cases where BNP API is not responding, which seems to
be quite often.
Note: the invest will still be scraped but we will not be able to fetch
some of its details.https://gitlab.com/woob/woob/-/commit/b16497f919b6280fbf5d9e7c02840d2564cf7563weboob.capabilities.base: Add '$US' to USD currency symbols2020-10-07T18:39:26+02:00Christophe Francoischristophe.francois@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/d9a8ed97fce5f5b90cd85f763342946a60d2beeb[creditmutuel] add url for revolving loan2020-10-07T18:39:26+02:00Julien Montagnatjulien.montagnat@budget-insight.com
add new url to match with the revolving loan pagehttps://gitlab.com/woob/woob/-/commit/e0fcfcd25d6e5209609c56189e813e7063a89bd8[boursorama] Handle error message on new transfers page2020-10-07T18:39:26+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.com
When no account can emit transfers, we have an error message where we
expected a form and the list of account.https://gitlab.com/woob/woob/-/commit/6fde01ae4661966bef01e29c731d7259a772cca5[boursorama] Feat: add card summaries on card accounts2020-10-07T18:39:26+02:00Damien Matdamien.mat@budget-insight.com
When CSV is not availble (most cases),
we get card_summaries transactions on parent accounts history page.
In order to do that, and get as well summary details,
we now start from parent account history page,
parse card_summaries,
and follow detail page for each one.https://gitlab.com/woob/woob/-/commit/0ffb9506effa97d5351ded132f02550afbede91c[bred] total_offset in history to avoid infinite loop and less max requests2020-10-07T18:39:26+02:00Dorian Rolydorian.roly@budget-insight.com
It seems that, in certain cases, the loop can be infinite when the
website is returning the transactions endlessly.
total_offset is here to block the while. If an account has more than
100 000 transactions, they will not have older transactions.
Because of the total_offset that previously was at 100 000 which led
at 4000 request max, I've decided to reduce the total_offset max to
50 000, halving the number of requests.https://gitlab.com/woob/woob/-/commit/fef831dc1972fafb0a5b4954f9cecd62a29f5c6c[creditdunord] Added Investment.unitprice for market accounts2020-10-07T18:39:26+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
I don't know why I didn't include it when I wrote this part, maybe this
column didn't appear back then. Or maybe I had no idea what I was doing.https://gitlab.com/woob/woob/-/commit/e2d88f103fad168fcd7c76800110deed48eb2202[cragr] Corrected get_profile2020-10-07T18:39:25+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
Sometimes profile details are not available, leading to a 404 error.https://gitlab.com/woob/woob/-/commit/4cb855f5ee0c17f8532cd2a250e5ae0989458273[unsplash] new CC0 image search module2020-09-27T16:40:06+02:00Vincent Adev@indigo.rehttps://gitlab.com/woob/woob/-/commit/6e4857310844a998d104c8a2959f07d4af36111f[docs] Cosmetic improvement + few minor fixes in the doc2020-09-26T12:50:36+02:00Florent Viardflorent.viard@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/5435400d9d19edd34a97d7dee619d70a8aef659c[docs] Add note in Contribute about the ipv6 restriction when using weboob gi...2020-09-26T12:48:58+02:00Florent Viardflorent.viard@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/75abe3cf0c6596d6ce068322760a171197a5f152[docs] Fixes typos in the code of 2 sections of the cookbook2020-09-26T12:48:58+02:00Florent Viardflorent.viard@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/be7e0390427ad125d598b680cb4403a5fd38e77b[boursorama] Fix login url2020-09-23T18:55:35+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.com
When doing a request on the login url with a `/` at the end, we receive
a 301 and are redirected to the same url without a `/` at the end. And
because of the `/` that is expected in the url, we end up not finding
the page.
Removing the `/` at the end in the regex leads to no redirection and it
matches both cases.https://gitlab.com/woob/woob/-/commit/77c713b45c7077bbcd019fe5e37c3ee38a4a91b5[banquepopulaire] add new cases of login retry without phase parameter2020-09-23T18:55:35+02:00Maxime Gasselinmaxime.gasselin@budget-insight.com
Numerous wrongpass are still present. We let the retry to decide if we are
in a real wrongpass case.https://gitlab.com/woob/woob/-/commit/57fbd3d07daba58a03e9c2429817d045ddf9be07tools: add har-to-old.py to convert HAR bundles into the old format2020-09-23T18:55:35+02:00Vincent Ardissonvardisson@budget-insight.com
Thanks to this tool, we will be able to drop dumping to the old, custom,
hard-to-parse format in favor of the new, ill-defined format: HAR!https://gitlab.com/woob/woob/-/commit/99ca803bd2b1d9ac96286b5ec7215005a6db109fweboob/browser/browsers: when dumping HAR, add a key to tell binary data2020-09-23T18:55:35+02:00Vincent Ardissonvardisson@budget-insight.com
For binary response, HAR allows base64 encoding so data is untouched.
But no such thing exists for binary posted data. Some generators write JSON
with the binary posted data as if it were latin-1 encoded text.
But when reading, what can we do? Think it's binary to encode back to latin-1?
Or think it's text?
HAR gives no answer, so we add a non-standard "x-binary" key to indicate
"text" is actually binary disguised with latin-1.https://gitlab.com/woob/woob/-/commit/8642772c01581167924119ad27e5e94b703ae9b5[axabanque] Fix iter_accounts condition2020-09-23T18:55:35+02:00Christophe Francoischristophe.francois@budget-insight.com
The condition was in ListElement instead of ItemElement and some
accounts were filtered out by mistake.https://gitlab.com/woob/woob/-/commit/976c68dd08d3e6a5221d0b943d6147103d043fc5[bp] Fix transfer execution with certicode2020-09-23T18:55:35+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/fdf714bd29bd4b8def714cb439abc77676c59be0[amazon] refactor the AppValidation with a specific url and global xpaths2020-09-23T18:55:35+02:00thibault dougethibault.douge@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/6a53adf1016d945d84e7ddba0c16b14b7df2c95f[lcl] Fix: avoid AttributeError on transactions page2020-09-23T18:55:35+02:00Damien Matdamien.mat@budget-insight.com
When amount is not found by the first parsing,
it must be done by a second one.
I changed the condition on which this is triggered,
since it not needed anymore,
and would throw an AttributeError when page is parsed a second time.https://gitlab.com/woob/woob/-/commit/ca86b06f5a5ee306b9e87d33cb0a934f68652900[netfinca] New method check_action_needed2020-09-23T18:55:35+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
This is mainly used during iter_accounts, but also in the cragr module
before calling is_account_present.https://gitlab.com/woob/woob/-/commit/8fa795efc1c8613d55a33346bb120033cdf0cdba[cragr] Handle netfinca action needed before checking if account is present2020-09-23T18:55:35+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
The xpath to check if an account is present does not appear if the
netfinca space is in action needed so I added a check_action_needed
method that is used for iter_investments and iter_market_orders.https://gitlab.com/woob/woob/-/commit/c3d10d049203723e631e43ba73773806efa34156[axabanque] Filter out closed wealth accounts2020-09-23T18:55:35+02:00Christophe Francoischristophe.francois@budget-insight.com
These accounts have no balance, we don't want to fetch them.https://gitlab.com/woob/woob/-/commit/474e3b0fd5c20b816fa6cda326872f3472f35952[oney] Set account balance for loans2020-09-23T18:55:34+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
Loans currently have balance == NotLoaded, which is a problem for the
backend: "TypeError: '>' not supported between instances of
'NotAvailableType' and 'int'"https://gitlab.com/woob/woob/-/commit/7cc50b7a0643e4cf93f6ea35eb7ba42a59b3b9ec[cmb] Add parent config attributes in module2020-09-23T18:55:34+02:00Maxime Gasselinmaxime.gasselin@budget-insight.com
It avoids bad config instantiation from parent.https://gitlab.com/woob/woob/-/commit/84c7372da914ab37b9d84a65641e7221b213d770[caissedepargne] Skip iter_market_orders for CENET browser2020-09-23T18:55:34+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
Otherwise some connections have 'CenetBrowser has no attribute
iter_market_orders'https://gitlab.com/woob/woob/-/commit/02adc40fe5f97e9ad9ae28f6291924914ab6dd31[creditmutuel] Handle new url for account summary2020-09-23T18:55:34+02:00Augustin Bouverotaugustin.bouverot@budget-insight.com
Some users are redirected to a new url for their por account summaryhttps://gitlab.com/woob/woob/-/commit/075b93ddd0cb5b37201006e44d1c290dabede60b[cragr] Type some accounts2020-09-23T18:55:34+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
A life insurance (ANAE) was not typed therefore its investments were not
scraped correctly.https://gitlab.com/woob/woob/-/commit/93d73e4d081118cec7860e114d3ec092cf8d62b6[banquepopulaire] Adapted regex to include 'Part sociale' as TYPE_MARKET2020-09-23T18:55:34+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/668b0690726c50e2f4aad31bc80d3a7a88398661[boursorama] Fix typo in log message2020-09-23T18:55:34+02:00Guillaume Risbourgguillaume.risbourg@budget-insight.comhttps://gitlab.com/woob/woob/-/commit/7393e4fb873fa592a1ac4078cf4a41886c361b68[myedenred] Repaired login2020-09-23T18:55:34+02:00Quentin Defenouillerequentin.defenouillere@budget-insight.com
The login URLs changed and once we reach login page there is now a
`<form>` instead of the former JSON login system.