- Oct 11, 2020
-
-
hydrargyrum authored
-
- Oct 08, 2020
-
-
- Oct 07, 2020
-
-
These methods were deprecated in stdlib xml.etree and also in lxml, and are ultimately removed in python 3.9.
-
-
Some accounts' ibans cannot be found anymore on the website. But since we kept the iban stored on our side, the 'old' transfer.account_iban is not empty when making a transfer. When we do not find the account based on its iban, we search it based on its id. So the account is valid, the iban is just different. This check allows to not have an assertion error when making a transfer from an account in this situation.
-
Some URLs have been updated on Bourse Direct but not yet on ING. For now we need to specify the old URLs in the abstract browser.
-
URLs for market accounts and history have been updated. The xpath for the history table has also changed, I took the time to make iter_history use TableElement.
-
This page may appear just after login. It is a 2FA method not handled yet. The message will inform the user to switch to 'Confirmation Mobile', which is an AppValidation method.
-
CloudCard (aka Secur'Pass) is a method that asks for an AppValidation when adding new recipients or making transfer when not on the Caisse d'Epargne application.
-
After login, we are now on the "inventory" page, which was not handled. There is nothing to do on the page, the rest of the browsing looks unchanged.
-
For those investments, trying the next request would lead to a 401 error. Also handle cases where BNP API is not responding, which seems to be quite often. Note: the invest will still be scraped but we will not be able to fetch some of its details.
-
-
add new url to match with the revolving loan page
-
When no account can emit transfers, we have an error message where we expected a form and the list of account.
-
When CSV is not availble (most cases), we get card_summaries transactions on parent accounts history page. In order to do that, and get as well summary details, we now start from parent account history page, parse card_summaries, and follow detail page for each one.
-
It seems that, in certain cases, the loop can be infinite when the website is returning the transactions endlessly. total_offset is here to block the while. If an account has more than 100 000 transactions, they will not have older transactions. Because of the total_offset that previously was at 100 000 which led at 4000 request max, I've decided to reduce the total_offset max to 50 000, halving the number of requests.
-
I don't know why I didn't include it when I wrote this part, maybe this column didn't appear back then. Or maybe I had no idea what I was doing.
-
Sometimes profile details are not available, leading to a 404 error.
-
- Sep 27, 2020
-
-
hydrargyrum authored
-
- Sep 26, 2020
-
-
- Sep 23, 2020
-
-
When doing a request on the login url with a `/` at the end, we receive a 301 and are redirected to the same url without a `/` at the end. And because of the `/` that is expected in the url, we end up not finding the page. Removing the `/` at the end in the regex leads to no redirection and it matches both cases.
-
Numerous wrongpass are still present. We let the retry to decide if we are in a real wrongpass case.
-
Thanks to this tool, we will be able to drop dumping to the old, custom, hard-to-parse format in favor of the new, ill-defined format: HAR!
-
For binary response, HAR allows base64 encoding so data is untouched. But no such thing exists for binary posted data. Some generators write JSON with the binary posted data as if it were latin-1 encoded text. But when reading, what can we do? Think it's binary to encode back to latin-1? Or think it's text? HAR gives no answer, so we add a non-standard "x-binary" key to indicate "text" is actually binary disguised with latin-1.
-
The condition was in ListElement instead of ItemElement and some accounts were filtered out by mistake.
-
-
-
When amount is not found by the first parsing, it must be done by a second one. I changed the condition on which this is triggered, since it not needed anymore, and would throw an AttributeError when page is parsed a second time.
-
This is mainly used during iter_accounts, but also in the cragr module before calling is_account_present.
-
The xpath to check if an account is present does not appear if the netfinca space is in action needed so I added a check_action_needed method that is used for iter_investments and iter_market_orders.
-
These accounts have no balance, we don't want to fetch them.
-
Loans currently have balance == NotLoaded, which is a problem for the backend: "TypeError: '>' not supported between instances of 'NotAvailableType' and 'int'"
-
It avoids bad config instantiation from parent.
-
Otherwise some connections have 'CenetBrowser has no attribute iter_market_orders'
-
Some users are redirected to a new url for their por account summary
-
A life insurance (ANAE) was not typed therefore its investments were not scraped correctly.
-
-
-
