Commit 1b65dd30 authored by Romain Bignon's avatar Romain Bignon

Weboob 2.0 released

parent c605ed0d
Pipeline #2971 passed with stages
in 14 minutes and 37 seconds
Weboob 2.0 (2020-02-13)
Applications: boobank
* keep Value for TransferStep and AddRecipientStep
* declare utf-8 in OFX formatter
* give account object to iter_transfer_recipient
Applications: boobill
* using a persistent storage for browser state
Applications: boobsize
* fix name
Applications: translaboob
* use babel to detect language names and pass ISO-639 codes
Applications: weboobdebug
* weboob-debug: fix name
Applications: weboobmain
* move scripts/weboob to a dedicated dir in weboob/applications
Applications: weboobrepos
* weboob-repos: port to python3
Browser
* set a parent logger to AbstractElement
* Allows SeleniumBrowser to use MANUAL proxies
* AUTHENTICATION_METHODS set in init
* delete prefix 'handle_' on authentication methods
* clean authentication config keys after login
* make variable name more understandable
* make get_expire stronger
* TWOFA_DURATION for the state expire date
* better names
* set expire from logged_date
* make authentication methods more flexible
* handle both 2FA and regular login methods
* add possibility to clear specific cookies
* handle NeedInteractiveFor2FA
* add documentation about otp, sms and resume
* add TwoFactorBrowser to handle multiple 2FA
* Set default logging level to ERROR for selenium
* Added the ability to overrride *Elements' env copy
* Allow passing custom headers in Form.submit()
* set attribute 'weboob' on browser
* Normalize encodings
* BrowserTooManyRequests: Remove class attributes for date helpers
* BrowserTooManyRequests: Raise a TypeError if the next_try is not a datetime
* BrowserTooManyRequests: Check whether the next_try is None after the date check
* BrowserTooManyRequests: Add message parameter
* [weboob/browser/browser+pages] Fixed doctest BASEURL
* add some helpers to TooManyRequests
* add rate limiting exception
* add build_refresh_token_parameters method
* python3 compatibility
* send redirect_uri when using the refresh_token
* allow multiple inheritance with AbstractBrowser
* Implemented wait_xpath_invisible
* Avoid crash when no BASEURL is set
* add klass attribute to decorated @method
* give state to the module at initialization
* introduce BrowserParamURL for passing params implictly
* Implement the empty_xpath attribute
* accept new parameter proxy_headers (but don't use it)
* Browser takes a new parameter 'proxy_headers'
* if use of refresh_token fails, forget it
* support when token_type key is missing
* if access_token_expire is None, the access_token has no expiration
* in case we got a 401 response, forget the access_token
* fix compatibility with python3
* Handle generic error for bad url callback
* Repair refresh token
* StatesMixin cookies should be str
* OAuth2PKCEMixin codes shouldn't be bytes
* Rewrite/improve PagesBrowser example
Browser: Filters
* Add the format .Italian() to CleanDecimal
* New keyword argument to remove accented characters on CleanText filter.
* Changed Capitalize filter to Title
* Updated & Reordered __all__
* Add dates to FormValue
* Add color to FormValue
* Add number/range type to FormValue, and tests
* MultiJoin accepts types other than str
* Regexp can use regex library
* Allow all dateutil.parser arguments in DateTime
* Strict dates by default
* Catch incomplete dates
* Explicit error message
* Add MultiJoin
* Join can use default argument
* Add doctests to Join
* TableCell(colspan=True) is now the only behavior desired
* add FromTimestamp to convert to datetime
* Fix error message for FormValue
* Add new types for FormValue
* Add MapIn filter, that allow the pattern of a selected value to another value using a dict.
* Reorganize imports / add missing to __all__
* Add Coalesce
Capabilities
* rename accepted_doc_types attribute of CapDocument
* Add KIID to DocumentTypes
* Added new capability CapBands
* Keep only PER
* add PER account type
* HousingPhoto inherits BaseImage
* use PostalAddress and GeoCoords types where possible
* introduce weboob.capabilities.address for a few object types
* Add firstname and lastname field
* add capability_to_string function to get capability name
* Add new SAR currency
* fix missing TransferInvalidOTP export
* add TransferInvalidOTP exception
* use StringField standard default value for transfer beneficiary label and number
* Indentation cosmetic changes
* Add new Investment attributes
* Add new document type 'income_tax'
* Add field 'original_gross_amount' to Transaction
* Add field 'gross_amount' to Transaction
* Remove performance_history default value
* Added new attributes for employee savings accounts
* Add RecaptchaV3Job to captcha capability
* add support for commission in currency in bank/Transaction
* add new add recipient exception for invalid otp
* add has_file field to Document class
* Add the relationship between the credentials owner (PSU) and the account
* Add bdate. Represent the date that the transaction appear in the website.
* prevent casting id from None to 'None' in constructor
Documentation
* add pip install
* fix compatibility with sphinx 2.0
* do not use deprecated defindex.html anymore
* Use correct word
General
* New airparif module (CapGauge)
* New aviva module (CapBank, CapBankWealth)
* New cesu module (CapDocument)
* New degiro module (CapBank, CapBankWealth)
* New hsbchk module (CapBank)
* New lendosphere module (CapBank, CapBankWealth)
* New lucca module (CapCalendarEvent, CapDocument)
* New lunchr module (CapBank)
* New metalarchives module (CapBands)
* New pajemploi module (CapDocument)
* New ticketscesu module (CapBank)
* New wiseed module (CapBank, CapBankWealth)
* Deleted biplan module
* Deleted figgo module
* Deleted lolix module
* Deleted wellsfargo module
General: Core
* send request_information to modules in all case, except if stdout is not a tty
* handle error NeedInteractiveFor2FA
* change way to bypass check, and set default value to None
* set default value to '' to disable checks if it is missing
* add ValueTransient for module parameters we don't want to keep in config
* AppValidation "resume" config should be set as a Value
* set request_information if app is interactive
* use _do_and_retry only for iter_resources
* handle interactive errors which allow to retry
* Ignore geckodriver.log file (used for Selenium-Firefox)
* Allow None as default value for ValueDate
* Replace BrowserQuestion __str__ with __unicode__
* add str method to BrowserQuestion exception
* progress should go from 20% to 30%, not the reverse
* tools.pdf Add blinkpdf function
* add str method to BrowserQuestion
* fix capabilities collision on AbstractModule class
* fix import for boobill and boobcoming
* rename Tenma
* refer to pip
* remove references to Qt as it's a separate repo
* gitlab-ci: disable python2
* move stuff from setup.py to setup.cfg
* remove qt apps
* remove qt5 stuff
* remove "scripts", use "entry_points"
* set default ADDITIONAL_CONFIG for AbstractModule
* New AppValidation errors
* Better logging
* Use the same logger
* Revert "gitlab-ci.yml: use python3 in before_script"
* Update mailmap
* Fixes #307: Remove remaining uses of is_command;
* python3/2 compatibility
* if 'repo_path' is None do not save it
* allow multiple inheritances with AbstractModule
* Add fullmatch compatibility
* add NeedInteractive exceptions when about to require OTP
* clarify missing number error
* gitlab-ci.yml: use python3 in before_script
* setup.py now requires python3.5 or higher
* handle ADDITIONAL_CONFIG on config property
* Modified valuation test for valuations = 0.00
* add CONFIG inheritance to AbstractModule
* fix iter_caps not returning only Capabilities
* Add new filters IsinCode & IsinType
* move external password fetch here
* Add missing test
* Avoid race conditions
* fix when BaseObjects don't have the same class
* introduce blinkpdf
* fix invalid email for va
* Add RecaptchaV3Question to weboob/exceptions
* read _proxy_headers from config
* add recipient sanity check
* Make BrowserHTTPSDowngrade inherit from Exception
* Raise exception instead of browser unavailable for 404 errors
* Add vscode to the .gitignore
* add Babel to dependencies
* map florian to budget-insight
* Add DecoupledValidation to handle decoupled DSP2 use case
* Allow SplitKeyboard to convert individual images
* Handle memoryview
* Force everything to be an iterator
* Better string formatting
* Move time_buffer to util
* Cross-platform file replace
* Introduce a dumper without aliases
* Allow setting Loader and Dumper at class level
* SQLiteConfig Python 3 compatibility
* Handle calls with only one level in SQLiteConfig
* Add SQLiteConfig
* The underlying storage class may have a sync method
* Add license headers
* Allow extra classes to be pickled properly
* Reduce filesystem calls in AutoCleanConfig
* Only create decorator as needed, more parameters
* Move time buffer logic out of the class
* Avoid altering the values dict by doing a get()
* No need to check if file exists
* Rename to DBMConfig
* Python 3 compatibility
* Avoid race conditions
* Add extra classes
* Allow usage as a context manager
* Add AnyDBMConfig, an implementation of IConfig for anydbm
* use more docstring instead of comments
* add more developers
* Add new WrongCaptchaResponse exception to weboob.exceptions.py
* forward parse_qs/parse_qsl extra args
* add `browser` parameter
* factor pattern parsing code
* fix import input for python3
Modules: afer
* Specified CapBankWealth legacy explicitly
* Module: remove CONFIG (now inherits from the aviva module)
* Made module an abstract of aviva
* remove accountnotfound
* Corrected wrongpass detection & handled ActionNeeded
* update regex to allow users credentials migration
* Raise BrowserPasswordExpired
* handle a new BrowserIncorrectPassword
Modules: airparif
* new CapGauge module for checking Paris pollution
Modules: allocine
* fix crap
Modules: amazon
* Add pagination and fix some values for documents
* Fix the url of the page for the documents
* change forgotten WRONGPASS_MESSAGES attribute
* fix link to download document
* handle more different messages for BrowserIncorrectPassword
* Add html bill
* fix login with better otp behavior
Modules: amazonstorecard
* ditch obsolete cmp function
Modules: ameli
* fix iter_documents which didn't accept str passed by boobill
* Change iter_subscriptions
* html document page has been put in a json
* handle error page to raise BrowserUnavailable
* new module
* total delete
Modules: amelipro
* harsh python3 port + cosmetics
Modules: americanexpress
* Re-write do_login in Selenium
* Modified login to avoid blocking accounts
* Add ActionNeeded detection on login
* Fixed login and better iter_accounts
* remove get_account
* Modified data index for iter_accounts
* get correct accounts currency
* A few code tweaks in browser.py and pages.py
* Fixed transactions parsing when faulty json
* Add bdate
Modules: amundi
* Fix 3 years performance
* Fetch performance history on CprPerformancePage
* Fix diff value
* Fix asset_category and add portfolio share
* Handle xpaths in English for CrpInvestmentPage
* Get performance columns dynamically
* Get investment details for Amundi Investments Page
* PER Account typing
* Implemented iter_pockets
* Correct Investment.diff
* Added Investment.diff
* Get performance_history for SGGestionPage
* Remove password regex
* Fetch 3 years performance on EE Investment Detail
* Added account type "Article 83"
* Handled investment attributes for SG Gestion space
* Handle specific characters when fetching account label
* Missing account type
* Implemented asset category & recommended period
* Skip investments without valuation
* remove get_account
* Repair SRRI fetching and investment.code check
* Implemented investment Level of Risk (SRRI)
* Skip iter_investments for accounts with null balance
* Implemented company_name and performance_history
* Compute correctly the amount of reimboursements.
Modules: anticaptcha
* manage Recaptcha v3
Modules: apivie
* Raise ActionNeeded when user info is too old
Modules: asana
* flake8 code linting
* correct some fields
Modules: aviva
* Modify browser's TIMEOUT to 120
* Browser: adapted aviva website to handle Afer abstract module
* Changed License to LGPL since module is now parent of an LGPL module
* Changed license for pages.py files and adapted the code for afer
* removed get_account and re-indented
* Fixed investments navigation and parsing
* Changed Capitalize to Title
* replace diff_percent field usage with diff_ratio
* Corrected condition to exclude 'Total'
* Adapt investment table to all account types
* Ignore insurances and accounts without balances
* Repaired module for iter_accounts and investments
* Correct iter_accounts xpath and account typing
* Added obj_code_type to investments if isin valid
* Add totomobile to list of ignore account types
* py3 compat (already compatible)
* added label for transactions
* raises a new actionneeded when we get redirected after login
* Add condition to filter investments
* label is not available on HistoryPage
* changes retirement's saving from type TYPE_MADELIN to TYPE_PERP
* patch after website design changes
* Skip habitation contract because it's not an account
* fetching currency
* wrong regexp
* fix for immediate 472462
* update aviva login because virtual keyboard doesn't exist anymore and not only digit password is now allowed
* add CapBankWealth and CapBankPockets in bank.py
* Getting the correct value off diff_percent
* adding some kinds to skip
* set default=NotAvailable for invests
* adapt module to new website
* fix stuff [mangled commit]
* integrate new login page
* FIx label xpath and get details for investment
* raise actionNeeded on cgu
* fix invalid xpath
* fix itering on <avances> transactions
* handle balance on some accounts
* raise wrongpass and force regexp on bad formatted password
* detect a new BrowserUnavailable case
* "arbitrage" are only one transaction
* fix "arbitrage" parsing
* some accounts have no history link
* add a new ActionNeededPage
* fix pyflakes
* skip non-financial accounts, and fix parsing of euro funds
* fix crash on iter_sub
* fix history link
* when user need to accept terms
* adding aviva module
Modules: axabanque
* Handle SMS OTP with ActionNeeded
* Add account type PER
* Remove cache from AXAAssurance browser
* New investment page
* Add default value to get_pid Attr()
* ordering transactions
* fix missing param
* Fixed documents retrieval and download
* Syntax for pep8 + cosmetic changes
* Use Floor division instead of Float division.
* add account ownership
* removed get_account
* Repair browser AxaAssurance and implement iter_history from JSON
* Formatted pages/bank.py for from future import unicode_litterals to work
* Fixed a market account being typed as checking account
* Fixed transactions parsing for lifeinsurances
* Fixed date for def card comings
* Revert "[axabanque] Repaired iter_history for AXAAssurance browser"
* update add recipient form parameters
* Set type to FrenchTransaction.TYPE_PAYBACK for Transactions which label starts with 'ANNUL' Those transactions were previously not typed. 12347@zendesk
* Added type FrenchTransaction.TYPE_BANK to all transactions which label starts with 'F-' All transactions that start with prefixed with F- seem to be bank fees, they were previously not typed. 12347@zendesk
* Changed regex to give a type to transactions which label starts with 'CB' Transactions starting in 'CB' weren't given a type so I added a case to the regex 12347@zendesk
* Add account type
* Add the bdate
* Get the real card number
* Change the way that we handle missing part for the account id
* Handle only the exception needed for account id
* Add profile name
* Type CEL account
* handle '/' in account id to avoid duplicates
* Repaired iter_history for AXAAssurance browser
* Return to the accounts page on the main website after iter_account
* Fix wrong typing Added 'epargne retraite agipi pair' which is a TYPE_PERP Changed obj_type to make use of MapIn filter
* handle new recipient error message
* Correct scraping of loan balance/currency
* dump transfer virtualkeyboard in session folder if needed
Modules: banquepopulaire
* Add one more location to find token
* Typed several Life Insurances & repaired their investments
* rename mis-backported iter_accounts method
* Fix transaction sorting on website
* Corrected some cosmetics in module.py
* Changed account Types & corrected redirection for Linebourse
* Rewrote iter_accounts and iter_investments
* Fix transaction being wrongfully deleted
* Syntax for pep8
* Fixed regex on profile names/company names
* Raise BrowserUnavailable when unable to access login page
* Modified owner_name account search for companies
* add ownership to account
* Handle dematerialization status for subscriptions
* Add CapDocument to get bank statements
* import unicode_literals in module.py and fix spelling
* typing transactions
* Add parent to card accounts
* Typed 'Parts sociales' as TYPE_MARKET
* rework previous patch, raise BrowserIncorrectPassword only in a specific case
* do not raise BrowserIncorrectPassword with legacy passwords
* Revert "[banquepopulaire] raise incorrect password if password is not digits only"
* factorize the back action
* raise incorrect password if password is not digits only
* fix BrowserUnavailable during iter_accounts
* Added bdate to transactions
* add condition to retrieve IBAN or not
* fix virtual keyboard giving not deterministic results
* add an additional hash to the virtual keyboard
* fix duplicated coming transactions between card accounts
* raise BrowserIncorrectPassword instead of BrowserPasswordExpired
* fix login cookies in javascript with spaces
* Fix error 400 during login
* Add support for BrowserIncorrectPassword when there is a Virtual Keyboard
* add support for virtual keyboard
* Fix login for some connections
* retrieve missing coming
Modules: barclays
* Added ActionNeeded exception
* Handle website unavailable
* add message to BrowserIncorrectPassword exception
* avoid crash in go_to_account if _btn is None
* fix login
Modules: becm
* remove useless file
Modules: bforbank
* Handle WrongPass and blocked account at login
* Fixed leave_espace_bourse behavior
* add account ownership
* fix missing param
* handle empty card page
* Ignore cards that are ordered but not activated yet
* Add capability CapProfile to the module
* Add bdate
* Repair cards URL
Modules: bibliothequesparis
* remove old stuff
* support new error location
* urls changed and json is not byte-equal strict
Modules: billetreduc
* sort events within a day
* pagination changed
Modules: binck
* Removed bugging Investment vdate
* Add new message that throws a BrowserIncorrectPassword
* Various fixes on investments vdate
* Fix investments vdate
* Added an url and a form xpath to handle questionnaire to skip at login
* fixed vdate for invests
* Factorized get_token() method
* Skip password change action needed
* Corrected module to work for all users
* Do not try to fetch investments if there is no money on the account
Modules: biplan
* Delete module concert hall is closing
Modules: bnpcards
* Handle 401 error on login (wrongpass)
* Add missing transaction type
* Added new site and siteswitch from old site to new site
* Fixed account label to include space in between first and last name
* Syntax for pep8 + cosmetic changes
* Removed unused test file
* Port to Python 3
* Added connection type logging at login
* Fix for iter_transactions non corporate
* set default value to 1 instead of unknown 'ges'
* Fixed transactions not scrapped for non corporate accounts.
* Fix iter accounts for multiple accounts for non corporate connections
Modules: bnporc
* reformat with yapf
* Handle new error code 1000
* Change ActionNeeded to BrowserPasswordExpired when we reach 100th
* [bnporc - pp] get documents also from first documents page
* Get rid of rdates in the future for coming transactions
* Get rid of useless parse_french_date import
* Corrected rdate parsed from label
* add account ownership
* Added missing coming transactions types
* make _transfer_id go through
* Removed unused transaction property
* Fix bug when loans data is empty
* Type summary_card that weren't typed
* Added missing TYPE_LIFE_INSURANCE account
* fix missing attributes for revolving credit in iter_subscription
* Fix for recipients with same iban
* Type card PAYBACKs as DEFERRED_CARD
* Update and clean document_pages.py
* Fixed BrowserPasswordExpired at login
* Fixed error for loans with empty balance
* Fixed old url used for HistoryPage
* Add documents to personnal pages
* Changed the URL for HistoryPage
* add 'Crédit Silo' as TYPE_REVOLVING_CREDIT
* Add revolving credit details
* Add detail for mortgage loan account
* Implement iter_accounts in browser2
* Corrected investments retrive on PEA 'Espèces'
* Fix wrong typing Added 'BNP Paribas Multihorizons' and 'BNP Paribas Libertéa Privilège' which are TYPE_LIFE_INSURANCE
* Fix broken history for entreprise website and avoid fetching 3 months if not needed
* fix regression on enterprise/browser, duplicating transactions
* Improve history navigation for bnp enterprise
* retrieve recipient transfer id
* ignore some non-transactions on life insurances
* 'idBeneficiaire' is not stable, change recipient id
* internal recipients don't have _web_state attribute
* remove all TransferError
* raise TransferInvalidRecipient when recipient is not active
* add provisional config 'digital_key'
* handle digital key to add new recipient
* catch websiteunavailable
Modules: bnppere
* Reindent some lines in module.py
* Implemented iter_investment & iter_history for multi-accounts
* Change account type to PER
* Visiogo: user last() column instead of [6] for amounts
* Implemented iter_investments
* Visiogo - implement Account.company_name
* Remove transactions ID
* raise get_profile NotImplementedError for classical bnppere
Modules: bolden
* retype lend documents and add fiscal docs
* Repair login form and keys
Modules: boursorama
* cleanup useless things, and use ValueTransient
* Fixed transactions xpaths
* Add missing transactions types
* add check_interactive before sms
* handle PSD2 SMS 2FA
* Fix card transactions
* Fixed loan currency xpath
* Fixed bug for loans that has no maturity date
* Change loan parsing to handle CLIC
* Fixed loans label xpath
* Modified handling of virtual keyboard
* repaired new recipient
* xpath changed for transfer confirmation
* update recipient label handling for transfers
* fixes on transfer
* Handle firstname and lastname
* send message for TransferBankError
* repaired iter_recipients
* Changed transactions amount format from US to French
* fix missing type for some withdrawal
* Fix deferred card transactions date
* Fix regex for Transaction.TYPE_CARD
* Skip checking coming transactions because 'date' disappeared
* Add Account type for "csl" accounts
* Typing '/livret-a/' urls-based accounts
* We don't get users custom transactions anymore
* Avoid to get 3 years of history for nothing when we get coming transaction
* update recipient category xpath
* Fix a bug that make deferred transaction with date == today to not show up
* Get investment liquidity when there are 0 on it.
* retrieve iban with new xpath
* Skip not deferred card
* Correct the way that we get the card number.
* Add pagination for Checking accounts
* Added the accounts ownership
* Add bdate to deferred card transaction
* some recipient category title has changed
* handle \ufffd character
* Add some "LDD" in account_types matching dict
* Reindent correctly the account type dict
* Change the label path to avoid duplicate text
* fix skipped accounts because DOM changed on account numbers
* get confirm transfer error
* Corrected transactions typing & card transaction filtering
* update iter recipient xpath
* Correct cards disappearance again