woob issueshttps://gitlab.com/woob/woob/-/issues2024-02-09T08:01:31Zhttps://gitlab.com/woob/woob/-/issues/216weboob-config update generates warnings2024-02-09T08:01:31ZWoob Importweboob-config update generates warnings```
weboob-config update
=== [ 0%] Getting http://updates.weboob.org/1.5/main/
=== [ 0%] Getting file:///home/leto/weboob/modules/
2019-01-28 16:50:54,394:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module am...```
weboob-config update
=== [ 0%] Getting http://updates.weboob.org/1.5/main/
=== [ 0%] Getting file:///home/leto/weboob/modules/
2019-01-28 16:50:54,394:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module americanexpress: [ImportError] cannot import name AccountsPage
2019-01-28 16:50:54,864:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module inrocks: [ImportError] cannot import name ArticlePage
2019-01-28 16:50:54,912:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module lefigaro: [ImportError] cannot import name ArticlePage
2019-01-28 16:50:54,913:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module liberation: [ImportError] cannot import name ArticlePage
2019-01-28 16:50:54,954:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module minutes20: [ImportError] cannot import name ArticlePage
2019-01-28 16:50:55,012:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module presseurop: [ImportError] cannot import name PresseuropPage
2019-01-28 16:50:55,086:WARNING:repository:1.5:repositories.py:304:build_index Unable to build module taz: [ImportError] cannot import name ArticlePage
```
How to remove them?https://gitlab.com/woob/woob/-/issues/270RawConfigParser.readfp is deprecated2024-02-09T07:59:26ZWoob ImportRawConfigParser.readfp is deprecated2.0Woob ImportWoob Importhttps://gitlab.com/woob/woob/-/issues/48Weboob on AWS LAMBDA2024-02-09T00:12:53ZWoob ImportWeboob on AWS LAMBDAHello,
I would use weboob in AWS (amazon web services) and serverless technologies, with lambdas AWS. The Lambdas support the python language 2.7 and 3.6
I did not find a schema of the general architecture of weboob, but installing it on...Hello,
I would use weboob in AWS (amazon web services) and serverless technologies, with lambdas AWS. The Lambdas support the python language 2.7 and 3.6
I did not find a schema of the general architecture of weboob, but installing it on a VM, I did not see any daemon linux.
Is it the applications that should just use weboob core lib only?
Do you think this is possible simply and do you have examples of single calls from a module?
Thank youhttps://gitlab.com/woob/woob/-/issues/39DIctElement behavior isn't good for iterating on dict values2024-02-09T00:09:03ZWoob ImportDIctElement behavior isn't good for iterating on dict valuesexample:
`{"data":
{"key1": {"name": "value"},
"key2": {"name": "value"}}}`
We want to iterate on the dict contained for each "key*" , but DictElement will iterate on the key of those dict.example:
`{"data":
{"key1": {"name": "value"},
"key2": {"name": "value"}}}`
We want to iterate on the dict contained for each "key*" , but DictElement will iterate on the key of those dict.1.4https://gitlab.com/woob/woob/-/issues/36Handle Referrer-Policy2024-02-09T00:06:32ZWoob ImportHandle Referrer-Policyhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy
Kinda complex, but at least detecting no-referrer could help us with privacy stuffhttps://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy
Kinda complex, but at least detecting no-referrer could help us with privacy stuff1.4Woob ImportWoob Importhttps://gitlab.com/woob/woob/-/issues/284Convert generators to lists when using Fields of type list, instead of crashing2024-02-09T00:02:51ZWoob ImportConvert generators to lists when using Fields of type list, instead of crashinghttps://gitlab.com/woob/woob/-/issues/288Redundant ways to use the DateTime filter2024-02-09T00:02:09ZWoob ImportRedundant ways to use the DateTime filterWe can change the ``parse_func`` but it seems only used with ``parse_french_date``. I think it would be simpler to replace those with usage of dayfirst=True + translations. And then completely remove ``parse_french_date``.
Of course, we...We can change the ``parse_func`` but it seems only used with ``parse_french_date``. I think it would be simpler to replace those with usage of dayfirst=True + translations. And then completely remove ``parse_french_date``.
Of course, we can use ``parse_func`` to use another library than dateutil (I am thinking about it). However, we already give it a bunch of parameters that are specific to dateutil so it's actually not usable.Woob ImportWoob Importhttps://gitlab.com/woob/woob/-/issues/234[fortuneo] more relevant information2024-02-08T23:58:11ZWoob Import[fortuneo] more relevant informationOn Fortuneo, :
- checks are now marked as "CHQ"
- the relevant information on "virements" is on the hidden div (visible with the magnifier icon). Without it, we get only dumb transfer numbers or things like "SEPA PAR INTERNET"On Fortuneo, :
- checks are now marked as "CHQ"
- the relevant information on "virements" is on the hidden div (visible with the magnifier icon). Without it, we get only dumb transfer numbers or things like "SEPA PAR INTERNET"Damien Mat (Powens)Maxime GasselinDamien Mat (Powens)https://gitlab.com/woob/woob/-/issues/235[creditdunord] better label cleaning2024-02-08T23:52:16ZWoob Import[creditdunord] better label cleaningThe keywords used by the bank have changed. To have clean labels in KMyMoney, "virements" and "deposits" have to be updatedThe keywords used by the bank have changed. To have clean labels in KMyMoney, "virements" and "deposits" have to be updatedDamien Mat (Powens)Maxime GasselinDamien Mat (Powens)https://gitlab.com/woob/woob/-/issues/413Get lydia entries for boobank2024-02-08T23:50:02ZWoob ImportGet lydia entries for boobankI'm using kresus (a libre personal finance manager) based upon weboob. And it's great.
But I'd like also to collect data from [lydia app](https://lydia-app.com/) and there is no module at the moment for lydia.
[There is an api](http://...I'm using kresus (a libre personal finance manager) based upon weboob. And it's great.
But I'd like also to collect data from [lydia app](https://lydia-app.com/) and there is no module at the moment for lydia.
[There is an api](http://supportpro.lydia-app.com/l/fr/article/o4zwbi4qr7-est-ce-que-lydia-a-une-api-si-oui-est-il-possible-d-y-acc-der) to fetch data account lydia.
What do you think about writing a lydia module based upon the api ? I can spend some time about it when I will be in holidays (even if I'm not a experienced python dev !)https://gitlab.com/woob/woob/-/issues/460[boobank] Idea for double authentification2024-02-08T23:45:08ZWoob Import[boobank] Idea for double authentificationHi,
I'm the main developer of Skrooge.
Skrooge uses boobank to download transactions from the banks.
The configuration file of weboob is very well done because it allows to put some code to request **interactively** the password.
Like ...Hi,
I'm the main developer of Skrooge.
Skrooge uses boobank to download transactions from the banks.
The configuration file of weboob is very well done because it allows to put some code to request **interactively** the password.
Like this:
```
[creditcooperatif]
_module = creditcooperatif
login = 12345
password = `kwallet-query --read-password creditcooperatif --folder Weboob kdewallet`
```
But,in case of a double authentication (SMS code, click in mobile application, ...) is needed nothing seems to be planned (I'm may be wrong).
So, my idea is the following one: use the same mechanism for double authentication.
Like this:
```
[creditcooperatif]
_module = creditcooperatif
login = 12345
password = `kwallet-query --read-password creditcooperatif --folder Weboob kdewallet`
doubleauthent =`zenity --entry --text "Code" --title "Enter code for account 12345" --entry-text=""`
```
This will allow to prompt the user with a specific action when double authentication is needed.
Do you think this could be done ?
Thank you for weboob.
Regards.Ludovic LANGELudovic LANGEhttps://gitlab.com/woob/woob/-/issues/200Documentation issue.2024-02-08T23:37:32ZWoob ImportDocumentation issue.I've been browsing the website and I have not been able to find any "example" or "getting started" or anything.
I want to gather tracking info from tracking websites and put the results in my "packages in transit" database. Some sites ...I've been browsing the website and I have not been able to find any "example" or "getting started" or anything.
I want to gather tracking info from tracking websites and put the results in my "packages in transit" database. Some sites allow me "lynx -dump <the-url>" but others will error out and just give me the same page no matter what the tracking number is. I think I need weboob, but I would first like the documentation to give me some sort of demo so that I can start looking for what needs to be done from there.
Sample URL: https://global.cainiao.com/detail.htm?mailNoList=LX593784117NL
This is really a documentation issue. For someone familiar with the package it is going to be perfectly obvious what to do, but for someone not familiar with the matter it is one big mess.
After installing weboob, weboob-qt (apt install) I tried "man weboob". No manual.
I tried weboob-qt No such executable. I tried weboob. It gives me a long list of things to choose and I have no clue what it wants me to do. And I've tried weboob-cli and it tells me the syntax is
Syntax: /usr/bin/weboob-cli capability method [args ..]
Well great! (NOT!)https://gitlab.com/woob/woob/-/issues/75[wish] Retrieve history for many account with one single command2024-02-08T23:33:18ZWoob Import[wish] Retrieve history for many account with one single commandI would like to retrieve all accounts from my bank in one single command, to avoid re-typing my password for each account. Would it be possible to do that with something like this ?
`boobank history XXXX@mybank YYYYYY@mybank -f ofx > m...I would like to retrieve all accounts from my bank in one single command, to avoid re-typing my password for each account. Would it be possible to do that with something like this ?
`boobank history XXXX@mybank YYYYYY@mybank -f ofx > myfile.ofx `
Thanks.https://gitlab.com/woob/woob/-/issues/155DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.x2024-02-08T23:27:58ZWoob ImportDeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.xhttps://stackoverflow.com/questions/15471333/how-to-eliminate-a-python3-deprecation-warning-for-the-equality-operator
Found it weboob/capabilities/base.py
Is it bad though? I got this warning on my other libs: future, tqdm, requests, etc.https://stackoverflow.com/questions/15471333/how-to-eliminate-a-python3-deprecation-warning-for-the-equality-operator
Found it weboob/capabilities/base.py
Is it bad though? I got this warning on my other libs: future, tqdm, requests, etc.https://gitlab.com/woob/woob/-/issues/123Get a single OFX/QFX for all accounts quickly2024-02-08T23:26:52ZWoob ImportGet a single OFX/QFX for all accounts quicklyThe following scripts will retrieve the history of all checking and credit cards accounts, and will generate a single (and valid) OFX/QFX file.
When there are many accounts to retrieve, `boobank` can be a little slow. So GNU `parall...The following scripts will retrieve the history of all checking and credit cards accounts, and will generate a single (and valid) OFX/QFX file.
When there are many accounts to retrieve, `boobank` can be a little slow. So GNU `parallel` is used to to retrieve data in parallel for each account. In my particular setup, download speed is increased by a factor of 10.
To use, you will need to put your accounts numbers in the following files:
* checkings.txt
* creditcards.txt
Please let me know how to upload this somewhere on the site, so it can easily be accessible by all.
[boobank.sh](/uploads/ffeae16c1fcbfa274ecc8ca703ab51cd/boobank.sh)
[boohistory.sh](/uploads/9d9fcc2cb34e9b623f31b7c20b105fe8/boohistory.sh)
[boocoming.sh](/uploads/552901410b2091b4729adf7e13667dbe/boocoming.sh)https://gitlab.com/woob/woob/-/issues/156check_xpath improvements2024-02-08T23:24:27ZWoob Importcheck_xpath improvements* maybe the "." check should not be there in some cases (obj_* when they are actual methods?)
* false positive when using Async
* sometimes the . is there but not at the start, try to parse those weird cases
* allow ignoring some line...* maybe the "." check should not be there in some cases (obj_* when they are actual methods?)
* false positive when using Async
* sometimes the . is there but not at the start, try to parse those weird cases
* allow ignoring some lines, for example "# noqa" comments (this would allow using check_xpath all the time, like our pyflakes script)
* make it into a flake8 plugin? (could solve ignores)https://gitlab.com/woob/woob/-/issues/160Repository update is not resilient to a failure of the module repo2024-02-08T22:59:51ZWoob ImportRepository update is not resilient to a failure of the module repoHello,
I open this ticket to report a problem about repository updates.
Currently, before doing an update of the repositories, weboob deletes the local cache of the repositories (in the `repository` folder) **before** performing an u...Hello,
I open this ticket to report a problem about repository updates.
Currently, before doing an update of the repositories, weboob deletes the local cache of the repositories (in the `repository` folder) **before** performing an update of the repository (see here : https://git.weboob.org/weboob/devel/blob/master/weboob/core/repositories.py#L600-601).
In case the update process fails, for example due to the update server being "down" or for example because the TLS certificate is not renewed in due time (yes the problem raised these days ;)), the user cannot use the local version of modules (in the `modules` folder), as weboob first looks in the repository cache to determine if it knows the module (and the cache was deleted). This results in raising an `ModuleLoadError` and breaks the local use of weboob due to a remote problem (the cached modules could still be valid or at least partially functional).
I understand the intend to delete everything, if we are certain that there won't be any problem when updating, however these kind of problems can happen. In my opinion, the local repository cache should only be deleted if :
- we got an updated version of the repo locally
- OR the repository is not within the repositories listed in the `sources.list` provided to weboob.
The second clause could be discussed, as for testing purposes, a repo could be disabled temporarily, and the cache would be deleted for nothing (making it impossible to come back to a normal situation, in case this repo is down).
I open this ticket to open the discussion on what should be the behaviour in case the update process fails for any reason, and keep weboob usable anyway. I will propose an implementation once I have a good view on what should be expected.https://gitlab.com/woob/woob/-/issues/676unable to load: No module named 'woob_applications' in a `pipx` install - dep...2024-02-06T19:51:35ZLudovic LANGEunable to load: No module named 'woob_applications' in a `pipx` install - dependency on setuptools ?When installing `woob` with `pipx`, the tool partially works with an error message:
```
# woob
usage: woob [--version] <command> [<args>]
Use one of this commands:
bands (unable to load: No module named 'woob_applications')...When installing `woob` with `pipx`, the tool partially works with an error message:
```
# woob
usage: woob [--version] <command> [<args>]
Use one of this commands:
bands (unable to load: No module named 'woob_applications')
bank (unable to load: No module named 'woob_applications')
...
video (unable to load: No module named 'woob_applications')
weather (unable to load: No module named 'woob_applications')
For more information about a command, use:
$ man woob-<command>
or
$ woob <command> --help
```
or
```
# woob config
Traceback (most recent call last):
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/launcher.py", line 52, in load_app
app_module = importlib.import_module("woob.applications.%s" % app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1325, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 929, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 994, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/applications/config/__init__.py", line 19, in <module>
from .config import AppConfig
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/applications/config/config.py", line 23, in <module>
from woob.tools.application.repl import ReplApplication
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/tools/application/repl.py", line 33, in <module>
from woob.core import CallErrors
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/core/__init__.py", line 21, in <module>
from .woob import Woob, WoobBase
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/core/woob.py", line 31, in <module>
from woob.core.modules import ModulesLoader, RepositoryModulesLoader
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/core/modules.py", line 26, in <module>
import pkg_resources
ModuleNotFoundError: No module named 'pkg_resources'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/.local/share/pipx/venvs/woob/bin/woob", line 8, in <module>
sys.exit(Launcher.run())
^^^^^^^^^^^^^^
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/launcher.py", line 106, in run
return cls.run_app(sys.argv[1], sys.argv[2:])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/launcher.py", line 60, in run_app
app_class = cls.load_app(app)
^^^^^^^^^^^^^^^^^
File "/root/.local/share/pipx/venvs/woob/lib/python3.12/site-packages/woob/launcher.py", line 54, in load_app
app_module = importlib.import_module("woob_applications.%s" % app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1304, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1381, in _gcd_import
File "<frozen importlib._bootstrap>", line 1354, in _find_and_load
File "<frozen importlib._bootstrap>", line 1318, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'woob_applications'
```
It seems that `pkg_resources` (which can be provided by `setuptools`) is mandatory.
We should make this dependency explicit.
A working fix (except for `smtp` module) is to inject `setuptools` in the `pipx` environment:
```
# pipx install woob
# pipx inject woob setuptools
# woob
usage: woob [--version] <command> [<args>]
Use one of this commands:
bands display bands and suggestions
...
smtp (unable to load: No module named 'woob_applications')
...
video search and play videos
weather display weather and forecasts
For more information about a command, use:
$ man woob-<command>
or
$ woob <command> --help
```Ludovic LANGELudovic LANGEhttps://gitlab.com/woob/woob/-/issues/583[bank][n26] Bug(n26): 404 Client Error: Not Found2024-01-28T04:57:48ZCarlos GOMES[bank][n26] Bug(n26): 404 Client Error: Not FoundHello,
I've got this error since some days
```
2022-08-29 09:38:11,031:DEBUG:bcall:3.0:bcall.py:101:backend_process <Backend 'bnp'>: Called function <bound method Application._do_complete of <woob.applications.bank.bank.Appbank object ...Hello,
I've got this error since some days
```
2022-08-29 09:38:11,031:DEBUG:bcall:3.0:bcall.py:101:backend_process <Backend 'bnp'>: Called function <bound method Application._do_complete of <woob.applications.bank.bank.Appbank object at 0x7f99a23760>> returned: <generator object Application._do_complete_iter at 0x7f9689a970>
Bug(n26): 404 Client Error: Not Found
2022-08-29 09:38:11,042:DEBUG:urllib3.util.retry:3.0:retry.py:333:from_int Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)
2022-08-29 09:38:11,042:DEBUG:urllib3.util.retry:3.0:retry.py:333:from_int Converted retries value: 2 -> Retry(total=2, connect=None, read=None, redirect=None, status=None)
=== [ 0%] Getting https://updates.woob.tech/3.0/main/
2022-08-29 09:38:11,057:DEBUG:urllib3.connectionpool:3.0:connectionpool.py:973:_new_conn Starting new HTTPS connection (1): updates.woob.tech:443
2022-08-29 09:38:11,243:DEBUG:urllib3.connectionpool:3.0:connectionpool.py:452:_make_request https://updates.woob.tech:443 "GET /3.0/main/modules.list HTTP/1.1" 200 67526
```
Thanks for you're helphttps://gitlab.com/woob/woob/-/issues/610[hsbc] when adding support for `coming` transactions, account balance is inco...2023-12-18T10:33:14ZLudovic LANGE[hsbc] when adding support for `coming` transactions, account balance is incorrect (it includes `coming` transactions) - request for commentsShort version: hsbc (the website) shows an account balance that include the sum of the coming transactions (i.e. : available balance). I'd like to split it in two : `balance` (ledger balance) and `coming` (available balance - ledger bala...Short version: hsbc (the website) shows an account balance that include the sum of the coming transactions (i.e. : available balance). I'd like to split it in two : `balance` (ledger balance) and `coming` (available balance - ledger balance).
Long version:
For the moment, `woob` does not distinguish between `coming` and non-`coming` transactions in `hsbc` module.
I would like to add support for those `coming` transactions.
Why ? I'm trying to split apart the `coming` transactions and the non-`coming` ones, because a `coming` transaction lacks details about the transaction itself (label only showing the transaction internal id, transaction type).
It's only after a transaction has been accounted for in hsbc that it gets its full label details.
e.g.:
* today is 2023-01-01, if have a transaction whose date is `2023-01-02` and label is only : `VR SEPA EMIS XXX0XXX`
* next day is 2023-01-02, the transaction date is unchanged, but the label is now : `VIREMENT SEPA EMIS YCO0 12345 COMMENT REFERENCE LABEL PAYEE`
Having a transaction that 'changes' from one day to the next one is difficult to work with - I have no way to reconcile those apart from date + amount, which is not enough - and so I want to completely get rid of those `coming` transactions.
Which looks quite easy, indeed, a raw patch looks like:
```
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
modified: modules/hsbc/browser.py
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
@ modules/hsbc/browser.py:662 @ def match_card(tr):
return history
elif self.life_insurance_useless.is_here():
return []
- elif not coming:
- return self._get_history()
else:
- raise NotImplementedError()
-
- def _get_history(self):
- for tr in self.page.get_history():
- yield tr
+ for tr in self.page.get_history():
+ if (tr._coming == coming):
+ yield tr
def get_investments(self, account, retry_li=True):
if not account.url:
```
With this, `iter_history` only returns accounted for transactions, and `iter_coming` only returns `coming` transactions.
So far, so good.
But the issue now is that the account balance is incorrect. Indeed, the account balance (as retrieved from the hsbc web site) *includes* the `coming` transactions.
I'd like to fix this, and was wondering what the best way was.
AFAIK the only "part" of the website that computes the value for us are the `ofx`, `txt` or `csv` data export which do show the correct balance (ledger balancer).
So that could be one possibility (albeit looks slow and complex).
Another approach would be, in the account retrieval, to open the history page, identify the `coming` transactions, compute the `coming` balance and substract it from the `available` balance.
What would be your recommended way ?
Notes: My hsbc account is a `new_space` one.