Skip to content
  1. Aug 26, 2020
  2. Aug 07, 2020
  3. Jul 12, 2020
  4. Jul 02, 2020
    • ntome's avatar
      weboob.browser.browsers: no pickling when saving cookie jar in StatesMixin · 140ce810
      ntome authored
      pickle may be unsafe for loading data. All we want is a cookie jar, we
      can just serialize its cookies, not necessarily the whole jar with its
      type and policy.
      
      Old format is base64(compress(pickle(jar))).
      New format is base64(compress(json(jar))) where JSON conversion is on
      the list of cookies (taking name, value, domain, path, secure (https)
      and expires).
      
      dump_state will now save using the new format while load_state supports
      both new and old format. This allows compatibility for some time, then
      later old format support in load_state will be dropped too.
      140ce810
    • ntome's avatar
      e72e641e
    • ntome's avatar
      weboob.browser.browsers: write new HAR entries without rewriting the full file · c2cda409
      ntome authored
      HAR file was rewritten on every request, because JSON libs don't allow inserting
      data in an existing JSON without rewriting the whole file.
      
      However, if we put request/response entries at the end of the HAR data, only a
      fixed suffix exists after the entries.
      Then we can seek near the file end to a computed position, write the new entry
      (which overwrites terminators), and we can rewrite the overwritten terminators
      right after.
      Not only can we write only the new data, but we do not need to keep track of
      the shifted bytes.
      
      If the HAR wasn't written with the exact same options, we won't seek accurately
      though.
      c2cda409
  5. Jun 17, 2020
  6. Jun 05, 2020
  7. Jun 03, 2020
  8. May 22, 2020
  9. May 08, 2020
  10. May 07, 2020
    • Quentin Defenouillere's avatar
      [capabilities/bank/test] Market Orders should have a quantity or an amount · ad363761
      Quentin Defenouillere authored and hydrargyrum's avatar hydrargyrum committed
      Some Market Orders have no indicated quantity and have a total amount
      instead.
      ad363761
    • Quentin Defenouillere's avatar
      [capabilites/bank] Added MarketOrder.amount · d57d947b
      Quentin Defenouillere authored and hydrargyrum's avatar hydrargyrum committed
      Sometimes there is not information on the quantity of stocks that has
      been bought or sold, only the total amount that has been bought or sold.
      d57d947b
    • ntome's avatar
      weboob.browser.nss: implement NSS database update instead of recreation · c036f657
      ntome authored and hydrargyrum's avatar hydrargyrum committed
      NSS uses a certificate database that is empty by default and not updated
      automatically when new CAs are added in /etc/ssl/certs.
      So we are forced to recreate the database from scratch and since it takes
      about 1 minute, due to "certutil" command being slow, we can't do it
      everytime.
      
      By implementing an update operation that only adds new certificates and
      removes obsolete ones, we run certutil much less, so the update is
      significantly faster.
      
      In order to detect changes to certificates, and because NSS databases and
      PEM data are very shitty to introspect, we will base ourselves mostly on
      NSS cert "nicknames".
      As /etc/ssl/certs contains a lot of duplicates, we will rely on PEM data
      hashing instead instead of filenames to detect different certificates, and
      they will be the nicknames.
      
      Simplified, an update operation goes like this:
      - list all db cert hashes (the nicknames)
      - hash all system certs
      - add in db all system certs for which the nickname was missing
      - remove from db the nicknames if the hash wasn't in system list
      
      For migrating to new nicknames format, basically we will have to purge the
      db and recreate it.
      c036f657
    • Léo Yvin's avatar
      [tools] robustify ColoredFormatter format method · 789cd17a
      Léo Yvin authored and hydrargyrum's avatar hydrargyrum committed
      Prevent a ColoredFormatter.format crash processing record with mix of unicode and bytes values.
      If UnicodeDecodeError occured, try to convert bytes record components into unicode.
      789cd17a