Commit d61e15cf authored by Romain Bignon's avatar Romain Bignon

rename things related to browsers

weboob.tools.browser -> weboob.deprecated.browser
weboob.tools.parsers -> weboob.deprecated.browser.parsers
weboob.tools.mech -> weboob.deprecated.mech
weboob.browser2 -> weboob.browser
weboob.core.exceptions -> weboob.exceptions

Also, the new tree for browser2 is:

weboob.browser: import weboob.browser.browsers.* and weboob.browser.url.*
weboob.browser.browsers: all browsers (including PagesBrowser and LoginBrowser)
weboob.browser.url: the URL class
weboob.browser.profiles: all Profile classes
weboob.browser.sessions: WeboobSession and FuturesSession
weboob.browser.cookies: that's a cookies thing
weboob.browser.pages: all Page and derivated classes, and Form class
weboob.browser.exceptions: specific browser exceptions
weboob.browser.elements: AbstractElement classes, and 'method' decorator
weboob.browser.filters.*: all filters
parent 1f95e763
......@@ -38,7 +38,7 @@ from mechanize import _headersutil as headersutil
from mechanize._html import EncodingFinder
from weboob.core import Weboob
from weboob.tools.browser import StandardBrowser, BrowserUnavailable
from weboob.deprecated.browser import StandardBrowser, BrowserUnavailable
from weboob.tools.misc import get_backtrace
from weboob.tools.misc import to_unicode
from weboob.tools.storage import StandardStorage
......
......@@ -65,7 +65,7 @@ In a module directory, there are commonly these files:
* **__init__.py** - needed in every python modules, it exports your :class:`Module <weboob.tools.backend.Module>` class.
* **module.py** - defines the main class of your module, which derives :class:`Module <weboob.tools.backend.Module>`.
* **browser.py** - your browser, derived from :class:`Browser <weboob.browser2.browser.Browser>`, is called by your module to interact with the supported website.
* **browser.py** - your browser, derived from :class:`Browser <weboob.browser.browsers.Browser>`, is called by your module to interact with the supported website.
* **pages.py** - all website's pages handled by the browser are defined here
* **test.py** - functional tests
* **favicon.png** - a 64x64 transparent PNG icon
......@@ -179,14 +179,14 @@ what are expected returned objects, and what exceptions it may raises.
Browser
*******
Most of modules use a class derived from :class:`PagesBrowser <weboob.browser2.page.PagesBrowser>` or
:class:`LoginBrowser <weboob.browser2.page.LoginBrowser>` (for authenticated websites) to interact with a website.
Most of modules use a class derived from :class:`PagesBrowser <weboob.browser.browsers.PagesBrowser>` or
:class:`LoginBrowser <weboob.browser.browsers.LoginBrowser>` (for authenticated websites) to interact with a website.
Edit ``browser.py``::
# -*- coding: utf-8 -*-
from weboob.browser2 import PagesBrowser
from weboob.browser import PagesBrowser
__all__ = ['ExampleBrowser']
......@@ -195,7 +195,7 @@ Edit ``browser.py``::
There are several possible class attributes:
* **BASEURL** - base url of website used for absolute paths given to :class:`open() <weboob.browser2.page.PagesBrowser.open>` or :class:`location() <weboob.browser2.page.PagesBrowser.location>`
* **BASEURL** - base url of website used for absolute paths given to :class:`open() <weboob.browser.browsers.PagesBrowser.open>` or :class:`location() <weboob.browser.browsers.PagesBrowser.location>`
* **PROFILE** - defines the behavior of your browser against the website. By default this is Firefox, but you can import other profiles
* **TIMEOUT** - defines the timeout for requests (defaults to 10 seconds)
* **VERIFY** - SSL verification (if the protocol used is **https**)
......@@ -205,15 +205,16 @@ Pages
For each page you want to handle, you have to create an associated class derived from one of these classes:
* :class:`HTMLPage <weboob.browser2.page.HTMLPage>` - a HTML page
* :class:`XMLPage <weboob.browser2.page.XMLPage>` - a XML document
* :class:`JsonPage <weboob.browser2.page.JsonPage>` - a Json object
* :class:`HTMLPage <weboob.browser.pages.HTMLPage>` - a HTML page
* :class:`XMLPage <weboob.browser.pages.XMLPage>` - a XML document
* :class:`JsonPage <weboob.browser.pages.JsonPage>` - a Json object
* :class:`CsvPage <weboob.browser.pages.CsvPage>` - a CSV table
In the file ``pages.py``, you can write, for example::
# -*- coding: utf-8 -*-
from weboob.browser2.page import HTMLPage
from weboob.browser.pages import HTMLPage
__all__ = ['IndexPage', 'ListPage']
......@@ -227,9 +228,9 @@ In the file ``pages.py``, you can write, for example::
``IndexPage`` is the class we will use to get information from the home page of the website, and ``ListPage`` will handle pages
which list accounts.
Then, you have to declare them in your browser, with the :class:`URL <weboob.browser2.page.URL>` object::
Then, you have to declare them in your browser, with the :class:`URL <weboob.browser.url.URL>` object::
from weboob.browser2.page import PagesBrowser, URL
from weboob.browser import PagesBrowser, URL
from .pages import IndexPage, ListPage
# ...
......@@ -257,9 +258,9 @@ For example, we can now implement some methods in ``ExampleBrowser``::
return self.page.iter_accounts()
When calling the :func:`go() <weboob.browser2.page.URL.go>` method, it reads the first regexp url of our :class:`URL <weboob.browser2.page.URL>` object, and go on the page.
When calling the :func:`go() <weboob.browser.url.URL.go>` method, it reads the first regexp url of our :class:`URL <weboob.browser.url.URL>` object, and go on the page.
:func:`stay_or_go() <weboob.browser2.page.URL.stay_or_go>` is used when you want to relocate on the page only if we aren't already on it.
:func:`stay_or_go() <weboob.browser.url.URL.stay_or_go>` is used when you want to relocate on the page only if we aren't already on it.
Once we are on the ``ListPage``, we can call every methods of the ``page`` object.
......@@ -295,8 +296,8 @@ the method :func:`create_default_browser <weboob.tools.backend.Module.create_def
def create_default_browser(self):
return self.create_browser(self.config['username'].get(), self.config['password'].get())
On the browser side, you need to inherit from :func:`LoginBrowser <weboob.browser2.page.LoginBrowser>` and to implement the function
:func:`do_login <weboob.browser2.page.LoginBrowser.do_login>`::
On the browser side, you need to inherit from :func:`LoginBrowser <weboob.browser.browsers.LoginBrowser>` and to implement the function
:func:`do_login <weboob.browser.browsers.LoginBrowser.do_login>`::
class ExampleBrowser(LoginBrowser):
login = URL('/login', LoginPage)
......@@ -319,7 +320,7 @@ Also, your ``LoginPage`` may look like::
form['password'] = password
form.submit()
Then, each method on your browser which need your user to be authenticated may be decorated by :func:`need_login <weboob.browser2.page.need_login>`::
Then, each method on your browser which need your user to be authenticated may be decorated by :func:`need_login <weboob.browser.browsers.need_login>`::
class ExampleBrowser(LoginBrowser):
accounts = URL('/accounts$', ListPage)
......@@ -329,9 +330,9 @@ Then, each method on your browser which need your user to be authenticated may b
self.accounts.stay_or_go()
return self.page.get_accounts()
The last thing to know is that :func:`need_login <weboob.browser2.page.need_login>` checks if the current page is a logged one by
reading the attribute :func:`logged <weboob.browser2.page.Page.logged>` of the instance. You can either define it yourself, as a
class boolean attribute or as a property, or to inherit your class from :class:`LoggedPage <weboob.browser2.page.LoggedPage>`.
The last thing to know is that :func:`need_login <weboob.browser.browsers.need_login>` checks if the current page is a logged one by
reading the attribute :func:`logged <weboob.browser.pages.Page.logged>` of the instance. You can either define it yourself, as a
class boolean attribute or as a property, or to inherit your class from :class:`LoggedPage <weboob.browser.pages.LoggedPage>`.
Parsing of pages
......@@ -342,7 +343,7 @@ Parsing of pages
When your browser locates on a page, an instance of the class related to the
:class:`URL <weboob.browser2.page.URL>` attribute which matches the url
:class:`URL <weboob.browser.url.URL>` attribute which matches the url
is created. You can declare methods on your class to allow your browser to
interact with it.
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Browser, BrowserHTTPNotFound
from weboob.deprecated.browser import Browser, BrowserHTTPNotFound
from .pages import RecipePage, ResultsPage
......
......@@ -20,7 +20,7 @@
from weboob.capabilities.recipe import Recipe, Comment
from weboob.capabilities.base import NotAvailable, NotLoaded
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
class ResultsPage(Page):
......
......@@ -17,8 +17,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser.decorators import id2url
from weboob.tools.browser import Browser
from weboob.deprecated.browser.decorators import id2url
from weboob.deprecated.browser import Browser
from .job import AdeccoJobAdvert
from .pages import SearchPage, AdvertPage
import urllib
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
from weboob.tools.html import html2text
from .job import AdeccoJobAdvert
import datetime
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.browser2 import PagesBrowser, URL
from weboob.browser import PagesBrowser, URL
from .pages import EventListPage, EventPage
from datetime import timedelta, date
......
......@@ -17,10 +17,10 @@
# You should have received a copy of the GNU Affero General Public License
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.browser2.page import HTMLPage, method, pagination
from weboob.browser2.elements import ItemElement, ListElement
from weboob.browser2.filters.standard import Regexp, CleanText, DateTime, Filter, Type, Env, Format, CombineDate
from weboob.browser2.filters.html import Link, XPath, CleanHTML
from weboob.browser.pages import HTMLPage, pagination
from weboob.browser.elements import ItemElement, ListElement, method
from weboob.browser.filters.standard import Regexp, CleanText, DateTime, Filter, Type, Env, Format, CombineDate
from weboob.browser.filters.html import Link, XPath, CleanHTML
from .calendar import AgendaDuLibreCalendarEvent
from datetime import time, datetime, date
......
......@@ -20,7 +20,7 @@
from weboob.capabilities.base import NotAvailable, NotLoaded
from weboob.capabilities.cinema import Movie, Person
from weboob.tools.browser import Browser
from weboob.deprecated.browser import Browser
from weboob.tools.json import json
import base64
import hashlib
......
......@@ -18,8 +18,8 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.browser2 import LoginBrowser, URL, need_login
from weboob.core.exceptions import BrowserIncorrectPassword
from weboob.browser import LoginBrowser, URL, need_login
from weboob.exceptions import BrowserIncorrectPassword
from .pages import LoginPage, AccountsPage
......
......@@ -21,9 +21,9 @@
import datetime
from decimal import Decimal
from weboob.browser2.page import HTMLPage, LoggedPage, method
from weboob.browser2.elements import ItemElement
from weboob.browser2.filters.standard import CleanDecimal, CleanText, Filter, TableCell
from weboob.browser.pages import HTMLPage, LoggedPage
from weboob.browser.elements import ItemElement, method
from weboob.browser.filters.standard import CleanDecimal, CleanText, Filter, TableCell
from weboob.capabilities.bank import Account
from weboob.tools.capabilities.bank.transactions import FrenchTransaction as Transaction
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Browser, BrowserHTTPNotFound
from weboob.deprecated.browser import Browser, BrowserHTTPNotFound
from .pages import RecipePage, ResultsPage, FourOFourPage
......
......@@ -20,7 +20,7 @@
from weboob.capabilities.recipe import Recipe
from weboob.capabilities.base import NotAvailable, NotLoaded
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
class FourOFourPage(Page):
......
......@@ -17,7 +17,7 @@
# You should have received a copy of the GNU Affero General Public License
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Browser, BrowserIncorrectPassword
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword
from weboob.capabilities.bill import Detail
from decimal import Decimal
from .pages import LoginPage, HomePage, AccountPage, LastPaymentsPage, PaymentDetailsPage, BillsPage
......
......@@ -22,7 +22,7 @@ from datetime import datetime
import re
import urllib
from decimal import Decimal
from weboob.tools.browser import Page, BrokenPageError
from weboob.deprecated.browser import Page, BrokenPageError
from weboob.capabilities.bill import Subscription, Detail, Bill
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
import urllib
from weboob.tools.browser import Browser, BrowserIncorrectPassword
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword
from weboob.capabilities.bill import Detail
from decimal import Decimal
from .pages import LoginPage, HomePage, AccountPage, HistoryPage, BillsPage
......
......@@ -22,7 +22,7 @@ from datetime import datetime
from decimal import Decimal
import re
import urllib
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
from weboob.capabilities.bill import Subscription, Detail, Bill
......
......@@ -20,7 +20,7 @@
from urlparse import urlsplit, parse_qsl
from weboob.tools.browser import Browser, BrowserIncorrectPassword
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword
from .pages import LoginPage, AccountsPage, TransactionsPage
......
......@@ -22,7 +22,7 @@ import datetime
from decimal import Decimal
import re
from weboob.tools.browser import Page, BrokenPageError
from weboob.deprecated.browser import Page, BrokenPageError
from weboob.capabilities.bank import Account
from weboob.capabilities import NotAvailable
from weboob.tools.capabilities.bank.transactions import FrenchTransaction as Transaction
......
......@@ -17,8 +17,8 @@
# You should have received a copy of the GNU Affero General Public License
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser.decorators import id2url
from weboob.tools.browser import Browser
from weboob.deprecated.browser.decorators import id2url
from weboob.deprecated.browser import Browser
import urllib
from .pages import SearchPage, AdvertPage
from .job import ApecJobAdvert
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
from weboob.tools.html import html2text
import dateutil.parser
import re
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Browser, BrowserIncorrectPassword
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword
from .pages import LoginPage, AccountsPage, OperationsPage
......
......@@ -21,7 +21,7 @@
from decimal import Decimal
from weboob.capabilities.bank import Account
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
from weboob.tools.capabilities.bank.transactions import FrenchTransaction
......
......@@ -18,8 +18,8 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Browser, BrowserIncorrectPassword
from weboob.tools.browser.decorators import id2url
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword
from weboob.deprecated.browser.decorators import id2url
from .pages import VideoPage, IndexPage, LoginPage, LoginRedirectPage
from .video import ArretSurImagesVideo
......
......@@ -21,7 +21,7 @@ import re
from weboob.capabilities.base import UserError
from weboob.capabilities.image import BaseImage
from weboob.tools.browser import Page, BrokenPageError
from weboob.deprecated.browser import Page, BrokenPageError
from weboob.capabilities import NotAvailable
......
......@@ -25,8 +25,8 @@ import urllib
from weboob.capabilities import NotAvailable
from weboob.capabilities.image import BaseImage
from weboob.tools.json import json as simplejson
from weboob.tools.browser import Browser
from weboob.tools.browser.decorators import id2url
from weboob.deprecated.browser import Browser
from weboob.deprecated.browser.decorators import id2url
from .pages import ArteLivePage, ArteLiveVideoPage
from .video import ArteVideo, ArteLiveVideo
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
from weboob.tools.html import html2text
from weboob.capabilities import NotAvailable
from weboob.capabilities.image import BaseImage
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import Browser, BrowserHTTPNotFound
from weboob.deprecated.browser import Browser, BrowserHTTPNotFound
from .pages import SubtitlesPage, SearchPage
......
......@@ -20,7 +20,7 @@
from weboob.capabilities.subtitle import Subtitle
from weboob.capabilities.base import NotAvailable
from weboob.tools.browser import Page
from weboob.deprecated.browser import Page
class SearchPage(Page):
......
......@@ -24,7 +24,7 @@ from weboob.tools.capabilities.streaminfo import StreamInfo
from weboob.capabilities.collection import CapCollection, Collection
from weboob.tools.backend import Module, BackendConfig
from weboob.tools.value import Value
from weboob.tools.browser import StandardBrowser
from weboob.deprecated.browser import StandardBrowser
import time
__all__ = ['AudioAddictModule']
......
......@@ -26,9 +26,10 @@ import re
import urllib
import urllib2
from weboob.tools.browser import Browser, BrowserIncorrectPassword, BrowserHTTPNotFound, BrowserUnavailable
from weboob.browser2.page import LoginBrowser, HTMLPage
from weboob.browser2.filters.standard import CleanText
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword, BrowserHTTPNotFound, BrowserUnavailable
from weboob.browser.browsers import LoginBrowser
from weboob.browser.pages import HTMLPage
from weboob.browser.filters.standard import CleanText
from weboob.tools.json import json
from weboob.tools.date import local2utc
from weboob.tools.misc import to_unicode
......
......@@ -31,7 +31,7 @@ from weboob.capabilities.dating import CapDating, OptimizationNotFound, Event
from weboob.capabilities.contact import CapContact, ContactPhoto, Query, QueryError
from weboob.capabilities.account import CapAccount, StatusField
from weboob.tools.backend import Module, BackendConfig
from weboob.tools.browser import BrowserUnavailable, BrowserHTTPNotFound
from weboob.deprecated.browser import BrowserUnavailable, BrowserHTTPNotFound
from weboob.tools.value import Value, ValueBool, ValueBackendPassword
from weboob.tools.date import local2utc
from weboob.tools.misc import to_unicode
......
......@@ -19,7 +19,7 @@
from random import randint
from weboob.tools.browser import BrowserUnavailable
from weboob.deprecated.browser import BrowserUnavailable
from weboob.capabilities.dating import Optimization
from weboob.tools.log import getLogger
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import BrowserUnavailable
from weboob.deprecated.browser import BrowserUnavailable
from weboob.capabilities.dating import Optimization
from weboob.capabilities.contact import QueryError
from weboob.tools.log import getLogger
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.tools.browser import BrowserUnavailable
from weboob.deprecated.browser import BrowserUnavailable
from weboob.capabilities.dating import Optimization
......
......@@ -19,7 +19,7 @@
from weboob.tools.test import BackendTest
from weboob.tools.browser import BrowserUnavailable
from weboob.deprecated.browser import BrowserUnavailable
class AuMTest(BackendTest):
......
......@@ -20,7 +20,7 @@
import urllib
from weboob.tools.browser import Browser, BrowserIncorrectPassword
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword
from .pages import LoginPage, AccountsPage, TransactionsPage, CBTransactionsPage, UnavailablePage
......
......@@ -22,7 +22,7 @@ import urllib
from decimal import Decimal, InvalidOperation
import re
from weboob.tools.browser import Page as _BasePage, BrowserUnavailable, BrokenPageError
from weboob.deprecated.browser import Page as _BasePage, BrowserUnavailable, BrokenPageError
from weboob.capabilities.bank import Account
from weboob.tools.capabilities.bank.transactions import FrenchTransaction
from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard
......
......@@ -18,8 +18,8 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.
from weboob.browser2 import LoginBrowser, need_login, URL
from weboob.core.exceptions import BrowserIncorrectPassword
from weboob.browser import LoginBrowser, need_login, URL
from weboob.exceptions import BrowserIncorrectPassword
from .pages import LoginPage, IndexPage, AccountsPage, OperationsPage
......
......@@ -24,13 +24,13 @@ import re
from cStringIO import StringIO
from weboob.capabilities.bank import Account
from weboob.browser2.page import HTMLPage, method, LoggedPage
from weboob.browser2.elements import ListElement, ItemElement
from weboob.browser2.filters.standard import CleanText, Regexp, CleanDecimal, Env
from weboob.browser2.filters.html import Attr
from weboob.browser.pages import HTMLPage, LoggedPage
from weboob.browser.elements import ListElement, ItemElement, method
from weboob.browser.filters.standard import CleanText, Regexp, CleanDecimal, Env
from weboob.browser.filters.html import Attr
from weboob.tools.captcha.virtkeyboard import MappedVirtKeyboard, VirtKeyboardError
from weboob.tools.capabilities.bank.transactions import FrenchTransaction
from weboob.core.exceptions import ParseError
from weboob.exceptions import ParseError
class Transaction(FrenchTransaction):
......
......@@ -20,7 +20,7 @@
import urllib
from weboob.tools.browser import Browser, BrowserIncorrectPassword, BrokenPageError
from weboob.deprecated.browser import Browser, BrowserIncorrectPassword, BrokenPageError
from .pages import LoginPage, IndexPage, AccountsPage, CardsPage, TransactionsPage, \
UnavailablePage, RedirectPage, HomePage
......
......@@ -24,7 +24,7 @@ from decimal import Decimal
import re
from mechanize import Cookie, FormNotFoundError
from weboob.tools.browser import Page as _BasePage, BrowserUnavailable, BrokenPageError
from weboob.deprecated.browser import Page as _BasePage, BrowserUnavailable, BrokenPageError
from weboob.capabilities.bank import Account
from weboob.tools.capabilities.bank.transactions import FrenchTransaction
......
......@@ -18,7 +18,7 @@
# along with weboob. If not, see <http://www.gnu.org/licenses/>.