pax_global_header 0000666 0000000 0000000 00000000064 11375270370 0014517 g ustar 00root root 0000000 0000000 52 comment=c5fef77bc8216a5aa8f0f2825fb79b577edfba4f
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/ 0000775 0000000 0000000 00000000000 11375270370 0020071 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/.gitignore 0000664 0000000 0000000 00000000066 11375270370 0022063 0 ustar 00root root 0000000 0000000 *_ui.py
*.pyc
*.swp
Session.vim
*.egg-info
build
dist
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/COPYING 0000664 0000000 0000000 00000104514 11375270370 0021131 0 ustar 00root root 0000000 0000000 GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc.
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
Copyright (C)
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see .
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
Copyright (C)
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
.
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/INSTALL 0000664 0000000 0000000 00000002114 11375270370 0021120 0 ustar 00root root 0000000 0000000 Weboob installation
===================
Like any setuptools package, Weboob can be installed in normal mode,
or in development mode.
Debian note
-----------
When using Debian, it is advised to use Python Debian packages, and not the PyPI ones.
To achieve this, please install the following packages before installing Weboob:
* python-gdata
normal mode
-----------
Install:
# ./setup.py install --record installed_files.txt
The files are simply copied in /usr/lib/python2.5/site-packages/Weboob-x.y-py2.5.egg
The --record flag is used to remember which files were copied.
Uninstall:
# cat installed_files.txt | xargs rm -rf
development mode
----------------
The development mode doesn't copy files, but it creates an egg-link
which points to the development directory.
It is useful for development when files often change.
# ./setup.py develop
Creates /usr/lib/python2.5/site-packages/Weboob.egg-link
bash completion
---------------
To enable bash completion, just source the tools/weboob_bash_completion
file from your ~/.bash_completion file (which is sourced by /etc/bash_completion).
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/ 0000775 0000000 0000000 00000000000 11375270370 0021560 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/boobank 0000775 0000000 0000000 00000001566 11375270370 0023131 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.boobank import Boobank
if __name__ == '__main__':
Boobank.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/chatoob 0000775 0000000 0000000 00000001606 11375270370 0023130 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.frontends.chatoob import Chatoob
if __name__ == '__main__':
Chatoob.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/havesex 0000775 0000000 0000000 00000001561 11375270370 0023154 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.havesex import HaveSex
if __name__ == '__main__':
HaveSex.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/masstransit 0000775 0000000 0000000 00000001576 11375270370 0024067 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Julien Hébert
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.masstransit import Masstransit
if __name__ == '__main__':
Masstransit.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/monboob 0000775 0000000 0000000 00000001604 11375270370 0023142 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.frontends.monboob import Monboob
if __name__ == '__main__':
Monboob.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/qvideoob 0000775 0000000 0000000 00000001564 11375270370 0023324 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.qvideoob import QVideoob
if __name__ == '__main__':
QVideoob.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/travel 0000775 0000000 0000000 00000001556 11375270370 0023012 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.travel import Travel
if __name__ == '__main__':
Travel.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/videoob 0000775 0000000 0000000 00000001563 11375270370 0023142 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.videoob import Videoob
if __name__ == '__main__':
Videoob.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/videoob-web-server 0000775 0000000 0000000 00000001575 11375270370 0025224 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.videoob_web import VideoobWeb
if __name__ == '__main__':
VideoobWeb.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/weboob-debug 0000775 0000000 0000000 00000001622 11375270370 0024050 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.frontends.weboobdebug import WeboobDebug
if __name__ == '__main__':
WeboobDebug.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/weboobcfg 0000775 0000000 0000000 00000001567 11375270370 0023454 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.weboobcfg import WeboobCfg
if __name__ == '__main__':
WeboobCfg.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/weboorrents 0000775 0000000 0000000 00000001575 11375270370 0024067 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.weboorrents import Weboorrents
if __name__ == '__main__':
Weboorrents.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/scripts/wetboobs 0000775 0000000 0000000 00000001564 11375270370 0023340 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.frontends.wetboobs import WetBoobs
if __name__ == '__main__':
WetBoobs.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/setup.py 0000775 0000000 0000000 00000002535 11375270370 0021613 0 ustar 00root root 0000000 0000000 #! /usr/bin/env python
# -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
try:
from setuptools import setup, find_packages
except ImportError:
from ez_setup import use_setuptools
use_setuptools()
from setuptools import setup, find_packages
import os
setup(
name='weboob',
version='0.1',
description='Weboob, web out of the browser',
author='Romain Bignon',
author_email='romain@peerfuse.org',
license='GPLv3',
url='http://www.weboob.org',
packages=find_packages(exclude=['ez_setup']),
scripts=[os.path.join('scripts', script) for script in os.listdir('scripts')],
package_data={'weboob.frontends.videoob_web': ['templates/*.mako']},
install_requires=[
]
)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/tools/ 0000775 0000000 0000000 00000000000 11375270370 0021231 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/tools/debpydep.py 0000775 0000000 0000000 00000002340 11375270370 0023401 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
import os
import subprocess
import sys
from cStringIO import StringIO
selection = set()
dependencies = set()
for root, dirs, files in os.walk(sys.argv[1]):
for f in files:
if f.endswith('.py') and f != '__init__.py':
s = "from %s import %s" % (root.strip('/').replace('/', '.'), f[:-3])
try:
exec s
except ImportError, e:
print >>sys.stderr, str(e)
else:
m = eval(f[:-3])
for attrname in dir(m):
try:
attr = getattr(m, attrname)
selection.add(attr.__file__)
except AttributeError:
pass
for f in selection:
f = f.replace('.pyc', '.py')
try:
f = os.path.abspath(os.path.join(os.path.split(f)[0], os.readlink(f)))
except OSError:
pass
p = subprocess.Popen(['/usr/bin/dpkg', '-S', f], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
if p.wait() == 0:
for line in p.stdout.readlines():
dependencies.add(line.strip().split(':')[0])
else:
print 'not found: %s' % f
for d in dependencies:
print d
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/tools/pyflakes.sh 0000775 0000000 0000000 00000000165 11375270370 0023410 0 ustar 00root root 0000000 0000000 #!/bin/bash
if [ `basename $PWD` == 'tools' ]; then
cd ..
fi
pyflakes weboob scripts/* | grep -v __init__.py
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/tools/pyreverse.sh 0000775 0000000 0000000 00000000363 11375270370 0023616 0 ustar 00root root 0000000 0000000 #!/bin/bash
#
# Examples:
# pyreverse.sh weboob.backends.aum
#
# pyreverse is included in pylint Debian package
function usage() {
echo "pyreverse.sh "
exit
}
[ -z "$1" ] && usage
pyreverse -p "$1" -a1 -s1 -o pdf "$1"
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/tools/weboob_bash_completion 0000664 0000000 0000000 00000002471 11375270370 0025663 0 ustar 00root root 0000000 0000000 # Weboob completion for Bash
#
# vim: filetype=sh expandtab softtabstop=4 shiftwidth=4
#
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
#
# This script can be distributed under the same license as the
# weboob or bash packages.
have weboobcfg &&
_weboob()
{
local cur words
COMPREPLY=()
cur=${COMP_WORDS[COMP_CWORD]}
words="$(${COMP_WORDS[0]} --shell-completion)"
case ${COMP_WORDS[1]} in
*)
COMPREPLY=( $( compgen -W "$words" | grep "^$cur" ) )
;;
esac
return 0
}
[ "$have" ] || return
weboob_applications=$(weboobcfg applications)
for application in $weboob_applications
do
complete -F _weboob $application
done
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/ 0000775 0000000 0000000 00000000000 11375270370 0021346 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/__init__.py 0000664 0000000 0000000 00000001345 11375270370 0023462 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .ouiboube import Weboob, CallErrors
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backend.py 0000664 0000000 0000000 00000011646 11375270370 0023317 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
import os
from threading import RLock
__all__ = ['BackendStorage', 'BaseBackend']
class BackendStorage(object):
def __init__(self, name, storage):
self.name = name
self.storage = storage
def set(self, *args):
if self.storage:
return self.storage.set(self.name, *args)
def get(self, *args, **kwargs):
if self.storage:
return self.storage.get(self.name, *args, **kwargs)
else:
return kwargs.get('default', None)
def load(self, default):
if self.storage:
return self.storage.load(self.name, default)
def save(self):
if self.storage:
return self.storage.save(self.name)
class BaseBackend(object):
# Module name.
NAME = None
# Name of the maintainer of this module.
MAINTAINER = ''
# Email address of the maintainer.
EMAIL = ''
# Version of module (for information only).
VERSION = ''
# Description
DESCRIPTION = ''
# License of this module.
LICENSE = ''
# Configuration required for this module. # Values must be ConfigField
# objects.
CONFIG = {}
# Storage
STORAGE = {}
# Browser class
BROWSER = None
class ConfigField(object):
def __init__(self, default=None, is_masked=False, regexp=None, description=None):
self.default = default
self.is_masked = is_masked
self.regexp = regexp
self.description = description
class ConfigError(Exception): pass
def __enter__(self):
self.lock.acquire()
def __exit__(self, t, v, tb):
self.lock.release()
def __repr__(self):
return u"" % self.name
def __init__(self, weboob, name, config, storage):
self.weboob = weboob
self.name = name
self.lock = RLock()
# Private fields (which start with '_')
self._private_config = dict([(key, value) for key, value in config.iteritems() if key.startswith('_')])
# Configuration of backend
self.config = {}
for name, field in self.CONFIG.iteritems():
value = config.get(name, field.default)
if value is None:
raise BaseBackend.ConfigError('Missing parameter "%s" (%s)' % (name, field.description))
if field.regexp and re.match(field.regexp, str(value)):
raise BaseBackend.ConfigError('Value of "%s" does not match regexp "%s"' % (name, field.regexp))
if not field.default is None:
if isinstance(field.default, bool) and not isinstance(value, bool):
value = value.lower() in ('1', 'true', 'on', 'yes')
elif isinstance(field.default, int) and not isinstance(value, int):
value = int(value)
elif isinstance(field.default, float) and not isinstance(value, float):
value = float(value)
self.config[name] = value
self.storage = BackendStorage(self.name, storage)
self.storage.load(self.STORAGE)
@property
def browser(self):
"""
Attribute 'browser'. The browser is created at the first call
of this attribute, to avoid useless pages access.
Note that the 'default_browser' method is called to create it.
"""
if not hasattr(self, '_browser'):
self._browser = self.default_browser()
return self._browser
def default_browser(self):
"""
Method to overload to build the default browser in
attribute 'browser'.
"""
return self.build_browser()
def build_browser(self, *args, **kwargs):
"""
Build a browser from the BROWSER class attribute and the
given arguments.
"""
if not self.BROWSER:
return None
if '_proxy' in self._private_config:
kwargs['proxy'] = self._private_config['_proxy']
elif 'HTTP_PROXY' in os.environ:
kwargs['proxy'] = os.environ['HTTP_PROXY']
return self.BROWSER(*args, **kwargs)
def has_caps(self, *caps):
for c in caps:
if isinstance(self, c):
return True
return False
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/ 0000775 0000000 0000000 00000000000 11375270370 0023120 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0025217 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/ 0000775 0000000 0000000 00000000000 11375270370 0023702 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/__init__.py 0000664 0000000 0000000 00000001375 11375270370 0026021 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .browser import AdopteUnMec
from .backend import AuMBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/backend.py 0000664 0000000 0000000 00000013525 11375270370 0025651 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from __future__ import with_statement
from datetime import datetime
from dateutil import tz
from weboob.backend import BaseBackend
from weboob.capabilities.chat import ICapChat
from weboob.capabilities.messages import ICapMessages, ICapMessagesReply, Message
from weboob.capabilities.dating import ICapDating
from weboob.tools.browser import BrowserUnavailable
from .browser import AdopteUnMec
from .optim.profiles_walker import ProfilesWalker
__all__ = ['AuMBackend']
class AuMBackend(BaseBackend, ICapMessages, ICapMessagesReply, ICapDating, ICapChat):
NAME = 'aum'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '1.0'
LICENSE = 'GPLv3'
DESCRIPTION = "French dating website"
CONFIG = {'username': BaseBackend.ConfigField(description='Username on website'),
'password': BaseBackend.ConfigField(description='Password of account', is_masked=True),
}
STORAGE = {'profiles_walker': {'viewed': []},
'sluts': {},
}
BROWSER = AdopteUnMec
def default_browser(self):
return self.build_browser(self.config['username'], self.config['password'])
# Private
_profiles_walker = None
def iter_messages(self, thread=None):
for message in self._iter_messages(thread, False):
yield message
def iter_new_messages(self, thread=None):
for message in self._iter_messages(thread, True):
yield message
def _iter_messages(self, thread, only_new):
with self.browser:
try:
profiles = {}
contacts = self.browser.get_contact_list()
for contact in contacts:
if not contact.get_id() in self.storage.get('sluts'):
slut = {'lastmsg': datetime(1970,1,1),
'msgstatus': ''}
else:
slut = self.storage.get('sluts', contact.get_id())
last_msg = slut['lastmsg'].replace(tzinfo=tz.tzutc())
new_lastmsg = last_msg
if only_new and contact.get_lastmsg_date() < last_msg and contact.get_status() == slut['msgstatus'] or \
not thread is None and int(thread) != contact.get_id():
continue
mails = self.browser.get_thread_mails(contact.get_id())
for mail in mails:
if only_new and mail.get_date() <= last_msg:
continue
if not mail.profile_link in profiles:
profiles[mail.profile_link] = self.browser.get_profile(mail.profile_link)
mail.signature += u'\n%s' % profiles[mail.profile_link].get_profile_text()
if new_lastmsg < mail.get_date():
new_lastmsg = mail.get_date()
yield mail
slut['lastmsg'] = new_lastmsg
slut['msgstatus'] = contact.get_status()
self.storage.set('sluts', contact.get_id(), slut)
self.storage.save()
# Send mail when someone added me in her basket.
# XXX possibly race condition if a slut adds me in her basket
# between the aum.nbNewBaskets() and aum.getBaskets().
new_baskets = self.browser.nb_new_baskets()
if new_baskets:
ids = self.browser.get_baskets()
while new_baskets > 0:
new_baskets -= 1
profile = self.browser.get_profile(ids[new_baskets])
yield Message(profile.get_id(), 1,
title='Basket of %s' % profile.get_name(),
sender=profile.get_name(),
content='You are taken in her basket!',
signature=profile.get_profile_text())
except BrowserUnavailable:
pass
def post_reply(self, thread_id, reply_id, title, message):
with self.browser:
self.browser.post_mail(thread_id, message)
def get_profile(self, _id):
try:
with self.browser:
return self.browser.get_profile(_id)
except BrowserUnavailable:
return None
def start_profiles_walker(self):
self._profile_walker = ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)
def stop_profiles_walker(self):
if self._profiles_walker:
self._profiles_walker.stop()
self._profiles_walker = None
def iter_chat_contacts(self, online=True, offline=True):
return self.browser.iter_chat_contacts(online=online, offline=offline)
def iter_chat_messages(self, _id=None):
return self.browser.iter_chat_messages(_id)
def send_chat_message(self, _id, message):
return self.browser.send_chat_message(_id, message)
#def start_chat_polling(self):
#self._profile_walker = ProfilesWalker(self.weboob.scheduler, self.storage, self.browser)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/browser.py 0000664 0000000 0000000 00000030630 11375270370 0025741 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2008-2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import datetime
import time
from logging import warning
import random
import simplejson
import urllib
from weboob.tools.browser import BaseBrowser
from weboob.tools.parsers.html5libparser import Html5libParser
from weboob.backends.aum.exceptions import AdopteWait
from weboob.backends.aum.pages.home import HomePage
from weboob.backends.aum.pages.contact_list import ContactListPage
from weboob.backends.aum.pages.contact_thread import ContactThreadPage
from weboob.backends.aum.pages.baskets import BasketsPage
from weboob.backends.aum.pages.profile import ProfilePage
from weboob.backends.aum.pages.search import SearchPage
from weboob.backends.aum.pages.login import LoginPage, RedirectPage, BanPage, ErrPage, RegisterPage, RegisterWaitPage, RegisterConfirmPage, ShopPage
from weboob.backends.aum.pages.edit import EditPhotoPage, EditPhotoCbPage, EditAnnouncePage, EditDescriptionPage, EditSexPage, EditPersonalityPage
from weboob.backends.aum.pages.wait import WaitPage
from weboob.capabilities.chat import ChatContact, ChatException, ChatMessage
__all__ = ['AdopteUnMec']
class AdopteUnMec(BaseBrowser):
DOMAIN = 'www.adopteunmec.com'
PROTOCOL = 'http'
ENCODING = 'iso-8859-1'
PAGES = {'http://www.adopteunmec.com/': LoginPage,
'http://www.adopteunmec.com/index.html': LoginPage,
'http://www.adopteunmec.com/index.php': LoginPage,
'http://www.adopteunmec.com/loginErr.php.*': ErrPage,
'http://www.adopteunmec.com/bans.php.*': BanPage,
'http://www.adopteunmec.com/redirect.php\?action=login': RedirectPage,
'http://www.adopteunmec.com/wait.php': WaitPage,
'http://www.adopteunmec.com/register2.php': RegisterPage,
'http://www.adopteunmec.com/register3.php.*': RegisterWaitPage,
'http://www.adopteunmec.com/register4.php.*': RegisterConfirmPage,
'http://www.adopteunmec.com/home.php': HomePage,
'http://www.adopteunmec.com/shop2c.php': ShopPage,
'http://www.adopteunmec.com/mails.php': ContactListPage,
'http://www.adopteunmec.com/mails.php\?type=1': BasketsPage,
'http://www.adopteunmec.com/thread.php\?id=([0-9]+)': ContactThreadPage,
'http://www.adopteunmec.com/edit.php\?type=1': EditPhotoPage,
'http://s\d+.adopteunmec.com/upload\d.php\?.*': EditPhotoCbPage,
'http://www.adopteunmec.com/edit.php\?type=2': EditAnnouncePage,
'http://www.adopteunmec.com/edit.php\?type=3': EditDescriptionPage,
'http://www.adopteunmec.com/edit.php\?type=4': EditSexPage,
'http://www.adopteunmec.com/edit.php\?type=5': EditPersonalityPage,
'http://www.adopteunmec.com/search.php.*': SearchPage,
'http://www.adopteunmec.com/searchRes.php.*': SearchPage,
'http://www.adopteunmec.com/rencontres-femmes/(.*)/([0-9]+)': ProfilePage,
'http://www.adopteunmec.com/catalogue-hommes/(.*)/([0-9]+)': ProfilePage,
'http://www.adopteunmec.com/view2.php': ProfilePage, # my own profile
'http://www.adopteunmec.com/(\w+)': ProfilePage, # a custom profile url
}
def __init__(self, *args, **kwargs):
kwargs['parser'] = Html5libParser(api='dom')
BaseBrowser.__init__(self, *args, **kwargs)
self.my_id = 0
def login(self):
if not self.is_on_page(LoginPage):
self.home()
self.page.login(self.username, self.password)
def is_logged(self):
return not self.is_on_page(LoginPage)
def home(self):
return self.location('http://www.adopteunmec.com/home.php')
def pageaccess(func):
def inner(self, *args, **kwargs):
if self.is_on_page(WaitPage):
if not self.page.check():
raise AdopteWait()
self.home()
if not self.page or self.is_on_page(LoginPage) and self.password:
self.home()
return func(self, *args, **kwargs)
return inner
def register(self, nickname, password, sex, birthday_d, birthday_m, birthday_y, zipcode, country, godfather=''):
if not self.is_on_page(RegisterPage):
self.location('http://www.adopteunmec.com/register2.php')
return self.page.register(nickname, password, sex, birthday_d, birthday_m, birthday_y, zipcode, country, godfather)
@pageaccess
def add_photo(self, name, f):
if not self.is_on_page(EditPhotoPage):
self.location('/edit.php?type=1')
return self.page.add_photo(name, f)
@pageaccess
def set_nickname(self, nickname):
if not self.is_on_page(EditAnnouncePage):
self.location('/edit.php?type=2')
return self.page.set_nickname(nickname)
@pageaccess
def set_announce(self, title=None, description=None, lookingfor=None):
if not self.is_on_page(EditAnnouncePage):
self.location('/edit.php?type=2')
return self.page.set_announce(title, description, lookingfor)
@pageaccess
def set_description(self, **args):
if not self.is_on_page(EditDescriptionPage):
self.location('/edit.php?type=3')
return self.page.set_description(**args)
@pageaccess
def score(self):
if time.time() - self.last_update > 60:
self.home()
return self.page.score()
@pageaccess
def get_my_name(self):
if time.time() - self.last_update > 60:
self.home()
return self.page.get_my_name()
@pageaccess
def get_my_id(self):
if self.my_id:
return self.my_id
if not self.is_on_page(HomePage):
self.home()
self.my_id = self.page.get_my_id()
return self.my_id
@pageaccess
def nb_new_mails(self):
if time.time() - self.last_update > 60:
self.home()
return self.page.nb_new_mails()
@pageaccess
def nb_new_baskets(self):
if time.time() - self.last_update > 60:
self.home()
return self.page.nb_new_baskets()
@pageaccess
def nb_new_visites(self):
if time.time() - self.last_update > 60:
self.home()
return self.page.nb_new_visites()
@pageaccess
def nb_available_charms(self):
self.home()
return self.page.nb_available_charms()
@pageaccess
def get_baskets(self):
self.location('/mails.php?type=1')
return self.page.get_profiles_ids_list()
@pageaccess
def flush_visits(self):
""" Does nothing, only flush new visits to increase my score """
self.openurl('/mails.php?type=3')
@pageaccess
def get_contact_list(self):
if not self.is_on_page(ContactListPage):
self.location('/mails.php')
return self.page.get_contact_list()
@pageaccess
def get_thread_mails(self, id):
if not self.is_on_page(ContactThreadPage) or self.page.get_id() != int(id):
self.page.open_thread_page(id)
return self.page.get_mails()
@pageaccess
def post_mail(self, id, content):
if not self.is_on_page(ContactThreadPage) or self.page.get_id() != int(id):
self.page.open_thread_page(id)
self.page.post(content)
@pageaccess
def send_charm(self, id):
result = self.openurl('http://www.adopteunmec.com/fajax_addBasket.php?id=%s' % id).read()
warning('Charm: %s' % result)
return result.find('noMoreFlashes') < 0
@pageaccess
def add_basket(self, id):
result = self.openurl('http://www.adopteunmec.com/fajax_addBasket.php?id=%s' % id).read()
warning('Basket: %s' % result)
# TODO check if it works (but it should)
return True
def deblock(self, id):
result = self.openurl('http://www.adopteunmec.com/fajax_postMessage.php?action=deblock&to=%s' % id).read()
warning('Deblock: %s' % result)
return True
@pageaccess
def rate(self, id, what, rating):
print 'rate "%s"' % id, what, rating
result = self.openurl('http://www.adopteunmec.com/fajax_vote.php', 'member=%s&what=%s&rating=%s' % (id, what, rating)).read()
print result
return float(result)
@pageaccess
def search_profiles(self, **kwargs):
self.location('/search.php?display=1')
self.page.search(**kwargs)
return self.page.get_profiles_ids()
@pageaccess
def get_profile(self, link):
if isinstance(link, (str,unicode)) and link.startswith('/'):
link = link[1:]
self.location('/%s' % link)
return self.page
@pageaccess
def get_slut_state(self, id):
result = self.openurl('http://www.adopteunmec.com/%s' % id).read()
if result.find('
en ligne
') >= 0:
r = 'online'
elif result.find('Cet utilisateur a quitt\xe9 le site ') >= 0:
r = 'removed'
elif result.find('ce profil a \xe9t\xe9 bloqu\xe9 par l\'\xe9quipe de mod\xe9ration ') >= 0:
r = 'removed'
elif result.find('
Cette personne vous a bloqu\xe9
') >= 0:
r = 'blocked'
else:
r = 'offline'
print 'getSlutState(%s) = %s' % (id, r)
return r
@pageaccess
def is_slut_online(self, id):
result = self.openurl('http://www.adopteunmec.com/%s' % id).read()
r = result.find('
en ligne
') >= 0
print 'isSlutOnline(%s) = %s' % (id, r)
return r
def _get_chat_infos(self):
json = simplejson.load(self.openurl('http://www.adopteunmec.com/1.1_cht_get.php?anticache=%f' % random.random()))
if json['error']:
raise ChatException(u'Error while getting chat infos. json:\n%s' % json)
return json
def iter_chat_contacts(self, online=True, offline=True):
def iter_dedupe(contacts):
yielded_ids = set()
for contact in contacts:
if contact['id'] not in yielded_ids:
yield contact
yielded_ids.add(contact['id'])
json = self._get_chat_infos()
for contact in iter_dedupe(json['contacts']):
if online and contact['cat'] == 1 or offline and contact['cat'] == 3:
if contact['cat'] == 1:
online = True
elif contact['cat'] == 3:
online = False
else:
raise ChatException(u'Unknown online status: contact=%s' % contact)
yield ChatContact(_id=contact['id'], pseudo=contact['pseudo'], online=online, avatar_url=contact['cover'], age=contact['birthday'])
def iter_chat_messages(self, _id=None):
json = self._get_chat_infos()
if json['messages'] is not None:
for message in json['messages']:
yield ChatMessage(id_from=message['id_from'], id_to=message['id_to'], message=message['message'], date=message['date'])
def send_chat_message(self, _id, message):
url = 'http://www.adopteunmec.com/1.1_cht_send.php?anticache=%f' % random.random()
data = dict(id=_id, message=message)
headers = {
'Content-type': 'application/x-www-form-urlencoded',
'Accept': 'text/plain',
'Referer': 'http://www.adopteunmec.com/chat.php',
'Origin': 'http://www.adopteunmec.com',
}
request = self.request_class(url, urllib.urlencode(data), headers)
response = self.openurl(request).read()
try:
datetime.datetime.strptime(response, '%Y-%m-%d %H:%M:%S')
return True
except ValueError:
return False
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/exceptions.py 0000664 0000000 0000000 00000001606 11375270370 0026440 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BrowserUnavailable
class AdopteWait(BrowserUnavailable):
pass
class AdopteBanned(BrowserUnavailable):
pass
class AdopteCantPostMail(Exception):
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/optim/ 0000775 0000000 0000000 00000000000 11375270370 0025032 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/optim/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0027131 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/optim/profiles_walker.py 0000664 0000000 0000000 00000006013 11375270370 0030574 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from __future__ import with_statement
from logging import info
from random import randint
from weboob.tools.browser import BrowserUnavailable
__all__ = ['ProfilesWalker']
class ProfilesWalker(object):
def __init__(self, sched, storage, browser):
self.sched = sched
self.storage = storage
self.browser = browser
self.visited_profiles = set(storage.get('profiles_walker', 'viewed'))
info(u'Loaded %d already visited profiles from storage.' % len(self.visited_profiles))
self.profiles_queue = set()
self.walk_cron = sched.repeat(60, self.enqueue_profiles)
self.view_cron = sched.schedule(randint(10,40), self.view_profile)
def save(self):
self.storage.set('profiles_walker', 'viewed', list(self.visited_profiles))
self.storage.save()
def stop(self):
self.event.cancel(self.event)
self.event = None
def enqueue_profiles(self):
with self.browser:
profiles_to_visit = self.browser.search_profiles().difference(self.visited_profiles)
info(u'Enqueuing profiles to visit: %s' % profiles_to_visit)
self.profiles_queue.update(profiles_to_visit)
self.save()
def view_profile(self):
try:
try:
id = self.profiles_queue.pop()
except KeyError:
return # empty queue
try:
with self.browser:
profile = self.browser.get_profile(id)
info(u'Visited profile %s (%s)' % (profile.get_name(), id))
# Get score from the aum_score module
#d = self.nucentral_core.callService(context.Context.fromComponent(self), 'aum_score', 'score', profile)
# d.addCallback(self.score_cb, profile.getID())
# deferredlist.append(d)
# do not forget that we visited this profile, to avoid re-visiting it.
self.visited_profiles.add(id)
self.save()
except BrowserUnavailable:
# We consider this profil hasn't been [correctly] analysed
self.profiles_queue.add(id)
return
except Exception, e:
print e
finally:
self.sched.schedule(randint(10,40), self.view_profile)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025001 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0027100 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/base.py 0000664 0000000 0000000 00000006065 11375270370 0026274 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
from logging import error, warning
from weboob.tools.browser import BasePage
class PageBase(BasePage):
def open_contact_list_page(self):
self.browser.follow_link(url_regex=r"/mails.php$")
def open_thread_page(self, id):
self.browser.location('/thread.php?id=%d' % int(id))
def score(self):
"""
popularité
7.230 pts
"""
l = self.document.getElementsByTagName('table')
for tag in l:
if tag.getAttribute('width') == '220':
#
child = tag.childNodes[1].childNodes[0].childNodes[3]
return child.childNodes[0].childNodes[0].data
error("Error: I can't find the score :(")
return '0'
def __get_indicator(self, elementName):
""" """
l = self.document.getElementsByTagName('span')
for tag in l:
if tag.getAttribute('id') == elementName:
child = tag.childNodes[0]
if not hasattr(child, 'data'):
if child.tagName != u'blink':
warning("Warning: %s counter isn't a blink and hasn't data" % elementName)
child = child.childNodes[0]
if not hasattr(child, 'data'):
break
return int(child.data)
error("Error: I can't find the %s counter :(" % elementName)
return 0
MYNAME_REGEXP = re.compile("Bonjour (.*)")
def get_my_name(self):
""" Bonjour Romain """
tags = self.document.getElementsByTagName('span')
for tag in tags:
if hasattr(tag.firstChild, 'data'):
m = self.MYNAME_REGEXP.match(tag.firstChild.data)
if m:
return m.group(1)
warning('Warning: Unable to fetch name')
return '?'
def nb_new_mails(self):
return self.__get_indicator(u'mailsCounter')
def nb_new_baskets(self):
return self.__get_indicator(u'flashsCounter')
def nb_new_visites(self):
return self.__get_indicator(u'visitesCounter')
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/baskets.py 0000664 0000000 0000000 00000001466 11375270370 0027016 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.profileslist_base import ProfilesListBase
class BasketsPage(ProfilesListBase):
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/contact_list.py 0000664 0000000 0000000 00000011736 11375270370 0030051 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2008-2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import re
from datetime import datetime, timedelta
from dateutil import tz
from logging import warning
from weboob.backends.aum.pages.base import PageBase
class ContactItem:
u"""
Hen
19ans, Montreuil
Comme ça, on est deux.
il y a 1 heure
nouveau
"""
fields = ['thread_link', 'photo', 'useless3', 'name', 'resume', 'status', 'useless', 'remove', 'useless2']
def __init__(self, tr):
self.tr = tr
def __get_element(self, id):
return self.tr.getElementsByTagName('td')[self.fields.index(id)]
def get_name(self):
tag = self.__get_element('name')
node = tag.getElementsByTagName('b')[0].firstChild
if node:
name = node.data
else:
# it is possible if the user has left site and hasn't any nickname
name = ''
return name
def get_status(self):
tag = self.__get_element('status')
return tag.firstChild.data
def is_new(self):
return self.get_status() == u'nouveau'
def is_answered(self):
return self.get_status() == u'répondu'
def get_resume(self):
tag = self.__get_element('resume')
return tag.getElementsByTagName('b')[0].firstChild.data.strip()
LASTMSG_RE = re.compile('il y a (\d+) (\w+)')
def get_lastmsg_date(self):
tag = self.__get_element('resume')
s = tag.childNodes[3].data
m = self.LASTMSG_RE.match(s)
if m:
d = {'secondes': 1,
'seconde': 1,
'minutes': 60,
'minute': 60,
'heures': 3600,
'heure': 3600,
'jours': 24*3600,
'jour': 24*3600,
}
try:
i = int(m.group(1)) * d[m.group(2)]
except KeyError:
warning('Unable to parse lastmsg ("%s" is not a valid unit)' % m.group(2))
return None
else:
return datetime.now(tz=tz.tzutc()) - timedelta(seconds=i)
else:
warning('Unable to parse lastmsg [%s]' % s)
return None
def get_id(self):
tag = self.__get_element('thread_link')
text = tag.getAttribute('onclick')
regexp = re.compile("window.location='/thread.php\?id=(.*)'")
m = regexp.match(text)
if m:
return int(m.group(1))
return 0
class ContactListPage(PageBase):
def on_loaded(self):
self.items = []
tags = self.document.getElementsByTagName('form')[0].childNodes[3].childNodes[1].childNodes
for tag in tags:
if not hasattr(tag, 'tagName') or tag.tagName != u'tr':
continue
if tag.hasAttribute('bgcolor'):
continue
self.items += [ContactItem(tag)]
def get_contact_list(self):
return self.items
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/contact_thread.py 0000664 0000000 0000000 00000033567 11375270370 0030353 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
from datetime import datetime
from dateutil import tz
from logging import error, warning
from mechanize import FormNotFoundError
from weboob.backends.aum.pages.base import PageBase
from weboob.backends.aum.exceptions import AdopteCantPostMail
from weboob.capabilities.messages import Message
class MailParser(Message):
"""
Romain
27 octobre 2008, 01:11:32, nouveau
Moi en g�n�ral j'aime sortir tout simplement dans des bars, pour discuter avec mes amis et/ou coll�gues, et rencontrer des gens. Sinon je fais de la guitare, et je d�veloppe des projets perso.
Message.__init__(self, id, 0, 'Discussion with %s' % name, name)
self.tr = tr.childNodes[0].childNodes[1].childNodes[0].childNodes[0]
tds = self.tr.childNodes
counter = 0
for td in tds:
if not hasattr(td, 'tagName') or td.tagName != u'td':
continue
counter += 1
if counter == 3:
date = u''
for child in td.childNodes[1].childNodes:
if hasattr(child, 'data'):
date += child.data
self.parse_date(date)
content = ''
for c in td.childNodes[3].childNodes:
if hasattr(c, 'data'):
content += ''.join(c.data.split('\n')) # to strip \n
elif hasattr(c, 'tagName'):
if c.tagName == 'br':
content += '\n'
elif c.tagName == 'img' and c.hasAttribute('src'):
m = self.SMILEY_REGEXP.match(c.getAttribute('src'))
if m and self.smileys.has_key(int(m.group(1))):
try:
content += self.smileys[int(m.group(1))]
except KeyError:
error('Mail: unable to find this smiley: %s' % c.getAttribute('src'))
self.content = content
break
self.parse_profile_link()
self.parse_from()
def set_reply_id(self, date):
self.reply_id = date
def parse_date(self, date_str):
# To match regexp, we have to remove any return chars in string
# before the status ('nouveau', 'lu', etc)
date_str = u''.join(date_str.split(u'\n'))
m = self.DATETIME_REGEXP.match(date_str)
if m:
day = int(m.group(1))
month = self.months.index(m.group(2))
year = int(m.group(3))
hour = int(m.group(4))
minute = int(m.group(5))
sec = int(m.group(6))
# build datetime object with local timezone
d = datetime(year, month, day, hour, minute, sec, tzinfo=tz.tzlocal())
# then, convert it to UTC timezone
d = d.astimezone(tz.tzutc())
# and get timestamp
self.date = d
self.id = self.get_date_int()
if m.group(7).find('nouveau') >= 0:
self.new = True
else:
error('Error: unable to parse the datetime string "%s"' % date_str)
def parse_profile_link(self):
tables = self.tr.getElementsByTagName('div')
for table in tables:
if table.hasAttribute('class') and table.getAttribute('class') == 'mini' and table.hasAttribute('onclick'):
text = table.getAttribute('onclick')
regexp = re.compile("window.location='(.*)'")
m = regexp.match(text)
if m:
self.profile_link = m.group(1)
self.signature = u'Profile: http://www.adopteunmec.com%s' % self.profile_link
return
warning('Unable to find the profile URL in the message %s@%s' % (self.get_from(), self.get_id()))
def parse_from(self):
tds = self.tr.getElementsByTagName('div')
for td in tds:
if not td.hasAttribute('class') or td.getAttribute('class') != 'mini_pseudo':
continue
if td.childNodes:
self.sender = td.childNodes[0].data
return
warning('Warning: unable to find from in the mail %s' % self.get_id())
class ContactThreadPage(PageBase):
"""
"""
def post(self, content):
try:
self.browser.select_form(name="sendMsg")
if isinstance(content, unicode):
content = content.encode('iso-8859-15', 'replace')
self.browser['message'] = content
self.browser.submit() # submit current form
except FormNotFoundError, e:
error = 'Unknown error (%s)' % e
p_list = self.document.getElementsByTagName('p')
for p in p_list:
if p.hasAttribute('align') and p.getAttribute('align') == 'center':
error = p.firstChild.data
break
raise AdopteCantPostMail(error)
"""
"""
id_regexp = re.compile("/thread.php\?id=([0-9]+)")
def on_loaded(self):
self.items = []
a_list = self.document.getElementsByTagName('a')
self.id = 0
for a in a_list:
if a.hasAttribute('href'):
m = self.id_regexp.match(a.getAttribute('href'))
if m:
self.id = int(m.group(1))
break
self.name = ''
big_list = self.document.getElementsByTagName('big')
for big in big_list:
child = big.firstChild
if hasattr(child, 'tagName') and child.tagName == u'b':
self.name = child.firstChild.data
break
tables = self.document.getElementsByTagName('table')
table = None
for t in tables:
if t.hasAttribute('style') and t.getAttribute('style') == 'width:700px;background:black':
table = t
break
if not table:
# It is probably the 'sent' page
return
for tag in table.childNodes[1].childNodes[1:]:
if not hasattr(tag, 'tagName') or tag.tagName != u'tr':
continue
if not tag.hasAttribute('valign'):
continue
mail = MailParser(self.id, self.name, tag)
if self.items:
self.items[-1].set_reply_id(mail.get_date_int())
self.items += [mail]
def get_id(self):
return self.id
def get_mails(self):
return self.items
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/edit.py 0000664 0000000 0000000 00000006673 11375270370 0026314 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.base import PageBase
class EditPhotoPage(PageBase):
def add_photo(self, name, f):
self.browser.select_form(name="form")
self.browser.find_control('uploaded').add_file(f, 'image/jpeg', name)
self.browser.submit()
self.browser.openurl('http://www.adopteunmec.com/home.php')
class EditPhotoCbPage(PageBase):
# Do nothing
pass
class EditAnnouncePage(PageBase):
def set_nickname(self, nickname):
self.browser.select_form(name="form")
self.browser['pseudo'] = nickname
self.browser.submit()
def set_announce(self, **kwargs):
self.browser.select_form(name="form")
self.browser.set_field(kwargs, 'title')
self.browser.set_field(kwargs, 'description', field='about1')
self.browser.set_field(kwargs, 'lookingfor', field='about2')
self.browser.submit()
class EditDescriptionPage(PageBase):
SHAPES = ['--', 'svelte', 'sportive', u'équilibrée', 'pulpeuse', u'généreuse', 'normale']
HAIR_COLORS = ['--', 'blancs', 'gris', 'noirs', 'bruns', 'chatains', 'roux', 'blonds', 'platines', u'colorés']
HAIR_SIZES = ['--', u'rasés', 'courts', 'mi-longs', 'longs']
EYES = ['--', 'noirs', 'marrons', 'noisettes', 'bleus', 'verts', 'gris']
ORIGINS = ['--', u'européennes', 'afro', 'maghrebines', 'asiatiques', u'métisses', 'eurasiennes', 'latines']
STYLES = ['--', 'fashion', 'chic', 'sport', u'décontracté', 'rock', u'bohème', 'masculin', 'dark', 'excentrique', 'electro', 'skate']
FOODS = ['--', 'mange de tout', 'piscovore', u'végétarien', u'végétalien', 'bio']
DRINKS = ['--', 'jamais', 'de temps en temps', 'souvent', 'pilier de bar']
SMOKES = ['--', u'ne tolère pas la fumée', u'tolère la fumée', 'fume de temps en temps', 'fume souvent']
def set_description(self, **kwargs):
self.browser.select_form(name='form')
self.browser.set_field(kwargs, 'height', field='size', is_list=True)
self.browser.set_field(kwargs, 'weight', is_list=True)
self.browser.set_field(kwargs, 'shape', is_list=self.SHAPES)
self.browser.set_field(kwargs, 'hair_color', is_list=self.HAIR_COLORS)
self.browser.set_field(kwargs, 'hair_size', is_list=self.HAIR_SIZES)
self.browser.set_field(kwargs, 'eyes', is_list=self.EYES)
self.browser.set_field(kwargs, 'origins', is_list=self.ORIGINS)
self.browser.set_field(kwargs, 'style', is_list=self.STYLES)
self.browser.set_field(kwargs, 'food', is_list=self.FOODS)
self.browser.set_field(kwargs, 'drink', is_list=self.DRINKS)
self.browser.set_field(kwargs, 'smoke', is_list=self.SMOKES)
self.browser.submit()
class EditSexPage(PageBase):
pass
class EditPersonalityPage(PageBase):
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/home.py 0000664 0000000 0000000 00000003607 11375270370 0026311 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
from weboob.backends.aum.pages.base import PageBase
from logging import error
class HomePage(PageBase):
MYID_REGEXP = re.compile("http://www.adopteunmec.com/\?mid=(\d+)")
def get_my_id(self):
fonts = self.document.getElementsByTagName('font')
for font in fonts:
m = self.MYID_REGEXP.match(font.firstChild.data)
if m:
return m.group(1)
error("Error: Unable to find my ID")
return 0
def nb_available_charms(self):
tables = self.document.getElementsByTagName('table')
for table in tables:
if table.hasAttribute('style') and table.getAttribute('style') == 'background-color:black;background-image:url(http://s.adopteunmec.com/img/barmec.gif);background-repeat:no-repeat':
fonts = table.getElementsByTagName('font')
i = 0
for font in fonts:
if font.hasAttribute('color') and font.getAttribute('color') == '#ff0198':
i += 1
if i == 3:
return int(font.firstChild.data)
error('Error: Unable to find the available charms counter')
return 0
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/login.py 0000664 0000000 0000000 00000005370 11375270370 0026470 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.base import PageBase
from weboob.tools.browser import BrowserIncorrectPassword
class LoginPage(PageBase):
def login(self, login, password):
self.browser.select_form(name="form_login")
self.browser['login'] = login
self.browser['password'] = password
self.browser.submit() # submit current form
class RegisterPage(PageBase):
def register(self, nickname, password, sex, birthday_d, birthday_m, birthday_y, zipcode, country, godfather):
""" form2:
- pseudo
- email
- password
- sex (0=m, 1=f)
- birthday0 (0-31)
- birthday1 (0-12)
- birthday2 (1930-1999)
- zip
- country (fr,be,ch,ca)
- godfather
"""
self.browser.select_form(name="form2")
self.browser.controls.pop() # pop the 'sex' control which is twice on page
self.browser.set_all_readonly(False)
if isinstance(nickname, unicode):
nickname = nickname.encode('iso-8859-15', 'ignore')
self.browser['pseudo'] = nickname
self.browser['email'] = self.browser.username
self.browser['pass'] = password
self.browser['sex0'] = [str(sex)]
self.browser['sex'] = str(sex)
self.browser['birthday0'] = [str(birthday_d)]
self.browser['birthday1'] = [str(birthday_m)]
self.browser['birthday2'] = [str(birthday_y)]
self.browser['zip'] = str(zipcode)
self.browser['country'] = [str(country)]
self.browser['godfather'] = godfather
self.browser.submit()
class RegisterWaitPage(PageBase):
pass
class RegisterConfirmPage(PageBase):
pass
class RedirectPage(PageBase):
def on_loaded(self):
for link in self.browser.links():
print link
self.browser.location('/wait.php')
class BanPage(PageBase):
pass
class ShopPage(PageBase):
pass
class ErrPage(PageBase):
def on_loaded(self):
raise BrowserIncorrectPassword('Incorrect login/password')
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/profile.py 0000664 0000000 0000000 00000037067 11375270370 0027030 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.base import PageBase
from weboob.capabilities.dating import Profile
from copy import deepcopy
from logging import warning
import re
class FieldBase:
def __init__(self, key):
self.key = key
def put_value(self, d, value):
raise NotImplementedError
class FieldString(FieldBase):
def put_value(self, d, value):
d[self.key] = unicode(value)
class FieldList(FieldBase):
def put_value(self, d, value):
d[self.key] = value.split(', ')
class FieldWideList(FieldBase):
def put_value(self, d, value):
d[self.key] += [value]
class FieldOld(FieldBase):
regexp = re.compile(u'([0-9]+) ans( \(Née le ([0-9]+) ([^ ]+) ([0-9]+)\))?')
month2i = ['', 'janvier', u'février', 'mars', 'avril', 'mai', 'juin', 'juillet', u'août', 'septembre', 'octobre', 'novembre', u'décembre']
def put_value(self, d, value):
m = self.regexp.match(value)
if not m:
return
d[self.key] = int(m.group(1))
if not m.group(2):
return
try:
d['birthday'] = (int(m.group(3)),
self.month2i.index(m.group(4)),
int(m.group(5)))
except ValueError, e:
print str(e)
class FieldLocation(FieldBase):
location = re.compile('(.*) \(([0-9]{5})\), (.*)')
def __init__(self):
FieldBase.__init__(self, '')
def put_value(self, d, value):
# TODO: determine distance, or something like
m = self.location.match(value)
if m:
d['location'] = m.group(1)
d['zipcode'] = int(m.group(2))
d['country'] = m.group(3)
else:
warning('Unable to parse the location "%s"' % value)
d['location'] = unicode(value)
class FieldMeasurements(FieldBase):
height = re.compile('([0-9]{1,3}) cm')
weight = re.compile('([0-9]{1,3}) kg')
# TODO: parse third parameter
def __init__(self):
FieldBase.__init__(self, '')
def put_value(self, d, value):
for s in value.split(', '):
m = self.height.match(s)
if m:
d['height'] = int(m.group(1))
continue
m = self.weight.match(s)
if m:
d['weight'] = int(m.group(1))
continue
if d['height'] and d['weight']:
bmi = (d['weight']/float(pow(d['height']/100.0, 2)))
if bmi < 15.5:
d['fat'] = 'severely underweight'
elif bmi < 18.4:
d['fat'] = 'underweight'
elif bmi < 24.9:
d['fat'] = 'normal'
elif bmi < 30:
d['fat'] = 'overweight'
else:
d['fat'] = 'obese'
d['BMI'] = bmi
class FieldParticularSignes(FieldBase):
def __init__(self): FieldBase.__init__(self, '')
def put_value(self, d, value):
for s in value.split(', '):
if s.find('tatouages') >= 0:
d['tatoos'] = True
elif s.find('piercing') >= 0:
d['piercing'] = True
elif s.find('lunettes') >= 0:
d['glasses'] = True
elif s.find('rousseur') >= 0:
d['freckle'] = True
class ProfilePage(PageBase, Profile):
empty_table = {'details': {'old': 0,
'birthday': (0,0,0),
'zipcode': 0,
'location': '',
'country': '',
'eyes': '',
'hairs': [],
'height': 0,
'weight': 0,
'BMI': 0,
'fat': '',
'from': '',
'tatoos': False,
'piercing': False,
'freckle': False,
'glasses': False,
'job': '',
'style': '',
'alimentation': '',
'alcool': '',
'tabac': '',
},
'liking': {'activities': '',
'music': [],
'cinema': [],
'books': [],
'tv': [],
},
'sex': {'underwear': [],
'top': [],
'bottom': [],
'interval': '',
'practices': [],
'favorite': '',
'toys': [],
},
'personality': {'snap': '',
'exciting': '',
'hate': '',
'vices': '',
'assets': '',
'fantasies': '',
'is': [],
},
}
tables = {'tab_0': 'details',
'tab_1': 'liking',
'tab_2': 'sex',
'tab_3': 'personality'
}
fields = {'details': {'Age': FieldOld('old'),
u'Réside à': FieldLocation(),
'Yeux': FieldString('eyes'),
'Cheveux ': FieldList('hairs'),
'Mensurations ': FieldMeasurements(),
'Origines ': FieldString('from'),
'Signes particuliers ': FieldParticularSignes(),
'Style ': FieldString('style'),
'Profession ': FieldString('job'),
'Alimentation': FieldString('alimentation'),
'Alcool': FieldString('alcool'),
'Tabac': FieldString('tabac'),
},
'liking': {'Hobbies ': FieldString('activities'),
'Musique ': FieldWideList('music'),
u'Cinéma': FieldWideList('cinema'),
'Livres ': FieldWideList('books'),
u'Télé': FieldWideList('tv'),
},
'sex': {u'Sous-v\xeatements ': FieldList('underwear'),
'... en haut ': FieldList('top'),
'... en bas ': FieldList('bottom'),
u'Fréquence idéale des rapports sexuels ':
FieldString('interval'),
'Pratiques sexuelles ': FieldList('practices'),
u'Accessoires préférés ':FieldList('toys'),
u'Position favorite ': FieldString('favorite'),
},
'personality': {u'Ça la fait craquer ': FieldString('snap'),
u'Ça l\'excite ': FieldString('exciting'),
u'Elle déteste ': FieldString('hate'),
'Ses vices ': FieldString('vices'),
'Ses atouts ': FieldString('assets'),
'Ses fantasmes ': FieldString('fantasies'),
'Elle est ': FieldList('is'),
},
}
ID_REGEXP = re.compile('(charm|addBasket|openAlbum)\(([0-9]+)(,[\s\'\d]+)?\)')
PHOTO_REGEXP = re.compile('http://(s|p)([0-9]+)\.adopteunmec\.com/(.*)')
STATS2ID = {'visites': 'visits',
'charmes': 'charms',
'paniers': 'baskets',
'mails': 'mails',
'POPULARIT': 'score',
}
STATS_VALUE_REGEXP = re.compile('([0-9\s]+).*')
def __repr__(self):
if isinstance(self.name, unicode):
name = self.name.encode('utf-8', 'backslashreplace')
else:
name = self.name
return '' % name
def on_loaded(self):
self.name = u''
self.description = u''
self.table = deepcopy(self.empty_table)
self.id = 0
self.photos = []
self.status = ''
self.stats = {'score': 0,
'visits': 0,
'charms': 0,
'baskets': 0,
'mails': 0,
}
divs = self.document.getElementsByTagName('td')
for div in divs:
if (div.hasAttribute('style') and
div.getAttribute('style') == "color:#ffffff;font-size:32px;font-weight:bold;letter-spacing:-2px" and
hasattr(div.firstChild, 'data')):
self.name = div.firstChild.data
if (div.hasAttribute('style') and
div.getAttribute('style') == "font-size:12px;font-weight:bold" and
hasattr(div.firstChild, 'data')):
self.status = div.firstChild.data
if div.hasAttribute('background'):
m = self.PHOTO_REGEXP.match(div.getAttribute('background'))
if m:
self.photos += [re.sub(u'thumb[0-2]_', u'image', div.getAttribute('background'))]
if div.hasAttribute('width') and str(div.getAttribute('width')) == '226':
trs = div.getElementsByTagName('tr')
for tr in trs:
tds = tr.getElementsByTagName('td')
if len(tds) > 2 and hasattr(tds[2].firstChild, 'data'):
label = tds[0].firstChild.data
value = tds[2].firstChild.data
elif len(tds) == 2:
label = unicode(tds[0].childNodes[1].data)
value = tds[1].childNodes[1].data
else:
continue
m = self.STATS_VALUE_REGEXP.match(value)
if m and self.STATS2ID.has_key(label):
self.stats[self.STATS2ID[label]] = int(m.group(1).replace(' ', ''))
divs = self.document.getElementsByTagName('div')
for div in divs:
if div.hasAttribute('id'):
if div.getAttribute('id') == 'about_div':
self.parse_description(div)
if div.getAttribute('id').startswith('tab_'):
self.parse_table(div)
for tag in ('img', 'td'):
imgs = self.document.getElementsByTagName(tag)
for img in imgs:
if img.hasAttribute('onclick'):
m = self.ID_REGEXP.match(img.getAttribute('onclick'))
if m:
self.id = int(m.group(2))
break
if self.id:
break
def parse_description(self, div):
# look for description
description = ''
for c in div.childNodes:
if hasattr(c, 'data'):
description += ''.join(c.data.split('\n')) # to strip \n
elif hasattr(c, 'tagName') and c.tagName == 'br':
description += '\n'
self.description = description
def parse_table(self, div):
d = self.table[self.tables[div.getAttribute('id')]]
fields = self.fields[self.tables[div.getAttribute('id')]]
table = div.getElementsByTagName('table')[1]
field1 = None
field2 = None
for tr in table.getElementsByTagName('tr'):
tds = tr.getElementsByTagName('td')
if len(tds) != 2:
continue
label1 = ''
label2 = ''
value1 = ''
value2 = ''
# Check for first column
if len(tds[0].childNodes) > 2:
b = tds[0].childNodes[2]
if hasattr(b, 'tagName') and b.tagName == 'b':
for child in b.childNodes:
label1 += child.data
else:
for child in tds[0].childNodes[2:]:
value1 += child.data
# Check for second column
if len(tds[1].childNodes) > 2:
b = tds[1].childNodes[2]
if hasattr(b, 'tagName') and b.tagName == 'b':
for child in b.firstChild.childNodes:
label2 += child.data
else:
for child in tds[1].childNodes[2:]:
if hasattr(child, 'data'):
value2 += child.data
if label1 and value2:
# This is a typically tuple of key/value on the line.
try:
fields[label1].put_value(d, value2)
except KeyError:
warning('Unable to find "%s" (%s)' % (label1, repr(label1)))
elif label1 and label2:
# two titles, so there will have a list of value in
# next lines on each columns
field1 = fields[label1]
field2 = fields[label2]
elif not label1 and not label1:
# two values, so it is a line of values
if field1 and value1:
field1.put_value(d, value1)
if field2 and value2:
field2.put_value(d, value2)
def get_name(self):
return self.name
def get_description(self):
return self.description
def get_table(self):
return self.table
def get_id(self):
return self.id
def get_photos(self):
return self.photos
def get_status(self):
return self.status
def is_online(self):
return self.status.find('en ligne') >= 0
def get_stats(self):
return self.stats
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/profileslist_base.py 0000664 0000000 0000000 00000003547 11375270370 0031075 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.base import PageBase
import re
class ProfilesListBase(PageBase):
PROFILE_REGEXP = re.compile(".*window\.location='(.*)'")
PHOTO_REGEXP = re.compile(".*background-image:url\(([A-Za-z0-9_\-:/\.]+)\).*")
WITHOUT_PHOTO = 'http://s.adopteunmec.com/img/thumb0.gif'
SHOW_WITHOUT_PHOTO = True
def on_loaded(self):
self.id_list = []
a_list = self.document.getElementsByTagName('div')
for a in a_list:
if a.hasAttribute('onclick') and a.hasAttribute('class') and a.getAttribute('class') in ('small', 'mini'):
m = self.PROFILE_REGEXP.match(a.getAttribute('onclick'))
if m:
url = m.group(1).split('/')[-1]
m = self.PHOTO_REGEXP.match(a.getElementsByTagName('div')[0].getAttribute('style'))
if url != 'home.php' and not url in self.id_list and \
m and (self.SHOW_WITHOUT_PHOTO or m.group(1) != self.WITHOUT_PHOTO):
self.id_list.append(url)
def get_profiles_ids(self):
return set(self.id_list)
def get_profiles_ids_list(self):
return self.id_list
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/search.py 0000664 0000000 0000000 00000002671 11375270370 0026626 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.profileslist_base import ProfilesListBase
class SearchPage(ProfilesListBase):
SHOW_WITHOUT_PHOTO = False
def search(self, **kwargs):
self.browser.select_form(name="form")
self.browser.set_all_readonly(False)
self.browser.set_field(kwargs, 'ageMin', is_list=True)
self.browser.set_field(kwargs, 'ageMax', is_list=True)
self.browser.set_field(kwargs, 'country', is_list=True)
self.browser.set_field(kwargs, 'dist', is_list=True)
self.browser.set_field(kwargs, 'nickname', field='pseudo')
self.browser.set_field(kwargs, 'save', value='true')
self.browser['originsV[]'] = ['1'] # excludes niggers (it doesn't work :( )
self.browser.submit()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/aum/pages/wait.py 0000664 0000000 0000000 00000002310 11375270370 0026313 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2008-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backends.aum.pages.base import PageBase
from weboob.backends.aum.exceptions import AdopteWait
from time import sleep
class WaitPage(PageBase):
def on_loaded(self):
raise AdopteWait()
def check(self):
result = self.browser.openurl('http://www.adopteunmec.com/fajax_checkEnter.php?anticache=0.46168455299380795').read()
return result == 'Ok'
def process_wait(self):
while not self.check(self):
sleep(10)
self.browser.location('/home.php')
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/ 0000775 0000000 0000000 00000000000 11375270370 0024403 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/__init__.py 0000664 0000000 0000000 00000001337 11375270370 0026520 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .backend import BNPorcBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/backend.py 0000664 0000000 0000000 00000003635 11375270370 0026353 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backend import BaseBackend
from weboob.capabilities.bank import ICapBank, AccountNotFound
from .browser import BNPorc
class BNPorcBackend(BaseBackend, ICapBank):
NAME = 'bnporc'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '1.0'
LICENSE = 'GPLv3'
DESCRIPTION = 'BNP Paribas french bank\' website'
CONFIG = {'login': BaseBackend.ConfigField(description='Account ID'),
'password': BaseBackend.ConfigField(description='Password of account', is_masked=True)
}
BROWSER = BNPorc
def default_browser(self):
return self.build_browser(self.config['login'], self.config['password'])
def iter_accounts(self):
for account in self.browser.get_accounts_list():
yield account
def get_account(self, _id):
try:
_id = long(_id)
except ValueError:
raise AccountNotFound()
else:
account = self.browser.get_account(_id)
if account:
return account
else:
raise AccountNotFound()
def iter_operations(self, account):
for coming in self.browser.get_coming_operations(account):
yield coming
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/browser.py 0000664 0000000 0000000 00000006316 11375270370 0026446 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BaseBrowser, BrowserIncorrectPassword
from weboob.backends.bnporc import pages
# Browser
class BNPorc(BaseBrowser):
DOMAIN = 'www.secure.bnpparibas.net'
PROTOCOL = 'https'
ENCODING = None # refer to the HTML encoding
PAGES = {'.*identifiant=DOSSIER_Releves_D_Operation.*': pages.AccountsList,
'.*identifiant=DSP_HISTOCPT.*': pages.AccountHistory,
'.*NS_AVEEC.*': pages.AccountComing,
'.*NS_AVEDP.*': pages.AccountPrelevement,
'.*Action=DSP_VGLOBALE.*': pages.LoginPage,
'.*type=homeconnex.*': pages.LoginPage,
'.*layout=HomeConnexion.*': pages.ConfirmPage,
}
is_logging = False
def __init__(self, *args, **kwargs):
kwargs['history'] = None # need history
BaseBrowser.__init__(self, *args, **kwargs)
def home(self):
self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/HomeConnexion?type=homeconnex')
def is_logged(self):
return not self.is_on_page(pages.LoginPage) or self.is_logging
def login(self):
assert isinstance(self.username, (str,unicode))
assert isinstance(self.password, (str,unicode))
assert self.password.isdigit()
if not self.is_on_page(pages.LoginPage):
self.location('https://www.secure.bnpparibas.net/banque/portail/particulier/HomeConnexion?type=homeconnex')
self.is_logging = True
self.page.login(self.username, self.password)
self.location('/NSFR?Action=DSP_VGLOBALE')
if self.is_on_page(pages.LoginPage):
raise BrowserIncorrectPassword()
self.is_logging = False
def get_accounts_list(self):
if not self.is_on_page(pages.AccountsList):
self.location('/NSFR?Action=DSP_VGLOBALE')
return self.page.get_list()
def get_account(self, id):
assert isinstance(id, (int, long))
if not self.is_on_page(pages.AccountsList):
self.location('/NSFR?Action=DSP_VGLOBALE')
l = self.page.get_list()
for a in l:
if a.id == id:
return a
return None
def get_coming_operations(self, account):
if not self.is_on_page(pages.AccountComing) or self.page.account.id != account.id:
self.location('/NS_AVEEC?ch4=%s' % account.link_id)
return self.page.get_operations()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/captcha.py 0000664 0000000 0000000 00000006654 11375270370 0026373 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import hashlib
import sys
import Image
class TileError(Exception):
def __init__(self, msg, tile = None):
Exception.__init__(self, msg)
self.tile = tile
class Captcha:
def __init__(self, file):
self.inim = Image.open(file)
self.nx,self.ny = self.inim.size
self.inmat = self.inim.load()
self.map = {}
self.tiles = [[Tile(x+5*y+1) for y in xrange(5)] for x in xrange(5)]
def __getitem__(self, (x, y)):
return self.inmat[x % self.nx, y % self.ny]
def all_coords(self):
for y in xrange(self.ny):
for x in xrange(self.nx):
yield x, y
def get_codes(self, code):
s = ''
for c in code:
s += '%02d' % self.map[int(c)].id
return s
def build_tiles(self):
y = 5
ty = 0
while y < self.ny:
x = 6
tx = 0
while x < self.nx:
if self[x,y] == 8:
tile = self.tiles[tx][ty]
tile.valid = True
yy = y
while not self[x,yy] in (3,7):
l = []
tile.map.append(l)
xx = x
while not self[xx,yy] in (3,7):
l.append(self[xx,yy])
xx += 1
yy += 1
self.map[tile.getNum()] = tile
x += 26
tx += 1
y += 25
ty += 1
class Tile:
hash = {'b2d25ae11efaaaec6dd6a4c00f0dfc29': 1,
'600873fa288e75ca6cca092ae95bf129': 2,
'da24ac28930feee169adcbc9bad4acaf': 3,
'76294dec2a3c6a7b8d9fcc7a116d1d4f': 4,
'd9531059e3834b6b8a97e29417a47dec': 5,
'8ba0c0cfe5e64d6b4afb1aa6f3612c1a': 6,
'19e0120231e7a9cf4544f96d8c388c8a': 7,
'83d8ad340156cb7f5c1e64454b66c773': 8,
'5ee8648d77eeb3e0979f6e59b2fbe66a': 9,
'3f3fb79bf61ebad096e05287119169df': 0
}
def __init__(self, _id):
self.id = _id
self.valid = False
self.map = []
def __repr__(self):
return "" % (self.id, self.valid)
def checksum(self):
s = ''
for pxls in self.map:
for pxl in pxls:
s += '%02d' % pxl
return hashlib.md5(s).hexdigest()
def getNum(self):
sum = self.checksum()
try:
return self.hash[sum]
except KeyError:
raise TileError('Tile not found', self)
def display(self):
for pxls in self.map:
for pxl in pxls:
sys.stdout.write('%02d' % pxl)
sys.stdout.write('\n')
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025502 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/pages/__init__.py 0000664 0000000 0000000 00000001624 11375270370 0027616 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .accounts_list import AccountsList
from .account_coming import AccountComing
from .login import LoginPage, ConfirmPage
class AccountHistory(AccountsList): pass
class AccountPrelevement(AccountsList): pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/pages/account_coming.py 0000664 0000000 0000000 00000003522 11375270370 0031046 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BasePage
from weboob.capabilities.bank import Operation
class AccountComing(BasePage):
def on_loaded(self):
self.operations = []
for tr in self.document.getiterator('tr'):
if tr.attrib.get('class', '') == 'hdoc1' or tr.attrib.get('class', '') == 'hdotc1':
tds = tr.findall('td')
if len(tds) != 3:
continue
date = tds[0].getchildren()[0].attrib.get('name', '')
label = u''
label += tds[1].text
for child in tds[1].getchildren():
if child.text: label += child.text
if child.tail: label += child.tail
if tds[1].tail: label += tds[1].tail
label = label.strip()
amount = tds[2].text.replace('.','').replace(',','.')
operation = Operation()
operation.setDate(date)
operation.setLabel(label)
operation.setAmount(float(amount))
self.operations.append(operation)
def get_operations(self):
return self.operations
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/pages/accounts_list.py 0000664 0000000 0000000 00000004475 11375270370 0030740 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
from weboob.capabilities.bank import Account
from weboob.tools.browser import BasePage
class AccountsList(BasePage):
LINKID_REGEXP = re.compile(".*ch4=(\w+).*")
def on_loaded(self):
pass
def get_list(self):
l = []
for tr in self.document.getiterator('tr'):
if tr.attrib.get('class', '') == 'comptes':
account = Account()
for td in tr.getiterator('td'):
if td.attrib.get('headers', '').startswith('Numero_'):
id = td.text
account.setID(long(''.join(id.split(' '))))
elif td.attrib.get('headers', '').startswith('Libelle_'):
a = td.findall('a')
label = unicode(a[0].text)
account.setLabel(label)
m = self.LINKID_REGEXP.match(a[0].attrib.get('href', ''))
if m:
account.setLinkID(m.group(1))
elif td.attrib.get('headers', '').startswith('Solde'):
a = td.findall('a')
balance = a[0].text
balance = balance.replace('.','').replace(',','.')
account.setBalance(float(balance))
elif td.attrib.get('headers', '').startswith('Avenir'):
a = td.findall('a')
coming = a[0].text
coming = coming.replace('.','').replace(',','.')
account.setComing(float(coming))
l.append(account)
return l
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/bnporc/pages/login.py 0000664 0000000 0000000 00000003167 11375270370 0027173 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import ClientForm
import sys
from weboob.tools.browser import BasePage
from weboob.backends.bnporc.captcha import Captcha, TileError
class LoginPage(BasePage):
def on_loaded(self):
pass
def login(self, login, password):
img = Captcha(self.browser.openurl('/NSImgGrille'))
self.browser.back()
try:
img.build_tiles()
except TileError, err:
print >>sys.stderr, "Error: %s" % err
if err.tile:
err.tile.display()
self.browser.select_form('logincanalnet')
# HACK because of fucking malformed HTML, the field isn't recognized by mechanize.
self.browser.controls.append(ClientForm.TextControl('text', 'ch1', {'value': ''}))
self.browser.set_all_readonly(False)
self.browser['ch1'] = login
self.browser['ch5'] = img.get_codes(password)
self.browser.submit()
class ConfirmPage(BasePage):
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/canaltp/ 0000775 0000000 0000000 00000000000 11375270370 0024542 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/canaltp/__init__.py 0000664 0000000 0000000 00000001340 11375270370 0026651 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .backend import CanalTPBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/canaltp/backend.py 0000664 0000000 0000000 00000003212 11375270370 0026501 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backend import BaseBackend
from weboob.capabilities.travel import ICapTravel, Station, Departure
from .browser import CanalTP
class CanalTPBackend(BaseBackend, ICapTravel):
NAME = 'canaltp'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '1.0'
LICENSE = 'GPLv3'
DESCRIPTION = "French trains"
BROWSER = CanalTP
def iter_station_search(self, pattern):
for _id, name in self.browser.iter_station_search(pattern):
yield Station(_id, name)
def iter_station_departures(self, station_id, arrival_id=None):
for i, d in enumerate(self.browser.iter_station_departures(station_id, arrival_id)):
departure = Departure(i, d['type'], d['time'])
departure.departure_station = d['departure']
departure.arrival_station = d['arrival']
departure.late = d['late']
departure.information = d['late_reason']
yield departure
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/canaltp/browser.py 0000664 0000000 0000000 00000005002 11375270370 0026574 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from datetime import datetime, date, time
from weboob.tools.browser import BaseBrowser
from weboob.tools.misc import toUnicode
class CanalTP(BaseBrowser):
DOMAIN = 'widget.canaltp.fr'
PROTOCOL = 'http'
PAGES = {}
def __init__(self):
BaseBrowser.__init__(self, '')
def iter_station_search(self, pattern):
result = self.openurl(u"http://widget.canaltp.fr/Prochains_departs_15122009/dev/gare.php?txtrech=%s" % unicode(pattern)).read()
for station in result.split('&'):
try:
_id, name = station.split('=')
except ValueError:
continue
else:
yield _id, toUnicode(name)
def iter_station_departures(self, station_id, arrival_id=None):
result = self.openurl(u"http://widget.canaltp.fr/Prochains_departs_15122009/dev/index.php?gare=%s" % unicode(station_id)).read()
result = result
departure = ''
for line in result.split('&'):
key, value = line.split('=', 1)
if key == 'nomgare':
departure = value
elif key.startswith('ligne'):
_type, unknown, _time, arrival, served, late, late_reason = value.split(';', 6)
yield {'type': toUnicode(_type),
'time': datetime.combine(date.today(), time(*[int(x) for x in _time.split(':')])),
'departure': toUnicode(departure),
'arrival': toUnicode(arrival).strip(),
'late': late and time(0, int(late.split()[0])) or time(),
'late_reason': toUnicode(late_reason).replace('\n', '').strip()}
def home(self):
pass
def login(self):
pass
def is_logged(self):
""" Do not need to be logged """
return True
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/ 0000775 0000000 0000000 00000000000 11375270370 0024216 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/__init__.py 0000664 0000000 0000000 00000001336 11375270370 0026332 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .backend import CragrBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/backend.py 0000664 0000000 0000000 00000004011 11375270370 0026153 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backend import BaseBackend
from weboob.capabilities.bank import ICapBank, AccountNotFound
from .browser import Cragr
class CragrBackend(BaseBackend, ICapBank):
NAME = 'cragr'
MAINTAINER = 'Laurent Bachelier'
EMAIL = 'laurent@bachelier.name'
VERSION = '1.0'
DESCRIPTION = 'Credit Agricole french bank\'s website'
LICENSE = 'GPLv3'
CONFIG = {'login': BaseBackend.ConfigField(description='Account ID'),
'password': BaseBackend.ConfigField(description='Password of account', is_masked=True),
'website': BaseBackend.ConfigField(description='What website to use', default='m.lefil.com'),
}
BROWSER = Cragr
def default_browser(self):
return self.build_browser(self.config['website'], self.config['login'], self.config['password'])
def iter_accounts(self):
for account in self.browser.get_accounts_list():
yield account
def get_account(self, _id):
try:
_id = long(_id)
except ValueError:
raise AccountNotFound()
else:
account = self.browser.get_account(_id)
if account:
return account
else:
raise AccountNotFound()
def iter_operations(self, account):
""" Not supported yet """
return iter([])
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/browser.py 0000664 0000000 0000000 00000005340 11375270370 0026255 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2009-2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BaseBrowser, BrowserIncorrectPassword
from weboob.backends.cragr import pages
# Browser
class Cragr(BaseBrowser):
PROTOCOL = 'https'
ENCODING = 'utf-8'
USER_AGENT = 'Wget/1.11.4'
is_logging = False
def __init__(self, website, *args, **kwargs):
self.DOMAIN = website
self.PAGES = {'https://%s/' % website: pages.LoginPage,
'https://%s/.*\.c.*' % website: pages.AccountsList,
'https://%s/login/process' % website: pages.AccountsList,
}
BaseBrowser.__init__(self, *args, **kwargs)
def viewing_html(self):
"""
As the fucking HTTP server returns a document in unknown mimetype
'application/vnd.wap.xhtml+xml' it is not recognized by mechanize.
So this is a fucking hack.
"""
return True
def home(self):
self.location('https://%s/' % self.DOMAIN)
def is_logged(self):
return self.page and self.page.is_logged() or self.is_logging
def login(self):
assert isinstance(self.username, (str,unicode))
assert isinstance(self.password, (str,unicode))
self.is_logging = True
if not self.is_on_page(pages.LoginPage):
self.home()
self.page.login(self.username, self.password)
self.is_logging = False
if not self.is_logged():
raise BrowserIncorrectPassword()
def get_accounts_list(self):
if not self.is_on_page(pages.AccountsList):
self.home()
return self.page.get_list()
def get_account(self, id):
assert isinstance(id, (int, long))
l = self.get_accounts_list()
for a in l:
if a.id == id:
return a
return None
#def get_coming_operations(self, account):
# if not self.is_on_page(pages.AccountComing) or self.page.account.id != account.id:
# self.location('/NS_AVEEC?ch4=%s' % account.link_id)
# return self.page.get_operations()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025315 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/pages/__init__.py 0000664 0000000 0000000 00000001401 11375270370 0027422 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .accounts_list import AccountsList
from .login import LoginPage
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/pages/accounts_list.py 0000664 0000000 0000000 00000003042 11375270370 0030540 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.capabilities.bank import Account
from .base import CragrBasePage
class AccountsList(CragrBasePage):
def on_loaded(self):
pass
def get_list(self):
l = []
for div in self.document.getiterator('div'):
if div.attrib.get('class', '') == 'dv' and div.getchildren()[0].tag == 'br':
account = Account()
account.setLabel(div.find('a').text.strip())
account.setID(long(div.findall('br')[1].tail.strip()))
s = div.find('div').find('span').find('b').text
balance = u''
for c in s:
if c.isdigit():
balance += c
if c == ',':
balance += '.'
account.setBalance(float(balance))
l.append(account)
return l
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/pages/base.py 0000664 0000000 0000000 00000001604 11375270370 0026602 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BasePage
class CragrBasePage(BasePage):
def is_logged(self):
for form in self.document.getiterator('form'):
return False
return True
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/cragr/pages/login.py 0000664 0000000 0000000 00000003132 11375270370 0026776 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import ClientForm
from .base import CragrBasePage
class LoginPage(CragrBasePage):
def on_loaded(self):
pass
def login(self, login, password):
self.browser.select_form(nr=0)
try:
self.browser['numero'] = login
self.browser['code'] = password
except ClientForm.ControlNotFoundError:
try:
self.browser['userLogin'] = login
self.browser['userPassword'] = password
except ClientForm.ControlNotFoundError:
self.browser.controls.append(ClientForm.TextControl('text', 'userLogin', {'value': ''}))
self.browser.controls.append(ClientForm.TextControl('text', 'userPassword', {'value': ''}))
self.browser.set_all_readonly(False)
self.browser['userLogin'] = login
self.browser['userPassword'] = password
self.browser.submit()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/ 0000775 0000000 0000000 00000000000 11375270370 0024045 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/__init__.py 0000664 0000000 0000000 00000001367 11375270370 0026165 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .browser import DLFP
from .backend import DLFPBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/backend.py 0000664 0000000 0000000 00000010757 11375270370 0026020 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backend import BaseBackend
from weboob.tools.browser import BrowserUnavailable
from weboob.capabilities.messages import ICapMessages, ICapMessagesReply, Message
from .feeds import ArticlesList
from .browser import DLFP
class DLFPBackend(BaseBackend, ICapMessages, ICapMessagesReply):
NAME = 'dlfp'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '1.0'
LICENSE = 'GPLv3'
DESCRIPTION = "Da Linux French Page"
CONFIG = {'username': BaseBackend.ConfigField(description='Username on website'),
'password': BaseBackend.ConfigField(description='Password of account', is_masked=True),
'get_news': BaseBackend.ConfigField(default=True, description='Get newspapers'),
'get_telegrams': BaseBackend.ConfigField(default=False, description='Get telegrams'),
}
STORAGE = {'seen': {}}
BROWSER = DLFP
def default_browser(self):
return self.build_browser(self.config['username'], self.config['password'])
def iter_messages(self, thread=None):
return self._iter_messages(thread, False)
def iter_new_messages(self, thread=None):
return self._iter_messages(thread, True)
def _iter_messages(self, thread, only_new):
if self.config['get_news']:
for message in self._iter_messages_of('newspaper', thread, only_new):
yield message
if self.config['get_telegrams']:
for message in self._iter_messages_of('telegram', thread, only_new):
yield message
def _iter_messages_of(self, what, thread_wanted, only_new):
if not what in self.storage.get('seen', default={}):
self.storage.set('seen', what, {})
seen = {}
for article in ArticlesList(what).iter_articles():
if thread_wanted and thread_wanted != article.id:
continue
if not article.id in self.storage.get('seen', what, default={}):
seen[article.id] = {'comments': []}
new = True
else:
seen[article.id] = self.storage.get('seen', what, article.id, default={})
new = False
try:
thread = self.browser.get_content(article.id)
except BrowserUnavailable:
continue
if not only_new or new:
yield Message(thread.id,
0,
thread.title,
thread.author,
article.datetime,
content=''.join([thread.body, thread.part2]),
signature='URL: %s' % article.url,
is_html=True)
for comment in thread.iter_all_comments():
if not comment.id in seen[article.id]['comments']:
seen[article.id]['comments'].append(comment.id)
new = True
else:
new = False
if not only_new or new:
yield Message(thread.id,
comment.id,
comment.title,
comment.author,
comment.date,
comment.reply_id,
comment.body,
'Score: %d' % comment.score,
is_html=True)
# If there is no articles seen, it's suspicious, probably I can't
# fetch the feed.
if seen:
self.storage.set('seen', what, seen)
self.storage.save()
def post_reply(self, thread_id, reply_id, title, message):
return self.browser.post_reply(thread_id, reply_id, title, message)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/browser.py 0000664 0000000 0000000 00000006006 11375270370 0026104 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import urllib
from cStringIO import StringIO
from weboob.tools.browser import BaseBrowser
from weboob.tools.parsers.lxmlparser import LxmlHtmlParser
from .pages.index import IndexPage, LoginPage
from .pages.news import ContentPage
from .tools import id2url, id2threadid, id2contenttype
class Parser(LxmlHtmlParser):
def parse(self, data, encoding=None):
# Want to kill templeet coders
data = StringIO(data.read().replace('<<', '<'))
return LxmlHtmlParser.parse(self, data, encoding)
# Browser
class DLFP(BaseBrowser):
DOMAIN = 'linuxfr.org'
PROTOCOL = 'https'
PAGES = {'https://linuxfr.org/': IndexPage,
'https://linuxfr.org/pub/': IndexPage,
'https://linuxfr.org/my/': IndexPage,
'https://linuxfr.org/login.html': LoginPage,
'https://linuxfr.org/.*/\d+.html': ContentPage
}
def __init__(self, *args, **kwargs):
kwargs['parser'] = Parser()
BaseBrowser.__init__(self, *args, **kwargs)
def home(self):
return self.location('https://linuxfr.org')
def get_content(self, _id):
self.location(id2url(_id))
return self.page.get_article()
def post_reply(self, thread, reply_id, title, message):
content_type = id2contenttype(thread)
thread_id = id2threadid(thread)
reply_id = int(reply_id)
if not content_type or not thread_id:
return False
# Define every data fields
data = {'news_id': thread_id,
'com_parent': reply_id,
'timestamp': '',
'res_type': content_type,
'referer': '%s://%s%s' % (self.PROTOCOL, self.DOMAIN, id2url(thread)),
'subject': title,
'body': message,
'format': 3,
'submit': 'Envoyer',
}
url = '%s://%s/submit/comments,%d,%d,%d.html#post' % (self.PROTOCOL, self.DOMAIN, thread_id, reply_id, content_type)
request = self.request_class(url, urllib.urlencode(data), {'Referer': url})
result = self.openurl(request).read()
# No message to send
return ()
def login(self):
self.location('/login.html', 'login=%s&passwd=%s&isauto=1' % (self.username, self.password))
def is_logged(self):
return (self.page and self.page.is_logged())
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/feeds.py 0000664 0000000 0000000 00000003413 11375270370 0025506 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import feedparser
from datetime import datetime
from .tools import url2id
class Article:
RSS = None
def __init__(self, _id, url, title, author, datetime):
self.id = _id
self.url = url
self.title = title
self.author = author
self.datetime = datetime
class Newspaper(Article):
RSS = 'https://linuxfr.org/backend/news/rss20.rss'
class Telegram(Article):
RSS = 'https://linuxfr.org/backend/journaux/rss20.rss'
class ArticlesList:
RSS = {'newspaper': Newspaper,
'telegram': Telegram
}
def __init__(self, section=None):
self.section = section
self.articles = []
def iter_articles(self):
for section, klass in self.RSS.iteritems():
if self.section and self.section != section:
continue
url = klass.RSS
feed = feedparser.parse(url)
for item in feed['items']:
article = klass(url2id(item['link']), item['link'], item['title'], item['author'], datetime(*item['date_parsed'][:7]))
yield article
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025144 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/pages/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0027243 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/pages/index.py 0000664 0000000 0000000 00000002524 11375270370 0026630 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BrowserIncorrectPassword, BasePage
class DLFPPage(BasePage):
def is_logged(self):
for form in self.document.getiterator('form'):
if form.attrib.get('id', None) == 'formulaire':
return False
return True
class IndexPage(DLFPPage):
pass
class LoginPage(DLFPPage):
def on_loaded(self):
if self.has_error():
raise BrowserIncorrectPassword()
def has_error(self):
for p in self.document.getiterator('p'):
if p.text and p.text.startswith(u'Vous avez rentré un mauvais mot de passe'):
return True
return False
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/pages/news.py 0000664 0000000 0000000 00000011023 11375270370 0026467 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from datetime import datetime
from logging import warning
from weboob.tools.misc import local2utc
from weboob.backends.dlfp.tools import url2id
from .index import DLFPPage
class Comment(object):
def __init__(self, browser, div, reply_id):
self.browser = browser
self.id = ''
self.reply_id = reply_id
self.title = u''
self.author = u''
self.date = u''
self.body = u''
self.score = 0
self.comments = []
for sub in div.getchildren():
if sub.tag == 'a':
self.id = sub.attrib['name']
elif sub.tag == 'h1':
try:
self.title = sub.find('b').text
except UnicodeError:
warning('Bad encoded title, but DLFP sucks')
elif sub.tag == 'div' and sub.attrib.get('class', '').startswith('comment'):
self.author = sub.find('a').text
self.date = self.parse_date(sub.find('i').tail)
self.score = int(sub.findall('i')[1].find('span').text)
self.body = self.browser.parser.tostring(sub.find('p'))
elif sub.attrib.get('class', '') == 'commentsul':
comment = Comment(self.browser, sub.find('li'), self.id)
self.comments.append(comment)
def parse_date(self, date_s):
return local2utc(datetime.strptime(unicode(date_s.strip()), u'le %d/%m/%Y à %H:%M.'))
def iter_all_comments(self):
for comment in self.comments:
yield comment
for c in comment.iter_all_comments():
yield c
def __repr__(self):
return u"" % (self.id, self.author, self.title)
class Article(object):
def __init__(self, browser, _id, tree):
self.browser = browser
self.id = _id
self.title = u''
self.author = u''
self.body = u''
self.part2 = u''
self.date = u''
self.comments = []
for div in tree.findall('div'):
if div.attrib.get('class', '').startswith('titlediv '):
self.author = div.find('a').text
for a in div.find('h1').getiterator('a'):
if a.text: self.title += a.text
if a.tail: self.title += a.tail
self.title = self.title.strip()
subdivs = div.findall('a')
if len(subdivs) > 1:
date_s = unicode(subdivs[1].text)
else:
date_s = unicode(div.find('i').tail)
#print date_s
if div.attrib.get('class', '').startswith('bodydiv '):
self.body = self.browser.parser.tostring(div)
def append_comment(self, comment):
self.comments.append(comment)
def iter_all_comments(self):
for comment in self.comments:
yield comment
for c in comment.iter_all_comments():
yield c
def parse_part2(self, div):
self.part2 = self.browser.parser.tostring(div)
class ContentPage(DLFPPage):
def on_loaded(self):
self.article = None
for div in self.document.find('body').find('div').findall('div'):
self.parse_div(div)
if div.attrib.get('class', '') == 'centraldiv':
for subdiv in div.findall('div'):
self.parse_div(subdiv)
def parse_div(self, div):
if div.attrib.get('class', '') in ('newsdiv', 'centraldiv'):
self.article = Article(self.browser, url2id(self.url), div)
if div.attrib.get('class', '') == 'articlediv':
self.article.parse_part2(div)
if div.attrib.get('class', '') == 'comments':
comment = Comment(self.browser, div, 0)
self.article.append_comment(comment)
def get_article(self):
return self.article
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/dlfp/tools.py 0000664 0000000 0000000 00000003561 11375270370 0025564 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
ID2URL_NEWSPAPER = re.compile('.*/(\d{4})/(\d{2})/(\d{2})/(\d+)\.html$')
ID2URL_TELEGRAM = re.compile('.*/~([A-Za-z0-9_]+)/(\d+)\.html$')
URL2ID_NEWSPAPER = re.compile('^N(\d{4})(\d{2})(\d{2}).(\d+)$')
URL2ID_TELEGRAM = re.compile('^T([A-Za-z0-9_]+).(\d+)$')
def url2id(url):
m = ID2URL_NEWSPAPER.match(url)
if m:
return 'N%04d%02d%02d.%d' % (int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4)))
m = ID2URL_TELEGRAM.match(url)
if m:
return 'T%s.%d' % (m.group(1), int(m.group(2)))
return None
def id2url(_id):
m = URL2ID_NEWSPAPER.match(_id)
if m:
return '/%04d/%02d/%02d/%d.html' % (int(m.group(1)), int(m.group(2)), int(m.group(3)), int(m.group(4)))
m = URL2ID_TELEGRAM.match(_id)
if m:
return '/~%s/%d.html' % (m.group(1), int(m.group(2)))
return None
def id2threadid(_id):
m = URL2ID_NEWSPAPER.match(_id)
if m:
return int(m.group(4))
m = URL2ID_TELEGRAM.match(_id)
if m:
return int(m.group(2))
return None
def id2contenttype(_id):
if not _id:
return None
if _id[0] == 'N':
return 1
if _id[0] == 'T':
return 5
return None
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/ 0000775 0000000 0000000 00000000000 11375270370 0024543 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/__init__.py 0000664 0000000 0000000 00000000044 11375270370 0026652 0 ustar 00root root 0000000 0000000 from .backend import GazelleBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/backend.py 0000664 0000000 0000000 00000004053 11375270370 0026506 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backend import BaseBackend
from weboob.capabilities.torrent import ICapTorrent
from .browser import GazelleBrowser
__all__ = ['GazelleBackend']
class GazelleBackend(BaseBackend, ICapTorrent):
NAME = 'gazelle'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '0.1'
DESCRIPTION = 'gazelle bittorrent tracker'
LICENSE = 'GPLv3'
CONFIG = {'username': BaseBackend.ConfigField(description='Username on website'),
'password': BaseBackend.ConfigField(description='Password of account', is_masked=True),
'protocol': BaseBackend.ConfigField(description='Protocol to use ("http" or "https")'),
'domain': BaseBackend.ConfigField(description='Domain (example "ssl.what.cd")'),
}
BROWSER = GazelleBrowser
def default_browser(self):
return self.build_browser(self.config['protocol'], self.config['domain'],
self.config['username'], self.config['password'])
def get_torrent(self, id):
return self.browser.get_torrent(id)
def get_torrent_file(self, id):
torrent = self.browser.get_torrent(id)
if not torrent:
return None
return self.browser.openurl(torrent.url).read()
def iter_torrents(self, pattern):
return self.browser.iter_torrents(pattern)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/browser.py 0000664 0000000 0000000 00000004325 11375270370 0026604 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BaseBrowser
from .pages.index import IndexPage, LoginPage
from .pages.torrents import TorrentsPage
__all__ = ['GazelleBrowser']
class GazelleBrowser(BaseBrowser):
DOMAIN = 'ssl.what.cd'
PROTOCOL = 'https'
PAGES = {'https?://%s/?(index.php)?': IndexPage,
'https?://%s/login.php': LoginPage,
'https?://%s/torrents.php.*': TorrentsPage,
}
def __init__(self, protocol, domain, *args, **kwargs):
self.DOMAIN = domain
self.PROTOCOL = protocol
self.PAGES = {}
for key, value in GazelleBrowser.PAGES.iteritems():
self.PAGES[key % domain] = value
BaseBrowser.__init__(self, *args, **kwargs)
def login(self):
if not self.is_on_page(LoginPage):
self.home()
self.page.login(self.username, self.password)
def is_logged(self):
if not self.page or self.is_on_page(LoginPage):
return False
if self.is_on_page(IndexPage):
return self.page.is_logged()
return True
def home(self):
return self.location('%s://%s/login.php' % (self.PROTOCOL, self.DOMAIN))
def iter_torrents(self, pattern):
self.location(self.buildurl('/torrents.php', searchstr=pattern))
assert self.is_on_page(TorrentsPage)
return self.page.iter_torrents()
def get_torrent(self, id):
self.location('/torrents.php?torrentid=%s' % id)
assert self.is_on_page(TorrentsPage)
return self.page.get_torrent(id)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025642 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/pages/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0027741 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/pages/index.py 0000664 0000000 0000000 00000002130 11375270370 0027317 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BasePage
__all__ = ['IndexPage', 'LoginPage']
class IndexPage(BasePage):
def is_logged(self):
return 'id' in self.document.find('body').attrib
class LoginPage(BasePage):
def login(self, login, password):
self.browser.select_form(nr=0)
self.browser['username'] = login
self.browser['password'] = password
self.browser.submit()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/gazelle/pages/torrents.py 0000664 0000000 0000000 00000014025 11375270370 0030076 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
from logging import warning
from weboob.tools.misc import html2text
from weboob.tools.browser import BasePage
from weboob.capabilities.torrent import Torrent
__all__ = ['TorrentsPage']
class TorrentsPage(BasePage):
TORRENTID_REGEXP = re.compile('torrents\.php\?action=download&id=(\d+)')
def unit(self, n, u):
m = {'KB': 1024,
'MB': 1024*1024,
'GB': 1024*1024*1024,
'TB': 1024*1024*1024*1024,
}
return float(n.replace(',', '')) * m.get(u, 1)
def format_url(self, url):
return '%s://%s/%s' % (self.browser.PROTOCOL,
self.browser.DOMAIN,
url)
def iter_torrents(self):
table = self.document.getroot().cssselect('table.torrent_table')
if not table:
table = self.document.getroot().cssselect('table#browse_torrent_table')
if table:
table = table[0]
current_group = None
for tr in table.findall('tr'):
if tr.attrib.get('class', '') == 'group':
tds = tr.findall('td')
current_group = u''
div = tds[-6]
if div.getchildren()[0].tag == 'div':
div = div.getchildren()[0]
for a in div.findall('a'):
if not a.text:
continue
if current_group:
current_group += ' - '
current_group += a.text
elif tr.attrib.get('class', '').startswith('group_torrent') or \
tr.attrib.get('class', '').startswith('torrent'):
tds = tr.findall('td')
title = current_group
if len(tds) == 7:
# Under a group
i = 0
elif len(tds) in (8,9):
# An alone torrent
i = len(tds) - 7
else:
# Useless title
continue
if title:
title += u' (%s)' % tds[i].find('a').text
else:
title = tds[i].find('a').text
url = tds[i].find('span').find('a').attrib['href']
id = self.TORRENTID_REGEXP.match(url)
if not id:
continue
id = id.group(1)
size = self.unit(*tds[i+3].text.split())
seeders = int(tds[i+5].text)
leechers = int(tds[i+6].text)
torrent = Torrent(id,
title,
url=self.format_url(url),
size=size,
seeders=seeders,
leechers=leechers)
yield torrent
else:
print tr.attrib
def get_torrent(self, id):
table = self.document.getroot().cssselect('div.thin')
if not table:
warning('No div.thin found')
return None
h2 = table[0].find('h2')
title = h2.text or ''
if h2.find('a') != None:
title += h2.find('a').text + h2.find('a').tail
torrent = Torrent(id, title)
table = self.document.getroot().cssselect('table.torrent_table')
if not table:
warning('No table found')
return None
for tr in table[0].findall('tr'):
if tr.attrib.get('class', '').startswith('group_torrent'):
tds = tr.findall('td')
if not len(tds) == 5:
continue
url = tds[0].find('span').find('a').attrib['href']
id = self.TORRENTID_REGEXP.match(url)
if not id:
warning('ID not found')
continue
id = id.group(1)
if id != torrent.id:
continue
torrent.url = self.format_url(url)
torrent.size = self.unit(*tds[1].text.split())
torrent.seeders = int(tds[3].text)
torrent.leechers = int(tds[4].text)
break
if not torrent.url:
warning('Torrent %d not found in list' % torrent.id)
return None
div = self.document.getroot().cssselect('div.main_column')
if not div:
warning('WTF')
return None
for box in div[0].cssselect('div.box'):
title = None
body = None
title_t = box.cssselect('div.head')
if title_t:
title = title_t[0].find('strong').text
body_t = box.cssselect('div.body')
if body_t:
body = html2text(self.browser.parser.tostring(body_t[0]))
if title and body:
torrent.description += '%s\n\n%s\n' % (title, body)
div = self.document.getroot().cssselect('div#files_%s' % torrent.id)
if div:
for tr in div[0].find('table'):
if tr.attrib.get('class', None) != 'colhead_dark':
torrent.files.append(tr.find('td').text)
return torrent
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/ 0000775 0000000 0000000 00000000000 11375270370 0025270 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/__init__.py 0000664 0000000 0000000 00000001363 11375270370 0027404 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Julien Hébert, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .backend import TransilienBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/backend.py 0000664 0000000 0000000 00000003470 11375270370 0027235 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Julien Hébert, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.backend import BaseBackend
from weboob.capabilities.travel import ICapTravel, Station, Departure
from .browser import Transilien
from .stations import STATIONS
class TransilienBackend(BaseBackend, ICapTravel):
NAME = 'transilien'
MAINTAINER = u'Julien Hébert'
EMAIL = 'juke@free.fr'
VERSION = '1.0'
LICENSE = 'GPLv3'
DESCRIPTION = "Transports in Paris"
BROWSER = Transilien
def iter_station_search(self, pattern):
pattern = pattern.lower()
for _id, name in STATIONS.iteritems():
if name.lower().find(pattern) >= 0:
yield Station(_id, name)
def iter_station_departures(self, station_id, arrival_id=None):
for i, d in enumerate(self.browser.iter_station_departures(station_id, arrival_id)):
departure = Departure(i, d['type'], d['time'])
departure.departure_station = d['departure']
departure.arrival_station = d['arrival']
departure.late = d['late']
departure.information = d['late_reason']
departure.plateform = d['plateform']
yield departure
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/browser.py 0000664 0000000 0000000 00000013310 11375270370 0027323 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Julien Hébert, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from datetime import datetime, date, time
import HTMLParser
from weboob.tools.browser import BaseBrowser
from weboob.tools.misc import toUnicode
from .pages.route import RoutePage
class Route(object):
"une ligne code_mission | time"
def __init__(self, code_mission, time, destination, platform):
self.code_mission = code_mission
self.time = time
self.destination = destination
self.platform = platform
def __repr__(self):
return "" % (self.code_mission,
self.time, self.destination, self.platform)
class Parser(HTMLParser.HTMLParser):
"Parse les tableaux html contenant les horaires"
def __init__(self):
HTMLParser.HTMLParser.__init__(self)
self.__table_horaires3 = False
self.__code_de_mission = False
self.__a_code_de_mission = False
self.__time = False
self.__destination = False
self.__platform = False
self.__liste_train = []
self.__liste_horaire = []
self.__liste_destination = []
self.__liste_platform = []
def parse(self, data, encoding):
self.feed(data.read())
return self
def handle_starttag(self, tag, attrs):
"execute a chaque balise ouvrante"
if (tag == 'table' and (dict(attrs)['class'] == 'horaires3')):
self.__table_horaires3 = True
elif self.__table_horaires3 and tag == 'td':
try:
self.__code_de_mission = (
dict(attrs)['headers'] == 'Code_de_mission')
self.__time = (
dict(attrs)['headers'] == 'Heure_de_passage')
self.__destination = (
dict(attrs)['headers'] == 'Destination')
self.__platform = (
dict(attrs)['headers'] == 'Voie')
except KeyError:
if dict(attrs).has_key('headers'):
raise
else:
pass
else:
self.__a_code_de_mission = (tag == 'a' and self.__code_de_mission)
def handle_data(self, data):
"execute pour chaque contenu de balise"
if self.__a_code_de_mission:
self.__liste_train.append(data.strip())
if self.__time and data.strip() != '*':
self.__liste_horaire.append(data.strip())
if self.__destination:
self.__liste_destination.append(data.strip())
if self.__platform:
self.__liste_platform.append(data.strip())
def handle_endtag(self, tag):
"execute à chaque balise fermante"
self.__a_code_de_mission ^= (self.__a_code_de_mission and tag == 'a')
self.__time ^= (self.__time and tag == 'td')
self.__destination ^= (self.__destination and tag == 'td')
self.__platform ^= (self.__platform and tag == 'td')
@property
def list_route(self):
"getter"
__list_route = []
__curseur_horaire = 0
for __i in self.__liste_train:
__list_route.append(Route(
code_mission=__i,
time=self.__liste_horaire[__curseur_horaire],
destination=self.__liste_destination[__curseur_horaire],
platform=self.__liste_platform[__curseur_horaire]
))
__curseur_horaire += 1
return __list_route
class Transilien(BaseBrowser):
DOMAIN = 'www.transilien.com'
PROTOCOL = 'http'
PAGES = {'http://www\.transilien\.com/web/ITProchainsTrainsAvecDest\.do\?.*': RoutePage,
'http://www\.transilien\.com/web/ITProchainsTrains\.do\?.*': RoutePage
}
def __init__(self):
BaseBrowser.__init__(self, '', parser=Parser())
def iter_station_search(self, pattern):
pass
def iter_station_departures(self, station_id, arrival_id=None):
if arrival_id:
self.location('http://www.transilien.com/web/ITProchainsTrainsAvecDest.do?codeTr3aDepart=%s&codeTr3aDest=%s&urlModule=/site/pid/184&gareAcc=true' % (station_id, arrival_id))
else:
self.location('http://www.transilien.com/web/ITProchainsTrains.do?tr3a=%s&urlModule=/site/pid/184' % station_id)
for route in self.page.document.list_route:
_late_reason = None
try :
_time = datetime.combine(date.today(), time(*[int(x) for x in route.time.split(':')]))
except ValueError:
_time = None
_late_reason = route.time
else:
yield {'type': toUnicode(route.code_mission),
'time': _time,
'departure': toUnicode(station_id),
'arrival': toUnicode(route.destination),
'late': time(),
'late_reason': _late_reason,
'plateform': toUnicode(route.platform)}
def home(self):
pass
def login(self):
pass
def is_logged(self):
""" Do not need to be logged """
return True
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/pages/ 0000775 0000000 0000000 00000000000 11375270370 0026367 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/pages/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0030466 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/pages/route.py 0000664 0000000 0000000 00000001472 11375270370 0030103 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Julien Hébert, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.tools.browser import BasePage
class RoutePage(BasePage):
def on_loaded(self):
return
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/pages/station.py 0000664 0000000 0000000 00000001275 11375270370 0030427 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/transilien/stations.py 0000664 0000000 0000000 00000024440 11375270370 0027512 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Julien Hébert, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
STATIONS = {
'BXN': u'BAGNEAUX SUR LOING',
'BJR': u'BOIS LE ROI',
'BOM': u'BOURRON MARLOTTE GREZ',
'BXI': u'BOUSSY ST ANTOINE',
'BNY': u'BRUNOY',
'CES': u'CESSON',
'CLY': u'CHANTILLY GOUVIEUX',
'CLX': u'CHATELET LES HALLES',
'CBV': u'COMBS LA VILLE QUINCY',
'COE': u'CORBEIL ESSONNES',
'CL': u'CREIL',
'DDI': u'DORDIVES',
'EVR': u'EVRY',
'EVC': u'EVRY COURCOURONNES',
'FFY': u'FERRIERES FONTENAY',
'FON': u'FONTAINEBLEAU AVON',
'GDS': u'GARE DU NORD',
'GAJ': u'GARGES SARCELLES',
'GOU': u'GOUSSAINVILLE',
'GBG': u'GRAND BOURG',
'GGG': u'GRIGNY CENTRE',
'JY': u'JUVISY',
'BBN': u'LA BORNE BLANCHE',
'BFX': u'LE BRAS DE FER',
'WEE': u'LE MEE',
'VD': u'LE VERT DE MAISONS',
'LNX': u'LES NOUES',
'LIU': u'LIEUSAINT MOISSY',
'LOV': u'LOUVRES',
'MFA': u'MAISONS ALFORT ALFORTVILLE',
'MEL': u'MELUN',
'MS': u'MONTARGIS',
'MTU': u'MONTEREAU',
'KRW': u'MONTGERON CROSNE',
'MKN': u'MONTIGNY SUR LOING',
'MOR': u'MORET VENEUX LES SABLONS',
'NSP': u'NEMOURS ST PIERRE',
'OBP': u'ORANGIS BOIS DE L EPINE',
'ORY': u'ORRY LA VILLE COYE LA FORET',
'PRF': u'PIERREFITTE STAINS',
'RIS': u'RIS ORANGIS',
'ZTN': u'SAVIGNY LE TEMPLE NANDY',
'SPP': u'SOUPPES CHÂTEAU LANDON',
'SDE': u'ST DENIS',
'SF': u'ST MAMMES',
'SFD': u'STADE DE FRANCE ST DENIS',
'SUR': u'SURVILLIERS FOSSES',
'TMR': u'THOMERY',
'VGS': u'VIGNEUX SUR SEINE',
'VP': u'VILLENEUVE PRAIRIE',
'VSG': u'VILLENEUVE ST GEORGES',
'VTV': u'VILLENEUVE TRIAGE',
'VIB': u'VILLIERS LE BEL GONESSE ARNOUVILLE',
'VWC': u'VIRY CHATILLON',
'YES': u'YERRES',
'ARW': u'ARGENTEUIL',
'AEH': u'AUBERGENVILLE ELISABETHVILLE',
'BEC': u'BECON LES BRUYERES',
'BCO': u'BOIS COLOMBES',
'CWJ': u'CHAVILLE RIVE DROITE',
'CLL': u'CLICHY LEVALLOIS',
'CBK': u'COLOMBES',
'CSH': u'CONFLANS SAINTE HONORINE',
'CPA': u'CORMEILLES EN PARISIS',
'KOU': u'COURBEVOIE',
'EPO': u'EPONE MEZIERES',
'ERA': u'ERAGNY NEUVILLE',
'ERE': u'ERMONT EAUBONNE',
'PSL': u'GARE ST LAZARE',
'HRY': u'HERBLAY',
'HAR': u'HOUILLES CARRIERES SUR SEINE',
'LDU': u'LA DEFENSE GARE SNCF',
'FMY': u'LA FRETTE MONTIGNY',
'LGK': u'LA GARENNE COLOMBES',
'LSD': u'LE STADE',
'VDO': u'LE VAL D OR',
'KVE': u'LES CLAIRIERES DE VERNEUIL',
'LMU': u'LES MUREAUX',
'LWA': u'LES VALLEES',
'MLF': u'MAISONS LAFFITTE',
'MTE': u'MANTES LA JOLIE',
'MTQ': u'MANTES STATION',
'MFL': u'MONTREUIL',
'NUN': u'NANTERRE UNIVERSITE',
'PSY': u'POISSY',
'PTC': u'PONT CARDINET',
'PSE': u'PONTOISE',
'PTX': u'PUTEAUX',
'SNN': u'SANNOIS',
'SVL': u'SARTROUVILLE',
'VDV': u'SEVRES VILLE D AVRAY',
'SCD': u'ST CLOUD',
'XOA': u'ST OUEN L AUMONE QUARTIER DE L EGLIS',
'MVH': u'SURESNES MONT VALERIEN',
'VDA': u'VAL D ARGENTEUIL',
'VET': u'VERNOUILLET VERNEUIL',
'VRD': u'VERSAILLES RIVE DROITE',
'VSW': u'VILLENNES SUR SEINE',
'VFD': u'VIROFLAY RIVE DROITE',
'ABL': u'ABLON',
'ARP': u'ARPAJON',
'ATH': u'ATHIS MONS',
'BFM': u'BIBLIOTHEQUE F. MITTERRAND',
'BIS': u'BIEVRES',
'BVI': u'BOULEVARD VICTOR',
'BY': u'BRETIGNY',
'BIH': u'BREUILLET BRUYERES LE CHATEL',
'BRW': u'BREUILLET VILLAGE',
'CPM': u'CHAMP DE MARS TOUR EIFFEL',
'CHV': u'CHAVILLE VELIZY',
'CAZ': u'CHILLY MAZARIN',
'CLR': u'CHOISY LE ROI',
'D': u'DOURDAN',
'ELY': u'EGLY',
'EYO': u'EPINAY SUR ORGE',
'GBI': u'GRAVIGNY BALIZY',
'IGY': u'IGNY',
'INV': u'INVALIDES',
'ISY': u'ISSY',
'ISP': u'ISSY VAL DE SEINE',
'IV': u'IVRY SUR SEINE',
'JVL': u'JAVEL',
'JAS': u'JOUY EN JOSAS',
'NG': u'LA NORVILLE ST GERMAIN LES ARPAJON',
'LAD': u'LES ARDOINES',
'LJU': u'LONGJUMEAU',
'MPU': u'MASSY PALAISEAU RER C',
'MFY': u'MEUDON VAL FLEURY',
'MDS': u'MUSEE D ORSAY',
'PJ': u'PETIT JOUY LES LOGES',
'PV': u'PETIT VAUX',
'PDM': u'PONT DE L ALMA',
'POA': u'PORCHEFONTAINE',
'SAO': u'SAVIGNY SUR ORGE',
'SXE': u'SERMAISE',
'SCW': u'ST CHERON',
'SHL': u'ST MICHEL NOTRE DAME',
'SHO': u'ST MICHEL SUR ORGE',
'SXG': u'STE GENEVIEVE DES BOIS',
'VBO': u'VAUBOYEN',
'VC': u'VERSAILLES CHANTIERS',
'VRG': u'VERSAILLES R G CHATEAU DE VERSAILLES',
'VRI': u'VILLENEUVE LE ROI',
'VFG': u'VIROFLAY RIVE GAUCHE',
'VY': u'VITRY SUR SEINE',
'BSO': u'BOURAY',
'CHK': u'CHAMARANDE',
'ETP': u'ETAMPES',
'ETY': u'ETRECHY',
'PZB': u'GARE D\'AUSTERLITZ',
'LYO': u'LARDY',
'MSX': u'MAROLLES EN HUREPOIX',
'SME': u'ST MARTIN D ETAMPES',
'CME': u'CHAMPAGNE SUR SEINE',
'CJR': u'CHARTRETTES',
'HER': u'HERICY',
'GPA': u'LA GRANDE PAROISSE',
'LYQ': u'LIVRY SUR SEINE',
'VSS': u'VERNOU SUR SEINE',
'VUN': u'VULAINES SUR SEINE SAMOREAU',
'PLY': u'GARE DE LYON',
'FPO': u'FONTAINE LE PORT',
'CJN': u'CHANGIS ST JEAN',
'CTH': u'CHATEAU THIERRY',
'CSG': u'CHELLES GOURNAY',
'CYZ': u'CHEZY SUR MARNE',
'CO': u'COULOMMIERS',
'EY': u'ESBLY',
'FMP': u'FAREMOUTIERS POMMEUSE',
'GCM': u'GUERARD LA CELLE SUR MORIN',
'LFJ': u'LA FERTE SOUS JOUARRE',
'LGY': u'LAGNY THORIGNY',
'MLB': u'MARLES EN BRIE',
'MEA': u'MEAUX',
'MOF': u'MORTCERF',
'MXK': u'MOUROUX',
'NAU': u'NANTEUIL SAACY',
'NAA': u'NOGENT L ARTAUD CHARLY',
'TOU': u'TOURNAN',
'TLP': u'TRILPORT',
'VAI': u'VAIRES TORCY',
'AEE': u'ASNIERES',
'AUU': u'AUBER',
'BQA': u'BRY SUR MARNE',
'BXG': u'BUSSY ST GEORGES',
'CGP': u'CHARLES DE GAULLE-ETOILE',
'GYN': u'GARE DE LYON',
'GAW': u'LA DEFENSE RER A',
'LQN': u'LOGNES',
'MVC': u'MARNE LA VALLEE CHESSY',
'NAF': u'NANTERRE PREFECTURE SNCF',
'NTN': u'NATION',
'NYP': u'NEUILLY PLAISANCE',
'NSL': u'NOISIEL',
'NYC': u'NOISY CHAMPS',
'NYG': u'NOISY LE GRAND MONT D EST',
'TOC': u'TORCY MARNE LA VALLEE',
'VDE': u'VAL D EUROPE',
'VFR': u'VAL DE FONTENAY RER A',
'VNC': u'VINCENNES',
'CJV': u'CERGY LE HAUT',
'CYP': u'CERGY PREFECTURE',
'CYC': u'CERGY ST CHRISTOPHE',
'CFD': u'CONFLANS FIN D OISE',
'NUE': u'NEUVILLE UNIVERSITE',
'RYR': u'AEROPORT CDG 2 TGV',
'ATW': u'ANTONY',
'ALC': u'AUBERVILLIERS LA COURNEUVE',
'AB': u'AULNAY SOUS BOIS',
'BAM': u'BLANC MESNIL',
'BQQ': u'BOURG LA REINE',
'BVJ': u'BURES SUR YVETTE',
'CUF': u'CITE UNIVERSITAIRE',
'CVW': u'COURCELLE SUR YVETTE',
'DFR': u'DENFERT ROCHEREAU',
'DRN': u'DRANCY',
'FMN': u'FONTAINE MICHALON',
'GIF': u'GIF SUR YVETTE',
'XBY': u'LA CROIX DE BERNY',
'HAQ': u'LA HACQUINIERE',
'LPN': u'LA PLAINE STADE DE FRANCE',
'LBT': u'LE BOURGET',
'GUW': u'LE GUICHET',
'BQC': u'LES BACONNETS',
'LZV': u'LOZERE',
'LXJ': u'LUXEMBOURG',
'MP': u'MASSY PALAISEAU RER B',
'MVP': u'MASSY VERRIERES RER B',
'ORS': u'ORSAY VILLE',
'PAX': u'PALAISEAU',
'PAW': u'PALAISEAU VILLEBON',
'PCX': u'PARC DE SCEAUX',
'PEX': u'PARC DES EXPOSITIONS',
'PWR': u'PORT ROYAL',
'BDE': u'SEVRAN BEAUDOTTES',
'XND': u'ST MICHEL NOTRE DAME',
'SNM': u'ST REMY LES CHEVREUSE',
'VPN': u'VILLEPINTE',
'RSY': u'AEROPORT CDG 1',
'ARK': u'ARCUEIL CACHAN',
'BGK': u'BAGNEUX',
'GTL': u'GENTILLY',
'LJA': u'LAPLACE',
'CVF': u'CHANTELOUP LES VIGNES',
'GGV': u'GARGENVILLE',
'IPO': u'ISSOU PORCHEVILLE',
'JUZ': u'JUZIERS',
'LIM': u'LIMAY',
'MHD': u'MEULAN HARDRICOURT',
'TPA': u'THUN LE PARADIS',
'TSS': u'TRIEL SUR SEINE',
'VXS': u'VAUX SUR SEINE',
'FNR': u'FONTENAY AUX ROSES',
'MY': u'MITRY CLAYE',
'RNS': u'ROBINSON',
'SKX': u'SCEAUX',
'SEV': u'SEVRAN LIVRY',
'VGL': u'VERT GALANT',
'VII': u'VILLEPARISIS MITRY LE NEUF',
'BOF': u'BOUFFEMONT MOISSELLES',
'DEU': u'DEUIL MONTMAGNY',
'DMO': u'DOMONT',
'ECZ': u'ECOUEN EZANVILLE',
'EPV': u'EPINAY VILLETANEUSE',
'PNB': u'GARE DU NORD',
'GRL': u'GROSLAY',
'LUZ': u'LUZARCHES',
'MSO': u'MONTSOULT MAFFLIERS',
'SLL': u'SARCELLES ST BRICE',
'SWY': u'SEUGY',
'VMS': u'VIARMES',
'VW': u'VILLAINES',
'CH': u'CHARTRES',
'CVI': u'CHAVILLE RIVE GAUCHE',
'CMA': u'CLAMART',
'CGW': u'COIGNIERES',
'DX': u'DREUX',
'EPN': u'EPERNON',
'FAF': u'FONTENAY LE FLEURY',
'GAQ': u'GARANCIERES LA QUEUE',
'GZA': u'GAZERAN',
'HOA': u'HOUDAN',
'JOY': u'JOUY',
'VYL': u'LA VERRIERE',
'LPE': u'LE PERRAY',
'LSI': u'LES ESSARTS LE ROI',
'MTN': u'MAINTENON',
'MBR': u'MARCHEZAIS BROUE',
'MDN': u'MEUDON',
'MLM': u'MONTFORT L AMAURY MERE',
'OGB': u'ORGERUS BEHOUST',
'PMP': u'PARIS MONTPARNASSE',
'PG': u'PLAISIR GRIGNON',
'PIE': u'PLAISIR LES CLAYES',
'RBT': u'RAMBOUILLET',
'SVR': u'SEVRES RIVE GAUCHE',
'SCR': u'ST CYR',
'SPI': u'ST PIAT',
'SQY': u'ST QUENTIN EN YVELINES',
'TAE': u'TACOIGNIERES RICHEBOURG',
'TVO': u'TRAPPES',
'VMK': u'VANVES MALAKOFF',
'VEP': u'VILLEPREUX LES CLAYES',
'VEH': u'VILLIERS NEAUPHLE PONTCHARTRAIN',
'CEG': u'CHAMP DE COURSES D ENGHIEN',
'CPO': u'CHAMPAGNE SUR OISE',
'EN': u'ENGHIEN LES BAINS',
'ERT': u'ERMONT EAUBONNE',
'ERM': u'ERMONT HALTE',
'FPN': u'FREPILLON',
'GNX': u'GROS NOYER ST PRIX',
'IAP': u'L ISLE ADAM PARMAIN',
'LBJ': u'LA BARRE ORMESSON',
'MLV': u'MERIEL',
'MWO': u'MERY SUR OISE',
'PEB': u'PERSAN BEAUMONT',
'SLF': u'ST LEU LA FORET',
'TVY': u'TAVERNY',
'VMD': u'VALMONDOIS',
'VCX': u'VAUCELLES',
'MJM': u'MAREIL SUR MAULDRE',
'MAE': u'MAULE',
'NZL': u'NEZEL AULNAY',
}
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/ 0000775 0000000 0000000 00000000000 11375270370 0024643 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/__init__.py 0000664 0000000 0000000 00000001365 11375270370 0026761 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Roger Philibert
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .backend import YoujizzBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/backend.py 0000664 0000000 0000000 00000003057 11375270370 0026611 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Roger Philibert
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.backend import BaseBackend
from weboob.capabilities.video import ICapVideoProvider
from .browser import YoujizzBrowser
from .video import YoujizzVideo
__all__ = ['YoujizzBackend']
class YoujizzBackend(BaseBackend, ICapVideoProvider):
NAME = 'youjizz'
MAINTAINER = 'Roger Philibert'
EMAIL = 'roger.philibert@gmail.com'
VERSION = '0.1'
DESCRIPTION = 'Youjizz videos website'
LICENSE = 'GPLv3'
CONFIG = {}
BROWSER = YoujizzBrowser
def get_video(self, _id):
return self.browser.get_video(_id)
def iter_page_urls(self, mozaic_url):
return self.browser.iter_page_urls(mozaic_url)
def iter_search_results(self, pattern=None, sortby=ICapVideoProvider.SEARCH_RELEVANCE, nsfw=False):
if not nsfw:
return iter(set())
return self.browser.iter_search_results(pattern)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/browser.py 0000664 0000000 0000000 00000005227 11375270370 0026706 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Roger Philibert
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import re
import urllib
from logging import warning
from weboob.tools.browser import BaseBrowser, BrowserUnavailable
from weboob.tools.browser.decorators import check_domain, id2url
from .pages.index import IndexPage
from .video import YoujizzVideo
__all__ = ['YoujizzBrowser']
class YoujizzBrowser(BaseBrowser):
DOMAIN = 'youjizz.com'
PROTOCOL = 'http'
PAGES = {r'http://.*youjizz\.com/?': IndexPage,
r'http://.*youjizz\.com/search/.+\.html': IndexPage,
}
@id2url(YoujizzVideo.id2url)
def get_video(self, url):
try:
data = self.openurl(url).read()
except BrowserUnavailable:
return None
def _get_url():
video_file_urls = re.findall(r'"(http://media[^ ,]+\.flv)"', data)
if len(video_file_urls) == 0:
return None
else:
if len(video_file_urls) > 1:
warning('Many video file URL found for given URL: %s' % video_file_urls)
return video_file_urls[0]
m = re.search(r'http://.*youjizz\.com/videos/(.+)\.html', url)
_id = unicode(m.group(1)) if m else None
m = re.search(r'(.+)', data)
title = unicode(m.group(1)) if m else None
m = re.search(r'.*Runtime.*(.+)', data)
if m:
minutes, seconds = (int(v) for v in unicode(m.group(1).strip()).split(':'))
duration = minutes * 60 + seconds
else:
duration = 0
return YoujizzVideo(_id=_id, title=title, url=_get_url(), duration=duration, nsfw=True)
@check_domain
def iter_page_urls(self, mozaic_url):
raise NotImplementedError()
def iter_search_results(self, pattern):
if not pattern:
self.home()
else:
self.location('/search/%s-1.html' % (urllib.quote_plus(pattern)))
assert self.is_on_page(IndexPage)
return self.page.iter_videos()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025742 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/pages/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0030041 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/pages/index.py 0000664 0000000 0000000 00000003715 11375270370 0027431 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Roger Philibert
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import re
from weboob.tools.browser import BasePage
from ..video import YoujizzVideo
__all__ = ['IndexPage']
class IndexPage(BasePage):
def iter_videos(self):
span_list = self.document.getroot().cssselect("span#miniatura")
if not span_list:
return
for span in span_list:
a = span.find('.//a')
if a is None:
continue
url = a.attrib['href']
_id = re.sub(r'/videos/(.+)\.html', r'\1', url)
thumbnail_url = span.find('.//img').attrib['src']
title1 = span.cssselect('span#title1')
if title1 is None:
title = None
else:
title = title1[0].text.strip()
duration = 0
thumbtime = span.cssselect('span.thumbtime')
if thumbtime is not None:
time_span = thumbtime[0].find('span')
minutes, seconds = time_span.text.strip().split(':')
duration = 60 * int(minutes) + int(seconds)
yield YoujizzVideo(_id,
title=title,
duration=duration,
thumbnail_url=thumbnail_url,
nsfw=True)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youjizz/video.py 0000664 0000000 0000000 00000002062 11375270370 0026323 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Roger Philibert
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.capabilities.video import BaseVideo
__all__ = ['YoujizzVideo']
class YoujizzVideo(BaseVideo):
def __init__(self, *args, **kwargs):
BaseVideo.__init__(self, *args, **kwargs)
self.id = u'%s@youjizz.com' % self.id
@classmethod
def id2url(cls, _id):
return 'http://www.youjizz.com/videos/%s.html' % _id
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/ 0000775 0000000 0000000 00000000000 11375270370 0024633 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/__init__.py 0000664 0000000 0000000 00000000044 11375270370 0026742 0 ustar 00root root 0000000 0000000 from .backend import YoupornBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/backend.py 0000664 0000000 0000000 00000003131 11375270370 0026572 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.backend import BaseBackend
from weboob.capabilities.video import ICapVideoProvider
from .browser import YoupornBrowser
__all__ = ['YoupornBackend']
class YoupornBackend(BaseBackend, ICapVideoProvider):
NAME = 'youporn'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '0.1'
DESCRIPTION = 'Youporn videos website'
LICENSE = 'GPLv3'
CONFIG = {}
BROWSER = YoupornBrowser
domain = u'youporn.com'
def get_video(self, _id):
return self.browser.get_video(_id)
SORTBY = ['relevance', 'rating', 'views', 'time']
def iter_search_results(self, pattern=None, sortby=ICapVideoProvider.SEARCH_RELEVANCE, nsfw=False):
if not nsfw:
return iter(set())
return self.browser.iter_search_results(pattern, self.SORTBY[sortby])
def iter_page_urls(self, mozaic_url):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/browser.py 0000664 0000000 0000000 00000003370 11375270370 0026673 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.tools.browser import BaseBrowser
from weboob.tools.browser.decorators import id2url
from .pages.index import IndexPage
from .pages.video import VideoPage
from .video import YoupornVideo
__all__ = ['YoupornBrowser']
class YoupornBrowser(BaseBrowser):
DOMAIN = 'youporn.com'
PROTOCOL = 'http'
PAGES = {'http://[w\.]*youporn\.com/?': IndexPage,
'http://[w\.]*youporn\.com/search.*': IndexPage,
'http://[w\.]*youporn\.com/watch/.+': VideoPage,
'http://[w\.]*youporngay\.com:80/watch/.+': VideoPage,
}
def __init__(self):
# Disallow arguments
BaseBrowser.__init__(self)
def iter_search_results(self, pattern, sortby):
if not pattern:
self.home()
else:
self.location(self.buildurl('/search/%s' % sortby, query=pattern))
assert self.is_on_page(IndexPage)
return self.page.iter_videos()
@id2url(YoupornVideo.id2url)
def get_video(self, url):
self.location(url)
return self.page.video
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025732 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/pages/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0030031 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/pages/base.py 0000664 0000000 0000000 00000002065 11375270370 0027221 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import ClientForm
from mechanize import FormNotFoundError
from weboob.tools.browser import BasePage
class PornPage(BasePage):
def on_loaded(self):
try:
self.browser.select_form(nr=0)
self.browser.submit(name='user_choice')
return False
except (ClientForm.ControlNotFoundError,FormNotFoundError):
return True
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/pages/index.py 0000664 0000000 0000000 00000004606 11375270370 0027421 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .base import PornPage
from ..video import YoupornVideo
__all__ = ['IndexPage']
class IndexPage(PornPage):
def iter_videos(self):
uls = self.document.getroot().cssselect("ul[class=clearfix]")
if not uls:
return
for ul in uls:
for li in ul.findall('li'):
a = li.find('a')
if a is None or a.find('img') is None:
continue
thumbnail_url = a.find('img').attrib['src']
h1 = li.find('h1')
a = h1.find('a')
if a is None:
continue
url = a.attrib['href']
_id = url[len('/watch/'):]
_id = _id[:_id.find('/')]
title = a.text.strip()
duration = 0
div = li.cssselect('div[class=duration_views]')
if div:
h2 = div[0].find('h2')
duration = 60 * int(h2.text.strip())
duration += int(h2.find('span').tail.strip())
rating = 0
rating_max = 0
div = li.cssselect('div[class=rating]')
if div:
p = div[0].find('p')
rating = float(p.text.strip())
rating_max = float(p.find('span').text.strip()[2:])
yield YoupornVideo(int(_id),
title=title,
rating=rating,
rating_max=rating_max,
duration=duration,
thumbnail_url=thumbnail_url,
nsfw=True)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/pages/video.py 0000664 0000000 0000000 00000006636 11375270370 0027425 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import re
import datetime
from logging import warning
from .base import PornPage
from ..video import YoupornVideo
class VideoPage(PornPage):
URL_REGEXP = re.compile("https?://[w\.]*youporn.com/watch/(\d+)/?.*")
def on_loaded(self):
if not PornPage.on_loaded(self):
return
self.video = YoupornVideo(self.get_id(),
self.get_title(),
self.get_url(),
nsfw=True)
self.set_details(self.video)
def get_id(self):
m = self.URL_REGEXP.match(self.url)
if m:
return int(m.group(1))
warning("Unable to parse ID")
return 0
def get_url(self):
el = self.document.getroot().cssselect('div[id=download]')
if el:
return el[0].cssselect('a')[0].attrib['href']
def get_title(self):
el = self.document.getroot().cssselect('h1')
if el:
return unicode(el[0].getchildren()[0].tail).strip()
DATE_REGEXP = re.compile("\w+ (\w+) (\d+) (\d+):(\d+):(\d+) (\d+)")
MONTH2I = ['', 'Jan', 'Feb', 'Mar', 'Apr', 'May', 'Jun', 'Jul', 'Aug', 'Sep', 'Oct', 'Nov', 'Dec']
def set_details(self, v):
div = self.document.getroot().cssselect('div[id=details]')
if not div:
return
for li in div[0].getiterator('li'):
span = li.find('span')
name = span.text.strip()
value = span.tail.strip()
if name == 'Duration:':
duration = 0
for word in value.split():
if word.endswith('min'):
duration += 60 * int(word[:word.find('min')])
elif word.endswith('sec'):
duration += int(word[:word.find('sec')])
v.duration = duration
elif name == 'Submitted:':
author = li.find('i')
if author is None:
author = li.find('a')
if author is None:
v.author = value
else:
v.author = author.text
elif name == 'Rating:':
r = value.split()
v.rating = float(r[0])
v.rating_max = float(r[2])
elif name == 'Date:':
m = self.DATE_REGEXP.match(value)
if m:
month = self.MONTH2I.index(m.group(1))
day = int(m.group(2))
hour = int(m.group(3))
minute = int(m.group(4))
second = int(m.group(5))
year = int(m.group(6))
v.date = datetime.datetime(year, month, day, hour, minute, second)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youporn/video.py 0000664 0000000 0000000 00000002165 11375270370 0026317 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Roger Philibert
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.capabilities.video import BaseVideo
__all__ = ['YoupornVideo']
class YoupornVideo(BaseVideo):
def __init__(self, *args, **kwargs):
BaseVideo.__init__(self, *args, **kwargs)
self.id = u'%s@youporn.com' % self.id
@classmethod
def id2url(cls, _id):
if _id.isdigit():
return 'http://www.youporn.com/watch/%d' % int(_id)
else:
return None
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/ 0000775 0000000 0000000 00000000000 11375270370 0024614 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/__init__.py 0000664 0000000 0000000 00000001365 11375270370 0026732 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .backend import YoutubeBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/backend.py 0000664 0000000 0000000 00000005221 11375270370 0026555 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz, Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import logging
from weboob.backend import BaseBackend
from weboob.capabilities.video import ICapVideoProvider
from .browser import YoutubeBrowser
from .video import YoutubeVideo
__all__ = ['YoutubeBackend']
class YoutubeBackend(BaseBackend, ICapVideoProvider):
NAME = 'youtube'
MAINTAINER = 'Christophe Benz'
EMAIL = 'christophe.benz@gmail.com'
VERSION = '0.1'
DESCRIPTION = 'Youtube videos website'
LICENSE = 'GPLv3'
CONFIG = {}
BROWSER = YoutubeBrowser
def get_video(self, _id):
return self.browser.get_video(_id)
def iter_search_results(self, pattern=None, sortby=ICapVideoProvider.SEARCH_RELEVANCE, nsfw=False):
try:
import gdata.youtube.service
except ImportError:
logging.error('Youtube backend search feature requires python-gdata package.')
return
yt_service = gdata.youtube.service.YouTubeService()
query = gdata.youtube.service.YouTubeVideoQuery()
query.orderby = ('relevance', 'rating', 'viewCount', 'published')[sortby]
query.racy = 'include' if nsfw else 'exclude'
if pattern:
query.categories.extend('/%s' % search_term.lower().encode('utf-8') for search_term in pattern.split())
feed = yt_service.YouTubeQuery(query)
for entry in feed.entry:
if entry.media.name:
author = entry.media.name.text.decode('utf-8').strip()
else:
author = None
yield YoutubeVideo(entry.id.text.split('/')[-1].decode('utf-8'),
title=entry.media.title.text.decode('utf-8').strip(),
author=author,
duration=int(entry.media.duration.seconds.decode('utf-8').strip()),
thumbnail_url=entry.media.thumbnail[0].url.decode('utf-8').strip())
def iter_page_urls(self, mozaic_url):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/browser.py 0000664 0000000 0000000 00000002252 11375270370 0026652 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz, Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.tools.browser import BaseBrowser
from weboob.tools.browser.decorators import check_domain, id2url
from .pages import VideoPage
from .video import YoutubeVideo
__all__ = ['YoutubeBrowser']
class YoutubeBrowser(BaseBrowser):
DOMAIN = u'youtube.com'
PAGES = {'.*youtube\.com/watch\?v=(.+)': VideoPage,
}
@id2url(YoutubeVideo.id2url)
def get_video(self, url):
self.location(url)
return self.page.video
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/pages/ 0000775 0000000 0000000 00000000000 11375270370 0025713 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/pages/__init__.py 0000664 0000000 0000000 00000001356 11375270370 0030031 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .video import VideoPage
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/pages/video.py 0000664 0000000 0000000 00000004314 11375270370 0027375 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz, Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import re
from logging import warning
from weboob.tools.browser import BasePage
from ..video import YoutubeVideo
__all__ = ['VideoPage']
class VideoPage(BasePage):
URL_REGEX = re.compile(r"https?://[w\.]*youtube.com/watch\?v=(.+)")
VIDEO_SIGNATURE_REGEX = re.compile(r'&t=([^ ,&]*)')
def on_loaded(self):
self.video = YoutubeVideo(self.get_id())
self.video.title = self.get_title()
self.video.url = self.get_url()
self.set_details(self.video)
def get_id(self):
m = self.URL_REGEX.match(self.url)
if m:
return m.group(1)
warning("Unable to parse ID")
return 0
def get_url(self):
video_signature = None
for data in self.document.getiterator('script'):
if not data.text:
continue
for m in re.finditer(self.VIDEO_SIGNATURE_REGEX, data.text):
video_signature = m.group(1)
return 'http://www.youtube.com/get_video?video_id=%s&t=%s&fmt=18' % (self.video.provider_id, video_signature)
def get_title(self):
found = self.document.getroot().cssselect('meta[name=title]')
if found:
content = found[0].attrib['content']
return unicode(content).strip()
return u''
def set_details(self, v):
div = self.document.getroot().cssselect('div[id=watch-description-body]')
if not div:
return
div = div[0]
v.author = div.find('a').find('strong').text
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/youtube/video.py 0000664 0000000 0000000 00000002056 11375270370 0026277 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.capabilities.video import BaseVideo
__all__ = ['YoutubeVideo']
class YoutubeVideo(BaseVideo):
def __init__(self, *args, **kwargs):
BaseVideo.__init__(self, *args, **kwargs)
self.id = u'%s@youtube.com' % self.id
@classmethod
def id2url(cls, _id):
return 'http://www.youtube.com/watch?v=%s' % _id
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/yweather/ 0000775 0000000 0000000 00000000000 11375270370 0024750 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/yweather/__init__.py 0000664 0000000 0000000 00000001341 11375270370 0027060 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .backend import YWeatherBackend
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/backends/yweather/backend.py 0000664 0000000 0000000 00000004330 11375270370 0026711 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import urllib2
from xml.dom import minidom
# TODO store datetime objects instead of strings
# from datetime import datetime
from weboob.backend import BaseBackend
from weboob.capabilities.weather import ICapWeather, CityNotFound, Current, Forecast
class YWeatherBackend(BaseBackend, ICapWeather):
NAME = 'yweather'
MAINTAINER = 'Romain Bignon'
EMAIL = 'romain@peerfuse.org'
VERSION = '1.0'
DESCRIPTION = 'Yahoo Weather'
LICENSE = 'GPLv3'
URL = 'http://weather.yahooapis.com/forecastrss?w=%s&u=%s'
def iter_city_search(self, pattern):
raise NotImplementedError()
def _get_dom(self, city_id):
handler = urllib2.urlopen(self.URL % (city_id, 'c'))
dom = minidom.parse(handler)
handler.close()
if not dom.getElementsByTagName('yweather:condition'):
raise CityNotFound()
return dom
def get_current(self, city_id):
dom = self._get_dom(city_id)
current = dom.getElementsByTagName('yweather:condition')[0]
return Current(current.getAttribute('date'), int(current.getAttribute('temp')), current.getAttribute('text'), 'C')
def iter_forecast(self, city_id):
dom = self._get_dom(city_id)
for forecast in dom.getElementsByTagName('yweather:forecast'):
yield Forecast(forecast.getAttribute('date'),
int(forecast.getAttribute('low')),
int(forecast.getAttribute('high')),
forecast.getAttribute('text'),
'C')
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/bcall.py 0000664 0000000 0000000 00000014607 11375270370 0023005 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from __future__ import with_statement
from copy import copy
import logging
from logging import debug
from threading import Thread, Event, RLock, Timer
from .tools.misc import get_backtrace
__all__ = ['BackendsCall', 'CallErrors']
class CallErrors(Exception):
def __init__(self, errors):
Exception.__init__(self, u'These errors have been raised in backend threads:\n%s' % (
u'\n'.join((u' * %s: %s%s' % (backend, error, backtrace + '\n'
if logging.root.level == logging.DEBUG else ''))
for backend, error, backtrace in self.errors)))
self.errors = copy(errors)
def __iter__(self):
return self.errors.__iter__()
class BackendsCall(object):
def __init__(self, backends, function, *args, **kwargs):
"""
@param backends list of backends to call
@param function backends' method name, or callable object
@param args, kwargs arguments given to called functions
"""
# Store if a backend is finished
self.backends = {}
for backend in backends:
self.backends[backend.name] = False
# Global mutex on object
self.mutex = RLock()
# Event set when every backends have give their data
self.finish_event = Event()
# Event set when there are new responses
self.response_event = Event()
# Waiting responses
self.responses = []
# Errors
self.errors = []
# Threads
self.threads = []
# Create jobs for each backend
with self.mutex:
for backend in backends:
debug('Creating a new thread for %s' % backend)
self.threads.append(Timer(0, self._caller, (backend, function, args, kwargs)).start())
if not backends:
self.finish_event.set()
def _store_error(self, backend, error):
with self.mutex:
backtrace = get_backtrace(error)
self.errors.append((backend, error, backtrace))
def _store_result(self, backend, result):
with self.mutex:
self.responses.append((backend, result))
self.response_event.set()
def _caller(self, backend, function, args, kwargs):
debug('%s: Thread created successfully' % backend)
with backend:
try:
# Call method on backend
try:
debug('%s: Calling function %s' % (backend, function))
if callable(function):
result = function(backend, *args, **kwargs)
else:
result = getattr(backend, function)(*args, **kwargs)
except Exception, error:
self._store_error(backend, error)
else:
debug('%s: Called function %s returned: "%s"' % (backend, function, result))
if hasattr(result, '__iter__'):
# Loop on iterator
try:
for subresult in result:
# Lock mutex only in loop in case the iterator is slow
# (for example if backend do some parsing operations)
self._store_result(backend, subresult)
except Exception, error:
self._store_error(backend, error)
else:
self._store_result(backend, result)
finally:
with self.mutex:
# This backend is now finished
self.backends[backend.name] = True
for finished in self.backends.itervalues():
if not finished:
return
self.finish_event.set()
self.response_event.set()
def _callback_thread_run(self, callback, errback):
responses = []
while not self.finish_event.isSet() or self.response_event.isSet():
self.response_event.wait()
with self.mutex:
responses = self.responses
self.responses = []
# Reset event
self.response_event.clear()
# Consume responses
while responses:
callback(*responses.pop(0))
if errback:
with self.mutex:
while self.errors:
errback(*self.errors.pop(0))
callback(None, None)
def callback_thread(self, callback, errback=None):
"""
Call this method to create a thread which will callback a
specified function everytimes a new result comes.
When the process is over, the function will be called with
both arguments set to None.
The functions prototypes:
def callback(backend, result)
def errback(backend, error)
"""
thread = Thread(target=self._callback_thread_run, args=(callback, errback))
thread.start()
return thread
def __iter__(self):
# Don't know how to factorize with _callback_thread_run
responses = []
while not self.finish_event.isSet() or self.response_event.isSet():
self.response_event.wait()
with self.mutex:
responses = self.responses
self.responses = []
# Reset event
self.response_event.clear()
# Consume responses
while responses:
yield responses.pop(0)
# Raise errors
with self.mutex:
if self.errors:
raise CallErrors(self.errors)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/ 0000775 0000000 0000000 00000000000 11375270370 0023777 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0026076 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/bank.py 0000664 0000000 0000000 00000004233 11375270370 0025266 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .cap import ICap
class AccountNotFound(Exception): pass
class Account(object):
def __init__(self):
self.id = 0
self.label = ''
self.balance = 0.0
self.coming = 0.0
self.link_id = ''
def setID(self, id):
assert isinstance(id, (int,long))
self.id = id
def setLabel(self, label): self.label = label
def setBalance(self, balance):
assert isinstance(balance, float)
self.balance = balance
def setComing(self, coming):
assert isinstance(coming, float)
self.coming = coming
def setLinkID(self, link):
self.link_id = link
def __repr__(self):
return u"" % (self.id, self.label)
class Operation(object):
def __init__(self):
self.date = None
self.label = u''
self.amount = 0.0
def __repr__(self):
return "" % (self.date, self.label, self.amount)
def setDate(self, date):
#assert isinstance(date, datetime.datetime)
self.date = date
def setLabel(self, label):
self.label = str(label)
def setAmount(self, amount):
assert isinstance(amount, float)
self.amount = amount
class ICapBank(ICap):
def iter_accounts(self):
raise NotImplementedError()
def get_account(self, _id):
raise NotImplementedError()
def iter_operations(self, account):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/cap.py 0000664 0000000 0000000 00000001331 11375270370 0025112 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
class ICap(object):
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/chat.py 0000664 0000000 0000000 00000003054 11375270370 0025272 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import datetime
from .cap import ICap
__all__ = ['ChatException', 'ICapChat']
class ChatException(Exception):
pass
class ChatContact(object):
def __init__(self, _id, pseudo, online, name=None, avatar_url=None, age=None):
self.id = _id
self.pseudo = pseudo
self.online = online
self.name = name
self.avatar_url = avatar_url
self.age = age
class ChatMessage(object):
def __init__(self, id_from, id_to, message, date=None):
self.id_from = id_from
self.id_to = id_to
self.message = message
self.date = datetime.datetime.utcnow() if date is None else date
class ICapChat(ICap):
def iter_chat_contacts(self, online=True, offline=True):
raise NotImplementedError()
def send_chat_message(self, _id, message):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/dating.py 0000664 0000000 0000000 00000003621 11375270370 0025621 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .cap import ICap
class Profile(object):
def get_profile_text(self):
body = u'Status: %s' % unicode(self.status)
if self.photos:
body += u'\nPhotos:'
for photo in self.photos:
body += u'\n\t\t%s' % unicode(photo)
body += u'\nStats:'
for label, value in self.get_stats().iteritems():
body += u'\n\t\t%-15s %s' % (label + ':', value)
body += u'\n\nInformations:'
for section, d in self.get_table().iteritems():
body += u'\n\t%s\n' % section
for key, value in d.items():
key = '%s:' % key
if isinstance(value, list):
body += u'\t\t%-15s %s\n' % (key, u', '.join([unicode(s) for s in value]))
elif isinstance(value, float):
body += u'\t\t%-15s %.2f\n' % (key, value)
else:
body += u'\t\t%-15s %s\n' % (key, unicode(value))
body += u'\n\nDescription:\n%s' % unicode(self.get_description())
return body
class ICapDating(ICap):
def get_profile(self, _id):
raise NotImplementedError()
def start_profile_walker(self):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/messages.py 0000664 0000000 0000000 00000006230 11375270370 0026161 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import datetime
import time
from .cap import ICap
class Message:
def __init__(self, thread_id, _id, title, sender, date=None, reply_id=u'', content=u'', signature=u'', is_html=False):
self.thread_id = unicode(thread_id)
self.id = unicode(_id)
self.reply_id = unicode(reply_id)
self.title = unicode(title)
self.sender = unicode(sender)
self.signature = unicode(signature)
self.new = False
self.content = content
if date is None:
date = datetime.datetime.utcnow()
self.date = date
self.is_html = is_html
def get_date_int(self):
return int(time.strftime('%Y%m%d%H%M%S', self.get_date().timetuple()))
def get_full_id(self):
return '%s.%s' % (self.thread_id, self.id)
def get_full_reply_id(self):
return '%s.%s' % (self.thread_id, self.reply_id)
def get_id(self):
return self.id
def get_thread_id(self):
return self.thread_id
def get_reply_id(self):
return self.reply_id
def get_title(self):
return self.title
def get_date(self):
return self.date
def get_from(self):
return self.sender
def get_content(self):
return self.content
def get_signature(self):
return self.signature
def is_new(self):
return self.new
def __eq__(self, msg):
return self.id == msg.id and self.thread_id == msg.thread_id
def __repr__(self):
result = '' % (
self.id, self.title, self.date, self.sender)
return result.encode('utf-8')
class ICapMessages(ICap):
def iter_new_messages(self, thread=None):
"""
Iterates on new messages from last time this function has been called.
@param thread thread name (optional)
@return [list] Message objects
"""
raise NotImplementedError()
def iter_messages(self, thread=None):
"""
Iterates on every messages
@param thread thread name (optional)
@return [list] Message objects
"""
raise NotImplementedError()
class ICapMessagesReply(ICap):
def post_reply(self, thread_id, reply_id, title, message):
"""
Post a reply.
@param thread_id ID of thread
@param reply_id message's id to reply
@param title title of message
@param message message to send
"""
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/torrent.py 0000664 0000000 0000000 00000002544 11375270370 0026053 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .cap import ICap
__all__ = ['ICapTorrent']
class Torrent(object):
def __init__(self, id, name, date=None, size=0.0, url=u'', seeders=0, leechers=0, files=[], description=u''):
self.id = id
self.name = name
self.date = date
self.size = size
self.url = url
self.seeders = seeders
self.leechers = leechers
self.files = files
self.description = description
class ICapTorrent(ICap):
def iter_torrents(self, pattern):
raise NotImplementedError()
def get_torrent(self, _id):
raise NotImplementedError()
def get_torrent_file(self, _id):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/travel.py 0000664 0000000 0000000 00000004557 11375270370 0025661 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon, Julien Hebert
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from datetime import time
from .cap import ICap
class ICapTravel(ICap):
def iter_station_search(self, pattern):
"""
Iterates on search results of stations.
@param pattern [str] the search pattern
@return [iter] the of Station objects
"""
raise NotImplementedError()
def iter_station_departures(self, station_id, arrival_id):
"""
Iterate on departures.
@param station_id [id] the station id
@param arrival_id [id] optionnal arrival station id
@return [iter] result of Departure objects
"""
raise NotImplementedError()
class Station(object):
def __init__(self, _id, name):
self.id = _id
self.name = name
def __repr__(self):
return "" % (self.id, self.name)
class Departure(object):
def __init__(self, _id, _type, _time):
self.id = _id
self.type = _type
self.time = _time
self.departure_station = u''
self.arrival_station = u''
self.late = time()
self.information = u''
self.plateform = u''
def __repr__(self):
return "" % (self.id,
self.type,
self.time.strftime('%H:%M'),
self.departure_station,
self.arrival_station)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/video.py 0000664 0000000 0000000 00000005114 11375270370 0025460 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .cap import ICap
__all__ = ['BaseVideo', 'ICapVideoProvider']
class BaseVideo(object):
def __init__(self, _id, title=None, url=None, author=None, duration=0, date=None,
rating=0.0, rating_max=0.0, thumbnail_url=None, nsfw=False):
self.id = _id
self.title = title
self.url = url
self.author = author
self.duration = int(duration)
self.date = date
self.rating = float(rating)
self.rating_max = float(rating_max)
self.thumbnail_url = thumbnail_url
self.nsfw = nsfw
@classmethod
def id2url(cls, _id):
"""Overloaded in child classes provided by backends."""
raise NotImplementedError()
@property
def formatted_duration(self):
return '%d:%02d:%02d' % (self.duration / 3600, (self.duration % 3600 / 60), self.duration % 60)
@property
def page_url(self):
return self.id2url(self.provider_id)
@property
def provider_id(self):
return self.id.split('@')[0]
class ICapVideoProvider(ICap):
def iter_page_urls(self, mozaic_url):
raise NotImplementedError()
(SEARCH_RELEVANCE,
SEARCH_RATING,
SEARCH_VIEWS,
SEARCH_DATE) = range(4)
def iter_search_results(self, pattern=None, sortby=SEARCH_RELEVANCE, nsfw=False):
"""
Iter results of a search on a pattern. Note that if pattern is None,
it get the latest videos.
@param pattern [str] pattern to search on
@param sortby [enum] sort by...
@param pattern [bool] include non-suitable for work videos if True
"""
raise NotImplementedError()
def get_video(self, _id):
"""
Get a Video from an ID.
@param _id the video id. It can be a numeric ID, or a page url, or so.
@return a Video object
"""
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/capabilities/weather.py 0000664 0000000 0000000 00000002737 11375270370 0026021 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .cap import ICap
class Forecast(object):
def __init__(self, date, low, high, text, unit):
self.date = date
self.low = low
self.high = high
self.text = text
self.unit = unit
class Current(object):
def __init__(self, date, temp, text, unit):
self.date = date
self.temp = temp
self.text = text
self.unit = unit
class City(object):
def __init__(self, city_id, name):
self.city_id = city_id
self.name = name
class CityNotFound(Exception):
pass
class ICapWeather(ICap):
def iter_city_search(self, pattern):
raise NotImplementedError()
def get_current(self, city_id):
raise NotImplementedError()
def iter_forecast(self, city_id):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/ 0000775 0000000 0000000 00000000000 11375270370 0023350 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0025447 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/boobank/ 0000775 0000000 0000000 00000000000 11375270370 0024763 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/boobank/__init__.py 0000664 0000000 0000000 00000001370 11375270370 0027075 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .boobank import Boobank
from .monfric import MonFric
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/boobank/boobank.py 0000664 0000000 0000000 00000005070 11375270370 0026752 0 ustar 00root root 0000000 0000000 #!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
# Copyright(C) 2009-2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from __future__ import with_statement
import logging
import weboob
from weboob.capabilities.bank import ICapBank, AccountNotFound
from weboob.tools.application import ConsoleApplication
__all__ = ['Boobank']
class Boobank(ConsoleApplication):
APPNAME = 'boobank'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
def main(self, argv):
self.load_backends(ICapBank)
return self.process_command(*argv[1:])
@ConsoleApplication.command('List every available accounts')
def command_list(self):
try:
for backend, account in self.weboob.do('iter_accounts'):
self.format(account)
except weboob.CallErrors, errors:
for backend, error, backtrace in errors:
if isinstance(error, weboob.tools.browser.BrowserIncorrectPassword):
logging.error(u'Error: Incorrect password for backend %s' % backend.name)
else:
logging.error(u'Error[%s]: %s\n%s' % (backend.name, error, backtrace))
@ConsoleApplication.command('Display all future operations')
def command_coming(self, id):
total = 0.0
def do(backend):
account = backend.get_account(id)
return backend.iter_operations(account)
try:
for backend, operation in self.weboob.do(do):
self.format(operation)
total += operation.amount
except weboob.CallErrors, errors:
for backend, error, backtrace in errors:
if isinstance(error, AccountNotFound):
logging.error(u'Error: account %s not found' % id)
else:
logging.error(u'Error[%s]: %s\n%s' % (backend.name, error, backtrace))
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/chatoob/ 0000775 0000000 0000000 00000000000 11375270370 0024767 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/chatoob/__init__.py 0000664 0000000 0000000 00000001356 11375270370 0027105 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .chatoob import Chatoob
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/chatoob/chatoob.py 0000664 0000000 0000000 00000004242 11375270370 0026762 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import logging
from weboob.tools.application import ConsoleApplication
from weboob.capabilities.chat import ICapChat
__all__ = ['Chatoob']
class Chatoob(ConsoleApplication):
APPNAME = 'chatoob'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Christophe Benz'
def main(self, argv):
self.load_backends(ICapChat)
#for backend, result in self.weboob.do('start_chat_polling', self.on_new_chat_message):
#logging.info(u'Polling chat messages for backend %s' % backend)
return self.process_command(*argv[1:])
def on_new_chat_message(self, message):
print 'on_new_chat_message: %s' % message
@ConsoleApplication.command('exit program')
def command_exit(self):
self.weboob.want_stop()
@ConsoleApplication.command('list online contacts')
def command_list(self):
for backend, contact in self.weboob.do('iter_chat_contacts', online=True, offline=False):
self.format(contact)
@ConsoleApplication.command('get messages')
def command_messages(self):
for backend, message in self.weboob.do('iter_chat_messages'):
self.format(message)
@ConsoleApplication.command('send message to contact')
def command_send(self, _id, message):
for backend, result in self.weboob.do('send_chat_message', _id, message):
if not result:
logging.error(u'Failed to send message to contact id="%s" on backend "%s"' % (_id, backend.name))
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/havesex/ 0000775 0000000 0000000 00000000000 11375270370 0025013 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/havesex/__init__.py 0000664 0000000 0000000 00000001331 11375270370 0027122 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .havesex import HaveSex
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/havesex/havesex.py 0000664 0000000 0000000 00000005077 11375270370 0027041 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from __future__ import with_statement
import logging
import sys
from weboob.tools.application import PromptApplication
from weboob.capabilities.dating import ICapDating
__all__ = ['HaveSex']
class HaveSex(PromptApplication):
APPNAME = 'havesex'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
STORAGE_FILENAME = 'dating.storage'
def main(self, argv):
self.load_config()
self.load_backends(ICapDating, storage=self.create_storage(self.STORAGE_FILENAME))
return self.loop()
def split_id(self, id):
try:
bname, id = id.split('.', 1)
except ValueError:
return None, None
return self.weboob.backends.get(bname, None), id
@PromptApplication.command("exit program")
def command_exit(self):
print 'Returning in real-life...'
self.weboob.want_stop()
@PromptApplication.command("show a profile")
def command_profile(self, id):
backend, _id = self.split_id(id)
if not backend:
print 'Invalid ID: %s' % id
return False
with backend:
profile = backend.get_profile(_id)
if not profile:
print 'Profile not found'
print profile.get_profile_text()
return True
def service(self, action, function):
sys.stdout.write('%s:' % action)
for backend, result in self.weboob.do(function):
sys.stdout.write(' ' + backend.name)
sys.stdout.flush()
sys.stdout.write('.\n')
@PromptApplication.command("start profiles walker")
def command_walker(self, action):
if action == 'start':
self.service('Starting walker', 'start_profiles_walker')
elif action == 'stop':
self.service('Stopping walker', 'stop_profiles_walker')
else:
logging.error(u'Syntax: walker (start|stop)')
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/masstransit/ 0000775 0000000 0000000 00000000000 11375270370 0025720 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/masstransit/__init__.py 0000664 0000000 0000000 00000001445 11375270370 0030035 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .masstransit import Masstransit
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/masstransit/masstransit.py 0000664 0000000 0000000 00000020472 11375270370 0030647 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Julien Hébert
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.capabilities.travel import ICapTravel
from weboob.tools.application import BaseApplication
import conic
import gtk
import hildon
from logging import debug
__all__ = ['Masstransit']
class MasstransitHildon():
"hildon interface"
def connect_event(self, connection, event=None, c=None, d=None):
debug("DBUS-DEBUG a: %s, b:%s, c:%s,d: %s" % (connection, event, c, d))
status = event.get_status()
if status == conic.STATUS_CONNECTED:
self.connected = True
if self.touch_selector_entry_filled == False:
debug("connected, now fill")
self.fill_touch_selector_entry()
if self.refresh_in_progress:
self.refresh()
elif status == conic.STATUS_DISCONNECTED:
self.connected = False
def __init__(self, weboob):
self.touch_selector_entry_filled = False
self.refresh_in_progress = False
self.connected = False
self.weboob = weboob
self.connection = conic.Connection()
self.connection.connect("connection-event", self.connect_event)
self.connection.set_property("automatic-connection-events", True)
self.connection.request_connection(conic.CONNECT_FLAG_NONE)
main_window = hildon.Window()
main_window.set_title("Horaires des Prochains Trains")
main_window.connect("destroy", self.on_main_window_destroy)
self.refresh_button = hildon.Button(
gtk.HILDON_SIZE_AUTO_WIDTH | gtk.HILDON_SIZE_FINGER_HEIGHT,
hildon.BUTTON_ARRANGEMENT_HORIZONTAL,
"Actualiser"
)
self.refresh_button.set_sensitive(False)
self.refresh_button.connect("clicked", self.on_refresh_button_clicked)
self.retour_button = hildon.Button(
gtk.HILDON_SIZE_AUTO_WIDTH | gtk.HILDON_SIZE_FINGER_HEIGHT,
hildon.BUTTON_ARRANGEMENT_HORIZONTAL,
"Retour"
)
self.retour_button.set_sensitive(False)
self.retour_button.connect("clicked", self.on_retour_button_clicked)
self.treestore = gtk.TreeStore(str, str, str, str, str)
treeview = gtk.TreeView(self.treestore)
treeview.append_column(
gtk.TreeViewColumn(
'Train',
gtk.CellRendererText(),
text=0
))
treeview.append_column(
gtk.TreeViewColumn(
'Horaire',
gtk.CellRendererText(),
text=1
))
treeview.append_column(
gtk.TreeViewColumn(
'Destination',
gtk.CellRendererText(),
text=2
))
treeview.append_column(
gtk.TreeViewColumn(
'Voie',
gtk.CellRendererText(),
text=3
))
treeview.append_column(
gtk.TreeViewColumn(
'Information',
gtk.CellRendererText(),
text=4
))
self.combo_source = hildon.TouchSelectorEntry(text=True)
self.combo_dest = hildon.TouchSelectorEntry(text=True)
self.picker_button_source = hildon.PickerButton(
gtk.HILDON_SIZE_AUTO,
hildon.BUTTON_ARRANGEMENT_VERTICAL)
self.picker_button_dest = hildon.PickerButton(
gtk.HILDON_SIZE_AUTO,
hildon.BUTTON_ARRANGEMENT_VERTICAL
)
self.picker_button_source.set_sensitive(False)
self.picker_button_dest.set_sensitive(False)
self.picker_button_source.set_title("Gare de Depart")
self.picker_button_dest.set_title("Gare d'arrivee")
self.picker_button_source.set_selector(self.combo_source)
self.picker_button_dest.set_selector(self.combo_dest)
vertical_box = gtk.VBox()
horizontal_box = gtk.HBox()
vertical_box.pack_start(horizontal_box)
horizontal_box.pack_start(self.picker_button_source)
horizontal_box.pack_start(self.picker_button_dest)
horizontal_box.pack_start(self.retour_button)
vertical_box.pack_start(treeview)
vertical_box.pack_start(self.refresh_button)
main_window.add(vertical_box)
main_window.show_all()
self.picker_button_source.connect("value-changed",
self.check_station_input,
self.picker_button_source)
self.picker_button_dest.connect("value-changed",
self.check_station_input,
self.picker_button_dest)
def fill_touch_selector_entry(self):
liste = []
for backend in self.weboob.iter_backends():
for station in backend.iter_station_search(""):
liste.append(station.name.capitalize())
liste.sort()
for station in liste:
self.combo_source.append_text(station)
self.combo_dest.append_text(station)
self.touch_selector_entry_filled = True
self.picker_button_source.set_sensitive(True)
def on_main_window_destroy(self, widget):
"exit application at the window close"
gtk.main_quit()
def on_main_window_show(self, param):
self.fill_touch_selector_entry()
def on_retour_button_clicked(self, widget):
"the button is clicked"
self.refresh_in_progress = True
col_source = self.combo_source.get_active(0)
col_dest = self.combo_dest.get_active(0)
self.combo_source.set_active(0, col_dest)
self.combo_dest.set_active(0, col_source)
self.refresh()
def on_refresh_button_clicked(self, widget):
"the refresh button is clicked"
self.refresh_in_progress = True
self.connection.request_connection(conic.CONNECT_FLAG_NONE)
def check_station_input(self, widget, user_data):
if self.combo_source.get_current_text() is None :
self.picker_button_dest.set_sensitive(False)
self.refresh_button.set_sensitive(False)
self.retour_button.set_sensitive(False)
else:
self.picker_button_dest.set_sensitive(True)
if self.combo_dest.get_current_text() is None:
self.refresh_button.set_sensitive(False)
self.retour_button.set_sensitive(False)
else:
self.refresh_button.set_sensitive(True)
self.retour_button.set_sensitive(True)
def refresh(self):
"update departures"
self.treestore.clear()
for backend in self.weboob.iter_backends():
for station in \
backend.iter_station_search(self.combo_source.get_current_text()):
for arrival in \
backend.iter_station_search(self.combo_dest.get_current_text()):
for departure in \
backend.iter_station_departures(station.id, arrival.id):
self.treestore.append(None,
[departure.type,
departure.time,
departure.arrival_station,
departure.plateform,
departure.information])
self.refresh_in_progress = False
class Masstransit(BaseApplication):
"Application Class"
APPNAME = 'masstransit'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Julien Hébert'
def main(self, argv):
"main fonction"
self.load_modules(ICapTravel)
MasstransitHildon(self.weboob)
gtk.main()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/monboob/ 0000775 0000000 0000000 00000000000 11375270370 0025003 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/monboob/__init__.py 0000664 0000000 0000000 00000001460 11375270370 0027115 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .monboob import Monboob
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/monboob/monboob.py 0000664 0000000 0000000 00000016276 11375270370 0027024 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2009-2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from email.mime.text import MIMEText
from smtplib import SMTP
from email.Header import Header, decode_header
from email.Utils import parseaddr, formataddr
from email import message_from_file
import time
import re
import sys
from weboob.capabilities.messages import ICapMessages, ICapMessagesReply, Message
from weboob.tools.application import ConsoleApplication
from weboob.tools.misc import html2text
__all__ = ['Monboob']
class Monboob(ConsoleApplication):
APPNAME = 'monboob'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
CONFIG = {'interval': 15,
'domain': 'weboob.example.org',
'recipient': 'weboob@example.org',
'smtp': 'localhost',
'html': 0}
def main(self, argv):
self.load_config()
self.load_backends(ICapMessages, storage=self.create_storage())
return self.process_command(*argv[1:])
@ConsoleApplication.command("pipe with a mail to post message")
def command_post(self):
msg = message_from_file(sys.stdin)
reply_to = msg.get('In-Reply-To')
if not reply_to:
print >>sys.stderr, 'This is not a reply (no Reply-To field)'
return 1
m = re.match('<(.*)@(.*)>', reply_to)
if m:
reply_to = m.group(1)
title = msg.get('Subject')
if title:
new_title = u''
for part in decode_header(title):
if part[1]:
new_title += unicode(part[0], part[1])
else:
new_title += unicode(part[0])
title = new_title
content = u''
for part in msg.walk():
if part.get_content_type() == 'text/plain':
s = part.get_payload(decode=True)
charsets = part.get_charsets() + msg.get_charsets()
for charset in charsets:
try:
content += unicode(s, charset)
except:
continue
else:
break
# remove signature
content = content.split(u'\n-- \n')[0]
bname, id = reply_to.split('.', 1)
try:
backend = self.weboob.backends[bname]
except KeyError:
print >>sys.stderr, 'Backend %s not found' % bname
return 1
if not backend.has_caps(ICapMessagesReply):
print >>sys.stderr, 'The backend %s does not implement ICapMessagesReply' % bname
return 1
thread_id, msg_id = id.rsplit('.', 1)
try:
backend.post_reply(thread_id, msg_id, title, content)
except Exception, e:
self.send_email(backend, Message(thread_id,
0,
title='Unable to send message',
sender='Monboob',
reply_id=msg_id,
content='Unable to send message to %s:\n\n\t%s' % (thread_id, e)))
@ConsoleApplication.command("run daemon")
def command_run(self):
self.weboob.repeat(int(self.config.get('interval')), self.process)
self.weboob.loop()
def process(self):
for backend, message in self.weboob.do('iter_new_messages'):
self.send_email(backend, message)
def send_email(self, backend, mail):
domain = self.config.get('domain')
recipient = self.config.get('recipient')
reply_id = ''
if mail.get_reply_id():
reply_id = u'<%s.%s@%s>' % (backend.name, mail.get_full_reply_id(), domain)
subject = mail.get_title()
sender = u'"%s" <%s@%s>' % (mail.get_from().replace('"', '""'), backend.name, domain)
# assume that get_date() returns an UTC datetime
date = time.strftime('%a, %d %b %Y %H:%M:%S +0000', mail.get_date().timetuple())
msg_id = u'<%s.%s@%s>' % (backend.name, mail.get_full_id(), domain)
if int(self.config.get('html')) and mail.is_html:
body = mail.get_content()
content_type = 'html'
else:
if mail.is_html:
body = html2text(mail.get_content())
else:
body = mail.get_content()
content_type = 'plain'
if mail.get_signature():
if int(self.config.get('html')) and mail.is_html:
body += u'
-- %s
' % mail.get_signature()
else:
body += u'\n\n-- \n'
if mail.is_html:
body += html2text(mail.get_signature())
else:
body += mail.get_signature()
# Header class is smart enough to try US-ASCII, then the charset we
# provide, then fall back to UTF-8.
header_charset = 'ISO-8859-1'
# We must choose the body charset manually
for body_charset in 'US-ASCII', 'ISO-8859-1', 'UTF-8':
try:
body.encode(body_charset)
except UnicodeError:
pass
else:
break
# Split real name (which is optional) and email address parts
sender_name, sender_addr = parseaddr(sender)
recipient_name, recipient_addr = parseaddr(recipient)
# We must always pass Unicode strings to Header, otherwise it will
# use RFC 2047 encoding even on plain ASCII strings.
sender_name = str(Header(unicode(sender_name), header_charset))
recipient_name = str(Header(unicode(recipient_name), header_charset))
# Make sure email addresses do not contain non-ASCII characters
sender_addr = sender_addr.encode('ascii')
recipient_addr = recipient_addr.encode('ascii')
# Create the message ('plain' stands for Content-Type: text/plain)
msg = MIMEText(body.encode(body_charset), content_type, body_charset)
msg['From'] = formataddr((sender_name, sender_addr))
msg['To'] = formataddr((recipient_name, recipient_addr))
msg['Subject'] = Header(unicode(subject), header_charset)
msg['Message-Id'] = msg_id
msg['Date'] = date
if reply_id:
msg['In-Reply-To'] = reply_id
# Send the message via SMTP to localhost:25
smtp = SMTP(self.config.get('smtp'))
print 'Send mail from <%s> to <%s>' % (sender, recipient)
smtp.sendmail(sender, recipient, msg.as_string())
smtp.quit()
return msg['Message-Id']
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ 0000775 0000000 0000000 00000000000 11375270370 0025160 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/Makefile 0000664 0000000 0000000 00000000061 11375270370 0026615 0 ustar 00root root 0000000 0000000 all:
$(MAKE) -C ui
clean:
$(MAKE) -C ui clean
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/__init__.py 0000664 0000000 0000000 00000000037 11375270370 0027271 0 ustar 00root root 0000000 0000000 from .qvideoob import QVideoob
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/main_window.py 0000664 0000000 0000000 00000011542 11375270370 0030050 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from PyQt4.QtCore import SIGNAL
from weboob.tools.application.qt import QtMainWindow
from weboob.frontends.qvideoob.ui.main_window_ui import Ui_MainWindow
from .video import Video
from .minivideo import MiniVideo
class MainWindow(QtMainWindow):
def __init__(self, config, weboob, parent=None):
QtMainWindow.__init__(self, parent)
self.ui = Ui_MainWindow()
self.ui.setupUi(self)
self.config = config
self.weboob = weboob
self.minivideos = []
self.ui.backendEdit.addItem('All backends', '')
for i, backend in enumerate(self.weboob.iter_backends()):
self.ui.backendEdit.addItem(backend.name, backend.name)
if backend.name == self.config.get('settings', 'backend'):
self.ui.backendEdit.setCurrentIndex(i+1)
self.ui.sortbyEdit.setCurrentIndex(int(self.config.get('settings', 'sortby')))
self.ui.nsfwCheckBox.setChecked(int(self.config.get('settings', 'nsfw')))
self.ui.sfwCheckBox.setChecked(int(self.config.get('settings', 'sfw')))
self.connect(self.ui.searchEdit, SIGNAL("returnPressed()"), self.search)
self.connect(self.ui.urlEdit, SIGNAL("returnPressed()"), self.openURL)
self.connect(self.ui.nsfwCheckBox, SIGNAL("stateChanged(int)"), self.nsfwChanged)
self.connect(self.ui.sfwCheckBox, SIGNAL("stateChanged(int)"), self.sfwChanged)
self.connect(self, SIGNAL('newData'), self.gotNewData)
def nsfwChanged(self, state):
self.config.set('settings', 'nsfw', int(self.ui.nsfwCheckBox.isChecked()))
self.updateVideosDisplay()
def sfwChanged(self, state):
self.config.set('settings', 'sfw', int(self.ui.sfwCheckBox.isChecked()))
self.updateVideosDisplay()
def updateVideosDisplay(self):
for minivideo in self.minivideos:
if (minivideo.video.nsfw and self.ui.nsfwCheckBox.isChecked() or
not minivideo.video.nsfw and self.ui.sfwCheckBox.isChecked()):
minivideo.show()
else:
minivideo.hide()
def search(self):
pattern = unicode(self.ui.searchEdit.text())
if not pattern:
return
for minivideo in self.minivideos:
self.ui.scrollAreaContent.layout().removeWidget(minivideo)
minivideo.hide()
self.minivideos = []
self.ui.searchEdit.setEnabled(False)
def cb(backend, video):
if backend and backend_name and backend.name != backend_name:
return
self.emit(SIGNAL('newData'), backend, video)
def eb(backend, err, backtrace):
print err
print backtrace
backend_name = str(self.ui.backendEdit.itemData(self.ui.backendEdit.currentIndex()).toString())
if backend_name:
process = self.weboob.do_backends(backend_name, 'iter_search_results', pattern, self.ui.sortbyEdit.currentIndex(), nsfw=True)
else:
process = self.weboob.do('iter_search_results', pattern, self.ui.sortbyEdit.currentIndex(), nsfw=True)
self.blah = process.callback_thread(cb, eb)
def gotNewData(self, backend, video):
if not backend:
self.ui.searchEdit.setEnabled(True)
return
minivideo = MiniVideo(backend, video)
self.ui.scrollAreaContent.layout().addWidget(minivideo)
self.minivideos.append(minivideo)
print backend
if (video.nsfw and not self.ui.nsfwCheckBox.isChecked() or
not video.nsfw and not self.ui.sfwCheckBox.isChecked()):
minivideo.hide()
def openURL(self):
url = unicode(self.ui.urlEdit.text())
if not url:
return
for backend in self.weboob.iter_backends():
video = backend.get_video(url)
if video:
video_widget = Video(video, self)
video_widget.show()
self.ui.urlEdit.clear()
def closeEvent(self, ev):
self.config.set('settings', 'backend', str(self.ui.backendEdit.itemData(self.ui.backendEdit.currentIndex()).toString()))
self.config.set('settings', 'sortby', self.ui.sortbyEdit.currentIndex())
self.config.save()
ev.accept()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/minivideo.py 0000664 0000000 0000000 00000004401 11375270370 0027514 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import urllib2
from PyQt4.QtGui import QFrame, QImage, QPixmap
from weboob.frontends.qvideoob.ui.minivideo_ui import Ui_MiniVideo
from .video import Video
class MiniVideo(QFrame):
def __init__(self, backend, video, parent=None):
QFrame.__init__(self, parent)
self.ui = Ui_MiniVideo()
self.ui.setupUi(self)
self.backend = backend
self.video = video
self.ui.titleLabel.setText(video.title)
self.ui.backendLabel.setText(backend.name)
self.ui.durationLabel.setText('%d:%02d:%02d' % (video.duration/3600, (video.duration%3600)/60, video.duration%60))
self.ui.authorLabel.setText(unicode(video.author))
self.ui.dateLabel.setText(video.date and unicode(video.date) or '')
if video.rating_max:
self.ui.ratingLabel.setText('%s / %s' % (video.rating, video.rating_max))
else:
self.ui.ratingLabel.setText('%s' % video.rating)
if video.thumbnail_url:
data = urllib2.urlopen(video.thumbnail_url).read()
img = QImage.fromData(data)
self.ui.imageLabel.setPixmap(QPixmap.fromImage(img))
def enterEvent(self, event):
self.setFrameShadow(self.Sunken)
QFrame.enterEvent(self, event)
def leaveEvent(self, event):
self.setFrameShadow(self.Raised)
QFrame.leaveEvent(self, event)
def mousePressEvent(self, event):
QFrame.mousePressEvent(self, event)
video = self.backend.get_video(self.video.id)
if video:
video_widget = Video(video, self)
video_widget.show()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/qvideoob.py 0000664 0000000 0000000 00000002575 11375270370 0027353 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.capabilities.video import ICapVideoProvider
from weboob.tools.application import QtApplication
from .main_window import MainWindow
class QVideoob(QtApplication):
APPNAME = 'qvideoob'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
CONFIG = {'settings': {'nsfw': True,
'sfw': True,
'sortby': 0,
'backend': ''
}
}
def main(self, argv):
self.load_modules(ICapVideoProvider)
self.load_config()
self.main_window = MainWindow(self.config, self.weboob)
self.main_window.show()
return self.weboob.loop()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ui/ 0000775 0000000 0000000 00000000000 11375270370 0025575 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ui/Makefile 0000664 0000000 0000000 00000000265 11375270370 0027240 0 ustar 00root root 0000000 0000000 UI_FILES = $(wildcard *.ui)
UI_PY_FILES = $(UI_FILES:%.ui=%_ui.py)
PYUIC = pyuic4
all: $(UI_PY_FILES)
%_ui.py: %.ui
$(PYUIC) -o $@ $^
clean:
rm -f *.pyc
rm -f $(UI_PY_FILES)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ui/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0027674 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ui/main_window.ui 0000664 0000000 0000000 00000011757 11375270370 0030462 0 ustar 00root root 0000000 0000000
MainWindow00785594QVideoobQFrame::StyledPanelQFrame::RaisedSearch: 00RelevanceRatingDurationDate10000Display:SFWtrueNSFWtrueQt::Horizontal4020true00763391QWidget#scrollAreaContent {
background-color: rgb(255, 255, 255);
}QFrame::StyledPanelQFrame::RaisedURL: 0078525
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ui/minivideo.ui 0000664 0000000 0000000 00000011337 11375270370 0030124 0 ustar 00root root 0000000 0000000
MiniVideo00464132FormQFrame::StyledPanelQFrame::Raised95500QFormLayout::ExpandingFieldsGrow75trueTitle0050truefalseTextLabel75trueDurationTextLabel75trueAuthorTextLabel75trueDateTextLabel75trueRatingTextLabel75trueWhereTextLabel
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/ui/video.ui 0000664 0000000 0000000 00000013711 11375270370 0027245 0 ustar 00root root 0000000 0000000
Video00647404Video1275trueQFrame::BoxQFrame::RaisedTextLabelQt::AlignCenter00background-color: rgb(255, 255, 255);QFrame::StyledPanelQFrame::Sunken00truetrueQFrame::StyledPanelQFrame::RaisedQFormLayout::ExpandingFieldsGrow75trueURLtrueArrowCursortruefalsetrue75trueDurationTextLabel75trueAuthorTextLabel75trueDateTextLabel75trueRatingTextLabelPhonon::VideoPlayerQWidgetphonon/videoplayer.hPhonon::SeekSliderQWidgetphonon/seekslider.h
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/qvideoob/video.py 0000664 0000000 0000000 00000003551 11375270370 0026644 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from PyQt4.QtCore import QUrl
from PyQt4.QtGui import QDialog
from PyQt4.phonon import Phonon
from weboob.frontends.qvideoob.ui.video_ui import Ui_Video
class Video(QDialog):
def __init__(self, video, parent=None):
QDialog.__init__(self, parent)
self.ui = Ui_Video()
self.ui.setupUi(self)
self.video = video
self.setWindowTitle("Video - %s" % video.title)
self.ui.urlEdit.setText(video.url)
self.ui.titleLabel.setText(video.title)
self.ui.durationLabel.setText('%d:%02d:%02d' % (video.duration/3600, (video.duration%3600)/60, video.duration%60))
self.ui.authorLabel.setText(unicode(video.author))
self.ui.dateLabel.setText(unicode(video.date))
if video.rating_max:
self.ui.ratingLabel.setText('%s / %s' % (video.rating, video.rating_max))
else:
self.ui.ratingLabel.setText('%s' % video.rating)
self.ui.seekSlider.setMediaObject(self.ui.videoPlayer.mediaObject())
self.ui.videoPlayer.load(Phonon.MediaSource(QUrl(video.url)))
self.ui.videoPlayer.play()
def closeEvent(self, event):
self.ui.videoPlayer.stop()
event.accept()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/travel/ 0000775 0000000 0000000 00000000000 11375270370 0024645 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/travel/__init__.py 0000664 0000000 0000000 00000001440 11375270370 0026755 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# vim: ft=python et softtabstop=4 cinoptions=4 shiftwidth=4 ts=4 ai
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .application import Travel
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/travel/application.py 0000664 0000000 0000000 00000003025 11375270370 0027522 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon, Julien Hébert
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.capabilities.travel import ICapTravel
from weboob.tools.application import ConsoleApplication
__all__ = ['Travel']
class Travel(ConsoleApplication):
APPNAME = 'travel'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
def main(self, argv):
self.load_modules(ICapTravel)
return self.process_command(*argv[1:])
@ConsoleApplication.command('Search stations')
def command_stations(self, pattern):
for backend, station in self.weboob.do('iter_station_search', pattern):
self.format(station)
@ConsoleApplication.command('List all departures on a special station')
def command_departures(self, station, arrival=None):
for backend, departure in self.weboob.do('iter_station_departures', station, arrival):
self.format(departure)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob/ 0000775 0000000 0000000 00000000000 11375270370 0024777 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob/__init__.py 0000664 0000000 0000000 00000001337 11375270370 0027114 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .application import Videoob
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob/application.py 0000664 0000000 0000000 00000003530 11375270370 0027655 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from weboob.capabilities.video import ICapVideoProvider
from weboob.tools.application import ConsoleApplication
__all__ = ['Videoob']
class Videoob(ConsoleApplication):
APPNAME = 'videoob'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Christophe Benz, Romain Bignon'
CONFIG = {}
def __init__(self):
ConsoleApplication.__init__(self)
self._parser.add_option('--nsfw', action='store_true', help='enable non-suitable for work videos')
def main(self, argv):
self.load_modules(ICapVideoProvider)
return self.process_command(*argv[1:])
@ConsoleApplication.command('Get video information (accept ID or URL)')
def command_info(self, id):
for backend, video in self.weboob.do('get_video', id):
if video is None:
continue
self.format(video)
@ConsoleApplication.command('Search videos')
def command_search(self, pattern=None):
print u'Search pattern: %s' % pattern if pattern else u'Last videos'
for backend, video in self.weboob.do('iter_search_results', pattern=pattern, nsfw=self.options.nsfw):
self.format(video)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/ 0000775 0000000 0000000 00000000000 11375270370 0025634 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/__init__.py 0000664 0000000 0000000 00000001342 11375270370 0027745 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .videoob_web import VideoobWeb
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/public/ 0000775 0000000 0000000 00000000000 11375270370 0027112 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/public/style.css 0000664 0000000 0000000 00000000076 11375270370 0030767 0 ustar 00root root 0000000 0000000 .video-item
{
margin-bottom: 5ex;
margin-left: 2em;
}
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/templates/ 0000775 0000000 0000000 00000000000 11375270370 0027632 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/templates/base.mako 0000664 0000000 0000000 00000000622 11375270370 0031415 0 ustar 00root root 0000000 0000000 ## -*- coding: utf-8 -*-
<%def name="title()" filter="trim">
Videoob Web
%def>
${self.title()}
${next.css()}
${next.body()}
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/templates/index.mako 0000664 0000000 0000000 00000001751 11375270370 0031616 0 ustar 00root root 0000000 0000000 ## -*- coding: utf-8 -*-
<%inherit file="base.mako"/>
<%def name="css()" filter="trim">
%def>
<%def name="video_item(item)">
% if merge:
% for item in results:
${video_item(item)}
% endfor
% else:
% for backend, items in sorted(results.iteritems()):
${backend}
% for item in items:
${video_item(item)}
% endfor
% endfor
% endif
%def>
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/videoob_web/videoob_web.py 0000664 0000000 0000000 00000010151 11375270370 0030470 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import os
from mako.lookup import TemplateLookup
from mako.runtime import Context
from routes import Mapper
from StringIO import StringIO
from webob.dec import wsgify
from webob import exc
from wsgiref.simple_server import make_server
from weboob.capabilities.video import ICapVideoProvider
from weboob.tools.application import BaseApplication
__all__ = ['VideoobWeb']
template_lookup = TemplateLookup(directories=[os.path.join(os.path.dirname(__file__), 'templates')],
output_encoding='utf-8', encoding_errors='replace')
class VideoobWeb(BaseApplication):
APPNAME = 'videoob-web'
CONFIG = dict(host='localhost', port=8080)
@wsgify
def make_app(self, req):
map = Mapper()
map.connect('index', '/', method='index')
results = map.routematch(environ=req.environ)
if results:
match, route = results
req.urlvars = ((), match)
kwargs = match.copy()
method = kwargs.pop('method')
return getattr(self, method)(req, **kwargs)
else:
public_path = os.path.join(os.path.dirname(__file__), 'public')
if not os.path.exists(public_path):
return exc.HTTPNotFound()
path = req.path
if path.startswith('/'):
path = path[1:]
public_file_path = os.path.join(public_path, path)
if os.path.exists(public_file_path):
if path.endswith('.css'):
req.response.content_type = 'text/css'
elif path.endswith('.js'):
req.response.content_type = 'text/javascript'
return open(public_file_path, 'r').read().strip()
else:
return exc.HTTPNotFound()
def main(self, argv):
self.load_config()
self.weboob.load_modules(ICapVideoProvider)
print 'Web server created. Listening on http://%s:%s' % (
self.config.get('host'), int(self.config.get('port')))
srv = make_server(self.config.get('host'), int(self.config.get('port')), self.make_app)
srv.serve_forever()
def index(self, req):
c = {}
nsfw = req.params.get('nsfw')
nsfw = False if not nsfw or nsfw == '0' else True
q = req.params.get('q', u'')
merge = req.params.get('merge')
merge = False if not merge or merge == '0' else True
c['merge'] = merge
c['form_data'] = dict(q=q)
c['results'] = [] if merge else {}
if q:
for backend in self.weboob.iter_backends():
videos = [dict(title=video.title,
page_url=video.page_url,
url=video.url if video.url else '/download?id=%s' % video.id,
thumbnail_url=video.thumbnail_url,
) \
for video in backend.iter_search_results(pattern=q, nsfw=nsfw)]
if videos:
if merge:
c['results'].extend(videos)
else:
c['results'][backend.name] = videos
if merge:
c['results'] = sorted(c['results'], key=lambda video: video['title'].lower())
template = template_lookup.get_template('index.mako')
buf = StringIO()
ctx = Context(buf, **c)
template.render_context(ctx)
return buf.getvalue().strip()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboobcfg/ 0000775 0000000 0000000 00000000000 11375270370 0025305 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboobcfg/__init__.py 0000664 0000000 0000000 00000001335 11375270370 0027420 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .weboobcfg import WeboobCfg
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboobcfg/weboobcfg.py 0000664 0000000 0000000 00000016245 11375270370 0027624 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import ConfigParser
import logging
import os
import subprocess
import weboob
from weboob.tools.application import ConsoleApplication
__all__ = ['WeboobCfg']
class WeboobCfg(ConsoleApplication):
APPNAME = 'weboobcfg'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
def main(self, argv):
return self.process_command(*argv[1:])
def caps_included(self, modcaps, caps):
modcaps = [x.__name__ for x in modcaps]
for cap in caps:
if not cap in modcaps:
return False
return True
@ConsoleApplication.command('List modules')
def command_modules(self, *caps):
print ' Name Capabilities Description '
print '+--------------+----------------------+----------------------------------------+'
self.weboob.modules_loader.load()
for name, module in self.weboob.modules_loader.modules.iteritems():
if caps and not self.caps_included(module.iter_caps(), caps):
continue
first_line = True
for cap in module.iter_caps():
if first_line:
print ' %-14s %-21s %s' % (name, cap.__name__, module.get_description())
first_line = False
else:
print ' %s' % cap.__name__
@ConsoleApplication.command('List applications')
def command_applications(self, *caps):
applications_path = os.path.abspath(os.path.join(os.path.dirname(weboob.__file__), '..', 'scripts'))
assert os.path.exists(applications_path)
print ' '.join(f for f in os.listdir(applications_path) if not f.startswith('.'))
@ConsoleApplication.command('Display a module')
def command_modinfo(self, name):
try:
module = self.weboob.modules_loader.get_or_load_module(name)
except KeyError:
logging.error('No such module: %s' % name)
return 1
print '.------------------------------------------------------------------------------.'
print '| Module %-69s |' % module.get_name()
print "+-----------------.------------------------------------------------------------'"
print '| Version | %s' % module.get_version()
print '| Maintainer | %s' % module.get_maintainer()
print '| License | %s' % module.get_license()
print '| Description | %s' % module.get_description()
print '| Capabilities | %s' % ', '.join([cap.__name__ for cap in module.iter_caps()])
first = True
for key, field in module.get_config().iteritems():
value = field.description
if not field.default is None:
value += ' (default: %s)' % field.default
if first:
print '| | '
print '| Configuration | %s: %s' % (key, value)
first = False
else:
print '| | %s: %s' % (key, value)
print "'-----------------' "
@ConsoleApplication.command('Add a configured backend')
def command_add(self, name, *options):
self.weboob.modules_loader.load()
if name not in [module_name for module_name, module in self.weboob.modules_loader.modules.iteritems()]:
logging.error(u'Backend "%s" does not exist.' % name)
return 1
params = {}
# set backend params from command-line arguments
for option in options:
try:
key, value = option.split('=', 1)
except ValueError:
logging.error(u'Parameters have to be formatted "key=value"')
return 1
params[key] = value
# ask for params non-specified on command-line arguments
module = self.weboob.modules_loader.get_or_load_module(name)
for key, value in module.get_config().iteritems():
if key not in params:
masked = key == 'password'
params[key] = self.ask(key, default=u'', masked=masked)
try:
self.weboob.backends_config.add_backend(name, name, params)
print u'Backend "%s" successfully added to file "%s".\n'\
'Please check configuration parameters values with "weboobcfg edit".' % (
name, self.weboob.backends_config.confpath)
except ConfigParser.DuplicateSectionError:
print u'Backend "%s" is already configured in file "%s"' % (name, self.weboob.backends_config.confpath)
response = raw_input(u'Add new instance of "%s" backend ? [yN] ' % name)
if response.lower() == 'y':
while True:
new_name = raw_input(u'Please give new instance name (could be "%s_1"): ' % name)
if not new_name:
continue
try:
self.weboob.backends_config.add_backend(new_name, name, params)
print u'Backend "%s" successfully added to file "%s".\n'\
'Please check configuration parameters values with "weboobcfg edit".' % (
name, self.weboob.backends_config.confpath)
break
except ConfigParser.DuplicateSectionError:
print u'Instance "%s" is already configured for backend "%s".' % (new_name, name)
@ConsoleApplication.command('List backends')
def command_list(self):
print ' Instance Name Name Params '
print '+---------------+--------------+------------------------------------------------+'
for instance_name, name, params in self.weboob.backends_config.iter_backends():
print ' %-15s %-14s %-47s' % (instance_name, name, ', '.join('%s=%s' % (key, value) for key, value in params.iteritems()))
@ConsoleApplication.command('Remove a backend')
def command_remove(self, instance_name):
try:
self.weboob.backends_config.remove_backend(instance_name)
except ConfigParser.NoSectionError:
logging.error("Backend '%s' does not exist" % instance_name)
return 1
@ConsoleApplication.command('Edit configuration file')
def command_edit(self):
subprocess.call([os.environ.get('EDITOR', 'vi'), self.weboob.backends_config.confpath])
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboobdebug/ 0000775 0000000 0000000 00000000000 11375270370 0025634 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboobdebug/__init__.py 0000664 0000000 0000000 00000001363 11375270370 0027750 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .weboobdebug import WeboobDebug
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboobdebug/weboobdebug.py 0000664 0000000 0000000 00000003310 11375270370 0030467 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import logging
from weboob.tools.application import ConsoleApplication
class WeboobDebug(ConsoleApplication):
APPNAME = 'weboobdebug'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Christophe Benz'
def main(self, argv):
return self.process_command(*argv[1:])
@ConsoleApplication.command('Debug backend')
def command_shell(self, backend_name):
try:
backend = self.weboob.load_modules(names=[backend_name])[backend_name]
except KeyError:
logging.error(u'Unable to load backend "%s"' % backend_name)
return 1
browser = backend.browser
from IPython.Shell import IPShellEmbed
shell = IPShellEmbed(argv=[])
locs = dict(backend=backend, browser=browser, frontend=self, weboob=self.weboob)
banner = 'Weboob debug shell\nBackend "%s" loaded.\nAvailable variables: %s' % (backend_name, locs)
shell.set_banner(shell.IP.BANNER + '\n\n' + banner)
shell(local_ns=locs, global_ns={})
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboorrents/ 0000775 0000000 0000000 00000000000 11375270370 0025721 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboorrents/__init__.py 0000664 0000000 0000000 00000001341 11375270370 0030031 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .weboorrents import Weboorrents
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/weboorrents/weboorrents.py 0000664 0000000 0000000 00000005310 11375270370 0030643 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from __future__ import with_statement
import logging
from weboob.capabilities.torrent import ICapTorrent
from weboob.tools.application import ConsoleApplication
__all__ = ['Weboorrents']
class Weboorrents(ConsoleApplication):
APPNAME = 'weboorrents'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
CONFIG = {}
def main(self, argv):
self.load_backends(ICapTorrent)
return self.process_command(*argv[1:])
def split_id(self, id):
if not '.' in id:
logging.error('ID must be in form .')
return None, None
backend_name, id = id.split('.', 1)
backend = self.weboob.backends.get(backend_name, None)
if not backend:
logging.error('Backends "%s" not found' % backend_name)
return None, None
return backend, id
@ConsoleApplication.command('Get information about a torrent')
def command_info(self, id):
backend, id = self.split_id(id)
if not backend:
return 1
with backend:
torrent = backend.get_torrent(id)
if not torrent:
logging.error('Torrent "%s" not found' % id)
return 1
self.format(torrent)
@ConsoleApplication.command('Get the torrent file')
def command_getfile(self, id, dest):
backend, id = self.split_id(id)
if not backend:
return 1
with backend:
s = backend.get_torrent_file(id)
if not s:
logging.error('Torrent "%s" not found' % id)
return 1
if dest == '-':
print s
else:
with open(dest, 'w') as f:
f.write(s)
@ConsoleApplication.command('Search torrents')
def command_search(self, pattern=None):
print u'Search pattern: %s' % pattern if pattern else u'Last torrents'
for backend, torrent in self.weboob.do('iter_torrents', pattern=pattern):
self.format(torrent)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/wetboobs/ 0000775 0000000 0000000 00000000000 11375270370 0025174 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/wetboobs/__init__.py 0000664 0000000 0000000 00000001333 11375270370 0027305 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .wetboobs import WetBoobs
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/frontends/wetboobs/wetboobs.py 0000664 0000000 0000000 00000004255 11375270370 0027400 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import logging
from weboob import CallErrors
from weboob.capabilities.weather import ICapWeather, CityNotFound
from weboob.tools.application import ConsoleApplication
__all__ = ['WetBoobs']
class WetBoobs(ConsoleApplication):
APPNAME = 'wetboobs'
VERSION = '1.0'
COPYRIGHT = 'Copyright(C) 2010 Romain Bignon'
def main(self, argv):
self.load_modules(ICapWeather)
return self.process_command(*argv[1:])
@ConsoleApplication.command('search cities')
def command_search(self, pattern):
for backend, city in self.weboob.do('iter_city_search', pattern):
self.format(city)
@ConsoleApplication.command('get current weather')
def command_current(self, city):
try:
for backend, current in self.weboob.do('get_current', city):
self.format(current)
except CallErrors, e:
for error in e:
if isinstance(error, CityNotFound):
logging.error('City "%s" not found' % city)
else:
raise error
@ConsoleApplication.command('get forecasts')
def command_forecasts(self, city):
try:
for backend, forecast in self.weboob.do('iter_forecast', city):
self.format(forecast)
except CallErrors, e:
for error in e:
if isinstance(error, CityNotFound):
logging.error('City "%s" not found' % city)
else:
raise error
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/modules.py 0000664 0000000 0000000 00000012027 11375270370 0023372 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from __future__ import with_statement
from ConfigParser import SafeConfigParser
import logging
from logging import debug, error, exception, warning
import os
import re
import stat
import weboob.backends
from weboob.backend import BaseBackend
from weboob.capabilities.cap import ICap
__all__ = ['Module']
class Module:
def __init__(self, name, module):
self.name = name
self.module = module
self.klass = None
for attrname in dir(self.module):
attr = getattr(self.module, attrname)
if isinstance(attr, type) and issubclass(attr, BaseBackend) and attr != BaseBackend:
self.klass = attr
if not self.klass:
raise ImportError("This is not a backend module (no BaseBackend class found)")
def get_name(self):
return self.klass.NAME
def get_maintainer(self):
return '%s <%s>' % (self.klass.MAINTAINER, self.klass.EMAIL)
def get_version(self):
return self.klass.VERSION
def get_description(self):
return self.klass.DESCRIPTION
def get_license(self):
return self.klass.LICENSE
def get_config(self):
return self.klass.CONFIG
def iter_caps(self):
for cap in self.klass.__bases__:
if issubclass(cap, ICap) and cap != ICap:
yield cap
def has_caps(self, *caps):
for c in caps:
if issubclass(self.klass, c):
return True
return False
def create_backend(self, weboob, name, config, storage):
debug('Created backend "%s"' % name)
return self.klass(weboob, name, config, storage)
class BackendsConfig(object):
class WrongPermissions(Exception):
pass
def __init__(self, confpath):
self.confpath = confpath
try:
mode = os.stat(confpath).st_mode
except OSError:
os.mknod(confpath, 0600)
else:
if mode & stat.S_IRGRP or mode & stat.S_IROTH:
raise self.WrongPermissions(
u'Weboob will not start until config file %s is readable by group or other users.' % confpath)
def iter_backends(self):
config = SafeConfigParser()
config.read(self.confpath)
for name in config.sections():
params = dict(config.items(name, raw=True))
try:
yield name, params.pop('_type'), params
except KeyError:
warning('Missing field "_type" for backend "%s"', name)
continue
def add_backend(self, name, _type, params):
if not name:
raise ValueError(u'Please give a name to the backend.')
config = SafeConfigParser()
config.read(self.confpath)
config.add_section(name)
config.set(name, '_type', _type)
for key, value in params.iteritems():
config.set(name, key, value)
with open(self.confpath, 'wb') as f:
config.write(f)
def remove_backend(self, name):
config = SafeConfigParser()
config.read(self.confpath)
config.remove_section(name)
with open(self.confpath, 'wb') as f:
config.write(f)
class ModulesLoader(object):
def __init__(self):
self.modules = {}
def get_or_load_module(self, name):
if name not in self.modules:
self.load_module('weboob.backends.%s' % name)
return self.modules[name]
def load(self):
path = weboob.backends.__path__[0]
regexp = re.compile('^%s/([\w\d_]+)$' % path)
for root, dirs, files in os.walk(path):
m = regexp.match(root)
if m and '__init__.py' in files:
self.load_module('weboob.backends.%s' % m.group(1))
def load_module(self, name):
try:
module = Module(name, __import__(name, fromlist=[str(name)]))
except ImportError, e:
msg = 'Unable to load module "%s": %s' % (name, e)
if logging.root.level == logging.DEBUG:
exception(msg)
return
else:
error(msg)
return
if name in self.modules:
warning('Module "%s" is already loaded (%s)' % self.modules[name].module)
return
self.modules[module.get_name()] = module
debug('Loaded module "%s" (%s)' % (name, module.module.__name__))
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/ouiboube.py 0000664 0000000 0000000 00000014411 11375270370 0023532 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from __future__ import with_statement
import os
from logging import warning
from weboob.bcall import BackendsCall, CallErrors
from weboob.modules import ModulesLoader, BackendsConfig
from weboob.scheduler import Scheduler
__all__ = ['Weboob', 'CallErrors']
class Weboob(object):
WORKDIR = os.path.join(os.path.expanduser('~'), '.weboob')
BACKENDS_FILENAME = 'backends'
def __init__(self, app_name, workdir=WORKDIR, backends_filename=None, scheduler=None):
self.app_name = app_name
self.workdir = workdir
self.backends = {}
# Scheduler
if scheduler is None:
scheduler = Scheduler()
self.scheduler = scheduler
# Create WORKDIR
if not os.path.exists(self.workdir):
os.mkdir(self.workdir, 0700)
elif not os.path.isdir(self.workdir):
warning('"%s" is not a directory' % self.workdir)
# Modules loader
self.modules_loader = ModulesLoader()
# Backends config
if not backends_filename:
backends_filename = os.path.join(self.workdir, self.BACKENDS_FILENAME)
elif not backends_filename.startswith('/'):
backends_filename = os.path.join(self.workdir, backends_filename)
self.backends_config = BackendsConfig(backends_filename)
def load_backends(self, caps=None, names=None, storage=None):
loaded_backends = {}
for name, _type, params in self.backends_config.iter_backends():
try:
module = self.modules_loader.get_or_load_module(_type)
except KeyError:
warning(u'Unable to find module "%s" for backend "%s"' % (_type, name))
continue
# Check conditions
if (not caps is None and not module.has_caps(caps)) or \
(not names is None and not name in names):
continue
try:
self.backends[name] = module.create_backend(self, name, params, storage)
loaded_backends[name] = self.backends[name]
except Exception, e:
warning(u'Unable to load "%s" backend: %s. filename=%s' % (name, e, self.backends_config.confpath))
return loaded_backends
def load_modules(self, caps=None, names=None, storage=None):
loaded_backends = {}
self.modules_loader.load()
for name, module in self.modules_loader.modules.iteritems():
if (caps is None or module.has_caps(caps)) and \
(names is None or module.get_name() in names):
try:
name = module.get_name()
self.backends[name] = module.create_backend(self, name, {}, storage)
loaded_backends[name] = self.backends[name]
except Exception, e:
warning(u'Unable to load "%s" module as backend with no config: %s' % (name, e))
return loaded_backends
def iter_backends(self, caps=None):
"""
Iter on each backends.
Note: each backend is locked when it is returned.
@param caps Optional list of capabilities to select backends
@return iterator on selected backends.
"""
for name, backend in sorted(self.backends.iteritems()):
if caps is None or backend.has_caps(caps):
with backend:
yield backend
def do(self, function, *args, **kwargs):
"""
Do calls on loaded backends with specified arguments, in separated
threads.
This function has two modes:
- If 'function' is a string, it calls the method with this name on
each backends with the specified arguments;
- If 'function' is a callable, it calls it in a separated thread with
the locked backend instance at first arguments, and *args and
**kwargs.
@param function backend's method name, or callable object
@return an iterator of results
"""
backends = list(self.iter_backends())
return BackendsCall(backends, function, *args, **kwargs)
def do_caps(self, caps, function, *args, **kwargs):
"""
Do calls on loaded modules with the specified capabilities, in
separated threads.
See also documentation of the 'do' method.
@param caps list of caps or cap to select backends
@param function backend's method name, or callable object
@return an iterator of results
"""
backends = list(self.iter_backends(caps))
return BackendsCall(backends, function, *args, **kwargs)
def do_backends(self, backends, function, *args, **kwargs):
if isinstance(backends, (str,unicode)):
backends = [backend for backend in self.iter_backends() if backend.name == backends]
elif isinstance(backends, (list,tuple)):
old_backends = backends
backends = []
for b in old_backends:
if isinstance(b, (str,unicode)):
try:
backends.append(self.backends[self.backends.index(b)])
except ValueError:
pass
else:
backends.append(b)
return BackendsCall(backends, function, *args, **kwargs)
def schedule(self, interval, function, *args):
return self.scheduler.schedule(interval, function, *args)
def repeat(self, interval, function, *args):
return self.scheduler.repeat(interval, function, *args)
def want_stop(self):
return self.scheduler.want_stop()
def loop(self):
return self.scheduler.run()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/scheduler.py 0000664 0000000 0000000 00000003652 11375270370 0023704 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import logging
from threading import Timer, Event
__all__ = ['Scheduler']
class IScheduler(object):
def schedule(self, interval, function, *args):
raise NotImplementedError()
def repeat(self, interval, function, *args):
raise NotImplementedError()
def run(self):
raise NotImplementedError()
def want_stop(self):
raise NotImplementedError()
class Scheduler(IScheduler):
def __init__(self):
self.stop_event = Event()
self.count = 0
self.queue = {}
def schedule(self, interval, function, *args):
self.count += 1
logging.debug('function "%s" will be called in %s seconds' % (function.__name__, interval))
timer = Timer(interval, function, args)
timer.start()
self.queue[self.count] = timer
return self.count
def repeat(self, interval, function, *args):
function(*args)
return self.schedule(interval, self._repeated_cb, interval, function, args)
def run(self):
self.stop_event.wait()
return True
def want_stop(self):
self.stop_event.set()
def _repeated_cb(self, interval, function, args):
self.repeat(interval, function, *args)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/ 0000775 0000000 0000000 00000000000 11375270370 0022506 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0024605 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/ 0000775 0000000 0000000 00000000000 11375270370 0025011 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/__init__.py 0000664 0000000 0000000 00000001576 11375270370 0027133 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from .base import BaseApplication, ConfigError
from .console import ConsoleApplication
from .prompt import PromptApplication
try:
from .qt import QtApplication
except ImportError:
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/base.py 0000664 0000000 0000000 00000014230 11375270370 0026275 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import sys, os
import logging
import optparse
from optparse import OptionGroup, OptionParser
from weboob import Weboob
from weboob.tools.config.iconfig import ConfigError
__all__ = ['BaseApplication', 'ConfigError']
class BaseApplication(object):
# Application name
APPNAME = ''
# Default configuration
CONFIG = {}
# Configuration directory
CONFDIR = os.path.join(os.path.expanduser('~'), '.weboob')
# Synopsis
SYNOPSIS = 'Usage: %prog [options (-h for help)] ...'
# Version
VERSION = None
# Copyright
COPYRIGHT = None
def __init__(self):
self.weboob = self.create_weboob()
self.config = None
version = None
if self.VERSION:
if self.COPYRIGHT:
version = '%s v%s (%s)' % (self.APPNAME, self.VERSION, self.COPYRIGHT)
else:
version = '%s v%s' % (self.APPNAME, self.VERSION)
self._parser = OptionParser(self.SYNOPSIS, version=version)
self._parser.add_option('-b', '--backends', help='what backend(s) to enable (comma separated)')
logging_options = OptionGroup(self._parser, 'Logging Options')
logging_options.add_option('-d', '--debug', action='store_true', help='display debug messages')
logging_options.add_option('-q', '--quiet', action='store_true', help='display only error messages')
logging_options.add_option('-v', '--verbose', action='store_true', help='display info messages')
self._parser.add_option_group(logging_options)
self._parser.add_option('--shell-completion', action='store_true', help=optparse.SUPPRESS_HELP)
def create_weboob(self):
return Weboob(self.APPNAME)
def create_storage(self, path=None, klass=None):
"""
Create a storage object.
@param path [str] an optional specific path.
@param klass [IStorage] what klass to instance.
@return a IStorage object
"""
if klass is None:
# load StandardStorage only here because some applications don't
# want to depend on yaml and do not use this function
from weboob.tools.storage import StandardStorage
klass = StandardStorage
if path is None:
path = os.path.join(self.CONFDIR, self.APPNAME + '.storage')
elif not path.startswith('/'):
path = os.path.join(self.CONFDIR, path)
return klass(path)
def load_config(self, path=None, klass=None):
"""
Load a configuration file and get his object.
@param path [str] an optional specific path.
@param klass [IConfig] what klass to instance.
@return a IConfig object
"""
if klass is None:
# load Config only here because some applications don't want
# to depend on yaml and do not use this function
# from weboob.tools.config.yamlconfig import YamlConfig
# klass = YamlConfig
from weboob.tools.config.iniconfig import INIConfig
klass = INIConfig
if path is None:
path = os.path.join(self.CONFDIR, self.APPNAME)
elif not path.startswith('/'):
path = os.path.join(self.CONFDIR, path)
self.config = klass(path)
self.config.load(self.CONFIG)
def main(self, argv):
""" Main function """
raise NotImplementedError()
def load_backends(self, caps=None, names=None, *args, **kwargs):
if names is None:
names = self._enabled_backends
return self.weboob.load_backends(caps, names, *args, **kwargs)
def load_modules(self, caps=None, names=None, *args, **kwargs):
if names is None:
names = self._enabled_backends
return self.weboob.load_modules(caps, names, *args, **kwargs)
def _get_completions(self):
"""
Overload this method in subclasses if you want to enrich shell completion.
@return a set object
"""
return set()
def _handle_app_options(self):
"""
Overload this method in subclasses if you want to handle options defined in subclass constructor.
"""
pass
@classmethod
def run(klass, args=None):
if args is None:
args = [(sys.stdin.encoding and arg.decode(sys.stdin.encoding) or arg) for arg in sys.argv]
app = klass()
app.options, args = app._parser.parse_args(args)
if app.options.shell_completion:
items = set()
for option in app._parser.option_list:
if not option.help is optparse.SUPPRESS_HELP:
items.update(str(option).split('/'))
items.update(app._get_completions())
print ' '.join(items)
sys.exit(0)
if app.options.debug:
level=logging.DEBUG
elif app.options.verbose:
level = logging.INFO
elif app.options.quiet:
level = logging.ERROR
else:
level = logging.WARNING
log_format = '%(asctime)s:%(levelname)s:%(filename)s:%(lineno)d:%(funcName)s %(message)s'
logging.basicConfig(stream=sys.stdout, level=level, format=log_format)
app._enabled_backends = app.options.backends.split(',') if app.options.backends else None
app._handle_app_options()
try:
sys.exit(app.main(args))
except KeyboardInterrupt:
print 'Program killed by SIGINT'
except ConfigError, e:
print 'Configuration error: %s' % e
sys.exit(1)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/console.py 0000664 0000000 0000000 00000017362 11375270370 0027036 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon, Julien Hébert, Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from functools import partial
import getpass
from inspect import getargspec
import logging
import re
import sys
import weboob
from weboob.modules import BackendsConfig
from .base import BaseApplication
from .formatters import formatters_classes
from .results import Results, WhereCondition, WhereConditionException
__all__ = ['ConsoleApplication']
class ConsoleApplication(BaseApplication):
SYNOPSIS = 'Usage: %prog [options (-h for help)] command [parameters...]'
def __init__(self):
try:
BaseApplication.__init__(self)
except BackendsConfig.WrongPermissions, e:
logging.error(u'Error: %s' % e)
sys.exit(1)
self._parser.format_description = lambda x: self._parser.description
if self._parser.description is None:
self._parser.description = ''
self._parser.description += 'Available commands:\n'
for name, arguments, doc_string in self._commands:
command = '%s %s' % (name, arguments)
self._parser.description += ' %-30s %s\n' % (command, doc_string)
self._parser.add_option('-f', '--formatter', default='simple', choices=formatters_classes.keys(),
help='select output formatter (%s)' % u','.join(formatters_classes.keys()))
self._parser.add_option('-s', '--select', help='select result item key(s) to display (comma-separated)')
self._parser.add_option('-w', '--where', help='filter results to display with boolean condition')
def _handle_app_options(self):
self._formatter = formatters_classes[self.options.formatter]
if self.options.select:
self.selected_fields = self.options.select.split(',')
else:
self.selected_fields = None
if self.options.where:
self.where_condition = WhereCondition(self.options.where)
else:
self.where_condition = None
def _get_completions(self):
return set(name for name, arguments, doc_string in self._commands)
def ask(self, question, default=None, masked=False, regexp=None):
"""
Ask a question to user.
@param question text displayed (str)
@param default optional default value (str)
@param masked if True, do not show typed text (bool)
@param regexp text must match this regexp (str)
@return entered text by user (str)
"""
if default is not None:
question = u'%s [%s]' % (question, default)
hidden_msg = u'(input chars are hidden) ' if masked else ''
question = u'%s%s: ' % (hidden_msg, question)
correct = False
while not correct:
line = getpass.getpass(question) if masked else raw_input(question)
if not line and default is not None:
line = default
correct = not regexp or re.match(regexp, unicode(line))
return line
def process_command(self, command=None, *args):
if command is None:
self._parser.print_help()
return 0
def f(x):
return x.startswith('command_' + command)
matching_commands = filter(f, dir(self))
if len(matching_commands) == 0:
sys.stderr.write("No such command: %s.\n" % command)
return 1
if len(matching_commands) != 1:
sys.stderr.write("Ambiguious command %s: %s.\n" % (command, ', '.join(
[s.replace('command_', '', 1) for s in matching_commands])))
return 1
func = getattr(self, matching_commands[0])
_args, varargs, varkw, defaults = getargspec(func)
nb_max_args = nb_min_args = len(_args) - 1
if defaults:
nb_min_args -= len(defaults)
if len(args) > nb_max_args and not varargs:
sys.stderr.write("Command '%s' takes at most %d arguments.\n" % (command, nb_max_args))
return 1
if len(args) < nb_min_args:
if varargs or defaults:
sys.stderr.write("Command '%s' takes at least %d arguments.\n" % (command, nb_min_args))
else:
sys.stderr.write("Command '%s' takes %d arguments.\n" % (command, nb_min_args))
return 1
try:
command_result = func(*args)
except weboob.CallErrors, errors:
logging.error(errors)
return 1
# Process result
if isinstance(command_result, Results):
self.format(command_result)
return 0
elif isinstance(command_result, (str, unicode)):
print command_result
return 0
elif isinstance(command_result, int):
return command_result
elif command_result is None:
return 0
else:
try:
print unicode(command_result)
except ValueError:
raise Exception('command_result type not expected: %s' % type(command_result))
_commands = []
def register_command(f, doc_string, register_to=_commands):
def get_arguments(func, skip=0):
"""
Get arguments of a function as a string.
skip is the number of skipped arguments.
"""
skip += 1
args, varargs, varkw, defaults = getargspec(func)
cut = len(args)
if defaults:
cut -= len(defaults)
args = ["<%s>" % a for a in args[skip:cut]] + \
["[%s]" % a for a in args[cut:]]
if varargs:
args.append("[%s..]" % varargs)
if varkw:
args.append("{WTF}" % varkw)
return " ".join(args)
command_name = f.func_name.replace('command_', '')
register_to.append((command_name, get_arguments(f), doc_string))
return f
def command(doc_string, f=register_command):
return partial(f, doc_string=doc_string)
def format(self, result):
try:
result = self._formatter.format(result, selected_fields=self.selected_fields, where_condition=self.where_condition)
if result is not None:
print result
except WhereConditionException, e:
logging.error(e)
register_command = staticmethod(register_command)
command = staticmethod(command)
def load_backends(self, caps=None, names=None, *args, **kwargs):
loaded_backends = BaseApplication.load_backends(self, caps, names, *args, **kwargs)
if not loaded_backends:
logging.error(u'Cannot start application: no configured backend was found.\nHere is a list of all available backends:')
from weboob.frontends.weboobcfg import WeboobCfg
weboobcfg = WeboobCfg()
if caps is not None:
if not isinstance(caps, (list, tuple, set)):
caps = (caps,)
caps = iter(cap.__name__ for cap in caps)
weboobcfg.command_modules(*caps)
logging.error(u'You can configure a backends using the "weboobcfg add" command:\nweboobcfg add [options..]')
sys.exit(0)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/formatters/ 0000775 0000000 0000000 00000000000 11375270370 0027177 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/formatters/__init__.py 0000664 0000000 0000000 00000001523 11375270370 0031311 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from .simple import SimpleFormatter
__all__ = ['formatters_classes']
formatters_classes = dict(
simple=SimpleFormatter,
)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/formatters/simple.py 0000664 0000000 0000000 00000003462 11375270370 0031047 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
__all__ = ['SimpleFormatter']
class SimpleFormatter(object):
@classmethod
def format(cls, obj, selected_fields=None, where_condition=None):
def iter_fields(obj):
import types
for attribute_name in dir(obj):
if attribute_name.startswith('_'):
continue
attribute = getattr(obj, attribute_name)
if not isinstance(attribute, types.MethodType):
yield attribute_name, attribute
if hasattr(obj, 'iter_fields'):
fields_iterator = obj.iter_fields()
else:
fields_iterator = iter_fields(obj)
d = dict((k, v) for k, v in fields_iterator)
if where_condition is not None:
if not where_condition.is_valid(d):
d = None
if not d:
return None
if selected_fields is None:
result = u'\n'.join(u'%s: %s' % (k, unicode(d[k])) for k in sorted(d)) + u'\n'
else:
result = u'\t'.join(unicode(d[k]) for k in selected_fields if d[k] is not None)
return result
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/prompt.py 0000664 0000000 0000000 00000004210 11375270370 0026701 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import sys
from weboob import Weboob
from weboob.scheduler import Scheduler
from .console import ConsoleApplication
__all__ = ['PromptApplication']
class PromptScheduler(Scheduler):
def __init__(self, prompt_cb, read_cb):
Scheduler.__init__(self)
self.read_cb = read_cb
self.prompt_cb = prompt_cb
def run(self):
try:
while not self.stop_event.isSet():
self.prompt_cb()
line = sys.stdin.readline()
if not line:
self.want_stop()
sys.stdout.write('\n')
else:
self.read_cb(line.strip())
except KeyboardInterrupt:
sys.stdout.write('\n')
class PromptApplication(ConsoleApplication):
SYNOPSIS = 'Usage: %prog [options (-h for help)]'
def create_weboob(self):
return Weboob(self.APPNAME, scheduler=PromptScheduler(self.prompt, self.read_cb))
@ConsoleApplication.command("Display this notice")
def command_help(self):
print 'Available commands:'
for name, arguments, doc_string in self._commands:
command = '%s %s' % (name, arguments)
print ' %-30s %s' % (command, doc_string)
def prompt(self):
sys.stdout.write('> ')
sys.stdout.flush()
def loop(self):
self.weboob.loop()
def read_cb(self, line):
line = line.split()
if line:
self.process_command(*line)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/qt.py 0000664 0000000 0000000 00000004313 11375270370 0026010 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import sys
from PyQt4.QtCore import QTimer, SIGNAL
from PyQt4.QtGui import QMainWindow, QApplication
from weboob import Weboob
from weboob.scheduler import IScheduler
from .base import BaseApplication
__all__ = ['QtApplication']
class QtScheduler(IScheduler):
def __init__(self, app):
self.app = app
self.timers = {}
def schedule(self, interval, function, *args):
timer = QTimer()
timer.setInterval(interval)
timer.setSingleShot(False)
self.app.connect(timer, SIGNAL("timeout()"), lambda: self.timeout(timer.timerId(), False, function, *args))
self.timers[timer.timerId()] = timer
def repeat(self, interval, function, *args):
timer = QTimer()
timer.setInterval(interval)
timer.setSingleShot(True)
self.app.connect(timer, SIGNAL("timeout()"), lambda: self.timeout(timer.timerId(), True, function, *args))
self.timers[timer.timerId()] = timer
def timeout(self, _id, single, function, *args):
function(*args)
if single:
self.timers.pop(_id)
def want_stop(self):
self.app.quit()
def run(self):
self.app.exec_()
class QtApplication(QApplication, BaseApplication):
def __init__(self):
QApplication.__init__(self, sys.argv)
BaseApplication.__init__(self)
def create_weboob(self):
return Weboob(self.APPNAME, scheduler=QtScheduler(self))
class QtMainWindow(QMainWindow):
def __init__(self, parent=None):
QMainWindow.__init__(self, parent)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/application/results.py 0000664 0000000 0000000 00000006602 11375270370 0027070 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
__all__ = ['Results', 'WhereCondition', 'WhereConditionException']
class Results(object):
def __init__(self, name=u'', header=None):
self.name = name
self._groups = []
self._items = []
self.header = header
def add_item(self, item):
self._items.append(item)
def add_items(self, items):
self._items.extend(items)
def iter_items(self):
return iter(self._items)
def deep_iter_items(self):
for i in self._items:
yield i
for g in self._groups:
for i in g.iter_items():
yield i
def add_group(self, group):
self._groups.append(group)
def get_group(self, name):
l = [group for group in self._groups if group.name == name]
if l:
return l[0]
else:
return None
def get_or_create_group(self, name, group_class=None):
if group_class is None:
group_class = Results
group = self.get_group(name)
if group:
return group
else:
new_group = group_class(name)
self.add_group(new_group)
return new_group
def iter_groups(self):
return iter(self._groups)
class WhereCondition(object):
def __init__(self, where_condition_str):
where_condition_str.replace('OR', 'or')
where_condition_str.replace('AND', 'and')
where_condition_str.replace('NOT', 'not')
or_list = []
for _or in where_condition_str.split('or'):
and_dict = {}
for _and in _or.split('and'):
if '!=' in _and:
k, v = _and.split('!=')
k += '!'
elif '=' in _and:
k, v = _and.split('=')
else:
raise WhereConditionException(u'Could not find = or != operator in sub-expression "%s"' % _and)
and_dict[k] = v
or_list.append(and_dict)
self.where_condition = or_list
def is_valid(self, d):
for _or in self.where_condition:
for k, v in _or.iteritems():
if k.endswith('!'):
k = k[:-1]
different = True
else:
different = False
if k in d:
if different:
if d[k] == v:
return False
else:
if d[k] != v:
return False
else:
raise WhereConditionException(u'Field "%s" is not valid.' % k)
return True
class WhereConditionException(Exception):
pass
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/browser/ 0000775 0000000 0000000 00000000000 11375270370 0024171 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/browser/__init__.py 0000664 0000000 0000000 00000001375 11375270370 0026310 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from weboob.tools.browser.browser import *
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/browser/browser.py 0000664 0000000 0000000 00000027405 11375270370 0026236 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import mechanize
import urllib
import urllib2
import ClientForm
import re
import time
from logging import warning, debug
from copy import copy
from threading import RLock
from weboob.tools.parsers import get_parser
from weboob.tools.decorators import retry
# Try to load cookies
try:
from weboob.tools.firefox_cookies import FirefoxCookieJar
except ImportError, e:
warning("Unable to store cookies: %s" % e)
HAVE_COOKIES = False
else:
HAVE_COOKIES = True
__all__ = ['BrowserIncorrectPassword', 'BrowserBanned', 'BrowserUnavailable', 'BrowserRetry',
'BasePage', 'BaseBrowser', ]
# Exceptions
class BrowserIncorrectPassword(Exception):
pass
class BrowserBanned(BrowserIncorrectPassword):
pass
class BrowserUnavailable(Exception):
pass
class BrowserRetry(Exception):
pass
class NoHistory(object):
"""
We don't want to fill memory with history
"""
def __init__(self): pass
def add(self, request, response): pass
def back(self, n, _response): pass
def clear(self): pass
def close(self): pass
class BasePage(object):
"""
Base page
"""
def __init__(self, browser, document, url=''):
self.browser = browser
self.document = document
self.url = url
def on_loaded(self):
"""
Called when the page is loaded.
"""
pass
class BaseBrowser(mechanize.Browser):
"""
Base browser class to navigate on a website.
"""
# ------ Class attributes --------------------------------------
DOMAIN = None
PROTOCOL = 'http'
ENCODING = 'utf-8'
PAGES = {}
USER_AGENT = 'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.4) Gecko/2008111318 Ubuntu/8.10 (intrepid) Firefox/3.0.3'
# ------ Abstract methods --------------------------------------
def home(self):
"""
Go to the home page.
"""
self.location('%s://%s' % (self.PROTOCOL, self.DOMAIN))
def login(self):
"""
Login to the website.
This function is called when is_logged() returns False and the password
attribute is not None.
"""
raise NotImplementedError()
def is_logged(self):
"""
Return True if we are logged on website. When Browser tries to access
to a page, if this method returns False, it calls login().
It is never called if the password attribute is None.
"""
raise NotImplementedError()
# ------ Browser methods ---------------------------------------
# I'm not a robot, so disable the check of permissions in robots.txt.
default_features = copy(mechanize.Browser.default_features)
default_features.remove('_robots')
def __init__(self, username=None, password=None, firefox_cookies=None,
parser=None, history=NoHistory(), proxy=None):
"""
Constructor of Browser.
@param username [str] username on website.
@param password [str] password on website. If it is None, Browser will
not try to login.
@param filefox_cookies [str] Path to cookies' sqlite file.
@param parser [IParser] parser to use on HTML files.
@param hisory [object] History manager. Default value is an object
which does not keep history.
"""
mechanize.Browser.__init__(self, history=history)
self.addheaders = [
['User-agent', self.USER_AGENT]
]
# Use a proxy
if proxy:
proto = 'http'
if proxy.find('://') >= 0:
proto, domain = proxy.split('://', 1)
else:
domain = proxy
self.set_proxies({proto: domain})
# Share cookies with firefox
if firefox_cookies and HAVE_COOKIES:
self._cookie = FirefoxCookieJar(self.DOMAIN, firefox_cookies)
self._cookie.load()
self.set_cookiejar(self._cookie)
else:
self._cookie = None
if parser is None:
parser = get_parser()()
elif isinstance(parser, (tuple,list)):
parser = get_parser(parser)()
self.parser = parser
self.page = None
self.last_update = 0.0
self.username = username
self.password = password
self.lock = RLock()
if self.password:
try:
self.home()
except BrowserUnavailable:
pass
def __enter__(self):
self.lock.acquire()
def __exit__(self, t, v, tb):
self.lock.release()
def pageaccess(func):
def inner(self, *args, **kwargs):
if not self.page or self.password and not self.page.is_logged():
self.home()
return func(self, *args, **kwargs)
return inner
@pageaccess
def keepalive(self):
self.home()
def check_location(func):
def inner(self, *args, **kwargs):
if args and isinstance(args[0], (str,unicode)) and args[0][0] == '/' and \
(not self.request or self.request.host != self.DOMAIN):
args = ('%s://%s%s' % (self.PROTOCOL, self.DOMAIN, args[0]),) + args[1:]
return func(self, *args, **kwargs)
return inner
@check_location
@retry(BrowserUnavailable, tries=3)
def openurl(self, *args, **kwargs):
"""
Open an URL but do not create a Page object.
"""
debug('Opening URL "%s", %s' % (args, kwargs))
try:
return mechanize.Browser.open(self, *args, **kwargs)
except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError), e:
raise BrowserUnavailable('%s (url="%s")' % (e, args and args[0] or 'None'))
except (mechanize.BrowserStateError, BrowserRetry):
self.home()
return mechanize.Browser.open(self, *args, **kwargs)
def submit(self, *args, **kwargs):
"""
Submit the selected form.
"""
try:
self._change_location(mechanize.Browser.submit(self, *args, **kwargs))
except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError), e:
self.page = None
raise BrowserUnavailable(e)
except (mechanize.BrowserStateError, BrowserRetry), e:
self.home()
raise BrowserUnavailable(e)
def is_on_page(self, pageCls):
return isinstance(self.page, pageCls)
def follow_link(self, *args, **kwargs):
try:
self._change_location(mechanize.Browser.follow_link(self, *args, **kwargs))
except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError), e:
self.page = None
raise BrowserUnavailable('%s (url="%s")' % (e, args and args[0] or 'None'))
except (mechanize.BrowserStateError, BrowserRetry), e:
self.home()
raise BrowserUnavailable(e)
@check_location
@retry(BrowserUnavailable, tries=3)
def location(self, *args, **kwargs):
"""
Change location of browser on an URL.
When the page is loaded, it looks up PAGES to find a regexp which
matches, and create the object. Then, the 'on_loaded' method of
this object is called.
If a password is set, and is_logged() returns False, it tries to login
with login() and reload the page.
"""
keep_args = copy(args)
keep_kwargs = kwargs.copy()
try:
self._change_location(mechanize.Browser.open(self, *args, **kwargs))
except BrowserRetry:
if not self.page or not args or self.page.url != args[0]:
self.location(keep_args, keep_kwargs)
except (mechanize.response_seek_wrapper, urllib2.HTTPError, urllib2.URLError), e:
self.page = None
raise BrowserUnavailable('%s (url="%s")' % (e, args and args[0] or 'None'))
except mechanize.BrowserStateError:
self.home()
self.location(*keep_args, **keep_kwargs)
def _change_location(self, result):
"""
This function is called when we have moved to a page, to load a Page
object.
"""
# Find page from url
pageCls = None
for key, value in self.PAGES.items():
regexp = re.compile('^%s$' % key)
m = regexp.match(result.geturl())
if m:
pageCls = value
break
# Not found
if not pageCls:
self.page = None
r = result.read()
if isinstance(r, unicode):
r = r.encode('iso-8859-15', 'replace')
print r
warning('Ho my fucking god, there isn\'t any page named %s' % result.geturl())
return
debug('[%s] Went on %s' % (self.username, result.geturl()))
self.last_update = time.time()
document = self.parser.parse(result, self.ENCODING)
self.page = pageCls(self, document, result.geturl())
self.page.on_loaded()
if self.password is not None and not self.is_logged():
debug('!! Relogin !!')
self.login()
return
if self._cookie:
self._cookie.save()
def buildurl(self, base, *args, **kwargs):
"""
Build an URL and escape arguments.
You can give a serie of tuples in *args (and the order is keept), or
a dict in **kwargs (but the order is lost).
Example:
>>> buildurl('/blah.php', ('a', '&'), ('c', '=')
'/blah.php?a=%26&b=%3D'
>>> buildurl('/blah.php', a='&', 'c'='=')
'/blah.php?b=%3D&a=%26'
"""
if not args:
args = kwargs
if not args:
return base
else:
return '%s?%s' % (base, urllib.urlencode(args))
def str(self, s):
if isinstance(s, unicode):
s = s.encode('iso-8859-15', 'replace')
return s
def set_field(self, args, label, field=None, value=None, is_list=False):
"""
Set a value to a form field.
@param args [dict] arguments where to look for value.
@param label [str] label in args.
@param field [str] field name. If None, use label instead.
@param value [str] value to give on field.
@param is_list [bool] the field is a list.
"""
try:
if not field:
field = label
if args.get(label, None) is not None:
if not value:
if is_list:
if isinstance(is_list, (list, tuple)):
try:
value = [self.str(is_list.index(args[label]))]
except ValueError, e:
if args[label]:
print '[%s] %s: %s' % (label, args[label], e)
return
else:
value = [self.str(args[label])]
else:
value = self.str(args[label])
self[field] = value
except ClientForm.ControlNotFoundError:
return
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/browser/decorators.py 0000664 0000000 0000000 00000003577 11375270370 0026724 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
__all__ = ['check_domain', 'id2url']
def check_domain(domain):
def wrapper(func):
def inner(self, *args, **kwargs):
if self.DOMAIN not in args[0]:
return None
return func(self, *args, **kwargs)
return inner
return wrapper
def id2url(id2url):
def wrapper(func):
def inner(self, *args, **kwargs):
arg = unicode(args[0])
if arg.startswith('http://'):
if self.DOMAIN in arg:
url = arg
else:
return None
else:
if '@' in arg:
_id, provider = arg.split('@')
if provider == self.DOMAIN:
url = id2url(_id)
if url is None:
return None
else:
return None
else:
url = id2url(arg)
if url is None:
return None
new_args = [url]
new_args.extend(args[1:])
return func(self, *new_args, **kwargs)
return inner
return wrapper
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/config/ 0000775 0000000 0000000 00000000000 11375270370 0023753 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/config/__init__.py 0000664 0000000 0000000 00000000000 11375270370 0026052 0 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/config/iconfig.py 0000664 0000000 0000000 00000001764 11375270370 0025753 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
class ConfigError(Exception): pass
class IConfig:
def load(self, default={}):
raise NotImplementedError()
def save(self):
raise NotImplementedError()
def set(self, *args):
raise NotImplementedError()
def get(self, *args, **kwargs):
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/config/iniconfig.py 0000664 0000000 0000000 00000006314 11375270370 0026276 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from __future__ import with_statement
from ConfigParser import SafeConfigParser
import logging
import os
from .iconfig import IConfig
__all__ = ['INIConfig']
class INIConfig(IConfig):
def __init__(self, path):
self.path = path
self.values = {}
self.config = SafeConfigParser()
def load(self, default={}):
def load_section(section):
sections = section.split(':')
if len(sections) > 1:
for s in sections:
values = load_section(s)
else:
return {section: dict(self.config.items(section))}
self.values = default.copy()
if os.path.exists(self.path):
self.config.read(self.path)
for section in self.config.sections():
self.values = load_section(section)
self.values.update(self.config.items('DEFAULT'))
logging.debug(u'Frontend configuration file loaded: %s.' % self.path)
else:
self.save()
logging.debug(u'Frontend configuration file created with default values: %s. '
'Please customize it.' % self.path)
return self.values
def save(self):
def save_section(values, root_section=None):
for k, v in values.iteritems():
if isinstance(v, (int, float, str, unicode)):
if root_section is not None and not self.config.has_section(root_section):
self.config.add_section(root_section)
self.config.set(root_section, k, unicode(v))
elif isinstance(v, dict):
new_section = ':'.join((root_section, k)) if root_section else k
if not self.config.has_section(new_section):
self.config.add_section(new_section)
save_section(v, new_section)
save_section(self.values)
with open(self.path, 'w') as f:
self.config.write(f)
def get(self, *args, **kwargs):
default = None
if 'default' in kwargs:
default = kwargs['default']
v = self.values
for k in args[:-1]:
if k in v:
v = v[k]
else:
return default
try:
return v[args[-1]]
except KeyError:
return default
def set(self, *args):
v = self.values
for k in args[:-2]:
if k not in v:
v[k] = {}
v = v[k]
v[args[-2]] = args[-1]
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/config/yamlconfig.py 0000664 0000000 0000000 00000004634 11375270370 0026464 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from __future__ import with_statement
import logging
import yaml
from .iconfig import IConfig, ConfigError
__all__ = ['YamlConfig']
class YamlConfig(IConfig):
def __init__(self, path):
self.path = path
self.values = {}
def load(self, default={}):
self.values = default.copy()
try:
with open(self.path, 'r') as f:
self.values = yaml.load(f)
logging.debug(u'Frontend configuration file loaded: %s.' % self.path)
except IOError:
self.save()
logging.debug(u'Frontend configuration file created with default values: %s. Please customize it.' % self.path)
if self.values is None:
self.values = {}
def save(self):
with open(self.path, 'w') as f:
yaml.dump(self.values, f)
def get(self, *args, **kwargs):
default = None
if 'default' in kwargs:
default = kwargs['default']
v = self.values
for a in args[:-1]:
try:
v = v[a]
except KeyError:
if not default is None:
v[a] = {}
v = v[a]
else:
raise ConfigError()
except TypeError:
raise ConfigError()
try:
v = v[args[-1]]
except KeyError:
v[args[-1]] = default
v = v[args[-1]]
return v
def set(self, *args):
v = self.values
for a in args[:-2]:
try:
v = v[a]
except KeyError:
v[a] = {}
v = v[a]
except TypeError:
raise ConfigError()
v[args[-2]] = args[-1]
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/decorators.py 0000664 0000000 0000000 00000003275 11375270370 0025234 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import logging
import time
__all__ = ['retry']
def retry(ExceptionToCheck, tries=4, delay=3, backoff=2):
"""
Retry decorator
from http://www.saltycrane.com/blog/2009/11/trying-out-retry-decorator-python/
original from http://wiki.python.org/moin/PythonDecoratorLibrary#Retry
"""
def deco_retry(f):
def f_retry(*args, **kwargs):
mtries, mdelay = tries, delay
try_one_last_time = True
while mtries > 1:
try:
return f(*args, **kwargs)
try_one_last_time = False
break
except ExceptionToCheck, e:
logging.debug(u'%s, Retrying in %d seconds...' % (e, mdelay))
time.sleep(mdelay)
mtries -= 1
mdelay *= backoff
if try_one_last_time:
return f(*args, **kwargs)
return
return f_retry # true decorator
return deco_retry
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/firefox_cookies.py 0000664 0000000 0000000 00000007207 11375270370 0026244 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
try:
import sqlite3 as sqlite
except ImportError, e:
from pysqlite2 import dbapi2 as sqlite
from mechanize import CookieJar, Cookie
__all__ = ['FirefoxCookieJar']
class FirefoxCookieJar(CookieJar):
def __init__(self, domain, sqlite_file=None, policy=None):
CookieJar.__init__(self, policy)
self.domain = domain
self.sqlite_file = sqlite_file
def __connect(self):
try:
db = sqlite.connect(database=self.sqlite_file, timeout=10.0)
except sqlite.OperationalError, err:
print 'Unable to open %s database: %s' % (self.sqlite_file, err)
return None
return db
def load(self):
db = self.__connect()
if not db: return
cookies = db.execute("""SELECT host, path, name, value, expiry, lastAccessed, isSecure
FROM moz_cookies
WHERE host LIKE '%%%s%%'""" % self.domain)
for entry in cookies:
domain = entry[0]
initial_dot = domain.startswith(".")
domain_specified = initial_dot
path = entry[1]
name = entry[2]
value = entry[3]
expires = entry[4]
secure = entry[6]
discard = False
c = Cookie(0, name, value,
None, False,
domain, domain_specified, initial_dot,
path, False,
secure,
expires,
discard,
None,
None,
{})
#if not ignore_discard and c.discard:
# continue
#if not ignore_expires and c.is_expired(now):
# continue
self.set_cookie(c)
def save(self):
db = self.__connect()
if not db: return
db.execute("DELETE FROM moz_cookies WHERE host LIKE '%%%s%%'" % self.domain)
for cookie in self:
if cookie.secure: secure = 1
else: secure = 0
if cookie.expires is not None:
expires = cookie.expires
else:
expires = 0
if cookie.value is None:
# cookies.txt regards 'Set-Cookie: foo' as a cookie
# with no name, whereas cookielib regards it as a
# cookie with no value.
name = ""
value = cookie.name
else:
name = cookie.name
value = cookie.value
# XXX ugly hack to keep this cookie
if name == 'PHPSESSID':
expires = 1854242393
db.execute("""INSERT INTO moz_cookies (host, path, name, value, expiry, isSecure)
VALUES (?, ?, ?, ?, ?, ?)""",
(cookie.domain, cookie.path, name, value, int(expires), int(secure)))
db.commit()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/misc.py 0000664 0000000 0000000 00000003741 11375270370 0024020 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import sys
import traceback
from dateutil import tz
__all__ = ['toUnicode', 'local2utc', 'html2text', 'get_backtrace']
def toUnicode(text):
r"""
>>> toUnicode('ascii')
u'ascii'
>>> toUnicode(u'utf\xe9'.encode('UTF-8'))
u'utf\xe9'
>>> toUnicode(u'unicode')
u'unicode'
"""
if isinstance(text, unicode):
return text
if not isinstance(text, str):
text = str(text)
try:
return unicode(text, "utf8")
except UnicodeError:
pass
return unicode(text, "ISO-8859-1")
def local2utc(d):
d = d.replace(tzinfo=tz.tzlocal())
d = d.astimezone(tz.tzutc())
return d
try:
import html2text as h2t
h2t.UNICODE_SNOB = 1
h2t.SKIP_INTERNAL_LINKS = True
html2text = h2t.html2text
except ImportError:
def html2text(s):
return s
def get_backtrace(empty="Empty backtrace."):
"""
Try to get backtrace as string.
Returns "Error while trying to get backtrace" on failure.
"""
try:
info = sys.exc_info()
trace = traceback.format_exception(*info)
sys.exc_clear()
if trace[0] != "None\n":
return "".join(trace)
except:
# No i18n here (imagine if i18n function calls error...)
return "Error while trying to get backtrace"
return empty
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/ 0000775 0000000 0000000 00000000000 11375270370 0024165 5 ustar 00root root 0000000 0000000 woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/__init__.py 0000664 0000000 0000000 00000003711 11375270370 0026300 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz, Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import logging
__all__ = ['get_parser', 'NoParserFound']
class NoParserFound(Exception): pass
def load_lxml():
from .lxmlparser import LxmlHtmlParser
return LxmlHtmlParser
def load_lxmlsoup():
from .lxmlsoupparser import LxmlSoupParser
return LxmlSoupParser
def load_html5lib():
from .html5libparser import Html5libParser
return Html5libParser
def load_elementtidy():
from .elementtidyparser import ElementTidyParser
return ElementTidyParser
def load_builtin():
from .htmlparser import HTMLParser
return HTMLParser
def get_parser(preference_order=('lxml', 'lxmlsoup', 'html5lib', 'elementtidy', 'builtin')):
"""
Get a parser from a preference order list.
This allows Weboob to run on systems without lxml, which is the default parser.
Return a parser implementing IParser.
"""
if not isinstance(preference_order, (tuple, list)):
preference_order = [preference_order]
for kind in preference_order:
if not 'load_%s' % kind in globals():
continue
try:
return globals()['load_%s' % kind]()
except ImportError:
logging.debug('%s is not installed.' % kind)
raise NoParserFound("No parser found (%s)" % ','.join(preference_order))
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/elementtidyparser.py 0000664 0000000 0000000 00000004302 11375270370 0030276 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
# XXX Currently, elementtidy segfaults when there are no error, because of
# the behavior of libtidy.
# A patch has been sent to Debian:
# http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=576343
#
# As it is not integrated in Debian yet, and as this problem persists on other
# systems, using elementtidy is for now to avoid.
from elementtidy import TidyHTMLTreeBuilder
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
from xml.etree import ElementTree
from .iparser import IParser
__all__ = ['ElementTidyParser']
class ElementTidyParser(IParser):
def parse(self, data, encoding=None):
TidyHTMLTreeBuilder.ElementTree = ElementTree
HTMLTreeBuilder = TidyHTMLTreeBuilder.TidyHTMLTreeBuilder
parser = HTMLTreeBuilder(encoding)
tree = ElementTree.parse(data, parser)
for elem in tree.getiterator():
if elem.tag.startswith('{'):
elem.tag = elem.tag[elem.tag.find('}')+1:]
return tree
def tostring(self, element):
e = ElementTree.Element('body')
e.text = element.text
e.tail = element.tail
for sub in element.getchildren():
e.append(sub)
s = ''
# XXX OK if it doesn't work with utf-8, the result will be fucking ugly.
for encoding in ('utf-8', 'ISO-8859-1'):
try:
s = ElementTree.tostring(e, encoding)
except UnicodeError:
continue
else:
break
return unicode(s)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/html5libparser.py 0000664 0000000 0000000 00000003251 11375270370 0027475 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon, Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from html5lib import treebuilders, HTMLParser
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
from xml.etree import ElementTree
from .iparser import IParser
__all__ = ['Html5libParser']
class Html5libParser(HTMLParser, IParser):
"""
Parser using html5lib.
Note that it is not available on every systems.
"""
# Default implementation for each type of API.
defaults = {'etree': ElementTree,
}
def __init__(self, api='etree'):
# if no default implementation is defined for this api, set it to None
# to let getTreeBuilder() using the corresponding implementation.
implementation = self.defaults.get(api, None)
HTMLParser.__init__(self, tree=treebuilders.getTreeBuilder(api, implementation))
def parse(self, data, encoding):
return HTMLParser.parse(self, data, encoding=encoding)
def tostring(self, elem):
# TODO
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/htmlparser.py 0000664 0000000 0000000 00000005324 11375270370 0026724 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
from HTMLParser import HTMLParser as _HTMLParser
import htmlentitydefs
try:
from xml.etree import cElementTree as ElementTree
except ImportError:
from xml.etree import ElementTree
from .iparser import IParser
__all__ = ['HTMLParser']
class HTMLTreeBuilder(_HTMLParser):
def __init__(self, encoding=None):
_HTMLParser.__init__(self)
self._target = ElementTree.TreeBuilder()
def doctype(self, name, pubid, system):
pass
def close(self):
tree = self._target.close()
return tree
def handle_starttag(self, tag, attrs):
self._target.start(tag, dict(attrs))
def handle_startendtag(self, tag, attrs):
self._target.start(tag, dict(attrs))
self._target.end(tag)
def handle_charref(self, name):
self._target.data(unichr(int(name)))
def handle_entityref(self, name):
try:
self._target.data(unichr(htmlentitydefs.name2codepoint[name]))
except KeyError:
self._target.data('&' + name)
def handle_data(self, data):
self._target.data(data)
def handle_endtag(self, tag):
try:
self._target.end(tag)
except:
pass
class HTMLParser(IParser):
def parse(self, data, encoding=None):
parser = HTMLTreeBuilder(encoding)
tree = ElementTree.parse(data, parser)
for elem in tree.getiterator():
if elem.tag.startswith('{'):
elem.tag = elem.tag[elem.tag.find('}')+1:]
return tree
def tostring(self, element):
e = ElementTree.Element('body')
e.text = element.text
e.tail = element.tail
for sub in element.getchildren():
e.append(sub)
s = ''
# XXX OK if it doesn't work with utf-8, the result will be fucking ugly.
for encoding in ('utf-8', 'ISO-8859-1'):
try:
s = ElementTree.tostring(e, encoding)
except UnicodeError:
continue
else:
break
return unicode(s)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/iparser.py 0000664 0000000 0000000 00000002224 11375270370 0026204 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Romain Bignon
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
class IParser(object):
def parse(self, data, encoding=None):
"""
Parse a HTML document with a specific encoding to get a tree.
@param data [str] HTML document
@param encoding [str] encoding to use
@return an object with the structured document
"""
raise NotImplementedError()
def tostring(self, elem):
"""
Get HTML string from an element.
"""
raise NotImplementedError()
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/lxmlparser.py 0000664 0000000 0000000 00000002171 11375270370 0026731 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
"""
Copyright(C) 2010 Christophe Benz
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, version 3 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
"""
import lxml.html
from .iparser import IParser
__all__ = ['LxmlHtmlParser']
class LxmlHtmlParser(IParser):
"""
Parser using lxml.
Note that it is not available on every systems.
"""
def parse(self, data, encoding=None):
parser = lxml.html.HTMLParser(encoding=encoding)
return lxml.html.parse(data, parser)
def tostring(self, element):
return lxml.html.tostring(element, encoding=unicode)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/parsers/lxmlsoupparser.py 0000664 0000000 0000000 00000002173 11375270370 0027642 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Christophe Benz
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
import lxml.html
import lxml.html.soupparser
from .iparser import IParser
__all__ = ['LxmlSoupParser']
class LxmlSoupParser(IParser):
"""
Parser using lxml elementsoup.
Note that it is not available on every systems.
"""
def parse(self, data, encoding=None):
return lxml.html.soupparser.parse(data)
def tostring(self, element):
return lxml.html.tostring(element, encoding=unicode)
woob-c5fef77bc8216a5aa8f0f2825fb79b577edfba4f/weboob/tools/storage.py 0000664 0000000 0000000 00000003433 11375270370 0024527 0 ustar 00root root 0000000 0000000 # -*- coding: utf-8 -*-
# Copyright(C) 2010 Romain Bignon
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 3 of the License.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.
from logging import error
class IStorage:
def load(self, backend, default={}):
raise NotImplementedError()
def save(self, backend):
raise NotImplementedError()
def set(self, backend, *args):
raise NotImplementedError()
def get(self, backend, *args, **kwargs):
raise NotImplementedError()
try:
from .config.yamlconfig import YamlConfig
except ImportError, e:
error('Import error for weboob.tools.config.yamlconfig: %s' % e)
else:
class StandardStorage(IStorage):
def __init__(self, path):
self.config = YamlConfig(path)
self.config.load()
def load(self, backend, default={}):
d = self.config.values.get(backend, {})
self.config.values[backend] = default.copy()
self.config.values[backend].update(d)
def save(self, backend):
self.config.save()
def set(self, backend, *args):
self.config.set(backend, *args)
def get(self, backend, *args, **kwargs):
return self.config.get(backend, *args, **kwargs)