From 7c433ca20d1b9f2df777af7f306c09142a866ae2 Mon Sep 17 00:00:00 2001 From: kwaaak Date: Thu, 22 Mar 2018 03:04:27 +0100 Subject: [PATCH] update2 (#2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * update movie.py to use omdbapi as imdbapi is non-functional * minor cosmetic change * Resolve #926 Also cleaned up dict value retrieval a little, the .get() calls were a bit unnecessary. * Remove feedparser dep from requirements * Remove feedparser from RPM spec * Make CAD currency code case-insensitive * Resolve #929 Ensures that FilenameAttribute's parse and serialize always have the parameters they need. * [isup] fix bad indexes preventing bot from recognizing http protocols `'http://'` is 7 characters long, so `site[:6]` can't ever match it. ditto for `'https://'` and `site[:7]` * Switch back to get() calls There's some places where there were already try catch blocks for KeyError, so I left those ones. * core: Fix issues with reloading folder modules Closes #899, obviates and closes #932. * Switch from *args to named args. * Properly fetch the xml before passing it to xmltodict I'm a moron. * Release 6.1.0 * Use flake8 for future checking, and add missing ones * Fix coding declaration in a few places it was still wrong Again, fuck windows. #821 * README: Update sopel package location in Arch * Fix crash during configure Reported by aam in IRC, http://pastebin.com/0rAQb7Kp * Fix TypeError: 'NoneType' object is not iterable Happened when an invalid language hint abbreviation was given. * Require a valid phrase to translate * [meetbot] fix crash when starting meeting Resolves #942 * Release 6.1.1 * Require chanmsg for all adminchannel commands * Fix #945 We should only get `socket.gaierror` if the user passes an invalid IP or hostname, but it could be in the event that we don't have DNS configured on the machine or a multitude of other things, so we can be somewhat vague in the error message. * Fix #860 Should resolve the inconsistencies between the implementation of unquote in python 2 and python 3. * [Formatting]Gray is the same as Grey. Which spelling is correct is a grey area. * Updated link for help command was a bad link, updated to correct one * [countdown] fix remaining time calculation * [docs] Rename * Add tracking of users and their accounts This uses some of the code that @maxpowa wrote for #941, but gives a somewhat more intuitive API. It also paves the way for potentially adding direct support for away-notify and metadata-notify. * rename references to willie in systemd service file * update project url in comments * update project url in comment * update project name in CONTRIBUTING.md (includes log files location, issue tracking url) * Update github URL to organization, Sopel-IRC, rather than Embolalia * [dice] Handle negative numbers and uppercase letters. First, add case-insensitivity to the "d" and "v" letters in the input regex by capturing [dD] and [vV] in regexes. Next, make sure we can handle negative numbers properly by capturing a possible "-" in front of any numbers. If dice_num or dice_type ends up negative, abort _roll_dice early (dice_type already had a negative check that wasn't being hit because "-" wasn't captured). * [dice] Handle negative numbers and uppercase letters. First, add case-insensitivity to the "d" and "v" letters in the input regex by capturing [dD] and [vV] in regexes. Next, make sure we can handle negative numbers properly by capturing a possible "-" in front of any numbers. If dice_num or dice_type ends up negative, abort _roll_dice early (dice_type already had a negative check that wasn't being hit because "-" wasn't captured). * [dice] Also handle negative drop_lowest values. * Make BTC currency code case-insensitive * Address comments on PR #961 This includes being more consistent about using pop rather than del to prevent key errors, and adding some locking around the privilege related things * Add user tracking for RFC WHO replies * Add away tracking * Add support for account-tag * Enable the account-tag capability * Replace channels list with channels dictionary Hopefully, nobody else is taking advantage of channels being a list, rather than a dict. If they are, well, oops. * Add enumeration of IRC events Closes #960 * Add cap-notify support See #971 * coretasks: Replace numeric events with their enums Also add the missing RPL_WHOSPCRPL to tools.events * [contrib] rename & edit willie out of contrib Fixes #963 * Huge cleanup of copyright headers and docstrings They're still super inconsistent and probably a lot are out of date, but at least there won't be random copyright info showing up in the docs anymore. Oh, and my domain and name are correct now, too… * [currency] Make arguments case-insensitive (close #979) * Fix URL excludes loading (#959) setup uses it as a list, and in previous versions of sopel it was a list, but in the UrlSection it's defined as a ValidatedAttribute. This was causing each character in the excludes list to be parsed as a regex exclude. Switching to ListAttribute fixes the issue. * [find_updates] Fix missing RPL_ prefix * [bot] Fix misleading message Coretasks is only one module, so if you loaded it and only one other, you'd get the "Couldn't load any modules" warning, even though there was a module loaded. * [trigger] IRCv3 server-time support * [tests] Create more trigger test cases * [tests] Create module tests * [tests] DB test cleanup * [tests] Add formatting tests * [tests] Fix coding declaration * [tests] Update .travis.yml * [translate] Fix bad test * Support using account name for auth * [tests] Remove pep8 dev-requirement Also fixed the critical error where py.test thought sopel.py (the entry script) was the sopel package. * [pep8] Minor clean up to conform with pep8 * [web] Ensure ca_certs is defined When running tests it's not defined at all, because the only place defining it was the config loader code. Now it's defined, but will still fail out since it's not a valid CA certificate file. * [pep8] Final pep8 run Everything should now conform to pep8 and pass flake8. * [tests] Improve trigger coverage * [reddit] Prevent specific commands in PM Resolves #789 * Documentation for target types * docs/core: Shift around functions to make autodoccing easier * docs: Start on a major cleanup of documentation * Make _ssl_recv always return bytes _ssl_recv returned empty strings instead of the empty bytes object if the socket was closed or upon ENOENT. This lead to exceptions when running sopel with python3 because asynchat.handle_read expects byte objects. This commit fixes #937. * docs: Majorly overhaul organization and format * Release 6.2.0 * Release 6.2.1 * dice: Allow comma delimiter Closes #998 * adminchannel: Remove totally useless commands Also add error messages to the somewhat useful ones * Make it a bit harder to run into the LC_ALL thing This behavior is stupid. Respecting LC_ALL, or anything else for that matter, over the encoding fucking noted in the fucking file is a bad decision, and someone should feel bad. I don't know why it makes things break in the specific and bizarre way it does, but it does, and there's no possible good reason for it. Closes #984. Fuck. * Escape nick before replacing it in regex Resolves #1004 * [weather] Fix YQL woeid lookup Handles an edge case that neither of the PRs handled Fixes #1006 Closes #1007, #1012 * CONTRIBUTING: Let's drop the [brackets] thing and do what everyone else does No point in being different from any other FOSS project out there. * contrib/rpm: willie->sopel * CONTRIBUTING: Update coding/future import guidelines `# coding=utf-8` is now the standard in Sopel & supports windows. The future import now conforms with the flake8 future imports (also conforms with @embolalia's formatting passes a bit back) * web: make web.py into a requests comaptibility layer Since web is deprecated and everyone should switch to requests, the first step is to make web.py a requests comaptibility layer. When web.py was new, requests was not ripe enough to use in Sopel. But now it's time to switch to requests like the rest of the python echo-system. web.py is no more. * find_updates: switch from web.py to requests * web: Fix typo This is why we have flake8 ;) * url: port to requests Now that web.py is deprecated, we can port url.py to requests. Originally from pull request #988, committed here with minor bugfixes and modified commit message. * translate: port to requests * movie: port to requests * xkcd: port to requests * Track the channel topic * Improve locale stupidity checking Thanks to @elad661's comment on b73fc6a408967b8fee1d78454b8f247daa1bbd5e * url: handle capitalized URLs Trigger rules are case-insensitive regexes, so the auto title responder will be triggered even for capitalized URLs such as "Http://google.com" (which can happen, for example, when a mobile device attempts to auto-capitalize the beginning of a sentence). Match URLs case insensitively for title lookup purposes and add error handling in case no URLs could be extracted from the match. * core: Tweak rate limiting to be more effective This doesn't solve the issue, but it should make it slightly less critical. #952 * reddit: fetch posts by submission_id Previously, the reddit module fetched posts by the full URL of the post. This led to RedirectExceptions in some cases, for example when someone links a naked reddit.com URL instead of www.reddit.com. Instead, match only the post ID and pass it to get_submission. * Release 6.3.0 * Don't warn about non-UTF8 locales when running on Python 2 Python 2 doesn't change string behavior according to the locale env, that's a py3 specific weirdness. Also, reword the error message to better explain the issue to the user. * Fix apparent typo in host_blocks initialization * Added missing import to xkcd.py * core: Fix print in handle_error when reaching the exception limit Fixes issue #1025 * trigger: Fix target for QUIT events This was fun to debug! Basically, Soepl encountered an exception when removing unknown users when they QUIT. While this shouldn't happen, it should still be handled gracefully. Since it was an exception, Sopel's response was to try and send the exception line to the channel (sender) the message came from, but QUIT events don't come from channels (or users by PRIVMSG)! Since QUIT was not special-cased, the naive assumption that the first argument is the "sender" was used, and when Sopel tried to send the exception line to the "sender", and the sender had a space in it, this would lead to spam if a user exists with a nick that is identical to the first word in the QUIT message. Ouch. The fix special-cases QUIT in pretrigger to never have a "sender". I also added a test to make sure we parse QUIT correctly. This solves issue #1026. * core: Never try to send an exception line when sender is None Just in case. * Make sure we're working with UTF8 string Depending on the URL, response.iter_content() in Python 2/3 will return either: - `type 'unicode'`/`class 'str'` (for "plain" HTML) - `type 'str'`/`class 'bytes'` (for binary streams, like file URLs) To distinguish between the two situations we're checking if we got string or bytes, and proceed accordingly. This also fixes #1021 * [doc] unify grammatical number of `@commands` example It seems there is no `@command` decorator. At any rate, examples for `@commands` should use the same (and correct) grammatical number. * Release 6.3.1 * FIX: Private BZ's - AttributeError: 'NoneType' * [calc] Remove .wa, as API now requires a key Will be moved to an external module that supports the new API * search: Remove ad URL results DDG changed their HTML output slightly and that threw us off, this *should* fix the r.search.yahoo.com URLs that .g was returning. * Fix issue #1048 * fix .set command for non filename attributes * Fix loading/reloading modules that share the name of the bot owner * Typo correction deamon -> daemon Squashed into a single commit. * Fix config loading in some edge cases Fixes #999 Usually where try/except wouldn't catch NoOptionError, happens when running tests in specific environments. * add groupdict function to triggers (#1061) * Add IRCv3 extended-join tests * Add regular join test * Replace e.message with str(e), e.message has been deprecated since python 2.6 * Fix nickname examples help_prefix shouldn't replace the first character if it's a nickname example. * Fix syntax error * weather: fix location yql query Resolves #1050 and #1029 * Implement proper extended-join support * search: tweak ad result blocking A slight regex change to avoid yahoo ad results from duck duck go if it ends up using the HTML search * irc: toggle error replies (#1071) Adds config option to toggle Sopel replying directly to the error source. * irc: always send exceptions to logger We don't really need to check if `trigger.sender` is `None` when we're sending to the logger -- as long as `trigger` is defined, we'll be fine. This just ensures that the `logging_channel` will always get the exception messages. Also pre-formats the message using format because it's more clear what's going on this way. * Create suppress-warnings.py Can be dropped into ~/.ipython/profile_default/startup/ to suppress the DeprecationWarnings you get when starting Sopel with iPython enabled * run_script: if argv is specified, use it * [announce] Confirm when all announces have been sent (#1044) * Add global and channel rate limits (#1065) * Add global and channel rate limits * Default user rate and compatibility with jenni modules * Fix critical keyerror bug in rate limiting * Simplify syntax for @rate() decorator and update docs * Don't reset function timer during cooldown * fix channel time diff variable * fix indentation in bot.py * weather: catch empty forecast results (#1077) e.g. when the user enters a continent for the location. * irc: treat error in connect as a disconnect (#845) * irc: test suite enhancement Comes with some tweaks to support tests Daemonizes the ping and timeout threads (they should have been in the first place) * coretasks: prevent KeyError when untracked user leaves Fixes #1005 * web: fix header bleed (#1092) Resolves #1091 * seen: be a smart-ass if people ask the bot about itself (#1086) * module: ignore privilege requirement in privmsg (#1093) Resolves #1087 * run_script: fix PID file checking logic when the file is empty This fixes issue #1075 I don't know why the elif explicitly negated the previous codnition, it's obviously not needed because else if already implies the previous condition is False. Also, whoever added the parenthesis there messed up the logic even further, before they were there, it worked okay, even if the condition was a bit more verbose than logically needed. Well, that's what you get when you blindly try to make code conform to PEP8 without actually reading it. * unicode_info: fallback if input is None (#958) Resolves #957 * db: raise ValueError in unalias_nick to match documentation (#1102) Documentation says that a ValueError should be raised if there is not at least one other nick in the group. Resolves #1101 * Update .gitignore (#1110) Renamed willie references to sopel Added .DS_Store ignore * coretasks: tweak topic tracking (#1111) Support different implementation of topic update, RPL_TOPIC appears to only be sent to the user who actually updated the topic. Resolves #1107 * meetbot/url: fix SSLError The core.verify_ssl was not passed to url.find_title(), resulting in SSL errors on sites with invalid certs when `verify_ssl = False` Slight refactor of @psachin's original code for backwards compatibility. Resolves #1113 * coretasks: add support for authentication on Quakenet (#1122) Added the necessary lines for authenticating Sopel with Q. The implementation is almost exactly like AuthServ's. Added Q to core_section along with the other authentication methods, since it is now supported. * coretasks: remove .lower() on auth_method auth_method may be None if it's unset, forgot about that case when merging. Resolves #1124 * setup: tweak requirements Remove unsupported requires statement in setup.py Pin requests dependency to 2.10.0 as 2.11.0 introduced a breaking change against the url.py module * sopel/trigger.py: fix intent_regex * url: make find_title more robust Previously, each 512-byte chunk is prone to decoding mishap when a UTF-8 sequence is incomplete. Now we decode all of content at once, ignoring errors. The old problem appears reliably for pages with many high codepoints: ~~~ http://www3.nhk.or.jp/news/easy/k10010665021000/k10010665021000.html [ NEWS WEB EASY|東南アジアでジカ熱になる人が増える ] - www3.nhk.or.jp ~~~ * [reddit] Change NSFW tag to SPOILERS for some subs Hard-coded rather than configured, since in theory the same list should apply to everyone, and we should merge in new ones. That and effort. * setup: Be more flexible about requests version * Release 6.4.0 * Notify if Bugzilla is private (#1115) Although the primary error no longer exist, but the bot shows nothing if the bugzilla has invalid alias, invaid id or if it has no valid permission to access the bug. The logs should show warnings such as, WARNING:sopel.modules.bugzilla:Bugzilla error: NotFound <- (Invalid ID) WARNING:sopel.modules.bugzilla:Bugzilla error: NotPermitted <- (No permission) This patch should notify about those errors. Closes-Bug: #1112 Signed-off-by: Sachin Patil * Lint imports (#1085) After realizing I'd left a dead import in calc.py after removing the .wa command, I decided to go through and clean up any other imports that didn't appear to be in use any more. * Fixed missing `verify_ssl` param - `verify_ssl` param was missing in few function calls Closes-Bug: #1118 Signed-off-by: Sachin Patil * Add a decorator for url handling Closes #761. Also add xkcd url handling as a demo. * Add docstring for url decorator * Create a gist with the command list Closes #1105 Closes #1080 * Cache help gist location * Use custom user agent for title requests * Add Travis badge * Increase timeout for DB locked error This doesn't fix #736, but should at least make it less common * Fix CI * Release 6.5.0 * [weather] Use help_prefix in hint text when no location given * Add a pronouns module If https://github.com/witch-house/pronoun.is/pull/40 gets merged, it's probably worth porting to use that, since there are a *lot* of pronoun sets. Yes, this should probably support other languages. Sopel's i18n is horrible and I know it. * Fix asking for another user's pronouns * Be a bit less snarky when asked for the bot's pronouns But only a little * [etymology] unescape all known HTML entities Replace bespoke implementation of unescape() with stdlib tools; fix #1153. * Fixes for pronouns.py Fixes setpronouns error on lack of trigger.group(2), fixes autocomplete of nicks with a space so that it's stripped out automatically, fixes that it will say the wrong username if you request someone else's pronouns. * Fixes ConnectionError url.find_title() throws ConnectionError when hostname/IPaddress is not readable thereby fails to read title Sample error ``` 15:11:05 psachin: https://10.65.177.15 15:11:09 BB-8: requests.exceptions.ConnectionError: HTTPSConnectionPool(host='10.65.177.15', port=443): Max retries exceeded with url: / (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 113] No route to host',)) (file "/home/tss/virtualenvs/sopel/lib64/python3.5/site-packages/requests/adapters.py", line 487, in send) ``` * Update web.py fix blank User-Agent, if a custom user-agent is set for web.get() * Use a common user-agent to get the proper results from DDG * Update search.py Hmm maybe single quotes would be better. * fix some missed stuff * Missed one more header copy. Hopefully last one. * Fix typo * Remove duplicate item in triggerable tupe check * Exclude File links from regex matching Fix https://github.com/sopel-irc/sopel/issues/1182 * Added default value to numbered_result Added missing default value of "True" for "verify_ssl" parameter on "number_result". * IP example Fixed broken IP module example * Fix API urls for Bank of Canada and BitcoinAverage * Upper/lowercase shouldn't matter for tell module * Release 6.5.1 * weather: update from deprecated sopel.web to requests * safety module - catch exception on urllib/parse * Fix reddit module * Actually fix reddit * [ip] Fix example/test (Google Inc. => Google LLC) Google changed to an LLC, and updated its AS information, which broke the test assertion. Changes cherry-picked from #1250 and reworded. * Update ignored files for tests Ignore movie.py module because it requires an API key (and will probably be moved out of core anyway). Fix ignores for entry script and ipython module (which were still using the old "willie" name and therefore weren't ignored). This also allows removing the command-line ignore from the Travis build script. --- .gitignore | 10 +- .travis.yml | 14 +- CONTRIBUTING.md | 18 +- CREDITS | 6 +- NEWS | 121 +++++ README.rst | 8 +- checkstyle.sh | 38 +- conftest.py | 2 +- contrib/README | 12 +- contrib/rpm/makerpm.py | 14 +- contrib/rpm/{willie.spec.in => sopel.spec.in} | 15 +- contrib/{willie.cfg => sopel.cfg} | 16 +- contrib/sopel.conf | 2 + contrib/sopel.service | 17 + contrib/suppress-warnings.py | 5 + contrib/willie.conf | 2 - contrib/willie.service | 16 - dev-requirements.txt | 2 +- docs/source/api.rst | 45 ++ docs/source/bot.rst | 18 + docs/source/conf.py | 11 +- docs/source/config.rst | 20 + docs/source/db.rst | 5 + docs/source/index.rst | 466 +----------------- docs/source/plugin.rst | 73 +++ docs/source/trigger.rst | 5 + pytest_run.py | 4 +- requirements.txt | 2 +- setup.py | 5 +- sopel/__init__.py | 34 +- sopel/bot.py | 354 ++++++++++--- sopel/config/__init__.py | 21 +- sopel/config/core_section.py | 25 +- sopel/config/types.py | 37 +- sopel/coretasks.py | 335 ++++++++++--- sopel/db.py | 8 +- sopel/formatting.py | 14 +- sopel/irc.py | 190 ++----- sopel/loader.py | 22 +- sopel/logger.py | 2 +- sopel/module.py | 142 +++--- sopel/modules/__init__.py | 4 +- sopel/modules/admin.py | 6 +- sopel/modules/adminchannel.py | 110 +---- sopel/modules/announce.py | 5 +- sopel/modules/bugzilla.py | 25 +- sopel/modules/calc.py | 55 +-- sopel/modules/clock.py | 15 +- sopel/modules/countdown.py | 10 +- sopel/modules/currency.py | 51 +- sopel/modules/dice.py | 36 +- sopel/modules/etymology.py | 21 +- sopel/modules/find.py | 17 +- sopel/modules/find_updates.py | 20 +- sopel/modules/help.py | 84 +++- sopel/modules/ipython.py | 6 +- sopel/modules/isup.py | 15 +- sopel/modules/lmgtfy.py | 4 +- sopel/modules/meetbot.py | 31 +- sopel/modules/movie.py | 19 +- sopel/modules/ping.py | 4 +- sopel/modules/pronouns.py | 96 ++++ sopel/modules/rand.py | 4 +- sopel/modules/reddit.py | 56 ++- sopel/modules/reload.py | 12 +- sopel/modules/remind.py | 6 +- sopel/modules/safety.py | 14 +- sopel/modules/search.py | 34 +- sopel/modules/seen.py | 7 +- sopel/modules/spellcheck.py | 4 +- sopel/modules/tell.py | 10 +- sopel/modules/tld.py | 4 +- sopel/modules/translate.py | 56 ++- sopel/modules/unicode_info.py | 25 +- sopel/modules/units.py | 4 +- sopel/modules/uptime.py | 4 +- sopel/modules/url.py | 79 ++- sopel/modules/version.py | 7 +- sopel/modules/weather.py | 92 ++-- sopel/modules/wikipedia.py | 18 +- sopel/modules/wiktionary.py | 4 +- sopel/modules/xkcd.py | 94 ++-- sopel/run_script.py | 17 +- sopel/test_tools.py | 2 +- sopel/tools/__init__.py | 18 +- sopel/tools/_events.py | 203 ++++++++ sopel/tools/calculation.py | 5 +- sopel/tools/jobs.py | 4 +- sopel/tools/target.py | 90 ++++ sopel/tools/time.py | 5 +- sopel/trigger.py | 68 ++- sopel/web.py | 180 +++---- test/test_db.py | 9 +- test/test_formatting.py | 26 + test/test_irc.py | 147 ++++++ test/test_module.py | 210 ++++++++ test/test_trigger.py | 229 ++++++++- 97 files changed, 2869 insertions(+), 1668 deletions(-) rename contrib/rpm/{willie.spec.in => sopel.spec.in} (89%) rename contrib/{willie.cfg => sopel.cfg} (51%) create mode 100644 contrib/sopel.conf create mode 100644 contrib/sopel.service create mode 100644 contrib/suppress-warnings.py delete mode 100644 contrib/willie.conf delete mode 100644 contrib/willie.service create mode 100644 docs/source/api.rst create mode 100644 docs/source/bot.rst create mode 100644 docs/source/config.rst create mode 100644 docs/source/db.rst create mode 100644 docs/source/plugin.rst create mode 100644 docs/source/trigger.rst create mode 100644 sopel/modules/pronouns.py create mode 100644 sopel/tools/_events.py create mode 100644 sopel/tools/target.py create mode 100644 test/test_formatting.py create mode 100644 test/test_irc.py create mode 100644 test/test_module.py diff --git a/.gitignore b/.gitignore index 74b07f2dc0..b53fdcbb5c 100644 --- a/.gitignore +++ b/.gitignore @@ -5,8 +5,8 @@ MANIFEST doc/build/* logs/* tests -willie.wiki/* -willie.egg-info/* +sopel.wiki/* +sopel.egg-info/* *.db *.pyc *.pyo @@ -25,3 +25,9 @@ willie.egg-info/* .settings .project .pydevproject + +.cache +.coverage + +# macOS +*.DS_Store diff --git a/.travis.yml b/.travis.yml index e37190d0ef..8f7e55ba12 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,18 +1,24 @@ language: python python: - "2.7" - - "3.3" + - "3.4" git: submodules: false sudo: false # Enables running on faster infrastructure. cache: directories: - $HOME/.cache/pip +addons: + apt: + packages: + - enchant install: - pip install -r requirements.txt -r dev-requirements.txt - pip install coveralls - - pip install pep8 + - pip install flake8 script: - - ./checkstyle.sh - - coverage run --source willie -m py.test . + - ./checkstyle.sh || true + - coverage run --source sopel -m py.test -v . - coverage report --show-missing +after_success: + coveralls diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7b0c812248..eec3436ae6 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -2,14 +2,14 @@ Submitting Issues ----------------- When submitting issues to our -[issue tracker](https://github.com/embolalia/willie/issues), it's important +[issue tracker](https://github.com/sopel-irc/sopel/issues), it's important that you do the following: 1. Describe your issue clearly and concisely. -2. Give Willie the .version command, and include the output in your issue. -3. Note the OS you're running Willie on, and how you installed Willie (via your +2. Give Sopel the .version command, and include the output in your issue. +3. Note the OS you're running Sopel on, and how you installed Sopel (via your package manager, pip, setup.py install, or running straight from source) -4. Include relevant output from the log files in ~/.willie/logs. +4. Include relevant output from the log files in ~/.sopel/logs. Committing Code --------------- @@ -33,12 +33,12 @@ include your changes. * Test your code before you commit. We don't have a formal testing plan in place, but you should make sure your code works as promised before you commit. * Make your commit messages clear and explicative. Our convention is to place - the name of the thing you're changing in [brackets] at the beginning of the - message: the module name for modules, [docs] for documentation files, - [coretasks] for coretasks.py, [db] for the database feature, and so on. -* Python files should always have `#coding: utf8` as the first line (or the + the name of the thing you're changing in at the beginning of the + message, followed by a colon: the module name for modules, docs for documentation files, + coretasks for coretasks.py, db for the database feature, and so on. +* Python files should always have `# coding=utf-8` as the first line (or the second, if the first is `#!/usr/bin/env python`), and - `from __future__ import unicode_literals` as the first line after the module + `from __future__ import unicode_literals, absolute_import, print_function, division` as the first line after the module docstring. Issue Tags diff --git a/CREDITS b/CREDITS index 14cf4a8f80..70a023cec4 100644 --- a/CREDITS +++ b/CREDITS @@ -1,6 +1,6 @@ This file is dedicated to the people who want to list their names (or handles) for their contribution to this work. This project is a fork of "phenny" from http://inamidst.com/phenny/ -This project's name is "Willie" +This project's name is "Sopel" The original creator: Sean B. Palmer deserves the most credit for originally creating phenny. He has done an extraordinary job at producing this project, without him this fork would not exist. @@ -19,7 +19,7 @@ Kenneth K. Sham (Kays) Joel Friedly (jfriedly) Samuel Clements (Ziaix) Dimitri Molenaars (Tyrope) -Edward Powell (Embolalia) +Elsie Powell (Embolalia) Elad Alfassa (elad661) Lior Ramati (FireRogue) -Syfaro Warraw (Syfaro) \ No newline at end of file +Syfaro Warraw (Syfaro) diff --git a/NEWS b/NEWS index 2b9f3fd125..8c9ccc456c 100644 --- a/NEWS +++ b/NEWS @@ -1,3 +1,124 @@ +Changes between 6.5.0 and 6.5.1 +=============================== +Module changes: +* A module to track users' pronouns is added +* A few bug and regression fixes + +Changes between 6.4.0 and 6.5.0 +=============================== +Module changes: +* xkcd module can now recognize xkcd.com urls +* SSL is verified for HTTP requests when not turned off in the config +* The command list is placed in a gist, to prevent flooding +* Title finding uses a custom user agent, to prevent issues with some sites + +Core changes: +* Intent handling is improved + +API changes: +* A @url decorator is added to simplify URL handling + +Changes between 6.3.1 and 6.4.0 +=============================== +Module changes: +* For some subreddits where NSFW is used to mark spoilers, an appropriate tag is shown. +* .ddg avoids giving ad results. +* .wa is fully removed; a replacement can be found on PyPI as sopel-modules.wolfram + +Core changes: +* Support for authenticating with Quakenet's Q is added. +* Errors from empty PID files are fixed. +* Issues with errors not being logged to the logging channel are fixed. +* Topic tracking is improved. +* extended-join is supported properly +* Error messages being reported to the triggering channel/user can be disabled. + +API changes: +* Channel priliveges are no longer checked in private messages. +* Rate limiting can now be done by channel and globally, not just per user. + +Changes between 6.3.0 and 6.3.1 +=============================== +Module changes: +* The xkcd module is working again +* Fix an issue causing unicode errors to show for some URLs when using Python 2 (but you should really switch to 3!) + +Core Changes: +* Fix a bug in QUIT message parsing which caused certain users to be flooded with PMs if their nick matched the first word of a user's QUIT message (such as "disconnected" or "ping") +* Fix a rare python 3 incompatibility bug when quitting due to too many core errors. +* We no longer show a warning when detecting a non-unicode system locale if you're still using Python 2 + +Changes between 6.2.0 and 6.3.0 +=============================== +Module changes: +* Many modules ported to use requests package for stability and security. +* Weather location lookup is fixed. +* Confusing and unnecessary commands like .op were removed. +* Splitting of options in .choice is now more intuitive. +* Some edge cases in reddit post information were fixed. + +Core changes: +* A check is added to warn about an obscure environment issue that can cause strange errors. +* Regex characters in the bot's nick no longer cause issues when a rule has the nickname added. +* Rate limiting is tweaked slightly, which should reduce the severity of the .commands flood bug until a proper solution is found. + +API changes: +* The current topic of a channel is now available as the Channel object's topic attribute. +* sopel.web has been reworked as a wrapper around requests; it remains deprecated. + +Changes between 6.1.1 and 6.2.0 +=============================== +Module changes: +* An error in excluding URLs from title display is fixed +* Case sensitivity issues in currency and dice commands are fixed +* Guards to require channel or private message are added to a number of commands, to avoid confusing errors +* A calculation bug in the countdown command is fixed. +* Misc minor bugfixes and improvements + +Core changes: +* An occasional error with SSL connections on Python 3 is fixed +* On servers which support IRCv3 account extensions, the services account name can be used to authenticate the owner +* Numerous additional IRCv3 features are supported + +API changes: +* bot.privileges is now deprecated in favor of bot.channels +* bot.channels contains more information about the channels the bot is in +* bot.users is now available with information about the users Sopel is aware of +* sopel.web is now deprecated in favor of the third-party `requests` library +* trigger.time is added with the current time, or server-time if the server supports it +* sopel.tools.events is now available as an enum of IRC numeric replies + +Changes between 6.1.0 and 6.1.1 +=============================== +If you are updating from a pre-6.0 version (i.e. Willie), there are backwards- +incompatible changes which you should be aware of. See +http://willie.dftba.net/willie_6.html for more information. + +Core changes: +* A regression which caused the config wizard to be unusable is fixed. + +Changes between 6.0.0 and 6.1.0 +=============================== +If you are updating from a pre-6.0 version (i.e. Willie), there are backwards- +incompatible changes which you should be aware of. See +http://willie.dftba.net/willie_6.html for more information. + +Module changes: +* A regression which prevented the URL safety detection from working is fixed. +* Issues with some special characters in DuckDuckGo searches are fixed +* lxml is no longer required by any modules, greatly simplifying the install process +* Misc minor bugfixes and improvements + +Core changes: +* A regression which disabled blocking functionality is fixed +* Examples are no longer mangled by the .help command, and show the correct prefix +* The listing from .commands is now separated by module +* Issues with reloading folder modules are fixed + +API changes: +* ListAttribute configs can be set to a list or set, with the same effect +* The configure method of validated config attributes now takes the parent config and section name + Changes between 5.5.1 and 6.0.0 =============================== This release contains backwards-incompatible changes. See diff --git a/README.rst b/README.rst index 55ad1ae564..9d9e4034b6 100644 --- a/README.rst +++ b/README.rst @@ -1,4 +1,4 @@ -|version| |downloads| |license| |issues| |forks| |stars| |ages| |works| |badges| +|version| |downloads| |license| |build| |issues| |forks| |stars| |ages| |works| |badges| Introduction ------------ @@ -12,7 +12,7 @@ Installation Latest stable release ===================== If you're on Arch, the easiest way to install is through your package -manager. The package is named ``sopel`` in AUR. On other +manager. The package is named ``sopel`` in [community] repository. On other distros, and pretty much any operating system you can run Python on, you can install `pip `_, and do ``pip install sopel``. Failing all that, you can download the latest tarball from @@ -69,6 +69,8 @@ Join us in `#sopel `_ on Freenode. :target: https://pypi.python.org/pypi/sopel .. |license| image:: https://img.shields.io/pypi/l/sopel.svg :target: https://github.com/sopel-irc/sopel/blob/master/COPYING +.. |build| image:: https://travis-ci.org/sopel-irc/sopel.svg?branch=master + :target: https://travis-ci.org/sopel-irc/sopel .. |issues| image:: https://img.shields.io/github/issues/sopel-irc/sopel.svg :target: https://github.com/sopel-irc/sopel/issues .. |forks| image:: https://img.shields.io/github/forks/sopel-irc/sopel.svg @@ -77,4 +79,4 @@ Join us in `#sopel `_ on Freenode. :target: https://github.com/sopel-irc/sopel/stargazers .. |ages| image:: https://img.shields.io/badge/ages-12%2B-green.svg .. |works| image:: https://img.shields.io/badge/works-usually-yellow.svg -.. |badges| image:: https://img.shields.io/badge/badges-9-green.svg +.. |badges| image:: https://img.shields.io/badge/badges-10-green.svg diff --git a/checkstyle.sh b/checkstyle.sh index b6881d07eb..b2c9e00906 100755 --- a/checkstyle.sh +++ b/checkstyle.sh @@ -6,15 +6,22 @@ find_source_files() { files=$(find_source_files) # These are acceptable (for now). 128 and 127 should be removed eventually. ignore='--ignore=E501,E128,E127' +# These are forbidding certain __future__ imports. The plugin has errors both +# for having and not having them; we want to always have them, so we ignore +# the having them errors and keep the not having them errors. +ignore=$ignore',FI50,FI51,FI52,FI53,FI54,FI55' +# F12 is with_statement, which is already in 2.7. F15 requires and F55 forbids +# generator_stop, which should probably be made mandatory at some point. +ignore=$ignore',F12,F15,F55' # These are rules that are relatively new or have had their definitions tweaked # recently, so we'll forgive them until versions of PEP8 in various developers' -#distros are updated +# distros are updated ignore=$ignore',E265,E713,E111,E113,E402,E731' # For now, go through all the checking stages and only die at the end exit_code=0 -if ! pep8 $ignore --filename=*.py $(find_source_files); then - echo "ERROR: PEP8 does not pass." +if ! flake8 $ignore --filename=*.py $(find_source_files); then + echo "ERROR: flake8 does not pass." exit_code=1 fi @@ -49,29 +56,4 @@ if $fail_py3_unicode; then exit_code=1 fi -check_future () { - fail_unicode_literals=false - for file in $files; do - if ! grep -L "from __future__ import $1" $file; then - fail_unicode_literals=true - fi - done - if $fail_unicode_literals; then - if $2; then - echo "ERROR: Above files do not have $1 import." - exit_code=1 - else - echo "WARNING: Above files do not have $1 import." - fi - fi -} -for mandatory in unicode_literals -do - check_future $mandatory true -done -for optional in division print_function absolute_import -do - check_future $optional false -done - exit $exit_code diff --git a/conftest.py b/conftest.py index 676b003208..e8c6406cd0 100644 --- a/conftest.py +++ b/conftest.py @@ -1,2 +1,2 @@ # This file lists files which should be ignored by pytest -collect_ignore = ["setup.py", "willie.py", "willie/modules/ipython.py"] +collect_ignore = ["setup.py", "sopel.py", "sopel/modules/ipython.py", "sopel/modules/movie.py"] diff --git a/contrib/README b/contrib/README index e4219347d7..21b4aa13cc 100644 --- a/contrib/README +++ b/contrib/README @@ -1,10 +1,10 @@ -This folder contains willie.service and willie.cfg designed to be distributed by 3rd party distrubtions such as Fedora Project or Arch Linux. +This folder contains sopel.service and sopel.cfg designed to be distributed by 3rd party distrubtions such as Fedora Project or Arch Linux. -willie.cfg is a default configuration file for willie, that assumes the OS is new enough to have /run and /usr/lib/tmpfiles.d +sopel.cfg is a default configuration file for sopel, that assumes the OS is new enough to have /run and /usr/lib/tmpfiles.d -willie.service is a systemd service file that assumes you are using a rather recent Willie and has no multiple instance support (TODO). It also assumes that the system has a special user named willie designated for running the bot and this user has access to /run/willie (should be setup by willie.conf in /usr/lib/tmpfiles.d), /var/log/willie and /var/lib/willie +sopel.service is a systemd service file that assumes you are using a rather recent Willie and has no multiple instance support (TODO). It also assumes that the system has a special user named sopel designated for running the bot and this user has access to /run/sopel (should be setup by sopel.conf in /usr/lib/tmpfiles.d), /var/log/sopel and /var/lib/sopel Default installation paths: - willie.cfg /etc - willie.conf /usr/lib/tmpfiles.d - willie.service /usr/lib/systemd/system + sopel.cfg /etc + sopel.conf /usr/lib/tmpfiles.d + sopel.service /usr/lib/systemd/system diff --git a/contrib/rpm/makerpm.py b/contrib/rpm/makerpm.py index ab1893952d..9aef427dc1 100755 --- a/contrib/rpm/makerpm.py +++ b/contrib/rpm/makerpm.py @@ -16,13 +16,13 @@ if len(sys.argv)>1: build = sys.argv[1] print 'Generating archive...' -f = open('willie-%s.tar' % version, 'w') -repo.archive(f, prefix='willie-%s/' % version) +f = open('sopel-%s.tar' % version, 'w') +repo.archive(f, prefix='sopel-%s/' % version) f.close() print 'Building spec file..' -spec_in = open('willie.spec.in', 'r') -spec_out = open('willie.spec', 'w') +spec_in = open('sopel.spec.in', 'r') +spec_out = open('sopel.spec', 'w') for line in spec_in: newline = line.replace('#GITTAG#', head_hash) newline = newline.replace('#BUILD#', build) @@ -32,12 +32,12 @@ spec_in.close() spec_out.close() print 'Starting rpmbuild...' -cmdline = 'rpmbuild --define="%_specdir @wd@" --define="%_rpmdir @wd@" --define="%_srcrpmdir @wd@" --define="%_sourcedir @wd@" -ba willie.spec'.replace('@wd@', os.getcwd()) +cmdline = 'rpmbuild --define="%_specdir @wd@" --define="%_rpmdir @wd@" --define="%_srcrpmdir @wd@" --define="%_sourcedir @wd@" -ba sopel.spec'.replace('@wd@', os.getcwd()) p = call(cmdline, shell=True) for item in os.listdir('noarch'): os.rename(os.path.join('noarch', item), item) print 'Cleaning...' os.removedirs('noarch') -os.remove('willie.spec') -os.remove('willie-%s.tar' % version) +os.remove('sopel.spec') +os.remove('sopel-%s.tar' % version) print 'Done' diff --git a/contrib/rpm/willie.spec.in b/contrib/rpm/sopel.spec.in similarity index 89% rename from contrib/rpm/willie.spec.in rename to contrib/rpm/sopel.spec.in index d1c7a3cf52..6f017cd025 100644 --- a/contrib/rpm/willie.spec.in +++ b/contrib/rpm/sopel.spec.in @@ -1,13 +1,13 @@ %{!?python_sitelib: %global python_sitelib %(%{__python} -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())")} %define gittag #GITTAG# -Name: willie +Name: sopel Version: #VERSION# Release: 0.#BUILD#%{gittag}%{?dist} Summary: Simple, lightweight and easy-to-use IRC Utility bot License: EFL 2.0 -URL: http://willie.dftba.net/ +URL: http://sopel.chat/ Source0: %{name}-%{version}.tar BuildArch: noarch @@ -17,7 +17,6 @@ BuildRequires: dos2unix BuildRequires: systemd Requires: pytz -Requires: python-feedparser Requires: python-enchant Requires: pyOpenSSL Requires: python-praw @@ -57,7 +56,7 @@ mkdir -p %{buildroot}/var/lib/%{name} %doc README.rst COPYING CREDITS NEWS docs/build/api-docs %config(noreplace) %{_sysconfdir}/%{name}.cfg %{python_sitelib}/* -%{_bindir}/willie +%{_bindir}/sopel %dir %attr(-,%{name}, %{name})/run/%{name}/ %{_prefix}/lib/tmpfiles.d/%{name}.conf %{_unitdir}/%{name}.service @@ -69,17 +68,17 @@ mkdir -p %{buildroot}/var/lib/%{name} getent group %{name} >/dev/null || groupadd -r %{name} getent passwd %{name} >/dev/null || \ useradd -r -g %{name} -d /var/lib/%{name} -s /sbin/nologin \ - -c "willie ircbot account " %{name} + -c "sopel ircbot account " %{name} exit 0 %post -%systemd_post willie.service +%systemd_post sopel.service %preun -%systemd_preun willie.service +%systemd_preun sopel.service %postun -%systemd_postun willie.service +%systemd_postun sopel.service %changelog * #LONGDATE# Elad Alfassa #VERSION#0.#BUILD##GITTAG# diff --git a/contrib/willie.cfg b/contrib/sopel.cfg similarity index 51% rename from contrib/willie.cfg rename to contrib/sopel.cfg index 7e5a3cc40c..f99eb68b9b 100644 --- a/contrib/willie.cfg +++ b/contrib/sopel.cfg @@ -1,24 +1,24 @@ -#Default willie configuration file for Fedora +#Default sopel configuration file for Fedora #For information related to possible configuration values see -# https://github.com/embolalia/willie/wiki/Core-configuration-settings -# https://github.com/embolalia/willie/wiki/Module-Configuration +# https://github.com/sopel-irc/sopel/wiki/Core-configuration-settings +# https://github.com/sopel-irc/sopel/wiki/Module-Configuration # #IMPORTANT NOTE! #You must delete the not_configured line in order for the bot to work, # otherwise it will refuse to start. [core] -nick=willie +nick=sopel not_configured=True host=chat.freenode.net port=6697 use_ssl=True verify_ssl=True owner= -logdir=/var/log/willie -pid_dir=/run/willie -homedir=/var/lib/willie +logdir=/var/log/sopel +pid_dir=/run/sopel +homedir=/var/lib/sopel [db] userdb_type='sqlite' -userdb_file='/var/lib/willie/user.db +userdb_file='/var/lib/sopel/user.db diff --git a/contrib/sopel.conf b/contrib/sopel.conf new file mode 100644 index 0000000000..bbff2ae1e3 --- /dev/null +++ b/contrib/sopel.conf @@ -0,0 +1,2 @@ +# Sopel temporary directory setup +d /run/sopel 0755 sopel sopel - diff --git a/contrib/sopel.service b/contrib/sopel.service new file mode 100644 index 0000000000..27e7d98a7e --- /dev/null +++ b/contrib/sopel.service @@ -0,0 +1,17 @@ +[Unit] +Description=Sopel IRC bot +Documentation=http://sopel.chat/ +After=network.target + +[Service] +Type=simple +User=sopel +PIDFile=/run/sopel/sopel-sopel.pid +ExecStart=/usr/bin/sopel -c /etc/sopel.cfg +Restart=on-failure +RestartPreventExitStatus=2 +RestartSec=30 +Environment=LC_ALL=en_US.UTF-8 + +[Install] +WantedBy=multi-user.target diff --git a/contrib/suppress-warnings.py b/contrib/suppress-warnings.py new file mode 100644 index 0000000000..61484fb84c --- /dev/null +++ b/contrib/suppress-warnings.py @@ -0,0 +1,5 @@ +# coding=utf-8 +# suppress-warnings.py +# Suppress iPython's DeprecationWarnings on Sopel start +import warnings +warnings.filterwarnings('ignore') diff --git a/contrib/willie.conf b/contrib/willie.conf deleted file mode 100644 index b97cc8e868..0000000000 --- a/contrib/willie.conf +++ /dev/null @@ -1,2 +0,0 @@ -# Willie tmporary directory setup -d /run/willie 0755 willie willie - diff --git a/contrib/willie.service b/contrib/willie.service deleted file mode 100644 index a3197a156c..0000000000 --- a/contrib/willie.service +++ /dev/null @@ -1,16 +0,0 @@ -[Unit] -Description=Willie IRC bot -Documentation=http://willie.dftba.net/ -After=network.target - -[Service] -Type=simple -User=willie -PIDFile=/run/willie/willie-willie.pid -ExecStart=/usr/bin/willie -c /etc/willie.cfg -Restart=on-failure -RestartPreventExitStatus=2 -RestartSec=30 - -[Install] -WantedBy=multi-user.target diff --git a/dev-requirements.txt b/dev-requirements.txt index 348f283d18..3248113dc4 100644 --- a/dev-requirements.txt +++ b/dev-requirements.txt @@ -1,2 +1,2 @@ pytest -pep8==1.4.6 +ipython diff --git a/docs/source/api.rst b/docs/source/api.rst new file mode 100644 index 0000000000..3741789985 --- /dev/null +++ b/docs/source/api.rst @@ -0,0 +1,45 @@ +Additional API features +======================= + +Sopel includes a number of additional functions that are useful for various +common IRC tasks. + +Note that ``sopel.web`` was deprecated in 6.2.0, and is not included in this +documentation, but is still in use in many modules. It's highly recommended +that you switch to `requests `_ +instead. + +.. contents:: + +sopel.tools +------------ + +.. automodule:: sopel.tools + :members: + +sopel.tools.time +---------------- +.. automodule:: sopel.tools.time + :members: + +sopel.tools.calculation +----------------------- +.. automodule:: sopel.tools.calculation + :members: + +sopel.tools.target +------------------ +.. automodule:: sopel.tools.target + :members: + +sopel.tools.events +------------------ +.. autoclass:: sopel.tools.events + :members: + :undoc-members: + +sopel.formatting +----------------- +.. automodule:: sopel.formatting + :members: + :undoc-members: diff --git a/docs/source/bot.rst b/docs/source/bot.rst new file mode 100644 index 0000000000..7077ba0d58 --- /dev/null +++ b/docs/source/bot.rst @@ -0,0 +1,18 @@ +The bot and its state +===================== + +.. autoclass:: sopel.bot.Sopel + :members: + + + .. py:attribute:: nick + + Sopel's current nick. Changing this while Sopel is running is unsupported. + + .. py:attribute:: user + + Sopel's user/ident. + + .. py:attribute:: name + + Sopel's "real name", as used for whois. diff --git a/docs/source/conf.py b/docs/source/conf.py index 6eb02a0ea1..37e7fbafea 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -28,7 +28,8 @@ # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions = ['sphinx.ext.autodoc'] +extensions = ['sphinx.ext.autodoc', 'sphinx.ext.intersphinx'] +intersphinx_mapping = {'python': ('https://docs.python.org/3', None)} # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates'] @@ -44,7 +45,7 @@ # General information about the project. project = u'Sopel IRC Bot' -copyright = u'2012, E. Powell, et al.' +copyright = u'2012-2015, Elsie Powell, et al.' # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the @@ -187,7 +188,7 @@ # (source start file, target name, title, author, documentclass [howto/manual]). latex_documents = [ ('index', 'sopel.tex', u'Sopel IRC Bot Documentation', - u'E. Powell, et al.', 'manual'), + u'Elsie Powell, et al.', 'manual'), ] # The name of an image file (relative to this directory) to place at the top of @@ -217,7 +218,7 @@ # (source start file, name, description, authors, manual section). man_pages = [ ('index', 'sopel', u'Sopel IRC Bot Documentation', - [u'E. Powell, et al.'], 1) + [u'Elsie Powell, et al.'], 1) ] # If true, show URL addresses after external links. @@ -231,7 +232,7 @@ # dir menu entry, description, category) texinfo_documents = [ ('index', 'sopel', u'Sopel IRC Bot Documentation', - u'E. Powell, et al.', 'SopelIRCBot', 'One line description of project.', + u'Elsie Powell, et al.', 'SopelIRCBot', 'One line description of project.', 'Miscellaneous'), ] diff --git a/docs/source/config.rst b/docs/source/config.rst new file mode 100644 index 0000000000..dbc763f2c5 --- /dev/null +++ b/docs/source/config.rst @@ -0,0 +1,20 @@ +Configuration functionality +=========================== + +.. automodule:: sopel.config + :members: + :undoc-members: + + +.. automodule:: sopel.config.types + :members: + :undoc-members: + +The [core] configuration section +-------------------------------- + +.. autoclass:: sopel.config.core_section.CoreSection + :members: + :undoc-members: + + diff --git a/docs/source/db.rst b/docs/source/db.rst new file mode 100644 index 0000000000..c67902761b --- /dev/null +++ b/docs/source/db.rst @@ -0,0 +1,5 @@ +The bot's database +================== + +.. automodule:: sopel.db + :members: diff --git a/docs/source/index.rst b/docs/source/index.rst index f5f7b5c9d5..41e0c8ccc3 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -1,453 +1,31 @@ -.. Sopel IRC Bot documentation master file, created by - sphinx-quickstart on Sat Jun 16 00:18:40 2012. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. +.. title:: Sopel IRC Bot -Introduction -============ +Sopel IRC Bot +============= -This package contains a framework for an easily set-up, multi-purpose IRC bot. -The intent with this package is not that it be included with other packages, -but rather be run as a standalone program. However, it is intended (and -encouraged) that users write new "modules" for it. This documentation is -intended to serve that goal. +`Sopel `_ is a Python IRC bot framework. It is designed to +enable easily writing new utilities and features for your IRC channels. -.. contents:: :depth: 2 +Quick links +----------- -Getting started: Your functions, ``sopel``, and ``trigger`` -============================================================ +* `Latest releases `_ +* `Source code `_ +* `Wiki `_ -At its most basic, writing a Sopel module involves creating a Python file with -some number of functions in it. Each of these functions will be passed a -``Sopel`` object (``Phenny`` in *1.x* and ``Jenni`` in *2.x*) and a ``Trigger`` -object (``CommandInput`` in *1.x* and *2.x*). By convention, these are named -``phenny`` and ``input`` in *1.x*, ``jenni`` and ``input`` in *2.x*, -``willie`` and ``trigger`` in *3.x*, and ``bot`` and ``trigger`` from version -*4.0* onward. For the purposes of this guide, the *4.0* names will be used. +.. + Eventually, add install instructions and Hello world here, and move the + tutorial into pages following this one. -Your modules ------------- - -A Sopel module contains one or more ``callable``\s. It may optionally contain -``configure``, ``setup``, and ``shutdown`` functions. ``callable``\s are given -a number of attributes, which determine when they will be executed. This is -done with decorators, imported from :py:mod:`sopel.module`. It may also be -done by adding the attributes directly, at the same indentation level as the -function's ``def`` line, following the last line of the function; this is the -only option in versions prior to *4.0*. - -.. py:method:: callable(bot, trigger) - - This is the general function format, called by Sopel when a command is - used, a rule is matched, or an event is seen, as determined by the - attributes of the function. The details of what this function does are - entirely up to the module writer - the only hard requirement from the bot - is that it be callable with a ``Sopel`` object and a ``Trigger`` object, - as noted above. Usually, methods of the Sopel object will be used in reply - to the trigger, but this isn't a requirement. - - The return value of a callable will usually be ``None``. This doesn't need - to be explicit; if the function has no ``return`` statement (or simply uses - ``return`` with no arguments), ``None`` will be returned. In *3.2+*, the - return value can be a constant; in prior versions return values were - ignored. Returning a constant instructs the bot to perform some action - after the ``callable``'s execution. For example, returning ``NOLIMIT`` will - suspend rate limiting on that call. These constants are defined in - :py:mod:`sopel.module`, except in version *3.2* in which they are defined - as attributes of the ``Sopel`` class. - - Note that the name can, and should, be anything - it doesn't need to be - called callable. - - .. py:attribute:: commands - - *See also:* :py:func:`sopel.module.commands` - - A list of commands which will trigger the function. If the Sopel instance - is in a channel, or sent a PRIVMSG, where one of these strings is said, - preceeded only by the configured prefix (a period, by default), the - function will execute. - - .. py:attribute:: rule - - *See also:* :py:func:`sopel.module.rule` - - A regular expression which will trigger the function. If the Sopel - instance is in a channel, or sent a PRIVMSG, where a string matching - this expression is said, the function will execute. Note that captured - groups here will be retrievable through the ``Trigger`` object later. - - Inside the regular expression, some special directives can be used. - ``$nick`` will be replaced with the nick of the bot and ``,`` or ``:``, - and ``$nickname`` will be replaced with the nick of the bot. - - Prior to *3.1*, rules could also be made one of three formats of tuple. - The values would be joined together to form a singular regular - expression. However, these kinds of rules add no functionality over - simple regular expressions, and are considered deprecated in *3.1*. - - .. py:attribute:: event - - *See also:* :py:func:`sopel.module.event` - - This is one of a number of events, such as ``'JOIN'``, ``'PART'``, - ``'QUIT'``, etc. (More details can be found in `RFC 1459`_.) When the - Sopel bot is sent one of these events, the function will execute. Note - that functions with an event must also be given a ``rule`` to match - (though it may be ``'.*'``, which will always match) or they will not - be triggered. - - .. _RFC 1459: http://www.irchelp.org/irchelp/rfc/rfc.html - - .. py:attribute:: rate - - *Availability: 2+* - - *See also:* :py:func:`sopel.module.rate` - - This limits the frequency with which a single user may use the - function. If a function is given a ``rate`` of ``20``, a single user - may only use that function once every 20 seconds. This limit applies to - each user individually. Users on the ``admin`` list in Sopel's - configuration are exempted from rate limits. - - .. py:attribute:: priority - - *See also:* :py:func:`sopel.module.priority` - - Priority can be one of ``high``, ``medium``, ``low``. It allows you to - control the order of callable execution, if your module needs it. - Defaults to ``medium`` - -.. py:method:: setup(sopel) - - This is an optional function of a module, which will be called while the - module is being loaded. The purpose of this function is to perform whatever - actions are needed to allow a module to function properly (e.g, ensuring - that the appropriate configuration variables exist and are set). Note that - this normally occurs prior to connection to the server, so the behavior of - the Sopel object's messaging functions is undefined. - - Throwing an exception from this function (such as a `ConfigurationError - <#sopel.config.ConfigurationError>`_) will prevent any callables in the - module from being registered, and provide an error message to the user. - This is useful when requiring the presence of configuration values or - making other environmental requirements. - - The bot will not continue loading modules or connecting during the - execution of this function. As such, an infinite loop (such as an - unthreaded polling loop) will cause the bot to hang. - -.. py:method:: shutdown(sopel) - - *Availability: 4.1+* - - This is an optional function of a module, which will be called while the - Sopel is quitting. Note that this normally occurs after closing connection - to the server, so the behavior of the Sopel object's messaging functions - is undefined. The purpose of this function is to perform whatever actions - are needed to allow a module to properly clean up (e.g, ensuring that any - temporary cache files are deleted). - - The bot will not continue notifying other modules or continue quitting - during the execution of this function. As such, an infinite loop (such as - an unthreaded polling loop) will cause the bot to hang. - -.. py:method:: configure(config) - - *Availability: 3+* - - This is an optional function of a module, which will be called during the - user's setup of the bot. It's intended purpose is to use the methods of the - passed ``Config`` object in order to create the configuration variables it - needs to function properly. - - In *3.1+*, the docstring of this function can be used to document the - configuration variables that the module uses. This is not currently used - by the bot itself; it is merely convention. - -The ``Sopel`` class --------------------- - -.. autoclass:: sopel.bot.Sopel - :members: - -.. py:function:: reply(text, destination=None, reply_to=None, notice=False) - - In a module function, send ``text`` to the channel in which the function was - triggered, preceeded by the nick of the user who triggered it. If - ``reply_to`` is specified and is not ``None``, this function will preceed - the message with the the value of ``reply_to``. - - If ``destination`` is specified and is not ``None``, this function will send - the message to ``destination`` instead of the originating channel. - ``destination`` can be either a channel or a user. - - If ``notice`` is set to True, this function will send the reply in an - IRC ``NOTICE`` instead of a regular IRC ``PRIVMSG``. - - This function is not available outside of module functions. It can not - be used, for example, in a module's ``setup`` or ``shutdown`` function. - - The same behavior regarding loop detection and length restrictions - apply to ``reply`` as to ``msg``, though ``reply`` does not offer - automatic message splitting. - -.. py:function:: say(text, destination=None, max_messages=1) - - In a module function, send ``text`` to the channel in which the - function was triggered. - - If ``destination`` is specified and is not ``None``, this function will send - the message to ``destination`` instead of the originating channel. - ``destination`` can be either a channel or a user. - - This function is not available outside of module functions. It can not - be used, for example, in a module's ``configure`` function. - - The same behavior regarding loop detection and length restrictions, as - well as message splitting, apply to ``say`` as to ``msg``. - -.. py:function:: action(text, destination=None) - - In a module function, send ``text`` to the channel in which the function - was triggered preceeded by CTCP ACTION directive (result identical to using - /me in most clients). - - If ``destination`` is specified and is not ``None``, this function will send - the message to ``destination`` instead of the originating channel. - ``destination`` can be either a channel or a user. - - This function is not available outside of module functions. It can not - be used, for example, in a module's ``configure`` function. - - The same behavior regarding loop detection and length restrictions apply - to ``action`` as to ``msg`` and ``say``, though like ``reply`` there is - no facility for message splitting. - -.. py:function:: notice(text, destination=None) - - In a module function, send ``text`` to the channel in which the function - was triggered as an IRC ``NOTICE``. - - If ``destination`` is specified and is not ``None``, this function will send - the message to ``destination`` instead of the originating channel. - ``destination`` can be either a channel or a user. - - This function is not available outside of module functions. It can not - be used, for example, in a module's ``configure`` function. - - The same behavior regarding loop detection and length restrictions apply - to ``action`` as to ``msg`` and ``say``, though like ``reply`` there is - no facility for message splitting. - -.. py:function:: quit(message) - - Gracefully quit and shutdown, using ``message`` as the quit message. - - Sopel will notify modules that it is quitting should the modules have - a ``shutdown`` method. - -.. py:function:: part(channel) - - Part ``channel`` - -.. py:function:: join(channel, password = None) - - Join a channel named ``channel``. - -.. py:attribute:: nick - - Sopel's current nick. Changing this while Sopel is running is unsupported. - -.. py:attribute:: name - - Sopel's "real name", as used for whois. - -.. py:attribute:: password - - Sopel's NickServ password - -.. py:attribute:: channels - - A list of Sopel's initial channels. This list will initially be the same - as the one given in the config file, but is not guaranteed to be kept - up-to-date. - -.. py:attribute:: ops -.. py:attribute:: halfplus - - *Availability: 3+* - - Dictionary mapping channels to a list of their ops, and half-ops and ops - respectively. - -.. py:function:: write(args, text=None) - - Send a command to the server - - ``args`` is an iterable of strings, which are joined by spaces. - ``text`` is treated as though it were the final item in ``args``, but - is preceeded by a ``:``. This is a special case which means that - ``text``, unlike the items in ``args`` may contain spaces (though this - constraint is not checked by ``write``). - - In other words, both ``sopel.write(('PRIVMSG',), 'Hello, world!')`` - and ``sopel.write(('PRIVMSG', ':Hello, world!'))`` will send - ``PRIVMSG :Hello, world!`` to the server. - - Newlines and carriage returns ('\\n' and '\\r') are removed before - sending. Additionally, if the message (after joining) is longer than - than 510 characters, any remaining characters will not be sent. - - .. py:function:: msg(destination, text, max_messages=1) - - Send a PRIVMSG of ``text`` to ``destination``. If the same ``text`` was - the message in 5 or more of the last 8 calls to ``msg``, ``'...'`` will - be sent instead. If this condition is met, and ``'...'`` is more than 3 - of the last 8 calls, no message will be sent. This is intended to prevent - Sopel from being caught in an infinite loop with another bot, or being - used to spam. - - If ``max_messages`` argument is optional, and defaults to 1. The - message will be split into that number of segments. Each segment will - be 400 bytes long or less (bearing in mind that messages are UTF-8 - encoded). The message will be split at the last space before the 400th - byte, or at the 400th byte if no such space exists. The remainder will - be split in the same manner until either the given number of segments - is reached or the remainder is less than 400 bytes. - - If the message is too long to fit into the given number of segments (or - if no number is given), the bot will send as many bytes to the server - as it can. The server, due to the structure of the protocol, will - likely truncate the message further, to a length that is not - determinable by Sopel (though you can generally rely on 400 bytes - making it through). - - Note that when a message is split not on a space but on a byte number, - no attention is given to Unicode character boundaries, and no other - word boundaries besides space will be split upon. This will not cause - problems in the vast majority of cases. - -.. py:function:: debug(tag, text, level) - - *Availability: 3+* - - Send ``text`` to Sopel's configured ``debug_target``. This can be either - an IRC channel (starting with ``#``) or ``stdio``. Suppress the message - if the given ``level`` is lower than Sopel's configured ``verbose`` - setting. Acceptable values for ``level`` are ``'verbose'`` (only send if - Sopel is in verbose mode), ``'warning'`` (send if Sopel is in verbose - or warning mode), ``always`` (send debug message regardless of the configured debug level). - Returns True if the message is sent or printed, and False if it - is not. - - If ``debug_target`` is a channel, the same behavior regarding loop - detection and length restrictions apply to ``debug`` as to ``msg``. - -.. py:function:: add_op(channel, name) -.. py:function:: add_halfop(channel, name) - - *Availability: 3+, deprecated in 4.1.0* - - Add ``name`` to ``channel``'s entry in the ``ops`` or ``halfplus`` - dictionaries, respectively. - -.. py:function:: del_op(channel, name) -.. py:function:: del_halfop(channel, name) - - *Availability: 3+, deprecated in 4.1.0* - - Remove ``name`` from ``channel``'s entry in the ``ops`` or ``halfplus`` - dictionaries, respectively. - -.. py:function:: flush_ops(channel) - - *Availability: 3+, deprecated in 4.1.0* - - Re-initialize and empty the ``ops`` and ``halfops`` entry for - ``channel``. - -.. py:function:: init_ops_list(self, channel) - - *Availability: 3+, deprecated in 4.1.0* - - Create an empty entry in ``ops`` and ``halfops`` for ``channel``. This - will not damage existing entries, but must be done before users can be - added to either dictionary. - -The ``Trigger`` class ---------------------- - -.. autoclass:: sopel.trigger.Trigger - :members: - -Database functionality -====================== - -.. automodule:: sopel.db - :members: - -Configuration functionality -=========================== - -.. automodule:: sopel.config - :members: - :undoc-members: - -Static configuration sections ------------------------------ - -.. automodule:: sopel.config.types - :members: - :undoc-members: - -The [core] configuration section --------------------------------- - -.. autoclass:: sopel.config.core_section.CoreSection - :members: - :undoc-members: - -Miscellaneous: ``web``, ``tools``, ``module``, ``formatting`` -============================================================= - -These provide a number of useful shortcuts for common tasks. - -sopel.web ----------- - -.. automodule:: sopel.web - :members: - -sopel.tools ------------- - -.. automodule:: sopel.tools - :members: - -.. automodule:: sopel.tools.time - :members: - -.. automodule:: sopel.tools.calculation - :members: - -sopel.module +Documentation ------------- -.. automodule:: sopel.module - :members: - -sopel.formatting ------------------ -.. automodule:: sopel.formatting - :members: - :undoc-members: - -Indices and tables -================== -* :ref:`genindex` -* :ref:`modindex` -* :ref:`search` +.. toctree:: + :titlesonly: -.. _re: https://docs.python.org/2/library/re.html + plugin + bot + trigger + config + db + api diff --git a/docs/source/plugin.rst b/docs/source/plugin.rst new file mode 100644 index 0000000000..9f68c61cc8 --- /dev/null +++ b/docs/source/plugin.rst @@ -0,0 +1,73 @@ +Plugin structure +================ + +A Sopel plugin consists of a Python module containing one or more +``callable``\s. It may optionally also contain ``configure``, ``setup``, and +``shutdown`` hooks. + +.. py:method:: callable(bot, trigger) + + A callable is any function which takes as its arguments a + :class:`sopel.bot.Sopel` object and a :class:`sopel.trigger.Trigger` + object, and is wrapped with appropriate decorators from + :mod:`sopel.module`. The ``bot`` provides the ability to send messages to + the network and check the state of the bot. The ``trigger`` provides + information about the line which triggered this function to be called. + + The return value of these function is ignored, unless it is + :const:`sopel.module.NOLIMIT`, in which case rate limiting will not be + applied for that call. + + Note that the name can, and should, be anything - it doesn't need to be + called "callable". + +.. py:method:: setup(bot) + + This is an optional function of a plugin, which will be called while the + module is being loaded. The purpose of this function is to perform whatever + actions are needed to allow a module to function properly (e.g, ensuring + that the appropriate configuration variables exist and are set). Note that + this normally occurs prior to connection to the server, so the behavior of + the messaging functions on the :class:`sopel.bot.Sopel` object it's passed + is undefined. + + Throwing an exception from this function (such as a + :exc:`sopel.config.ConfigurationError`) will prevent any callables in the + module from being registered, and provide an error message to the user. + This is useful when requiring the presence of configuration values or + making other environmental requirements. + + The bot will not continue loading modules or connecting during the + execution of this function. As such, an infinite loop (such as an + unthreaded polling loop) will cause the bot to hang. + +.. py:method:: shutdown(bot) + + This is an optional function of a module, which will be called while the + bot is quitting. Note that this normally occurs after closing connection + to the server, so the behavior of the messaging functions on the + :class:`sopel.bot.Sopel` object it's passed is undefined. The purpose of + this function is to perform whatever actions are needed to allow a module + to properly clean up (e.g, ensuring that any temporary cache files are + deleted). + + The bot will not continue notifying other modules or continue quitting + during the execution of this function. As such, an infinite loop (such as + an unthreaded polling loop) will cause the bot to hang. + + .. versionadded:: 4.1 + +.. py:method:: configure(config) + + This is an optional function of a module, which will be called during the + user's setup of the bot. It's intended purpose is to use the methods of the + passed :class:`sopel.config.Config` object in order to create the + configuration variables it needs to function properly. + + .. versionadded:: 3.0 + +sopel.module +------------ +.. automodule:: sopel.module + :members: + diff --git a/docs/source/trigger.rst b/docs/source/trigger.rst new file mode 100644 index 0000000000..98a8845631 --- /dev/null +++ b/docs/source/trigger.rst @@ -0,0 +1,5 @@ +Triggers +======== + +.. autoclass:: sopel.trigger.Trigger + :members: diff --git a/pytest_run.py b/pytest_run.py index 94d6523ab2..c936ce2987 100755 --- a/pytest_run.py +++ b/pytest_run.py @@ -1,5 +1,5 @@ #!/usr/bin/env python -# coding=utf8 +# coding=utf-8 """This is a script for running pytest from the command line. This script exists so that the project directory gets added to sys.path, which @@ -9,7 +9,7 @@ Copyright 2013, Ari Koivula, Licensed under the Eiffel Forum License 2. -http://willie.dfbta.net +http://sopel.chat """ from __future__ import unicode_literals diff --git a/requirements.txt b/requirements.txt index 7501e7e17f..b124b8a2f4 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,6 +1,6 @@ -feedparser xmltodict pytz praw pyenchant pygeoip +requests>=2.0.0,<2.11.0 diff --git a/setup.py b/setup.py index 415de45077..3d084ca3f3 100755 --- a/setup.py +++ b/setup.py @@ -1,5 +1,5 @@ #!/usr/bin/env python -# coding=utf8 +# coding=utf-8 from __future__ import unicode_literals, print_function from sopel import __version__ @@ -38,7 +38,7 @@ def read_reqs(path): name='sopel', version=__version__, description='Simple and extendible IRC bot', - author='Edward Powell', + author='Elsie Powell', author_email='powell.518@gmail.com', url='http://sopel.chat/', long_description=( @@ -52,7 +52,6 @@ def read_reqs(path): str('sopel.config'), str('sopel.tools')], license='Eiffel Forum License, version 2', platforms='Linux x86, x86-64', - requires=requires, install_requires=requires, entry_points={'console_scripts': ['sopel = sopel.run_script:main']}, ) diff --git a/sopel/__init__.py b/sopel/__init__.py index 45eb94bdae..168ec3f0cd 100644 --- a/sopel/__init__.py +++ b/sopel/__init__.py @@ -1,16 +1,26 @@ # coding=utf-8 -""" -__init__.py - Sopel Init Module -Copyright 2008, Sean B. Palmer, inamidst.com -Copyright 2012, Edward Powell, http://embolalia.net -Copyright © 2012, Elad Alfassa +# ASCII ONLY IN THIS FILE THOUGH!!!!!!! +# Python does some stupid bullshit of respecting LC_ALL over the encoding on the +# file, so in order to undo Python's ridiculous fucking idiocy, we have to have +# our own check. -Licensed under the Eiffel Forum License 2. +# Copyright 2008, Sean B. Palmer, inamidst.com +# Copyright 2012, Elsie Powell, http://embolalia.com +# Copyright 2012, Elad Alfassa +# +# Licensed under the Eiffel Forum License 2. + +from __future__ import unicode_literals, absolute_import, print_function, division +import locale +import sys +loc = locale.getlocale() +if sys.version_info.major > 2: + if not loc[1] or 'UTF-8' not in loc[1]: + print('WARNING!!! You are running with a non-UTF8 locale environment ' + 'variables (e.g. LC_ALL is set to "C"), which makes Python 3 do ' + 'stupid things. If you get strange errors, please set it to ' + 'something like "en_US.UTF-8".', file=sys.stderr) -http://sopel.chat/ -""" -from __future__ import unicode_literals -from __future__ import absolute_import from collections import namedtuple import os @@ -19,7 +29,7 @@ import traceback import signal -__version__ = '6.0.0' +__version__ = '6.5.1' def _version_info(version=__version__): @@ -46,7 +56,6 @@ def _version_info(version=__version__): def run(config, pid_file, daemon=False): import sopel.bot as bot - import sopel.web as web import sopel.logger from sopel.tools import stderr delay = 20 @@ -54,7 +63,6 @@ def run(config, pid_file, daemon=False): if not config.core.ca_certs: stderr('Could not open CA certificates file. SSL will not ' 'work properly.') - web.ca_certs = config.core.ca_certs def signal_handler(sig, frame): if sig == signal.SIGUSR1 or sig == signal.SIGTERM: diff --git a/sopel/bot.py b/sopel/bot.py index 49ae4ffa96..fc98dd9ec7 100644 --- a/sopel/bot.py +++ b/sopel/bot.py @@ -1,17 +1,11 @@ # coding=utf-8 -""" -bot.py - Sopel IRC Bot -Copyright 2008, Sean B. Palmer, inamidst.com -Copyright 2012, Edward Powell, http://embolalia.net -Copyright © 2012, Elad Alfassa +# Copyright 2008, Sean B. Palmer, inamidst.com +# Copyright © 2012, Elad Alfassa +# Copyright 2012-2015, Elsie Powell, http://embolalia.com +# +# Licensed under the Eiffel Forum License 2. -Licensed under the Eiffel Forum License 2. - -http://sopel.chat/ -""" -from __future__ import unicode_literals -from __future__ import print_function -from __future__ import absolute_import +from __future__ import unicode_literals, absolute_import, print_function, division import collections import os @@ -41,6 +35,18 @@ py3 = False +class _CapReq(object): + def __init__(self, prefix, module, failure=None, arg=None, success=None): + def nop(bot, cap): + pass + # TODO at some point, reorder those args to be sane + self.prefix = prefix + self.module = module + self.arg = arg + self.failure = failure or nop + self.success = success or nop + + class Sopel(irc.Bot): def __init__(self, config, daemon=False): irc.Bot.__init__(self, config) @@ -54,7 +60,7 @@ def __init__(self, config, daemon=False): 'low': collections.defaultdict(list) } self.config = config - """The ``Config`` for the current Sopel instance.""" + """The :class:`sopel.config.Config` for the current Sopel instance.""" self.doc = {} """ A dictionary of command names to their docstring and example, if @@ -62,19 +68,14 @@ def __init__(self, config, daemon=False): key in version *3.2* onward. Prior to *3.2*, the name of the function as declared in the source code was used. """ - self.command_groups = collections.defaultdict(list) + self._command_groups = collections.defaultdict(list) """A mapping of module names to a list of commands in it.""" - self.stats = {} - """ - A dictionary which maps a tuple of a function name and where it was - used to the nuber of times it was used there. - """ - self.times = {} + self.stats = {} # deprecated, remove in 7.0 + self._times = {} """ A dictionary mapping lower-case'd nicks to dictionaries which map funtion names to the time which they were last used by that nick. """ - self.acivity = {} self.server_capabilities = {} """A dict mapping supported IRCv3 capabilities to their options. @@ -87,27 +88,43 @@ def __init__(self, config, daemon=False): self.enabled_capabilities = set() """A set containing the IRCv3 capabilities that the bot has enabled.""" self._cap_reqs = dict() - """A dictionary of capability requests - - Maps the capability name to a list of tuples of the prefix ('-', '=', - or ''), the name of the requesting module, the function to call if the - the request is rejected, and the argument to the capability (or None). - """ + """A dictionary of capability names to a list of requests""" self.privileges = dict() """A dictionary of channels to their users and privilege levels - The value associated with each channel is a dictionary of Identifiers to a - bitwise integer value, determined by combining the appropriate constants - from `module`.""" + The value associated with each channel is a dictionary of + :class:`sopel.tools.Identifier`\s to + a bitwise integer value, determined by combining the appropriate + constants from :mod:`sopel.module`. + + .. deprecated:: 6.2.0 + Use :attr:`channels` instead. + """ + + self.channels = tools.SopelMemory() # name to chan obj + """A map of the channels that Sopel is in. + + The keys are Identifiers of the channel names, and map to + :class:`sopel.tools.target.Channel` objects which contain the users in + the channel and their permissions. + """ + self.users = tools.SopelMemory() # name to user obj + """A map of the users that Sopel is aware of. + + The keys are Identifiers of the nicknames, and map to + :class:`sopel.tools.target.User` instances. In order for Sopel to be + aware of a user, it must be in at least one channel which they are also + in. + """ self.db = SopelDB(config) - """The bot's database.""" + """The bot's database, as a :class:`sopel.db.SopelDB` instance.""" self.memory = tools.SopelMemory() """ A thread-safe dict for storage of runtime data to be shared between - modules. See `SopelMemory <#tools.Sopel.SopelMemory>`_ + modules. See :class:`sopel.tools.Sopel.SopelMemory` """ self.scheduler = sopel.tools.jobs.JobScheduler(self) @@ -117,10 +134,34 @@ def __init__(self, config, daemon=False): # Default to empty if not self.config.core.nick_blocks: self.config.core.nick_blocks = [] - if not self.config.core.nick_blocks: + if not self.config.core.host_blocks: self.config.core.host_blocks = [] self.setup() + # Backwards-compatibility aliases to attributes made private in 6.2. Remove + # these in 7.0 + times = property(lambda self: getattr(self, '_times')) + command_groups = property(lambda self: getattr(self, '_command_groups')) + + def write(self, args, text=None): # Shim this in here for autodocs + """Send a command to the server. + + ``args`` is an iterable of strings, which are joined by spaces. + ``text`` is treated as though it were the final item in ``args``, but + is preceeded by a ``:``. This is a special case which means that + ``text``, unlike the items in ``args`` may contain spaces (though this + constraint is not checked by ``write``). + + In other words, both ``sopel.write(('PRIVMSG',), 'Hello, world!')`` + and ``sopel.write(('PRIVMSG', ':Hello, world!'))`` will send + ``PRIVMSG :Hello, world!`` to the server. + + Newlines and carriage returns ('\\n' and '\\r') are removed before + sending. Additionally, if the message (after joining) is longer than + than 510 characters, any remaining characters will not be sent. + """ + irc.Bot.write(self, args, text=text) + def setup(self): stderr("\nWelcome to Sopel. Loading modules...\n\n") @@ -158,24 +199,29 @@ def setup(self): self.register(*relevant_parts) success_count += 1 - if len(modules) > 2: # coretasks is counted + if len(modules) > 1: # coretasks is counted stderr('\n\nRegistered %d modules,' % (success_count - 1)) stderr('%d modules failed to load\n\n' % error_count) else: stderr("Warning: Couldn't load any modules") def unregister(self, obj): + if not callable(obj): + return if hasattr(obj, 'rule'): # commands and intents have it added for rule in obj.rule: - self._callables[obj.priority][rule].remove(obj) + callb_list = self._callables[obj.priority][rule] + if obj in callb_list: + callb_list.remove(obj) if hasattr(obj, 'interval'): # TODO this should somehow find the right job to remove, rather than # clearing the entire queue. Issue #831 self.scheduler.clear_jobs() - if getattr(obj, '__name__', None) == 'shutdown': + if (getattr(obj, '__name__', None) == 'shutdown' + and obj in self.shutdown_methods): self.shutdown_methods.remove(obj) - def register(self, callables, jobs, shutdowns): + def register(self, callables, jobs, shutdowns, urls): self.shutdown_methods = shutdowns for callbl in callables: for rule in callbl.rule: @@ -185,7 +231,7 @@ def register(self, callables, jobs, shutdowns): # TODO doc and make decorator for this. Not sure if this is how # it should work yet, so not making it public for 6.0. category = getattr(callbl, 'category', module_name) - self.command_groups[category].append(callbl.commands[0]) + self._command_groups[category].append(callbl.commands[0]) for command, docs in callbl._docs.items(): self.doc[command] = docs for func in jobs: @@ -193,6 +239,149 @@ def register(self, callables, jobs, shutdowns): job = sopel.tools.jobs.Job(interval, func) self.scheduler.add_job(job) + if not self.memory.contains('url_callbacks'): + self.memory['url_callbacks'] = tools.SopelMemory() + for func in urls: + self.memory['url_callbacks'][func.url_regex] = func + + def part(self, channel, msg=None): + """Part a channel.""" + self.write(['PART', channel], msg) + + def join(self, channel, password=None): + """Join a channel + + If `channel` contains a space, and no `password` is given, the space is + assumed to split the argument into the channel to join and its + password. `channel` should not contain a space if `password` is given. + + """ + if password is None: + self.write(('JOIN', channel)) + else: + self.write(['JOIN', channel, password]) + + def msg(self, recipient, text, max_messages=1): + # Deprecated, but way too much of a pain to remove. + self.say(text, recipient, max_messages) + + def say(self, text, recipient, max_messages=1): + """Send ``text`` as a PRIVMSG to ``recipient``. + + In the context of a triggered callable, the ``recipient`` defaults to + the channel (or nickname, if a private message) from which the message + was received. + + By default, this will attempt to send the entire ``text`` in one + message. If the text is too long for the server, it may be truncated. + If ``max_messages`` is given, the ``text`` will be split into at most + that many messages, each no more than 400 bytes. The split is made at + the last space character before the 400th byte, or at the 400th byte if + no such space exists. If the ``text`` is too long to fit into the + specified number of messages using the above splitting, the final + message will contain the entire remainder, which may be truncated by + the server. + """ + # We're arbitrarily saying that the max is 400 bytes of text when + # messages will be split. Otherwise, we'd have to acocunt for the bot's + # hostmask, which is hard. + max_text_length = 400 + # Encode to bytes, for propper length calculation + if isinstance(text, unicode): + encoded_text = text.encode('utf-8') + else: + encoded_text = text + excess = '' + if max_messages > 1 and len(encoded_text) > max_text_length: + last_space = encoded_text.rfind(' '.encode('utf-8'), 0, max_text_length) + if last_space == -1: + excess = encoded_text[max_text_length:] + encoded_text = encoded_text[:max_text_length] + else: + excess = encoded_text[last_space + 1:] + encoded_text = encoded_text[:last_space] + # We'll then send the excess at the end + # Back to unicode again, so we don't screw things up later. + text = encoded_text.decode('utf-8') + try: + self.sending.acquire() + + # No messages within the last 3 seconds? Go ahead! + # Otherwise, wait so it's been at least 0.8 seconds + penalty + + recipient_id = Identifier(recipient) + + if recipient_id not in self.stack: + self.stack[recipient_id] = [] + elif self.stack[recipient_id]: + elapsed = time.time() - self.stack[recipient_id][-1][0] + if elapsed < 3: + penalty = float(max(0, len(text) - 40)) / 70 + wait = 0.8 + penalty + if elapsed < wait: + time.sleep(wait - elapsed) + + # Loop detection + messages = [m[1] for m in self.stack[recipient_id][-8:]] + + # If what we about to send repeated at least 5 times in the + # last 2 minutes, replace with '...' + if messages.count(text) >= 5 and elapsed < 120: + text = '...' + if messages.count('...') >= 3: + # If we said '...' 3 times, discard message + return + + self.write(('PRIVMSG', recipient), text) + self.stack[recipient_id].append((time.time(), self.safe(text))) + self.stack[recipient_id] = self.stack[recipient_id][-10:] + finally: + self.sending.release() + # Now that we've sent the first part, we need to send the rest. Doing + # this recursively seems easier to me than iteratively + if excess: + self.msg(recipient, excess, max_messages - 1) + + def notice(self, text, dest): + """Send an IRC NOTICE to a user or a channel. + + Within the context of a triggered callable, ``dest`` will default to + the channel (or nickname, if a private message), in which the trigger + happened. + """ + self.write(('NOTICE', dest), text) + + def action(self, text, dest): + """Send ``text`` as a CTCP ACTION PRIVMSG to ``dest``. + + The same loop detection and length restrictions apply as with + :func:`say`, though automatic message splitting is not available. + + Within the context of a triggered callable, ``dest`` will default to + the channel (or nickname, if a private message), in which the trigger + happened. + """ + self.say('\001ACTION {}\001'.format(text), dest) + + def reply(self, text, dest, reply_to, notice=False): + """Prepend ``reply_to`` to ``text``, and send as a PRIVMSG to ``dest``. + + If ``notice`` is ``True``, send a NOTICE rather than a PRIVMSG. + + The same loop detection and length restrictions apply as with + :func:`say`, though automatic message splitting is not available. + + Within the context of a triggered callable, ``reply_to`` will default to + the nickname of the user who triggered the call, and ``dest`` to the + channel (or nickname, if a private message), in which the trigger + happened. + """ + text = '%s: %s' % (reply_to, text) + if notice: + self.notice(text, dest) + else: + self.say(text, dest) + class SopelWrapper(object): def __init__(self, sopel, trigger): # The custom __setattr__ for this class sets the attribute on the @@ -236,22 +425,46 @@ def reply(self, message, destination=None, reply_to=None, notice=False): def call(self, func, sopel, trigger): nick = trigger.nick - if nick not in self.times: - self.times[nick] = dict() - - if not trigger.admin and \ - not func.unblockable and \ - func.rate > 0 and \ - func in self.times[nick]: - timediff = time.time() - self.times[nick][func] - if timediff < func.rate: - self.times[nick][func] = time.time() - LOGGER.info( - "%s prevented from using %s in %s: %d < %d", - trigger.nick, func.__name__, trigger.sender, timediff, - func.rate - ) - return + current_time = time.time() + if nick not in self._times: + self._times[nick] = dict() + if self.nick not in self._times: + self._times[self.nick] = dict() + if not trigger.is_privmsg and trigger.sender not in self._times: + self._times[trigger.sender] = dict() + + if not trigger.admin and not func.unblockable: + if func in self._times[nick]: + usertimediff = current_time - self._times[nick][func] + if func.rate > 0 and usertimediff < func.rate: + #self._times[nick][func] = current_time + LOGGER.info( + "%s prevented from using %s in %s due to user limit: %d < %d", + trigger.nick, func.__name__, trigger.sender, usertimediff, + func.rate + ) + return + if func in self._times[self.nick]: + globaltimediff = current_time - self._times[self.nick][func] + if func.global_rate > 0 and globaltimediff < func.global_rate: + #self._times[self.nick][func] = current_time + LOGGER.info( + "%s prevented from using %s in %s due to global limit: %d < %d", + trigger.nick, func.__name__, trigger.sender, globaltimediff, + func.global_rate + ) + return + + if not trigger.is_privmsg and func in self._times[trigger.sender]: + chantimediff = current_time - self._times[trigger.sender][func] + if func.channel_rate > 0 and chantimediff < func.channel_rate: + #self._times[trigger.sender][func] = current_time + LOGGER.info( + "%s prevented from using %s in %s due to channel limit: %d < %d", + trigger.nick, func.__name__, trigger.sender, chantimediff, + func.channel_rate + ) + return try: exit_code = func(sopel, trigger) @@ -260,11 +473,14 @@ def call(self, func, sopel, trigger): self.error(trigger) if exit_code != NOLIMIT: - self.times[nick][func] = time.time() + self._times[nick][func] = current_time + self._times[self.nick][func] = current_time + if not trigger.is_privmsg: + self._times[trigger.sender][func] = current_time def dispatch(self, pretrigger): args = pretrigger.args - event, args, text = pretrigger.event, args, args[-1] + event, args, text = pretrigger.event, args, args[-1] if args else '' if self.config.core.nick_blocks or self.config.core.host_blocks: nick_blocked = self._nick_blocked(pretrigger.nick) @@ -280,7 +496,9 @@ def dispatch(self, pretrigger): match = regexp.match(text) if not match: continue - trigger = Trigger(self.config, pretrigger, match) + user_obj = self.users.get(pretrigger.nick) + account = user_obj.account if user_obj else None + trigger = Trigger(self.config, pretrigger, match, account) wrapper = self.SopelWrapper(self, trigger) for func in funcs: @@ -361,7 +579,8 @@ def _shutdown(self): ) ) - def cap_req(self, module_name, capability, arg=None, failure_callback=None): + def cap_req(self, module_name, capability, arg=None, failure_callback=None, + success_callback=None): """Tell Sopel to request a capability when it starts. By prefixing the capability with `-`, it will be ensured that the @@ -384,7 +603,12 @@ def cap_req(self, module_name, capability, arg=None, failure_callback=None): request, the `failure_callback` function will be called, if provided. The arguments will be a `Sopel` object, and the capability which was rejected. This can be used to disable callables which rely on the - capability. In future versions + capability. It will be be called either if the server NAKs the request, + or if the server enabled it and later DELs it. + + The `success_callback` function will be called upon acknowledgement of + the capability from the server, whether during the initial capability + negotiation, or later. If ``arg`` is given, and does not exactly match what the server provides or what other modules have requested for that capability, it is @@ -395,16 +619,17 @@ def cap_req(self, module_name, capability, arg=None, failure_callback=None): prefix = capability[0] entry = self._cap_reqs.get(cap, []) - if any((ent[3] != arg for ent in entry)): + if any((ent.arg != arg for ent in entry)): raise Exception('Capability conflict') if prefix == '-': if self.connection_registered and cap in self.enabled_capabilities: raise Exception('Can not change capabilities after server ' 'connection has been completed.') - if any((ent[0] != '-' for ent in entry)): + if any((ent.prefix != '-' for ent in entry)): raise Exception('Capability conflict') - entry.append((prefix, module_name, failure_callback, arg)) + entry.append(_CapReq(prefix, module_name, failure_callback, arg, + success_callback)) self._cap_reqs[cap] = entry else: if prefix != '=': @@ -416,7 +641,8 @@ def cap_req(self, module_name, capability, arg=None, failure_callback=None): 'connection has been completed.') # Non-mandatory will callback at the same time as if the server # rejected it. - if any((ent[0] == '-' for ent in entry)) and prefix == '=': + if any((ent.prefix == '-' for ent in entry)) and prefix == '=': raise Exception('Capability conflict') - entry.append((prefix, module_name, failure_callback, arg)) + entry.append(_CapReq(prefix, module_name, failure_callback, arg, + success_callback)) self._cap_reqs[cap] = entry diff --git a/sopel/config/__init__.py b/sopel/config/__init__.py index 015909bf29..f930365ce4 100644 --- a/sopel/config/__init__.py +++ b/sopel/config/__init__.py @@ -1,7 +1,5 @@ -# coding=utf8 +# coding=utf-8 """ -*Availability: 3+; 6+ for configuration section definitions.* - The config object provides a simplified to access Sopel's configuration file. The sections of the file are attributes of the object, and the keys in the section are attributes of that. So, for example, the ``eggs`` attribute in the @@ -12,14 +10,14 @@ defined keys will be available. A section can not be given more than one definition. The ``[core]`` section is defined with ``CoreSection`` when the object is initialized. + +.. versionadded:: 6.0.0 """ -# Copyright 2012-2015, Edward Powell, embolalia.net +# Copyright 2012-2015, Elsie Powell, embolalia.com # Copyright © 2012, Elad Alfassa # Licensed under the Eiffel Forum License 2. -from __future__ import unicode_literals -from __future__ import print_function -from __future__ import absolute_import +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.tools import iteritems, stderr import sopel.tools @@ -27,9 +25,10 @@ import sopel.loader import os import sys -try: +if sys.version_info.major < 3: import ConfigParser -except ImportError: +else: + basestring = str import configparser as ConfigParser import sopel.config.core_section from sopel.config.types import StaticSection @@ -71,10 +70,8 @@ def homedir(self): # Technically it's the other way around, so we can bootstrap filename # attributes in the core section, but whatever. configured = None - try: + if self.parser.has_option('core', 'homedir'): configured = self.parser.get('core', 'homedir') - except ConfigParser.NoOptionError: - pass if configured: return configured else: diff --git a/sopel/config/core_section.py b/sopel/config/core_section.py index 09348c3e38..844f054f7b 100644 --- a/sopel/config/core_section.py +++ b/sopel/config/core_section.py @@ -1,7 +1,6 @@ -# coding=utf8 +# coding=utf-8 -from __future__ import unicode_literals -from __future__ import print_function +from __future__ import unicode_literals, absolute_import, print_function, division import os.path @@ -44,11 +43,17 @@ class CoreSection(StaticSection): admins = ListAttribute('admins') """The list of people (other than the owner) who can administer the bot""" + admin_accounts = ListAttribute('admin_accounts') + """The list of accounts (other than the owner's) who can administer the bot. + + This should not be set for networks that do not support IRCv3 account + capabilities.""" + auth_method = ChoiceAttribute('auth_method', choices=[ - 'nickserv', 'authserv', 'sasl', 'server']) + 'nickserv', 'authserv', 'Q', 'sasl', 'server']) """The method to use to authenticate with the server. - Can be ``nickserv``, ``authserv``, ``sasl``, or ``server``.""" + Can be ``nickserv``, ``authserv``, ``Q``, ``sasl``, or ``server``.""" auth_password = ValidatedAttribute('auth_password') """The password to use to authenticate with the server.""" @@ -151,6 +156,13 @@ def homedir(self): owner = ValidatedAttribute('owner', default=NO_DEFAULT) """The IRC name of the owner of the bot.""" + owner_account = ValidatedAttribute('owner_account') + """The services account name of the owner of the bot. + + This should only be set on networks which support IRCv3 account + capabilities. + """ + pid_dir = FilenameAttribute('pid_dir', directory=True, default='.') """The directory in which to put the file Sopel uses to track its process ID. @@ -166,6 +178,9 @@ def homedir(self): It is a regular expression (so the default, ``\.``, means commands start with a period), though using capturing groups will create problems.""" + reply_errors = ValidatedAttribute('reply_errors', bool, default=True) + """Whether to message the sender of a message that triggered an error with the exception.""" + throttle_join = ValidatedAttribute('throttle_join', int) """Slow down the initial join of channels to prevent getting kicked. diff --git a/sopel/config/types.py b/sopel/config/types.py index e94eccd16b..55d5167886 100644 --- a/sopel/config/types.py +++ b/sopel/config/types.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """Types for creating section definitions. A section definition consists of a subclass of ``StaticSection``, on which any @@ -24,7 +24,7 @@ ValueError: ListAttribute value must be a list. """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import os.path import sys from sopel.tools import get_input @@ -46,7 +46,7 @@ class NO_DEFAULT(object): class StaticSection(object): """A configuration section with parsed and validated settings. - This class is intended to be subclassed with added ``ValidatedAttribute``s. + This class is intended to be subclassed with added ``ValidatedAttribute``\s. """ def __init__(self, config, section_name, validate=True): if not config.parser.has_section(section_name): @@ -60,7 +60,7 @@ def __init__(self, config, section_name, validate=True): except ValueError as e: raise ValueError( 'Invalid value for {}.{}: {}'.format(section_name, value, - e.message) + str(e)) ) except AttributeError: if validate: @@ -92,7 +92,7 @@ def configure_setting(self, name, prompt, default=NO_DEFAULT): default = clazz.default while True: try: - value = clazz.configure(prompt, default) + value = clazz.configure(prompt, default, self._parent, self._section_name) except ValueError as exc: print(exc) else: @@ -111,7 +111,7 @@ def __init__(self, name, default=None): self.name = name self.default = default - def configure(self, prompt, default): + def configure(self, prompt, default, parent, section_name): """With the prompt and default, parse and return a value from terminal. """ if default is not NO_DEFAULT and default is not None: @@ -143,9 +143,9 @@ def __get__(self, instance, owner=None): # instance here. return self - try: + if instance._parser.has_option(instance._section_name, self.name): value = instance._parser.get(instance._section_name, self.name) - except configparser.NoOptionError: + else: if self.default is not NO_DEFAULT: return self.default raise AttributeError( @@ -202,11 +202,11 @@ def serialize(self, value): def parse(self, value): return value - def configure(self, prompt, default): + def configure(self, prompt, default, parent, section_name): if self.parse == _parse_boolean: prompt += ' (y/n)' default = 'y' if default else 'n' - return super(ValidatedAttribute, self).configure(prompt, default) + return super(ValidatedAttribute, self).configure(prompt, default, parent, section_name) class ListAttribute(BaseValidated): @@ -233,7 +233,7 @@ def serialize(self, value): raise ValueError('ListAttribute value must be a list.') return ','.join(value) - def configure(self, prompt, default): + def configure(self, prompt, default, parent, section_name): each_prompt = '?' if isinstance(prompt, tuple): each_prompt = prompt[1] @@ -290,9 +290,9 @@ def __init__(self, name, relative=True, directory=False, default=None): def __get__(self, instance, owner=None): if instance is None: return self - try: + if instance._parser.has_option(instance._section_name, self.name): value = instance._parser.get(instance._section_name, self.name) - except configparser.NoOptionError: + else: if self.default is not NO_DEFAULT: value = self.default else: @@ -311,6 +311,17 @@ def __set__(self, instance, value): value = self.serialize(main_config, this_section, value) instance._parser.set(instance._section_name, self.name, value) + def configure(self, prompt, default, parent, section_name): + """With the prompt and default, parse and return a value from terminal. + """ + if default is not NO_DEFAULT and default is not None: + prompt = '{} [{}]'.format(prompt, default) + value = get_input(prompt + ' ') + if not value and default is NO_DEFAULT: + raise ValueError("You must provide a value for this option.") + value = value or default + return self.parse(parent, section_name, value) + def parse(self, main_config, this_section, value): if value is None: return diff --git a/sopel/coretasks.py b/sopel/coretasks.py index 2d45ec7f87..d683255e1f 100644 --- a/sopel/coretasks.py +++ b/sopel/coretasks.py @@ -1,27 +1,27 @@ # coding=utf-8 -""" -coretasks.py - Sopel Routine Core tasks -Copyright 2008-2011, Sean B. Palmer (inamidst.com) and Michael Yanovich -(yanovich.net) -Copyright © 2012, Elad Alfassa -Copyright 2012, Edward Powell (embolalia.net) -Licensed under the Eiffel Forum License 2. - -Sopel: http://sopel.chat/ +"""Tasks that allow the bot to run, but aren't user-facing functionality This is written as a module to make it easier to extend to support more responses to standard IRC codes without having to shove them all into the dispatch function in bot.py and making it easier to maintain. """ -from __future__ import unicode_literals +# Copyright 2008-2011, Sean B. Palmer (inamidst.com) and Michael Yanovich +# (yanovich.net) +# Copyright © 2012, Elad Alfassa +# Copyright 2012-2015, Elsie Powell embolalia.com +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division +from random import randint import re import sys import time import sopel import sopel.module -from sopel.tools import Identifier, iteritems +from sopel.bot import _CapReq +from sopel.tools import Identifier, iteritems, events +from sopel.tools.target import User, Channel import base64 from sopel.logger import get_logger @@ -31,6 +31,7 @@ LOGGER = get_logger(__name__) batched_caps = {} +who_reqs = {} # Keeps track of reqs coming from this module, rather than others def auth_after_register(bot): @@ -49,9 +50,17 @@ def auth_after_register(bot): 'AUTHSERV auth', account + ' ' + password )) + + elif bot.config.core.auth_method == 'Q': + account = bot.config.core.auth_username + password = bot.config.core.auth_password + bot.write(( + 'AUTH', + account + ' ' + password + )) -@sopel.module.event('001', '251') +@sopel.module.event(events.RPL_WELCOME, events.RPL_LUSERCLIENT) @sopel.module.rule('.*') @sopel.module.thread(False) @sopel.module.unblockable @@ -90,8 +99,39 @@ def startup(bot, trigger): for channel in bot.config.core.channels: bot.join(channel) + if (not bot.config.core.owner_account and + 'account-tag' in bot.enabled_capabilities and + '@' not in bot.config.core.owner): + msg = ( + "This network supports using network services to identify you as " + "my owner, rather than just matching your nickname. This is much " + "more secure. If you'd like to do this, make sure you're logged in " + "and reply with \"{}useserviceauth\"" + ).format(bot.config.core.help_prefix) + bot.msg(bot.config.core.owner, msg) + + +@sopel.module.require_privmsg() +@sopel.module.require_owner() +@sopel.module.commands('useserviceauth') +def enable_service_auth(bot, trigger): + if bot.config.core.owner_account: + return + if 'account-tag' not in bot.enabled_capabilities: + bot.say('This server does not fully support services auth, so this ' + 'command is not available.') + return + if not trigger.account: + bot.say('You must be logged in to network services before using this ' + 'command.') + return + bot.config.core.owner_account = trigger.account + bot.config.save() + bot.say('Success! I will now use network services to identify you as my ' + 'owner.') + -@sopel.module.event('477') +@sopel.module.event(events.ERR_NOCHANMODES) @sopel.module.rule('.*') @sopel.module.priority('high') def retry_join(bot, trigger): @@ -115,11 +155,9 @@ def retry_join(bot, trigger): time.sleep(6) bot.join(channel) -#Functions to maintain a list of chanops in all of sopel's channels. - @sopel.module.rule('(.*)') -@sopel.module.event('353') +@sopel.module.event(events.RPL_NAMREPLY) @sopel.module.priority('high') @sopel.module.thread(False) @sopel.module.unblockable @@ -234,6 +272,11 @@ def track_nicks(bot, trigger): value = bot.privileges[channel].pop(old) bot.privileges[channel][new] = value + for channel in bot.channels.values(): + channel.rename_user(old, new) + if old in bot.users: + bot.users[new] = bot.users.pop(old) + @sopel.module.rule('(.*)') @sopel.module.event('PART') @@ -241,14 +284,9 @@ def track_nicks(bot, trigger): @sopel.module.thread(False) @sopel.module.unblockable def track_part(bot, trigger): - if trigger.nick == bot.nick: - bot.channels.remove(trigger.sender) - del bot.privileges[trigger.sender] - else: - try: - del bot.privileges[trigger.sender][trigger.nick] - except KeyError: - pass + nick = trigger.nick + channel = trigger.sender + _remove_from_channel(bot, nick, channel) @sopel.module.rule('.*') @@ -258,17 +296,55 @@ def track_part(bot, trigger): @sopel.module.unblockable def track_kick(bot, trigger): nick = Identifier(trigger.args[1]) + channel = trigger.sender + _remove_from_channel(bot, nick, channel) + + +def _remove_from_channel(bot, nick, channel): if nick == bot.nick: - bot.channels.remove(trigger.sender) - del bot.privileges[trigger.sender] + bot.privileges.pop(channel, None) + bot.channels.pop(channel, None) + + lost_users = [] + for nick_, user in bot.users.items(): + user.channels.pop(channel, None) + if not user.channels: + lost_users.append(nick_) + for nick_ in lost_users: + bot.users.pop(nick_, None) else: - # Temporary fix to stop KeyErrors from being sent to channel - # The privileges dict may not have all nicks stored at all times - # causing KeyErrors - try: - del bot.privileges[trigger.sender][nick] - except KeyError: - pass + bot.privileges[channel].pop(nick, None) + + user = bot.users.get(nick) + if user and channel in user.channels: + bot.channels[channel].clear_user(nick) + if not user.channels: + bot.users.pop(nick, None) + + +def _whox_enabled(bot): + # Either privilege tracking or away notification. For simplicity, both + # account notify and extended join must be there for account tracking. + return (('account-notify' in bot.enabled_capabilities and + 'extended-join' in bot.enabled_capabilities) or + 'away-notify' in bot.enabled_capabilities) + + +def _send_who(bot, channel): + if _whox_enabled(bot): + # WHOX syntax, see http://faerion.sourceforge.net/doc/irc/whox.var + # Needed for accounts in who replies. The random integer is a param + # to identify the reply as one from this command, because if someone + # else sent it, we have no fucking way to know what the format is. + rand = str(randint(0, 999)) + while rand in who_reqs: + rand = str(randint(0, 999)) + who_reqs[rand] = channel + bot.write(['WHO', channel, 'a%nuachtf,' + rand]) + else: + # We might be on an old network, but we still care about keeping our + # user list updated + bot.write(['WHO', channel]) @sopel.module.rule('.*') @@ -278,10 +354,25 @@ def track_kick(bot, trigger): @sopel.module.unblockable def track_join(bot, trigger): if trigger.nick == bot.nick and trigger.sender not in bot.channels: - bot.channels.append(trigger.sender) + bot.write(('TOPIC', trigger.sender)) + bot.privileges[trigger.sender] = dict() + bot.channels[trigger.sender] = Channel(trigger.sender) + _send_who(bot, trigger.sender) + bot.privileges[trigger.sender][trigger.nick] = 0 + user = bot.users.get(trigger.nick) + if user is None: + user = User(trigger.nick, trigger.user, trigger.host) + bot.users[trigger.nick] = user + bot.channels[trigger.sender].add_user(user) + + if len(trigger.args) > 1 and trigger.args[1] != '*' and ( + 'account-notify' in bot.enabled_capabilities and + 'extended-join' in bot.enabled_capabilities): + user.account = trigger.args[1] + @sopel.module.rule('.*') @sopel.module.event('QUIT') @@ -290,8 +381,10 @@ def track_join(bot, trigger): @sopel.module.unblockable def track_quit(bot, trigger): for chanprivs in bot.privileges.values(): - if trigger.nick in chanprivs: - del chanprivs[trigger.nick] + chanprivs.pop(trigger.nick, None) + for channel in bot.channels.values(): + channel.clear_user(trigger.nick) + bot.users.pop(trigger.nick, None) @sopel.module.rule('.*') @@ -300,24 +393,54 @@ def track_quit(bot, trigger): @sopel.module.priority('high') @sopel.module.unblockable def recieve_cap_list(bot, trigger): + cap = trigger.strip('-=~') # Server is listing capabilites if trigger.args[1] == 'LS': recieve_cap_ls_reply(bot, trigger) # Server denied CAP REQ elif trigger.args[1] == 'NAK': - entry = bot._cap_reqs.get(trigger, None) + entry = bot._cap_reqs.get(cap, None) # If it was requested with bot.cap_req if entry: for req in entry: # And that request was mandatory/prohibit, and a callback was # provided - if req[0] and req[2]: + if req.prefix and req.failure: + # Call it. + req.failure(bot, req.prefix + cap) + # Server is removing a capability + elif trigger.args[1] == 'DEL': + entry = bot._cap_reqs.get(cap, None) + # If it was requested with bot.cap_req + if entry: + for req in entry: + # And that request wasn't prohibit, and a callback was + # provided + if req.prefix != '-' and req.failure: # Call it. - req[2](bot, req[0] + trigger) - # Server is acknowledinge SASL for us. - elif (trigger.args[0] == bot.nick and trigger.args[1] == 'ACK' and - 'sasl' in trigger.args[2]): - recieve_cap_ack_sasl(bot) + req.failure(bot, req.prefix + cap) + # Server is adding new capability + elif trigger.args[1] == 'NEW': + entry = bot._cap_reqs.get(cap, None) + # If it was requested with bot.cap_req + if entry: + for req in entry: + # And that request wasn't prohibit + if req.prefix != '-': + # Request it + bot.write(('CAP', 'REQ', req.prefix + cap)) + # Server is acknowledging a capability + elif trigger.args[1] == 'ACK': + caps = trigger.args[2].split() + for cap in caps: + cap.strip('-~= ') + bot.enabled_capabilities.add(cap) + entry = bot._cap_reqs.get(cap, []) + for req in entry: + if req.success: + req.success(bot, req.prefix + trigger) + if cap == 'sasl': # TODO why is this not done with bot.cap_req? + recieve_cap_ack_sasl(bot) def recieve_cap_ls_reply(bot, trigger): @@ -342,32 +465,46 @@ def recieve_cap_ls_reply(bot, trigger): # If some other module requests it, we don't need to add another request. # If some other module prohibits it, we shouldn't request it. - if 'multi-prefix' not in bot._cap_reqs: - # Whether or not the server supports multi-prefix doesn't change how we - # parse it, so we don't need to worry if it fails. - bot._cap_reqs['multi-prefix'] = (['', 'coretasks', None, None],) + core_caps = ['multi-prefix', 'away-notify', 'cap-notify', 'server-time'] + for cap in core_caps: + if cap not in bot._cap_reqs: + bot._cap_reqs[cap] = [_CapReq('', 'coretasks')] + + def acct_warn(bot, cap): + LOGGER.info('Server does not support %s, or it conflicts with a custom ' + 'module. User account validation unavailable or limited.', + cap[1:]) + if bot.config.core.owner_account or bot.config.core.admin_accounts: + LOGGER.warning( + 'Owner or admin accounts are configured, but %s is not ' + 'supported by the server. This may cause unexpected behavior.', + cap[1:]) + auth_caps = ['account-notify', 'extended-join', 'account-tag'] + for cap in auth_caps: + if cap not in bot._cap_reqs: + bot._cap_reqs[cap] = [_CapReq('=', 'coretasks', acct_warn)] for cap, reqs in iteritems(bot._cap_reqs): # At this point, we know mandatory and prohibited don't co-exist, but # we need to call back for optionals if they're also prohibited prefix = '' for entry in reqs: - if prefix == '-' and entry[0] != '-': - entry[2](bot, entry[0] + cap) + if prefix == '-' and entry.prefix != '-': + entry.failure(bot, entry.prefix + cap) continue - if entry[0]: - prefix = entry[0] + if entry.prefix: + prefix = entry.prefix # It's not required, or it's supported, so we can request it if prefix != '=' or cap in bot.server_capabilities: # REQs fail as a whole, so we send them one capability at a time - bot.write(('CAP', 'REQ', entry[0] + cap)) + bot.write(('CAP', 'REQ', entry.prefix + cap)) # If it's required but not in server caps, we need to call all the # callbacks else: for entry in reqs: - if entry[2] and entry[0] == '=': - entry[2](bot, entry[0] + cap) + if entry.failure and entry.prefix == '=': + entry.failure(bot, entry.prefix + cap) # If we want to do SASL, we have to wait before we can send CAP END. So if # we are, wait on 903 (SASL successful) to send it. @@ -401,7 +538,7 @@ def auth_proceed(bot, trigger): bot.write(('AUTHENTICATE', base64.b64encode(sasl_token.encode('utf-8')))) -@sopel.module.event('903') +@sopel.module.event(events.RPL_SASLSUCCESS) @sopel.module.rule('.*') def sasl_success(bot, trigger): bot.write(('CAP', 'END')) @@ -493,3 +630,91 @@ def blocks(bot, trigger): return else: bot.reply(STRINGS['huh']) + + +@sopel.module.event('ACCOUNT') +@sopel.module.rule('.*') +def account_notify(bot, trigger): + if trigger.nick not in bot.users: + bot.users[trigger.nick] = User(trigger.nick, trigger.user, trigger.host) + account = trigger.args[0] + if account == '*': + account = None + bot.users[trigger.nick].account = account + + +@sopel.module.event(events.RPL_WHOSPCRPL) +@sopel.module.rule('.*') +@sopel.module.priority('high') +@sopel.module.unblockable +def recv_whox(bot, trigger): + if len(trigger.args) < 2 or trigger.args[1] not in who_reqs: + # Ignored, some module probably called WHO + return + if len(trigger.args) != 8: + return LOGGER.warning('While populating `bot.accounts` a WHO response was malformed.') + _, _, channel, user, host, nick, status, account = trigger.args + away = 'G' in status + _record_who(bot, channel, user, host, nick, account, away) + + +def _record_who(bot, channel, user, host, nick, account=None, away=None): + nick = Identifier(nick) + channel = Identifier(channel) + if nick not in bot.users: + bot.users[nick] = User(nick, user, host) + user = bot.users[nick] + if account == '0': + user.account = None + else: + user.account = account + user.away = away + if channel not in bot.channels: + bot.channels[channel] = Channel(channel) + bot.channels[channel].add_user(user) + + +@sopel.module.event(events.RPL_WHOREPLY) +@sopel.module.rule('.*') +@sopel.module.priority('high') +@sopel.module.unblockable +def recv_who(bot, trigger): + channel, user, host, _, nick, = trigger.args[1:6] + _record_who(bot, channel, user, host, nick) + + +@sopel.module.event(events.RPL_ENDOFWHO) +@sopel.module.rule('.*') +@sopel.module.priority('high') +@sopel.module.unblockable +def end_who(bot, trigger): + if _whox_enabled(bot): + who_reqs.pop(trigger.args[1], None) + + +@sopel.module.rule('.*') +@sopel.module.event('AWAY') +@sopel.module.priority('high') +@sopel.module.thread(False) +@sopel.module.unblockable +def track_notify(bot, trigger): + if trigger.nick not in bot.users: + bot.users[trigger.nick] = User(trigger.nick, trigger.user, trigger.host) + user = bot.users[trigger.nick] + user.away = bool(trigger.args) + + +@sopel.module.rule('.*') +@sopel.module.event('TOPIC') +@sopel.module.event(events.RPL_TOPIC) +@sopel.module.priority('high') +@sopel.module.thread(False) +@sopel.module.unblockable +def track_topic(bot, trigger): + if trigger.event != 'TOPIC': + channel = trigger.args[1] + else: + channel = trigger.args[0] + if channel not in bot.channels: + return + bot.channels[channel].topic = trigger.args[-1] diff --git a/sopel/db.py b/sopel/db.py index bd6602ac47..32fb767fb7 100644 --- a/sopel/db.py +++ b/sopel/db.py @@ -1,5 +1,5 @@ # coding=utf-8 -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import json import os.path @@ -52,7 +52,7 @@ def __init__(self, config): def connect(self): """Return a raw database connection object.""" - return sqlite3.connect(self.filename) + return sqlite3.connect(self.filename, timeout=10) def execute(self, *args, **kwargs): """Execute an arbitrary SQL query against the database. @@ -140,7 +140,7 @@ def alias_nick(self, nick, alias): values = [nick_id, alias.lower(), alias] try: self.execute(sql, values) - except sqlite3.IntegrityError as e: + except sqlite3.IntegrityError: raise ValueError('Alias already exists.') def set_nick_value(self, nick, key, value): @@ -174,7 +174,7 @@ def unalias_nick(self, alias): nick_id = self.get_nick_id(alias, False) count = self.execute('SELECT COUNT(*) FROM nicknames WHERE nick_id = ?', [nick_id]).fetchone()[0] - if count == 0: + if count <= 1: raise ValueError('Given alias is the only entry in its group.') self.execute('DELETE FROM nicknames WHERE slug = ?', [alias.lower()]) diff --git a/sopel/formatting.py b/sopel/formatting.py index 03a8bd51d9..b1733ea761 100644 --- a/sopel/formatting.py +++ b/sopel/formatting.py @@ -1,10 +1,11 @@ # coding=utf-8 -"""*Availability: 4.5+* +"""The formatting module includes functions to apply IRC formatting to text. -The formatting module includes functions to apply IRC formatting to text.""" -# Copyright 2014, Edward D. Powell, embolalia.net +*Availability: 4.5+* +""" +# Copyright 2014, Elsie Powell, embolalia.com # Licensed under the Eiffel Forum License 2. -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import sys if sys.version_info.major >= 3: unicode = str @@ -45,11 +46,14 @@ class colors: PINK = '13' LIGHT_PURPLE = PINK FUCHSIA = PINK - # Nobody has complained that this is grey not gray yet, so go with it? GREY = '14' LIGHT_GREY = '15' SILVER = LIGHT_GREY + #Create aliases. + GRAY = GREY + LIGHT_GRAY = LIGHT_GREY + def _get_color(color): if color is None: diff --git a/sopel/irc.py b/sopel/irc.py index 7d856337ef..d7b04e2e1b 100644 --- a/sopel/irc.py +++ b/sopel/irc.py @@ -1,23 +1,16 @@ # coding=utf-8 -""" -irc.py - An Utility IRC Bot -Copyright 2008, Sean B. Palmer, inamidst.com -Copyright 2012, Edward Powell, http://embolalia.net -Copyright © 2012, Elad Alfassa - -Licensed under the Eiffel Forum License 2. - -Sopel: http://sopel.chat/ - -When working on core IRC protocol related features, consult protocol -documentation at http://www.irchelp.org/irchelp/rfc/ -""" -from __future__ import unicode_literals -from __future__ import print_function -from __future__ import absolute_import +# irc.py - An Utility IRC Bot +# Copyright 2008, Sean B. Palmer, inamidst.com +# Copyright 2012, Elsie Powell, http://embolalia.com +# Copyright © 2012, Elad Alfassa +# +# Licensed under the Eiffel Forum License 2. +# +# When working on core IRC protocol related features, consult protocol +# documentation at http://www.irchelp.org/irchelp/rfc/ +from __future__ import unicode_literals, absolute_import, print_function, division import sys -import re import time import socket import asyncore @@ -27,9 +20,8 @@ import traceback from sopel.logger import get_logger from sopel.tools import stderr, Identifier -from sopel.trigger import PreTrigger, Trigger +from sopel.trigger import PreTrigger try: - import select import ssl if not hasattr(ssl, 'match_hostname'): # Attempt to import ssl_match_hostname from python-backports @@ -66,11 +58,9 @@ def __init__(self, config): self.name = config.core.name """Sopel's "real name", as used for whois.""" - self.channels = [] - """The list of channels Sopel is currently in.""" - self.stack = {} self.ca_certs = ca_certs + self.enabled_capabilities = set() self.hasquit = False self.sending = threading.RLock() @@ -80,20 +70,13 @@ def __init__(self, config): # Right now, only accounting for two op levels. # This might be expanded later. # These lists are filled in startup.py, as of right now. + # Are these even touched at all anymore? Remove in 7.0. self.ops = dict() - """ - A dictionary mapping channels to a ``Identifier`` list of their operators. - """ + """Deprecated. Use bot.channels instead.""" self.halfplus = dict() - """ - A dictionary mapping channels to a ``Identifier`` list of their half-ops and - ops. - """ + """Deprecated. Use bot.channels instead.""" self.voices = dict() - """ - A dictionary mapping channels to a ``Identifier`` list of their voices, - half-ops and ops. - """ + """Deprecated. Use bot.channels instead.""" # We need this to prevent error loops in handle_error self.error_count = 0 @@ -139,23 +122,6 @@ def safe(self, string): return string def write(self, args, text=None): - """Send a command to the server. - - ``args`` is an iterable of strings, which are joined by spaces. - ``text`` is treated as though it were the final item in ``args``, but - is preceeded by a ``:``. This is a special case which means that - ``text``, unlike the items in ``args`` may contain spaces (though this - constraint is not checked by ``write``). - - In other words, both ``sopel.write(('PRIVMSG',), 'Hello, world!')`` - and ``sopel.write(('PRIVMSG', ':Hello, world!'))`` will send - ``PRIVMSG :Hello, world!`` to the server. - - Newlines and carriage returns ('\\n' and '\\r') are removed before - sending. Additionally, if the message (after joining) is longer than - than 510 characters, any remaining characters will not be sent. - - """ args = [self.safe(arg) for arg in args] if text is not None: text = self.safe(text) @@ -189,7 +155,6 @@ def run(self, host, port=6667): self.initiate_connect(host, port) except socket.error as e: stderr('Connection error: %s' % e) - self.hasquit = True def initiate_connect(self, host, port): stderr('Connecting to %s:%s...' % (host, port)) @@ -225,7 +190,8 @@ def quit(self, message): def handle_close(self): self.connection_registered = False - self._shutdown() + if hasattr(self, '_shutdown'): + self._shutdown() stderr('Closed!') # This will eventually call asyncore dispatchers close method, which @@ -233,23 +199,6 @@ def handle_close(self): # race conditions. self.close() - def part(self, channel, msg=None): - """Part a channel.""" - self.write(['PART', channel], msg) - - def join(self, channel, password=None): - """Join a channel - - If `channel` contains a space, and no `password` is given, the space is - assumed to split the argument into the channel to join and its - password. `channel` should not contain a space if `password` is given. - - """ - if password is None: - self.write(('JOIN', channel)) - else: - self.write(['JOIN', channel, password]) - def handle_connect(self): if self.config.core.use_ssl and has_ssl: if not self.config.core.verify_ssl: @@ -284,8 +233,10 @@ def handle_connect(self): stderr('Connected.') self.last_ping_time = datetime.now() timeout_check_thread = threading.Thread(target=self._timeout_check) + timeout_check_thread.daemon = True timeout_check_thread.start() ping_thread = threading.Thread(target=self._send_ping) + ping_thread.daemon = True ping_thread.start() def _timeout_check(self): @@ -328,7 +279,7 @@ def _ssl_recv(self, buffer_size): data = self.socket.read(buffer_size) if not data: self.handle_close() - return '' + return b'' return data except ssl.SSLError as why: if why[0] in (asyncore.ECONNRESET, asyncore.ENOTCONN, @@ -337,7 +288,7 @@ def _ssl_recv(self, buffer_size): return '' elif why[0] == errno.ENOENT: # Required in order to keep it non-blocking - return '' + return b'' else: raise @@ -368,6 +319,8 @@ def found_terminator(self): self.buffer = '' self.last_ping_time = datetime.now() pretrigger = PreTrigger(self.nick, line) + if all(cap not in self.enabled_capabilities for cap in ['account-tag', 'extended-join']): + pretrigger.tags.pop('account', None) if pretrigger.event == 'PING': self.write(('PONG', pretrigger.args[-1])) @@ -384,89 +337,6 @@ def found_terminator(self): def dispatch(self, pretrigger): pass - def msg(self, recipient, text, max_messages=1): - # Deprecated, but way too much of a pain to remove. - self.say(text, recipient, max_messages) - - def say(self, text, recipient, max_messages=1): - # We're arbitrarily saying that the max is 400 bytes of text when - # messages will be split. Otherwise, we'd have to acocunt for the bot's - # hostmask, which is hard. - max_text_length = 400 - # Encode to bytes, for propper length calculation - if isinstance(text, unicode): - encoded_text = text.encode('utf-8') - else: - encoded_text = text - excess = '' - if max_messages > 1 and len(encoded_text) > max_text_length: - last_space = encoded_text.rfind(' '.encode('utf-8'), 0, max_text_length) - if last_space == -1: - excess = encoded_text[max_text_length:] - encoded_text = encoded_text[:max_text_length] - else: - excess = encoded_text[last_space + 1:] - encoded_text = encoded_text[:last_space] - # We'll then send the excess at the end - # Back to unicode again, so we don't screw things up later. - text = encoded_text.decode('utf-8') - try: - self.sending.acquire() - - # No messages within the last 3 seconds? Go ahead! - # Otherwise, wait so it's been at least 0.8 seconds + penalty - - recipient_id = Identifier(recipient) - - if recipient_id not in self.stack: - self.stack[recipient_id] = [] - elif self.stack[recipient_id]: - elapsed = time.time() - self.stack[recipient_id][-1][0] - if elapsed < 3: - penalty = float(max(0, len(text) - 50)) / 70 - wait = 0.7 + penalty - if elapsed < wait: - time.sleep(wait - elapsed) - - # Loop detection - messages = [m[1] for m in self.stack[recipient_id][-8:]] - - # If what we about to send repeated at least 5 times in the - # last 2 minutes, replace with '...' - if messages.count(text) >= 5 and elapsed < 120: - text = '...' - if messages.count('...') >= 3: - # If we said '...' 3 times, discard message - return - - self.write(('PRIVMSG', recipient), text) - self.stack[recipient_id].append((time.time(), self.safe(text))) - self.stack[recipient_id] = self.stack[recipient_id][-10:] - finally: - self.sending.release() - # Now that we've sent the first part, we need to send the rest. Doing - # this recursively seems easier to me than iteratively - if excess: - self.msg(recipient, excess, max_messages - 1) - - def notice(self, text, dest): - """Send an IRC NOTICE to a user or a channel. - - See IRC protocol documentation for more information. - - """ - self.write(('NOTICE', dest), text) - - def action(self, text, dest): - self.say('\001ACTION {}\001'.format(text), dest) - - def reply(self, text, dest, reply_to, notice=False): - text = '%s: %s' % (reply_to, text) - if notice: - self.notice(text, dest) - else: - self.say(text, dest) - def error(self, trigger=None): """Called internally when a module causes an error.""" try: @@ -501,12 +371,15 @@ def error(self, trigger=None): stderr("Could not save full traceback!") LOGGER.error("Could not save traceback from %s to file: %s", trigger.sender, str(e)) - if trigger: + if trigger and self.config.core.reply_errors and trigger.sender is not None: self.msg(trigger.sender, signature) - except Exception as e: if trigger: + LOGGER.error('Exception from {}: {} ({})'.format(trigger.sender, str(signature), trigger.raw)) + except Exception as e: + if trigger and self.config.core.reply_errors and trigger.sender is not None: self.msg(trigger.sender, "Got an error.") - LOGGER.error("Exception from %s: %s", trigger.sender, str(e)) + if trigger: + LOGGER.error('Exception from {}: {} ({})'.format(trigger.sender, str(e), trigger.raw)) def handle_error(self): """Handle any uncaptured error in the core. @@ -532,8 +405,7 @@ def handle_error(self): logfile.close() if self.error_count > 10: if (datetime.now() - self.last_error_timestamp).seconds < 5: - print >> sys.stderr, "Too many errors, can't continue" + stderr("Too many errors, can't continue") os._exit(1) self.last_error_timestamp = datetime.now() self.error_count = self.error_count + 1 - diff --git a/sopel/loader.py b/sopel/loader.py index d1473f2fbe..cae9cc921f 100644 --- a/sopel/loader.py +++ b/sopel/loader.py @@ -1,5 +1,5 @@ # coding=utf-8 -from __future__ import unicode_literals, absolute_import +from __future__ import unicode_literals, absolute_import, print_function, division import imp import os.path @@ -11,6 +11,9 @@ if sys.version_info.major >= 3: basestring = (str, bytes) +# Can be implementation-dependent +_regex_type = type(re.compile('')) + def get_module_description(path): good_file = (os.path.isfile(path) and path.endswith('.py') @@ -108,6 +111,11 @@ def enumerate_modules(config, show_all=False): def compile_rule(nick, pattern): + # Not sure why this happens on reloads, but it shouldn't cause problems… + if isinstance(pattern, _regex_type): + return pattern + + nick = re.escape(nick) pattern = pattern.replace('$nickname', nick) pattern = pattern.replace('$nick', r'{}[,:]\s+'.format(nick)) flags = re.IGNORECASE @@ -151,6 +159,8 @@ def clean_callable(func, config): func.priority = getattr(func, 'priority', 'medium') func.thread = getattr(func, 'thread', True) func.rate = getattr(func, 'rate', 0) + func.channel_rate = getattr(func, 'channel_rate', 0) + func.global_rate = getattr(func, 'global_rate', 0) if not hasattr(func, 'event'): func.event = ['PRIVMSG'] @@ -173,7 +183,7 @@ def clean_callable(func, config): if hasattr(func, 'example'): example = func.example[0]["example"] example = example.replace('$nickname', nick) - if example[0] != help_prefix: + if example[0] != help_prefix and not example.startswith(nick): example = help_prefix + example[len(help_prefix):] if doc or example: for command in func.commands: @@ -193,14 +203,14 @@ def load_module(name, path, type_): def is_triggerable(obj): - return any(hasattr(obj, attr) for attr in ('rule', 'rule', 'intent', - 'commands')) + return any(hasattr(obj, attr) for attr in ('rule', 'intent', 'commands')) def clean_module(module, config): callables = [] shutdowns = [] jobs = [] + urls = [] for obj in itervalues(vars(module)): if callable(obj): if getattr(obj, '__name__', None) == 'shutdown': @@ -211,4 +221,6 @@ def clean_module(module, config): elif hasattr(obj, 'interval'): clean_callable(obj, config) jobs.append(obj) - return callables, jobs, shutdowns + elif hasattr(obj, 'url_regex'): + urls.append(obj) + return callables, jobs, shutdowns, urls diff --git a/sopel/logger.py b/sopel/logger.py index e4e1f290a5..84ba1495ff 100644 --- a/sopel/logger.py +++ b/sopel/logger.py @@ -1,5 +1,5 @@ # coding=utf-8 -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import logging diff --git a/sopel/module.py b/sopel/module.py index 2c58a21be8..73d61a1d98 100644 --- a/sopel/module.py +++ b/sopel/module.py @@ -1,34 +1,25 @@ # coding=utf-8 -"""This module is meant to be imported from sopel modules. - -It defines the following decorators for defining sopel callables: -sopel.module.rule -sopel.module.thread -sopel.module.commands -sopel.module.nickname_commands -sopel.module.priority -sopel.module.event -sopel.module.rate -sopel.module.example +"""This contains decorators and tools for creating callable plugin functions. """ -#Copyright 2013, Ari Koivula, -#Copyright © 2013, Elad Alfassa -#Copyright 2013, Lior Ramati -#Licensed under the Eiffel Forum License 2. +# Copyright 2013, Ari Koivula, +# Copyright © 2013, Elad Alfassa +# Copyright 2013, Lior Ramati +# Licensed under the Eiffel Forum License 2. -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division +import re import sopel.test_tools import functools NOLIMIT = 1 """Return value for ``callable``\s, which supresses rate limiting for the call. -*Avalability: 4.0+; available as ``Sopel.NOLIMIT`` in 3.2* - Returning this value means the triggering user will not be prevented from triggering the command again within the rate limit. This can be used, for example, to allow a user to rety a failed command immediately. + +.. versionadded:: 4.0 """ VOICE = 1 @@ -39,28 +30,25 @@ def unblockable(function): - """Decorator. Equivalent to func.unblockable = True. - - If this decorator is used, the function will be called, even if the bot has - been configured to ignore commands from the user. This can be used to - ensure events such as JOIN are always recorded. + """Decorator which exempts the function from nickname and hostname blocking. + This can be used to ensure events such as JOIN are always recorded. """ function.unblockable = True return function def interval(*args): - """Decorator. Equivalent to func.interval.append(value). + """Decorates a function to be called by the bot every X seconds. - A function that uses this decorator will be called every X seconds, where X - is the argument. This decorator can be used multiple times for multiple - intervals, or all intervals can be given at once as arguments. The first - time the function will be called is X seconds after the bot was started. + This decorator can be used multiple times for multiple intervals, or all + intervals can be given at once as arguments. The first time the function + will be called is X seconds after the bot was started. - For the callable, the first argument will be the bot itself, but it will - not have the say, reply or action methods as would be the case when called - due to rule or command. + Unlike other plugin functions, ones decorated by interval must only take a + :class:`sopel.bot.Sopel` as their argument; they do not get a trigger. The + bot argument will not have a context, so functions like ``bot.say()`` will + not have a default destination. There is no guarantee that the bot is connected to a server or joined a channel when the function is called, so care must be taken. @@ -85,7 +73,7 @@ def add_attribute(function): def rule(value): - """Decorator. Equivalent to func.rule.append(value). + """Decorate a function to be called when a line matches the given pattern This decorator can be used multiple times to add more rules. @@ -110,7 +98,12 @@ def add_attribute(function): def thread(value): - """Decorator. Equivalent to func.thread = value. + """Decorate a function to specify if it should be run in a separate thread. + + Functions run in a separate thread (as is the default) will not prevent the + bot from executing other functions at the same time. Functions not run in a + separate thread may be started while other functions are still running, but + additional functions will not start until it is completed. Args: value: Either True or False. If True the function is called in @@ -124,7 +117,7 @@ def add_attribute(function): def commands(*command_list): - """Decorator. Sets a command list for a callable. + """Decorate a function to set one or more commands to trigger it. This decorator can be used to add multiple commands to one callable in a single line. The resulting match object will have the command as the first @@ -139,7 +132,7 @@ def commands(*command_list): attribute. If there is no commands attribute, it is added. Example: - @command("hello"): + @commands("hello"): If the command prefix is "\.", this would trigger on lines starting with ".hello". @@ -157,7 +150,7 @@ def add_attribute(function): def nickname_commands(*command_list): - """Decorator. Triggers on lines starting with "$nickname: command". + """Decorate a function to trigger on lines starting with "$nickname: command". This decorator can be used multiple times to add multiple rules. The resulting match object will have the command as the first group, rest of @@ -207,7 +200,7 @@ def add_attribute(function): def priority(value): - """Decorator. Equivalent to func.priority = value. + """Decorate a function to be executed with higher or lower priority. Args: value: Priority can be one of "high", "medium", "low". Defaults to @@ -224,7 +217,7 @@ def add_attribute(function): def event(*event_list): - """Decorator. Equivalent to func.event = value. + """Decorate a function to be triggered on specific IRC events. This is one of a number of events, such as 'JOIN', 'PART', 'QUIT', etc. (More details can be found in RFC 1459.) When the Sopel bot is sent one of @@ -232,6 +225,8 @@ def event(*event_list): must also be given a rule to match (though it may be '.*', which will always match) or they will not be triggered. + :class:`sopel.tools.events` provides human-readable names for many of the + numeric events, which may help your code be clearer. """ def add_attribute(function): if not hasattr(function, "event"): @@ -242,9 +237,9 @@ def add_attribute(function): def intent(*intent_list): - """Make a callable trigger on a message with any of the given intents. + """Decorate a callable trigger on a message with any of the given intents. - *Availability: 5.2.0+* + .. versionadded:: 5.2.0 """ def add_attribute(function): if not hasattr(function, "intents"): @@ -254,32 +249,29 @@ def add_attribute(function): return add_attribute -def rate(value): - """Decorator. Equivalent to func.rate = value. - - Availability: 2+ - - This limits the frequency with which a single user may use the function. If - a function is given a rate of 20, a single user may only use that function - once every 20 seconds. This limit applies to each user individually. Users - on the admin list in Sopel’s configuration are exempted from rate limits. +def rate(user=0, channel=0, server=0): + """Decorate a function to limit how often it can be triggered on a per-user + basis, in a channel, or across the server (bot). A value of zero means no + limit. If a function is given a rate of 20, that function may only be used + once every 20 seconds in the scope corresponding to the parameter. + Users on the admin list in Sopel’s configuration are exempted from rate + limits. Rate-limited functions that use scheduled future commands should import threading.Timer() instead of sched, or rate limiting will not work properly. - """ def add_attribute(function): - function.rate = value + function.rate = user + function.channel_rate = channel + function.global_rate = server return function return add_attribute def require_privmsg(message=None): - """ - Decorator, this allows functions to specify if they should be only - allowed via private message. + """Decorate a function to only be triggerable from a private message. - If it is not, `message` will be said if given. + If it is triggered in a channel message, `message` will be said if given. """ def actual_decorator(function): @functools.wraps(function) @@ -299,11 +291,9 @@ def _nop(*args, **kwargs): def require_chanmsg(message=None): - """ - Decorator, this allows functions to specify if they should be only - allowed via channel message. + """Decorate a function to only be triggerable from a channel message. - If it is not, `message` will be said if given. + If it is triggered in a private message, `message` will be said if given. """ def actual_decorator(function): @functools.wraps(function) @@ -323,8 +313,7 @@ def _nop(*args, **kwargs): def require_privilege(level, message=None): - """Decorator. Require at lesat the given channel privilege level to execute - the function. + """Decorate a function to require at least the given channel permission. `level` can be one of the privilege levels defined in this module. If the user does not have the privilege, `message` will be said if given. If it is @@ -332,6 +321,9 @@ def require_privilege(level, message=None): def actual_decorator(function): @functools.wraps(function) def guarded(bot, trigger, *args, **kwargs): + # If this is a privmsg, ignore privilege requirements + if trigger.is_privmsg: + return function(bot, trigger, *args, **kwargs) channel_privs = bot.privileges[trigger.sender] allowed = channel_privs.get(trigger.nick, 0) >= level if not trigger.is_privmsg and not allowed: @@ -344,7 +336,7 @@ def guarded(bot, trigger, *args, **kwargs): def require_admin(message=None): - """Decorator. Require the user triggering the message to be a bot admin. + """Decorate a function to require the triggering user to be a bot admin. If they are not, `message` will be said if given.""" def actual_decorator(function): @@ -363,7 +355,7 @@ def guarded(bot, trigger, *args, **kwargs): def require_owner(message=None): - """Decorator. Require the user triggering the message to be the bot owner. + """Decorate a function to require the triggering user to be the bot owner. If they are not, `message` will be said if given.""" def actual_decorator(function): @@ -381,13 +373,31 @@ def guarded(bot, trigger, *args, **kwargs): return actual_decorator +def url(url_rule): + """Decorate a function to handle URLs. + + This decorator takes a regex string that will be matched against URLs in a + message. The function it decorates, in addition to the bot and trigger, + must take a third argument ``match``, which is the regular expression match + of the url. This should be used rather than the matching in trigger, in + order to support e.g. the ``.title`` command. + """ + def actual_decorator(function): + @functools.wraps(function) + def helper(bot, trigger, match=None): + match = match or trigger + return function(bot, trigger, match) + helper.url_regex = re.compile(url_rule) + return helper + return actual_decorator + + class example(object): - """Decorator. Add an example. + """Decorate a function with an example. Add an example attribute into a function and generate a test. - """ - + # TODO dat doc doe >_< def __init__(self, msg, result=None, privmsg=False, admin=False, owner=False, repeat=1, re=False, ignore=None): """Accepts arguments for the decorator. diff --git a/sopel/modules/__init__.py b/sopel/modules/__init__.py index 127fb3de4d..b6f85b6e5e 100644 --- a/sopel/modules/__init__.py +++ b/sopel/modules/__init__.py @@ -1,2 +1,2 @@ -# coding=utf8 -from __future__ import unicode_literals +# coding=utf-8 +from __future__ import unicode_literals, absolute_import, print_function, division diff --git a/sopel/modules/admin.py b/sopel/modules/admin.py index 83256f416f..14e84aab83 100644 --- a/sopel/modules/admin.py +++ b/sopel/modules/admin.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ admin.py - Sopel Admin Module Copyright 2010-2011, Sean B. Palmer (inamidst.com) and Michael Yanovich @@ -10,7 +10,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.config.types import ( StaticSection, ValidatedAttribute, FilenameAttribute @@ -213,7 +213,7 @@ def set_config(bot, trigger): if isinstance(descriptor, FilenameAttribute): value = descriptor.parse(bot.config, descriptor, value) else: - value = descriptor.parse(descriptor, value) + value = descriptor.parse(value) except ValueError as exc: bot.say("Can't set attribute: " + str(exc)) return diff --git a/sopel/modules/adminchannel.py b/sopel/modules/adminchannel.py index 83a4fb68d8..e3fe10f441 100644 --- a/sopel/modules/adminchannel.py +++ b/sopel/modules/adminchannel.py @@ -1,18 +1,12 @@ -# coding=utf8 -""" -admin.py - Sopel Admin Module -Copyright 2010-2011, Michael Yanovich, Alek Rollyson, and Edward Powell -Copyright © 2012, Elad Alfassa -Licensed under the Eiffel Forum License 2. - -http://sopel.chat/ - -""" -from __future__ import unicode_literals +# coding=utf-8 +# Copyright 2010-2011, Michael Yanovich, Alek Rollyson, and Elsie Powell +# Copyright © 2012, Elad Alfassa +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division import re from sopel import formatting -from sopel.module import commands, priority, OP, HALFOP, require_privilege +from sopel.module import commands, priority, OP, HALFOP, require_privilege, require_chanmsg from sopel.tools import Identifier @@ -25,71 +19,8 @@ def default_mask(trigger): return '{} {} {} {}'.format(welcome, chan, topic_, arg) -@require_privilege(OP) -@commands('op') -def op(bot, trigger): - """ - Command to op users in a room. If no nick is given, - sopel will op the nick who sent the command - """ - if bot.privileges[trigger.sender][bot.nick] < OP: - return bot.reply("I'm not a channel operator!") - nick = trigger.group(2) - channel = trigger.sender - if not nick: - nick = trigger.nick - bot.write(['MODE', channel, "+o", nick]) - - -@require_privilege(OP) -@commands('deop') -def deop(bot, trigger): - """ - Command to deop users in a room. If no nick is given, - sopel will deop the nick who sent the command - """ - if bot.privileges[trigger.sender][bot.nick] < OP: - return bot.reply("I'm not a channel operator!") - nick = trigger.group(2) - channel = trigger.sender - if not nick: - nick = trigger.nick - bot.write(['MODE', channel, "-o", nick]) - - -@require_privilege(OP) -@commands('voice') -def voice(bot, trigger): - """ - Command to voice users in a room. If no nick is given, - sopel will voice the nick who sent the command - """ - if bot.privileges[trigger.sender][bot.nick] < HALFOP: - return bot.reply("I'm not a channel operator!") - nick = trigger.group(2) - channel = trigger.sender - if not nick: - nick = trigger.nick - bot.write(['MODE', channel, "+v", nick]) - - -@require_privilege(OP) -@commands('devoice') -def devoice(bot, trigger): - """ - Command to devoice users in a room. If no nick is given, - sopel will devoice the nick who sent the command - """ - if bot.privileges[trigger.sender][bot.nick] < HALFOP: - return bot.reply("I'm not a channel operator!") - nick = trigger.group(2) - channel = trigger.sender - if not nick: - nick = trigger.nick - bot.write(['MODE', channel, "-v", nick]) - - -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('kick') @priority('high') def kick(bot, trigger): @@ -139,7 +70,8 @@ def configureHostMask(mask): return '' -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('ban') @priority('high') def ban(bot, trigger): @@ -167,7 +99,8 @@ def ban(bot, trigger): bot.write(['MODE', channel, '+b', banmask]) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('unban') def unban(bot, trigger): """ @@ -194,7 +127,8 @@ def unban(bot, trigger): bot.write(['MODE', channel, '-b', banmask]) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('quiet') def quiet(bot, trigger): """ @@ -221,7 +155,8 @@ def quiet(bot, trigger): bot.write(['MODE', channel, '+q', quietmask]) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('unquiet') def unquiet(bot, trigger): """ @@ -248,7 +183,8 @@ def unquiet(bot, trigger): bot.write(['MODE', channel, '-q', quietmask]) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('kickban', 'kb') @priority('high') def kickban(bot, trigger): @@ -283,14 +219,14 @@ def kickban(bot, trigger): bot.write(['KICK', channel, nick], reason) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('topic') def topic(bot, trigger): """ This gives ops the ability to change the topic. The bot must be a Channel Operator for this command to work. """ - purple, green, bold = '\x0306', '\x0310', '\x02' if bot.privileges[trigger.sender][bot.nick] < HALFOP: return bot.reply("I'm not a channel operator!") if not trigger.group(2): @@ -318,7 +254,8 @@ def topic(bot, trigger): bot.write(('TOPIC', channel + ' :' + topic)) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('tmask') def set_mask(bot, trigger): """ @@ -329,7 +266,8 @@ def set_mask(bot, trigger): bot.say("Gotcha, " + trigger.nick) -@require_privilege(OP) +@require_chanmsg +@require_privilege(OP, 'You are not a channel operator.') @commands('showmask') def show_mask(bot, trigger): """Show the topic mask for the current channel.""" diff --git a/sopel/modules/announce.py b/sopel/modules/announce.py index 76f51c71ac..0f3f217ace 100644 --- a/sopel/modules/announce.py +++ b/sopel/modules/announce.py @@ -1,11 +1,11 @@ -# coding=utf8 +# coding=utf-8 """ announce.py - Send a message to all channels Copyright © 2013, Elad Alfassa, Licensed under the Eiffel Forum License 2. """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.module import commands, example @@ -21,3 +21,4 @@ def announce(bot, trigger): return for channel in bot.channels: bot.msg(channel, '[ANNOUNCEMENT] %s' % trigger.group(2)) + bot.reply('Announce complete.') diff --git a/sopel/modules/bugzilla.py b/sopel/modules/bugzilla.py index 0fa6d0e1b0..c87ff08e09 100644 --- a/sopel/modules/bugzilla.py +++ b/sopel/modules/bugzilla.py @@ -1,19 +1,23 @@ -# coding=utf8 +# coding=utf-8 """Bugzilla issue reporting module Copyright 2013-2015, Embolalia, embolalia.com Licensed under the Eiffel Forum License 2. """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division -import xmltodict import re + +import xmltodict + from sopel import web, tools -from sopel.module import rule from sopel.config.types import StaticSection, ListAttribute +from sopel.logger import get_logger +from sopel.module import rule regex = None +LOGGER = get_logger(__name__) class BugzillaSection(StaticSection): @@ -63,6 +67,13 @@ def show_bug(bot, trigger, match=None): url = 'https://%s%sctype=xml&%s' % match.groups() data = web.get(url, dont_decode=True) bug = xmltodict.parse(data).get('bugzilla').get('bug') + error = bug.get('@error', None) # error="NotPermitted" + + if error: + LOGGER.warning('Bugzilla error: %s' % error) + bot.say('[BUGZILLA] Unable to get infomation for ' + 'linked bug (%s)' % error) + return message = ('[BUGZILLA] %s | Product: %s | Component: %s | Version: %s | ' + 'Importance: %s | Status: %s | Assigned to: %s | ' + @@ -74,10 +85,14 @@ def show_bug(bot, trigger, match=None): else: status = bug.get('bug_status') + assigned_to = bug.get('assigned_to') + if isinstance(assigned_to, dict): + assigned_to = assigned_to.get('@name') + message = message % ( bug.get('short_desc'), bug.get('product'), bug.get('component'), bug.get('version'), (bug.get('priority') + ' ' + bug.get('bug_severity')), - status, bug.get('assigned_to').get('@name'), bug.get('creation_ts'), + status, assigned_to, bug.get('creation_ts'), bug.get('delta_ts')) bot.say(message) diff --git a/sopel/modules/calc.py b/sopel/modules/calc.py index 9acba453d3..421a655979 100644 --- a/sopel/modules/calc.py +++ b/sopel/modules/calc.py @@ -1,23 +1,19 @@ -# coding=utf8 +# coding=utf-8 """ calc.py - Sopel Calculator Module Copyright 2008, Sean B. Palmer, inamidst.com Licensed under the Eiffel Forum License 2. -http://sopel.dfbta.net +http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division -import re from sopel import web from sopel.module import commands, example from sopel.tools.calculation import eval_equation -from socket import timeout import sys -if sys.version_info.major < 3: - import HTMLParser -else: - import html.parser as HTMLParser +if sys.version_info.major >= 3: + unichr = chr BASE_TUMBOLIA_URI = 'https://tumbolia-two.appspot.com/' @@ -64,47 +60,6 @@ def py(bot, trigger): bot.reply('Sorry, no result.') -@commands('wa', 'wolfram') -@example('.wa sun mass / earth mass', - '[WOLFRAM] M_sun\/M_earth (solar mass per Earth mass) = 332948.6') -def wa(bot, trigger): - """Wolfram Alpha calculator""" - if not trigger.group(2): - return bot.reply("No search term.") - query = trigger.group(2) - uri = BASE_TUMBOLIA_URI + 'wa/' - try: - answer = web.get(uri + web.quote(query.replace('+', 'plus')), 45, - dont_decode=True) - except timeout as e: - return bot.say('[WOLFRAM ERROR] Request timed out') - if answer: - answer = answer.decode('unicode_escape') - answer = HTMLParser.HTMLParser().unescape(answer) - # This might not work if there are more than one instance of escaped - # unicode chars But so far I haven't seen any examples of such output - # examples from Wolfram Alpha - match = re.search('\\\:([0-9A-Fa-f]{4})', answer) - if match is not None: - char_code = match.group(1) - char = unichr(int(char_code, 16)) - answer = answer.replace('\:' + char_code, char) - waOutputArray = answer.split(";") - if(len(waOutputArray) < 2): - if(answer.strip() == "Couldn't grab results from json stringified precioussss."): - # Answer isn't given in an IRC-able format, just link to it. - bot.say('[WOLFRAM]Couldn\'t display answer, try http://www.wolframalpha.com/input/?i=' + query.replace(' ', '+')) - else: - bot.say('[WOLFRAM ERROR]' + answer) - else: - - bot.say('[WOLFRAM] ' + waOutputArray[0] + " = " - + waOutputArray[1]) - waOutputArray = [] - else: - bot.reply('Sorry, no result.') - - if __name__ == "__main__": from sopel.test_tools import run_example_tests run_example_tests(__file__) diff --git a/sopel/modules/clock.py b/sopel/modules/clock.py index b90d0a799c..d7fa8bbc65 100644 --- a/sopel/modules/clock.py +++ b/sopel/modules/clock.py @@ -1,13 +1,8 @@ -# coding=utf8 -""" -clock.py - Sopel Clock Module -Copyright 2008-9, Sean B. Palmer, inamidst.com -Copyright 2012, Edward Powell, embolalia.net -Licensed under the Eiffel Forum License 2. - -http://sopel.dfbta.net -""" -from __future__ import unicode_literals +# coding=utf-8 +# Copyright 2008-9, Sean B. Palmer, inamidst.com +# Copyright 2012, Elsie Powell, embolalia.com +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division try: import pytz diff --git a/sopel/modules/countdown.py b/sopel/modules/countdown.py index e7a5c8008b..23a6df8759 100644 --- a/sopel/modules/countdown.py +++ b/sopel/modules/countdown.py @@ -1,12 +1,12 @@ -# coding=utf8 +# coding=utf-8 """ countdown.py - Sopel Countdown Module Copyright 2011, Michael Yanovich, yanovich.net Licensed under the Eiffel Forum License 2. -http://sopel.dfbta.net +http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.module import commands, NOLIMIT import datetime @@ -29,9 +29,9 @@ def generic_countdown(bot, trigger): except: bot.say("Please use correct format: .countdown 2012 12 21") return NOLIMIT - bot.say(str(diff.days) + " days, " + str(diff.seconds / 60 / 60) + bot.say(str(diff.days) + " days, " + str(diff.seconds // 3600) + " hours and " - + str(diff.seconds / 60 - diff.seconds / 60 / 60 * 60) + + str(diff.seconds % 3600 // 60) + " minutes until " + text[0] + " " + text[1] + " " + text[2]) else: diff --git a/sopel/modules/currency.py b/sopel/modules/currency.py index 59764fb829..c81cee2cf9 100644 --- a/sopel/modules/currency.py +++ b/sopel/modules/currency.py @@ -1,17 +1,11 @@ -# coding=utf8 -"""currency.py - Sopel Exchange Rate Module -Copyright 2013 Edward Powell, embolalia.com -Licensed under the Eiffel Forum License 2 +# coding=utf-8 +# Copyright 2013 Elsie Powell, embolalia.com +# Licensed under the Eiffel Forum License 2 +from __future__ import unicode_literals, absolute_import, print_function, division -http://sopel.chat -""" -from __future__ import unicode_literals - -import json -import xmltodict import re -from sopel import web +from requests import get from sopel.module import commands, example, NOLIMIT # The Canadian central bank has better exchange rate data than the Fed, the @@ -26,25 +20,21 @@ def get_rate(code): + code = code.upper() if code == 'CAD': return 1, 'Canadian Dollar' elif code == 'BTC': - rates = json.loads(web.get('https://api.bitcoinaverage.com/ticker/all')) - return 1 / rates['CAD']['24h_avg'], 'Bitcoin—24hr average' - - data, headers = web.get(base_url.format(code), dont_decode=True, return_headers=True) - if headers['_http_status'] == 404: - return False, False - namespaces = { - 'http://www.cbwiki.net/wiki/index.php/Specification_1.1': 'cb', - 'http://purl.org/rss/1.0/': None, - 'http://www.w3.org/1999/02/22-rdf-syntax-ns#': 'rdf' } - xml = xmltodict.parse(data, process_namespaces=True, namespaces=namespaces).get('rdf:RDF') - namestring = xml.get('channel').get('title').get('#text') - name = namestring[len('Bank of Canada noon rate: '):] - name = re.sub(r'\s*\(noon\)\s*', '', name) - rate = xml.get('item').get('cb:statistics').get('cb:exchangeRate').get('cb:value').get('#text') - return float(rate), name + btc_rate = get('https://apiv2.bitcoinaverage.com/indices/global/ticker/BTCCAD') + rates = btc_rate.json() + return 1 / rates['averages']['day'], 'Bitcoin—24hr average' + + data = get("http://www.bankofcanada.ca/valet/observations/FX{}CAD/json".format(code)) + name = data.json()['seriesDetail']['FX{}CAD'.format(code)]['description'] + name = name.split(" to Canadian")[0] + json = data.json()['observations'] + for element in reversed(json): + if 'v' in element['FX{}CAD'.format(code)]: + return 1 / float(element['FX{}CAD'.format(code)]['v']), name @commands('cur', 'currency', 'exchange') @@ -79,14 +69,13 @@ def display(bot, amount, of, to): if not to_name: bot.reply("Unknown currency: %s" % to) return - except Exception as e: - raise + except Exception: bot.reply("Something went wrong while I was getting the exchange rate.") return NOLIMIT result = amount / of_rate * to_rate - bot.say("{} {} ({}) = {} {} ({})".format(amount, of, of_name, - result, to, to_name)) + bot.say("{} {} ({}) = {} {} ({})".format(amount, of.upper(), of_name, + result, to.upper(), to_name)) @commands('btc', 'bitcoin') diff --git a/sopel/modules/dice.py b/sopel/modules/dice.py index c55cff4584..2ac0d063f0 100644 --- a/sopel/modules/dice.py +++ b/sopel/modules/dice.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ dice.py - Dice Module Copyright 2010-2013, Dimitri "Tyrope" Molenaars, TyRope.nl @@ -7,7 +7,7 @@ http://sopel.chat/ """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import random import re import operator @@ -124,10 +124,10 @@ def get_number_of_faces(self): def _roll_dice(bot, dice_expression): result = re.search( r""" - (?P\d*) + (?P-?\d*) d - (?P\d+) - (v(?P\d+))? + (?P-?\d+) + (v(?P-?\d+))? $""", dice_expression, re.IGNORECASE | re.VERBOSE) @@ -140,6 +140,11 @@ def _roll_dice(bot, dice_expression): bot.reply("I don't have any dice with %d sides. =(" % dice_type) return None # Signal there was a problem + # Can't roll a negative number of dice. + if dice_num < 0: + bot.reply("I'd rather not roll a negative amount of dice. =(") + return None # Signal there was a problem + # Upper limit for dice should be at most a million. Creating a dict with # more than a million elements already takes a noticeable amount of time # on a fast computer and ~55kB of memory. @@ -151,7 +156,10 @@ def _roll_dice(bot, dice_expression): if result.group('drop_lowest'): drop = int(result.group('drop_lowest')) - dice.drop_lowest(drop) + if drop >= 0: + dice.drop_lowest(drop) + else: + bot.reply("I can't drop the lowest %d dice. =(" % drop) return dice @@ -176,7 +184,7 @@ def roll(bot, trigger): """ # This regexp is only allowed to have one captured group, because having # more would alter the output of re.findall. - dice_regexp = r"\d*d\d+(?:v\d+)?" + dice_regexp = r"-?\d*[dD]-?\d+(?:[vV]-?\d+)?" # Get a list of all dice expressions, evaluate them and then replace the # expressions in the original string with the results. Replacing is done @@ -231,9 +239,19 @@ def choose(bot, trigger): """ if not trigger.group(2): return bot.reply('I\'d choose an option, but you didn\'t give me any.') - choices = re.split('[\|\\\\\/]', trigger.group(2)) + choices = [trigger.group(2)] + for delim in '|\\/,': + choices = trigger.group(2).split(delim) + if len(choices) > 1: + break + # Use a different delimiter in the output, to prevent ambiguity. + for show_delim in ',|/\\': + if show_delim not in trigger.group(2): + show_delim += ' ' + break + pick = random.choice(choices) - return bot.reply('Your options: %s. My choice: %s' % (', '.join(choices), pick)) + return bot.reply('Your options: %s. My choice: %s' % (show_delim.join(choices), pick)) if __name__ == "__main__": diff --git a/sopel/modules/etymology.py b/sopel/modules/etymology.py index 784f42bd49..1c17231d1c 100644 --- a/sopel/modules/etymology.py +++ b/sopel/modules/etymology.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ etymology.py - Sopel Etymology Module Copyright 2007-9, Sean B. Palmer, inamidst.com @@ -6,8 +6,16 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division +try: + from html import unescape +except ImportError: + from HTMLParser import HTMLParser + + # pep8 dictates a blank line here... + def unescape(s): + return HTMLParser.unescape.__func__(HTMLParser, s) import re from sopel import web from sopel.module import commands, example, NOLIMIT @@ -29,13 +37,6 @@ r_sentence = re.compile(t_sentence % ')(?') - s = s.replace('<', '<') - s = s.replace('&', '&') - return s - - def text(html): html = r_tag.sub('', html) html = r_whitespace.sub(' ', html) @@ -74,7 +75,7 @@ def etymology(word): @commands('ety') -@example('word') +@example('.ety word') def f_etymology(bot, trigger): """Look up the etymology of a word""" word = trigger.group(2) diff --git a/sopel/modules/find.py b/sopel/modules/find.py index f6ee0b7779..8a1f58ab68 100644 --- a/sopel/modules/find.py +++ b/sopel/modules/find.py @@ -1,17 +1,14 @@ -# coding=utf8 -""" -find.py - Sopel Spelling correction module -Copyright 2011, Michael Yanovich, yanovich.net -Copyright 2013, Edward Powell, embolalia.net -Licensed under the Eiffel Forum License 2. - -http://sopel.chat +# coding=utf-8 +"""Sopel Spelling correction module -Contributions from: Matt Meinwald and Morgan Goose This module will fix spelling errors if someone corrects them using the sed notation (s///) commonly found in vi/vim. """ -from __future__ import unicode_literals +# Copyright 2011, Michael Yanovich, yanovich.net +# Copyright 2013, Elsie Powell, embolalia.com +# Licensed under the Eiffel Forum License 2. +# Contributions from: Matt Meinwald and Morgan Goose +from __future__ import unicode_literals, absolute_import, print_function, division import re from sopel.tools import Identifier, SopelMemory diff --git a/sopel/modules/find_updates.py b/sopel/modules/find_updates.py index 026af3a43b..2d0ba3307a 100644 --- a/sopel/modules/find_updates.py +++ b/sopel/modules/find_updates.py @@ -1,20 +1,18 @@ -# coding=utf8 -""" -find_updates.py - Update checking module for Sopel. +# coding=utf-8 +"""Update checking module for Sopel. This is separated from version.py, so that it can be easily overridden by distribution packagers, and they can check their repositories rather than the Sopel website. """ -# Copyright 2014, Edward D. Powell, embolalia.net +# Copyright 2014, Elsie Powell, embolalia.com # Licensed under the Eiffel Forum License 2. -from __future__ import unicode_literals - -import json +from __future__ import unicode_literals, absolute_import, print_function, division import sopel import sopel.module -import sopel.web +import requests +import sopel.tools wait_time = 24 * 60 * 60 # check once per day startup_check_run = False @@ -29,7 +27,7 @@ ) -@sopel.module.event('251') +@sopel.module.event(sopel.tools.events.RPL_LUSERCLIENT) @sopel.module.rule('.*') def startup_version_check(bot, trigger): global startup_check_run @@ -42,7 +40,9 @@ def startup_version_check(bot, trigger): def check_version(bot): version = sopel.version_info - info = json.loads(sopel.web.get(version_url)) + # TODO: Python3 specific. Disable urllib warning from config file. + # requests.packages.urllib3.disable_warnings() + info = requests.get(version_url, verify=bot.config.core.verify_ssl).json() if version.releaselevel == 'final': latest = info['version'] notes = info['release_notes'] diff --git a/sopel/modules/help.py b/sopel/modules/help.py index 53b8db9a92..cc588e9881 100644 --- a/sopel/modules/help.py +++ b/sopel/modules/help.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ help.py - Sopel Help Module Copyright 2008, Sean B. Palmer, inamidst.com @@ -7,14 +7,19 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import textwrap import collections +import json -from sopel.formatting import bold +import requests + +from sopel.logger import get_logger from sopel.module import commands, rule, example, priority +logger = get_logger(__name__) + @rule('$nick' '(?i)(help|doc) +([A-Za-z]+)(?:\?+)?$') @example('.help tell') @@ -42,23 +47,62 @@ def help(bot, trigger): if bot.doc[name][1]: msgfun('e.g. ' + bot.doc[name][1]) else: - if not trigger.is_privmsg: - bot.reply("I'm sending you a list of my commands in a private message!") - bot.say( - 'You can see more info about any of these commands by doing .help ' - ' (e.g. .help time)', - trigger.nick - ) - - name_length = max(6, max(len(k) for k in bot.command_groups.keys())) - for category, cmds in collections.OrderedDict(sorted(bot.command_groups.items())).items(): - category = category.upper().ljust(name_length) - cmds = ' '.join(cmds) - msg = bold(category) + ' ' + cmds - indent = ' ' * (name_length + 2) - msg = textwrap.wrap(msg, subsequent_indent=indent) - for line in msg: - bot.say(line, trigger.nick) + # This'll probably catch most cases, without having to spend the time + # actually creating the list first. Maybe worth storing the link and a + # heuristic in config, too, so it persists across restarts. Would need a + # command to regenerate, too... + if 'command-gist' in bot.memory and bot.memory['command-gist'][0] == len(bot.command_groups): + url = bot.memory['command-gist'][1] + else: + bot.say("Hang on, I'm creating a list.") + msgs = [] + + name_length = max(6, max(len(k) for k in bot.command_groups.keys())) + for category, cmds in collections.OrderedDict(sorted(bot.command_groups.items())).items(): + category = category.upper().ljust(name_length) + cmds = ' '.join(cmds) + msg = category + ' ' + cmds + indent = ' ' * (name_length + 2) + # Honestly not sure why this is a list here + msgs.append('\n'.join(textwrap.wrap(msg, subsequent_indent=indent))) + + url = create_gist(bot, '\n\n'.join(msgs)) + if not url: + return + bot.memory['command-gist'] = (len(bot.command_groups), url) + bot.say("I've posted a list of my commands at {} - You can see " + "more info about any of these commands by doing .help " + " (e.g. .help time)".format(url)) + + +def create_gist(bot, msg): + payload = { + 'description': 'Command listing for {}@{}'.format(bot.nick, bot.config.core.host), + 'public': 'true', + 'files': { + 'commands.txt': { + "content": msg, + }, + }, + } + try: + result = requests.post('https://api.github.com/gists', + data=json.dumps(payload)) + except requests.RequestException: + bot.say("Sorry! Something went wrong.") + logger.exception("Error posting commands gist") + return + if not result.status_code != '201': + bot.say("Sorry! Something went wrong.") + logger.error("Error %s posting commands gist: %s", + result.status_code, result.text) + return + result = result.json() + if 'html_url' not in result: + bot.say("Sorry! Something went wrong.") + logger.error("Invalid result %s", result) + return + return result['html_url'] @rule('$nick' r'(?i)help(?:[?!]+)?$') diff --git a/sopel/modules/ipython.py b/sopel/modules/ipython.py index 994e94e7b5..6df6112dbf 100644 --- a/sopel/modules/ipython.py +++ b/sopel/modules/ipython.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ ipython.py - sopel ipython console! Copyright © 2014, Elad Alfassa @@ -6,7 +6,7 @@ Sopel: http://sopel.chat/ """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import sopel import sys if sys.version_info.major >= 3: @@ -50,7 +50,7 @@ def interactive_shell(bot, trigger): bot.say('A tty is required to start the console') return if bot._daemon: - bot.say('Can\'t start console when running as a deamon') + bot.say('Can\'t start console when running as a daemon') return # Backup stderr/stdout wrappers diff --git a/sopel/modules/isup.py b/sopel/modules/isup.py index e7ca86939d..d0c939d877 100644 --- a/sopel/modules/isup.py +++ b/sopel/modules/isup.py @@ -1,12 +1,7 @@ -# coding=utf8 -""" -isup.py - Simple website status check with isup.me -Author: Edward Powell http://embolalia.net -About: http://sopel.chat - -This allows users to check if a website is up through isup.me. -""" -from __future__ import unicode_literals +# coding=utf-8 +"""Simple website status check with isup.me""" +# Author: Elsie Powell http://embolalia.com +from __future__ import unicode_literals, absolute_import, print_function, division from sopel import web from sopel.module import commands @@ -19,7 +14,7 @@ def isup(bot, trigger): if not site: return bot.reply("What site do you want to check?") - if site[:6] != 'http://' and site[:7] != 'https://': + if site[:7] != 'http://' and site[:8] != 'https://': if '://' in site: protocol = site.split('://')[0] + '://' return bot.reply("Try it again without the %s" % protocol) diff --git a/sopel/modules/lmgtfy.py b/sopel/modules/lmgtfy.py index 46e35fe5cf..7d0caebbca 100644 --- a/sopel/modules/lmgtfy.py +++ b/sopel/modules/lmgtfy.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ lmgtfy.py - Sopel Let me Google that for you module Copyright 2013, Dimitri Molenaars http://tyrope.nl/ @@ -6,7 +6,7 @@ http://sopel.chat/ """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.module import commands diff --git a/sopel/modules/meetbot.py b/sopel/modules/meetbot.py index 41c5c2af87..e43d56da6d 100644 --- a/sopel/modules/meetbot.py +++ b/sopel/modules/meetbot.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ meetbot.py - Sopel meeting logger module Copyright © 2012, Elad Alfassa, @@ -6,7 +6,7 @@ This module is an attempt to implement at least some of the functionallity of Debian's meetbot """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import time import os from sopel.config.types import ( @@ -148,7 +148,7 @@ def ischair(nick, channel): def startmeeting(bot, trigger): """ Start a meeting. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if ismeetingrunning(trigger.sender): bot.say('Can\'t do that, there is already a meeting in progress here!') @@ -156,9 +156,6 @@ def startmeeting(bot, trigger): if trigger.is_privmsg: bot.say('Can only start meetings in channels') return - if not bot.config.has_section('meetbot'): - bot.say('Meetbot not configured, make sure meeting_log_path and meeting_log_baseurl are defined') - return #Start the meeting meetings_dict[trigger.sender]['start'] = time.time() if not trigger.group(2): @@ -180,7 +177,7 @@ def startmeeting(bot, trigger): if not os.path.isdir(meeting_log_path + trigger.sender): try: os.makedirs(meeting_log_path + trigger.sender) - except Exception as e: + except Exception: bot.say("Can't create log directory for this channel, meeting not started!") meetings_dict[trigger.sender] = Ddict(dict) raise @@ -201,7 +198,7 @@ def startmeeting(bot, trigger): def meetingsubject(bot, trigger): """ Change the meeting subject. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -226,7 +223,7 @@ def meetingsubject(bot, trigger): def endmeeting(bot, trigger): """ End a meeting. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -251,7 +248,7 @@ def endmeeting(bot, trigger): def chairs(bot, trigger): """ Set the meeting chairs. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -275,7 +272,7 @@ def chairs(bot, trigger): def meetingaction(bot, trigger): """ Log an action in the meeting log - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -308,7 +305,7 @@ def listactions(bot, trigger): def meetingagreed(bot, trigger): """ Log an agreement in the meeting log. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -330,7 +327,7 @@ def meetingagreed(bot, trigger): def meetinglink(bot, trigger): """ Log a link in the meeing log. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -345,7 +342,7 @@ def meetinglink(bot, trigger): if not link.startswith("http"): link = "http://" + link try: - title = find_title(link) + title = find_title(link, verify=bot.config.core.verify_ssl) except: title = '' logplain('LINK: %s [%s]' % (link, title), trigger.sender) @@ -359,7 +356,7 @@ def meetinglink(bot, trigger): def meetinginfo(bot, trigger): """ Log an informational item in the meeting log - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): bot.say('Can\'t do that, start meeting first') @@ -395,7 +392,7 @@ def take_comment(bot, trigger): in the meeting. Used in private message only, as `.comment <#channel> ` - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not trigger.sender.is_nick(): return @@ -418,7 +415,7 @@ def take_comment(bot, trigger): def show_comments(bot, trigger): """ Show the comments that have been logged for this meeting with .comment. - https://github.com/embolalia/sopel/wiki/Using-the-meetbot-module + https://github.com/sopel-irc/sopel/wiki/Using-the-meetbot-module """ if not ismeetingrunning(trigger.sender): return diff --git a/sopel/modules/movie.py b/sopel/modules/movie.py index 1cda5e68c7..0abcdaeb01 100644 --- a/sopel/modules/movie.py +++ b/sopel/modules/movie.py @@ -1,14 +1,13 @@ -# coding=utf8 +# coding=utf-8 """ imdb.py - Sopel Movie Information Module Copyright © 2012-2013, Elad Alfassa, Licensed under the Eiffel Forum License 2. -This module relies on imdbapi.com +This module relies on omdbapi.com """ -from __future__ import unicode_literals -import json -import sopel.web as web +from __future__ import unicode_literals, absolute_import, print_function, division +import requests import sopel.module from sopel.logger import get_logger @@ -25,17 +24,17 @@ def movie(bot, trigger): if not trigger.group(2): return word = trigger.group(2).rstrip() - uri = "http://www.imdbapi.com/?t=" + word - u = web.get(uri, 30) - data = json.loads(u) # data is a Dict containing all the information we need + uri = "http://www.omdbapi.com/" + data = requests.get(uri, params={'t': word}, timeout=30, + verify=bot.config.core.verify_ssl).json() if data['Response'] == 'False': if 'Error' in data: message = '[MOVIE] %s' % data['Error'] else: LOGGER.warning( - 'Got an error from the imdb api, search phrase was %s; data was %s', + 'Got an error from the OMDb api, search phrase was %s; data was %s', word, str(data)) - message = '[MOVIE] Got an error from imdbapi' + message = '[MOVIE] Got an error from OMDbapi' else: message = '[MOVIE] Title: ' + data['Title'] + \ ' | Year: ' + data['Year'] + \ diff --git a/sopel/modules/ping.py b/sopel/modules/ping.py index 9f3d36d2c4..ee9c3d24e0 100644 --- a/sopel/modules/ping.py +++ b/sopel/modules/ping.py @@ -1,10 +1,10 @@ -# coding=utf8 +# coding=utf-8 """ ping.py - Sopel Ping Module Author: Sean B. Palmer, inamidst.com About: http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import random from sopel.module import rule, priority, thread diff --git a/sopel/modules/pronouns.py b/sopel/modules/pronouns.py new file mode 100644 index 0000000000..0b101811dc --- /dev/null +++ b/sopel/modules/pronouns.py @@ -0,0 +1,96 @@ +# coding=utf-8 +""" +pronouns.py - Sopel Pronouns Module +Copyright © 2016, Elsie Powell +Licensed under the Eiffel Forum License 2. + +http://sopel.chat +""" +from __future__ import unicode_literals, absolute_import, print_function, division + +from sopel.logger import get_logger +from sopel.module import commands, example + +logger = get_logger(__name__) +# Copied from pronoun.is, leaving a *lot* out. If +# https://github.com/witch-house/pronoun.is/pull/40 gets merged, using that +# would be a lot easier. +KNOWN_SETS = { + 'ze': 'ze/hir/hir/hirs/hirself', + 'ze/hir': 'ze/hir/hir/hirs/hirself', + 'ze/zir': 'ze/zir/zir/zirs/zirself', + 'they': 'they/them/their/theirs/themselves', + 'they/.../themselves': 'they/them/their/theirs/themselves', + 'they/.../themself': 'they/them/their/theirs/themself', + 'she': 'she/her/her/hers/herself', + 'he': 'he/him/his/his/himself', + 'xey': 'xey/xem/xyr/xyrs/xemself', + 'sie': 'sie/hir/hir/hirs/hirself', + 'it': 'it/it/its/its/itself', + 'ey': 'ey/em/eir/eirs/eirslef', +} + + +@commands('pronouns') +@example('.pronouns Embolalia') +def pronouns(bot, trigger): + if not trigger.group(2): + pronouns = bot.db.get_nick_value(trigger.nick, 'pronouns') + if pronouns: + say_pronouns(bot, trigger.nick, pronouns) + else: + bot.reply("I don't know your pronouns! You can set them with " + ".setpronouns") + else: + pronouns = bot.db.get_nick_value(trigger.group(2).replace(" ", ""), 'pronouns') + if pronouns: + say_pronouns(bot, trigger.group(2).replace(" ", ""), pronouns) + elif trigger.group(2) == bot.nick: + # You can stuff an entry into the database manually for your bot's + # gender, but like… it's a bot. + # https://twitter.com/hopefulcyborg/status/728231494116773889 + bot.say( + "I am a bot. Beep boop. My pronouns are it/it/its/its/itself. " + "See http://pronoun.is/it for examples." + ) + else: + bot.say("I don't know {}'s pronouns. They can set them with " + ".setpronouns".format(trigger.group(2).replace(" ", ""))) + + +def say_pronouns(bot, nick, pronouns): + for short, set_ in KNOWN_SETS.items(): + if pronouns == set_: + break + short = pronouns + + bot.say("{}'s pronouns are {}. See http://pronoun.is/{} for " + "examples.".format(nick, pronouns, short)) + + +@commands('setpronouns') +@example('.setpronouns they/them/their/theirs/themselves') +def set_pronouns(bot, trigger): + if trigger.group(2): + pronouns = trigger.group(2) + disambig = '' + if pronouns == 'they': + disambig = ' You can also use they/.../themself, if you prefer.' + pronouns = KNOWN_SETS.get(pronouns) + elif pronouns == 'ze': + disambig = ' I have ze/hir. If you meant ze/zir, you can use that instead.' + pronouns = KNOWN_SETS.get(pronouns) + elif len(pronouns.split('/')) != 5: + pronouns = KNOWN_SETS.get(pronouns) + if not pronouns: + bot.say( + "I'm sorry, I don't know those pronouns. You can give me a set " + "I don't know by formatting it " + "subject/object/possessive-determiner/posessive-pronoun/" + "reflexive, as in they/them/their/theirs/themselves" + ) + return + bot.db.set_nick_value(trigger.nick, 'pronouns', pronouns) + bot.reply("Thanks for telling me!" + disambig) + else: + bot.reply("What?") diff --git a/sopel/modules/rand.py b/sopel/modules/rand.py index 0b3e76ad73..b9b0a6ec7b 100644 --- a/sopel/modules/rand.py +++ b/sopel/modules/rand.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ rand.py - Rand Module Copyright 2013, Ari Koivula, @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.module import commands, example import random diff --git a/sopel/modules/reddit.py b/sopel/modules/reddit.py index 3e40866885..114765b727 100644 --- a/sopel/modules/reddit.py +++ b/sopel/modules/reddit.py @@ -1,15 +1,8 @@ -# coding=utf8 -""" -reddit-info.py - Sopel Reddit module -Author: Edward Powell, embolalia.net -About: http://sopel.chat - -This module provides special tools for reddit, namely showing detailed -info about reddit posts -""" -from __future__ import unicode_literals - -from sopel.module import commands, rule, example, NOLIMIT, OP +# coding=utf-8 +# Author: Elsie Powell, embolalia.com +from __future__ import unicode_literals, absolute_import, print_function, division + +from sopel.module import commands, rule, example, require_chanmsg, NOLIMIT, OP from sopel.formatting import bold, color, colors from sopel.web import USER_AGENT from sopel.tools import SopelMemory, time @@ -30,10 +23,14 @@ domain = r'https?://(?:www\.|np\.)?reddit\.com' -post_url = '(%s/r/.*?/comments/[\w-]+)' % domain +post_url = '%s/r/(.*?)/comments/([\w-]+)' % domain user_url = '%s/u(ser)?/([\w-]+)' % domain post_regex = re.compile(post_url) user_regex = re.compile(user_url) +spoiler_subs = [ + 'stevenuniverse', + 'onepunchman', +] def setup(bot): @@ -50,21 +47,30 @@ def shutdown(bot): @rule('.*%s.*' % post_url) def rpost_info(bot, trigger, match=None): - r = praw.Reddit(user_agent=USER_AGENT) + r = praw.Reddit( + user_agent=USER_AGENT, + client_id='6EiphT6SSQq7FQ', + client_secret=None, + ) match = match or trigger - s = r.get_submission(url=match.group(1)) + s = r.submission(id=match.group(2)) message = ('[REDDIT] {title} {link}{nsfw} | {points} points ({percent}) | ' '{comments} comments | Posted by {author} | ' 'Created at {created}') + subreddit = s.subreddit.display_name if s.is_self: - link = '(self.{})'.format(s.subreddit.display_name) + link = '(self.{})'.format(subreddit) else: - link = '({}) to r/{}'.format(s.url, s.subreddit.display_name) + link = '({}) to r/{}'.format(s.url, subreddit) if s.over_18: - nsfw = bold(color(' [NSFW]', colors.RED)) + if subreddit.lower() in spoiler_subs: + nsfw = bold(color(' [SPOILERS]', colors.RED)) + else: + nsfw = bold(color(' [NSFW]', colors.RED)) + sfw = bot.db.get_channel_value(trigger.sender, 'sfw') if sfw: link = '(link hidden)' @@ -105,7 +111,11 @@ def rpost_info(bot, trigger, match=None): def redditor_info(bot, trigger, match=None): """Show information about the given Redditor""" commanded = re.match(bot.config.core.prefix + 'redditor', trigger) - r = praw.Reddit(user_agent=USER_AGENT) + r = praw.Reddit( + user_agent=USER_AGENT, + client_id='6EiphT6SSQq7FQ', + client_secret=None, + ) match = match or trigger try: u = r.get_redditor(match.group(2)) @@ -153,6 +163,7 @@ def auto_redditor_info(bot, trigger): redditor_info(bot, trigger) +@require_chanmsg('.setsfw is only permitted in channels') @commands('setsafeforwork', 'setsfw') @example('.setsfw true') @example('.setsfw false') @@ -164,7 +175,10 @@ def update_channel(bot, trigger): if bot.privileges[trigger.sender][trigger.nick] < OP: return else: - sfw = trigger.group(3).strip().lower() == 'true' + param = 'true' + if trigger.group(2) and trigger.group(3): + param = trigger.group(3).strip().lower() + sfw = param == 'true' bot.db.set_channel_value(trigger.sender, 'sfw', sfw) if sfw: bot.reply('Got it. %s is now flagged as SFW.' % trigger.sender) @@ -182,6 +196,8 @@ def get_channel_sfw(bot, trigger): channel = trigger.group(2) if not channel: channel = trigger.sender + if channel.is_nick(): + return bot.say('.getsfw with no channel param is only permitted in channels') channel = channel.strip() diff --git a/sopel/modules/reload.py b/sopel/modules/reload.py index f99e1593c6..53bd7207d3 100644 --- a/sopel/modules/reload.py +++ b/sopel/modules/reload.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ reload.py - Sopel Module Reloader Module Copyright 2008, Sean B. Palmer, inamidst.com @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import collections import sys @@ -26,8 +26,6 @@ def f_reload(bot, trigger): return name = trigger.group(2) - if name == bot.config.core.owner: - return bot.reply('What?') if not name or name == '*' or name.upper() == 'ALL THE THINGS': bot._callables = { @@ -35,7 +33,7 @@ def f_reload(bot, trigger): 'medium': collections.defaultdict(list), 'low': collections.defaultdict(list) } - bot.command_groups = collections.defaultdict(list) + bot._command_groups = collections.defaultdict(list) bot.setup() return bot.reply('done') @@ -102,8 +100,8 @@ def f_load(bot, trigger): name = trigger.group(2) path = '' - if name == bot.config.core.owner: - return bot.reply('What?') + if not name: + return bot.reply('Load what?') if name in sys.modules: return bot.reply('Module already loaded, use reload') diff --git a/sopel/modules/remind.py b/sopel/modules/remind.py index 028dedcc15..34dca48f90 100644 --- a/sopel/modules/remind.py +++ b/sopel/modules/remind.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ remind.py - Sopel Reminder Module Copyright 2011, Sean B. Palmer, inamidst.com @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import os import re @@ -194,7 +194,7 @@ def at(bot, trigger): timediff = at_time - now else: if tz and tz.upper() != 'UTC': - bot.reply("I don't have timzeone support installed.") + bot.reply("I don't have timezone support installed.") return NOLIMIT now = datetime.now() at_time = datetime(now.year, now.month, now.day, diff --git a/sopel/modules/safety.py b/sopel/modules/safety.py index a201d0a85a..dc2c3b847c 100644 --- a/sopel/modules/safety.py +++ b/sopel/modules/safety.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ safety.py - Alerts about malicious URLs Copyright © 2014, Elad Alfassa, @@ -6,8 +6,8 @@ This module uses virustotal.com """ -from __future__ import unicode_literals -from __future__ import print_function +from __future__ import unicode_literals, absolute_import, print_function, division + import sopel.web as web from sopel.config.types import StaticSection, ValidatedAttribute, ListAttribute from sopel.formatting import color, bold @@ -115,7 +115,11 @@ def url_handler(bot, trigger): if not check: return # Not overriden by DB, configured default off - netloc = urlparse(trigger.group(1)).netloc + try: + netloc = urlparse(trigger.group(1)).netloc + except ValueError: + return # Invalid IPv6 URL + if any(regex.search(netloc) for regex in known_good): return # Whitelisted @@ -143,7 +147,7 @@ def url_handler(bot, trigger): result = bot.memory['safety_cache'][trigger] positives = result['positives'] total = result['total'] - except Exception as e: + except Exception: LOGGER.debug('Error from checking URL with VT.', exc_info=True) pass # Ignoring exceptions with VT so MalwareDomains will always work diff --git a/sopel/modules/search.py b/sopel/modules/search.py index badad0d29b..677f53fa3f 100644 --- a/sopel/modules/search.py +++ b/sopel/modules/search.py @@ -1,13 +1,8 @@ -# coding=utf8 -""" -search.py - Sopel Web Search Module -Copyright 2008-9, Sean B. Palmer, inamidst.com -Copyright 2012, Edward Powell, embolalia.net -Licensed under the Eiffel Forum License 2. - -http://sopel.chat -""" -from __future__ import unicode_literals +# coding=utf-8 +# Copyright 2008-9, Sean B. Palmer, inamidst.com +# Copyright 2012, Elsie Powell, embolalia.com +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division import re from sopel import web @@ -19,7 +14,8 @@ from urllib import quote_plus else: from urllib.parse import quote_plus - + + def formatnumber(n): """Format a number with beautiful commas.""" parts = list(str(n)) @@ -37,15 +33,15 @@ def bing_search(query, lang='en-GB'): if m: return m.group(1) -r_duck = re.compile(r'nofollow" class="[^"]+" href="(.*?)">') +r_duck = re.compile(r'nofollow" class="[^"]+" href="(?!https?:\/\/r\.search\.yahoo)(.*?)">') def duck_search(query): query = query.replace('!', '') uri = 'http://duckduckgo.com/html/?q=%s&kl=uk-en' % query - bytes = web.get(uri) - if 'web-result"' in bytes: # filter out the adds on top of the page - bytes = bytes.split('web-result"')[1] + bytes = web.get(uri,headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36'}) + if 'web-result' in bytes: # filter out the adds on top of the page + bytes = bytes.split('web-result')[1] m = r_duck.search(bytes) if m: return web.decode(m.group(1)) @@ -57,11 +53,11 @@ def duck_search(query): def duck_api(query): if '!bang' in query.lower(): return 'https://duckduckgo.com/bang.html' - + # This fixes issue #885 (https://github.com/sopel-irc/sopel/issues/885) - # It seems that duckduckgo api redirects to its Instant answer API html page + # It seems that duckduckgo api redirects to its Instant answer API html page # if the query constains special charactares that aren't urlencoded. - # So in order to always get a JSON response back the query is urlencoded + # So in order to always get a JSON response back the query is urlencoded query = quote_plus(query) uri = 'http://api.duckduckgo.com/?q=%s&format=json&no_html=1&no_redirect=1' % query results = json.loads(web.get(uri)) @@ -107,7 +103,7 @@ def search(bot, trigger): du = duck_search(query) or '-' if bu == du: - result = '%s (b, d)' % gu + result = '%s (b, d)' % bu else: if len(bu) > 150: bu = '(extremely long link)' diff --git a/sopel/modules/seen.py b/sopel/modules/seen.py index 581aeb40f3..f5fd787c92 100644 --- a/sopel/modules/seen.py +++ b/sopel/modules/seen.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ seen.py - Sopel Seen Module Copyright 2008, Sean B. Palmer, inamidst.com @@ -7,7 +7,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import time import datetime @@ -23,6 +23,9 @@ def seen(bot, trigger): bot.say(".seen - Reports when was last seen.") return nick = trigger.group(2).strip() + if nick == bot.nick: + bot.reply("I'm right here!") + return timestamp = bot.db.get_nick_value(nick, 'seen_timestamp') if timestamp: channel = bot.db.get_nick_value(nick, 'seen_channel') diff --git a/sopel/modules/spellcheck.py b/sopel/modules/spellcheck.py index 6023bfd796..f7225955a8 100644 --- a/sopel/modules/spellcheck.py +++ b/sopel/modules/spellcheck.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ spellcheck.py - Sopel spell check Module Copyright © 2012, Elad Alfassa, @@ -9,7 +9,7 @@ This module relies on pyenchant, on Fedora and Red Hat based system, it can be found in the package python-enchant """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division try: import enchant except ImportError: diff --git a/sopel/modules/tell.py b/sopel/modules/tell.py index 19f6aba802..ed3907ac68 100644 --- a/sopel/modules/tell.py +++ b/sopel/modules/tell.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ tell.py - Sopel Tell and Ask Module Copyright 2008, Sean B. Palmer, inamidst.com @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import os import time @@ -80,7 +80,7 @@ def setup(self): @commands('tell', 'ask') @nickname_commands('tell', 'ask') -@example('Sopel, tell Embolalia he broke something again.') +@example('$nickname, tell Embolalia he broke something again.') def f_remind(bot, trigger): """Give someone a message the next time they're seen""" teller = trigger.nick @@ -166,9 +166,9 @@ def message(bot, trigger): for remkey in remkeys: if not remkey.endswith('*') or remkey.endswith(':'): - if tellee == remkey: + if tellee.lower() == remkey.lower(): reminders.extend(getReminders(bot, channel, remkey, tellee)) - elif tellee.startswith(remkey.rstrip('*:')): + elif tellee.lower().startswith(remkey.lower().rstrip('*:')): reminders.extend(getReminders(bot, channel, remkey, tellee)) for line in reminders[:maximum]: diff --git a/sopel/modules/tld.py b/sopel/modules/tld.py index 67fd757930..62284b5a8c 100644 --- a/sopel/modules/tld.py +++ b/sopel/modules/tld.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ tld.py - Sopel TLD Module Copyright 2009-10, Michael Yanovich, yanovich.net @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel import web from sopel.module import commands, example diff --git a/sopel/modules/translate.py b/sopel/modules/translate.py index 609e57e9f4..c143aa530c 100644 --- a/sopel/modules/translate.py +++ b/sopel/modules/translate.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ translate.py - Sopel Translation Module Copyright 2008, Sean B. Palmer, inamidst.com @@ -7,19 +7,23 @@ http://sopel.chat """ -from __future__ import unicode_literals -from sopel import web -from sopel.module import rule, commands, priority, example +from __future__ import (unicode_literals, absolute_import, + print_function, division) import json -import sys import random -import os +import sys +import requests + + +from sopel import web +from sopel.module import rule, commands, priority, example + mangle_lines = {} if sys.version_info.major >= 3: unicode = str -def translate(text, in_lang='auto', out_lang='en'): +def translate(text, in_lang='auto', out_lang='en', verify_ssl=True): raw = False if unicode(out_lang).endswith('-raw'): out_lang = out_lang[:-4] @@ -31,19 +35,19 @@ def translate(text, in_lang='auto', out_lang='en'): 'Gecko/20071127 Firefox/2.0.0.11' } - url_query = { + query = { "client": "gtx", "sl": in_lang, "tl": out_lang, "dt": "t", "q": text, } - query_string = "&".join( - "{key}={value}".format(key=key, value=value) - for key, value in url_query.items() - ) - url = "http://translate.googleapis.com/translate_a/single?{query}".format(query=query_string) - result = web.get(url, timeout=40, headers=headers) + url = "http://translate.googleapis.com/translate_a/single" + result = requests.get(url, params=query, timeout=40, headers=headers, + verify=verify_ssl).text + + if result == '[,,""]': + return None, in_lang while ',,' in result: result = result.replace(',,', ',null,') @@ -72,18 +76,22 @@ def tr(bot, trigger): if (len(phrase) > 350) and (not trigger.admin): return bot.reply('Phrase must be under 350 characters.') + if phrase.strip() == '': + return bot.reply('You need to specify a string for me to translate!') + in_lang = in_lang or 'auto' out_lang = out_lang or 'en' if in_lang != out_lang: - msg, in_lang = translate(phrase, in_lang, out_lang) + msg, in_lang = translate(phrase, in_lang, out_lang, + verify_ssl=bot.config.core.verify_ssl) if sys.version_info.major < 3 and isinstance(msg, str): msg = msg.decode('utf-8') if msg: msg = web.decode(msg) # msg.replace(''', "'") msg = '"%s" (%s to %s, translate.google.com)' % (msg, in_lang, out_lang) else: - msg = 'The %s to %s translation failed, sorry!' % (in_lang, out_lang) + msg = 'The %s to %s translation failed, are you sure you specified valid language abbreviations?' % (in_lang, out_lang) bot.reply(msg) else: @@ -92,7 +100,7 @@ def tr(bot, trigger): @commands('translate', 'tr') @example('.tr :en :fr my dog', '"mon chien" (en to fr, translate.google.com)') -@example('.tr היי', '"Hi" (iw to en, translate.google.com)') +@example('.tr היי', '"Hey" (iw to en, translate.google.com)') @example('.tr mon chien', '"my dog" (fr to en, translate.google.com)') def tr2(bot, trigger): """Translates a phrase, with an optional language hint.""" @@ -118,16 +126,20 @@ def langcode(p): if (len(phrase) > 350) and (not trigger.admin): return bot.reply('Phrase must be under 350 characters.') + if phrase.strip() == '': + return bot.reply('You need to specify a string for me to translate!') + src, dest = args if src != dest: - msg, src = translate(phrase, src, dest) + msg, src = translate(phrase, src, dest, + verify_ssl=bot.config.core.verify_ssl) if sys.version_info.major < 3 and isinstance(msg, str): msg = msg.decode('utf-8') if msg: msg = web.decode(msg) # msg.replace(''', "'") msg = '"%s" (%s to %s, translate.google.com)' % (msg, src, dest) else: - msg = 'The %s to %s translation failed, sorry!' % (src, dest) + msg = 'The %s to %s translation failed, are you sure you specified valid language abbreviations?' % (src, dest) bot.reply(msg) else: @@ -147,6 +159,7 @@ def get_random_lang(long_list, short_list): @commands('mangle', 'mangle2') def mangle(bot, trigger): """Repeatedly translate the input until it makes absolutely no sense.""" + verify_ssl = bot.config.core.verify_ssl global mangle_lines long_lang_list = ['fr', 'de', 'es', 'it', 'no', 'he', 'la', 'ja', 'cy', 'ar', 'yi', 'zh', 'nl', 'ru', 'fi', 'hi', 'af', 'jw', 'mr', 'ceb', 'cs', 'ga', 'sv', 'eo', 'el', 'ms', 'lv'] lang_list = [] @@ -167,7 +180,8 @@ def mangle(bot, trigger): for lang in lang_list: backup = phrase try: - phrase = translate(phrase[0], 'en', lang) + phrase = translate(phrase[0], 'en', lang, + verify_ssl=verify_ssl) except: phrase = False if not phrase: @@ -175,7 +189,7 @@ def mangle(bot, trigger): break try: - phrase = translate(phrase[0], lang, 'en') + phrase = translate(phrase[0], lang, 'en', verify_ssl=verify_ssl) except: phrase = backup continue diff --git a/sopel/modules/unicode_info.py b/sopel/modules/unicode_info.py index a046ad7b31..6f7b3bfaa8 100644 --- a/sopel/modules/unicode_info.py +++ b/sopel/modules/unicode_info.py @@ -1,13 +1,9 @@ -# coding=utf8 -""" -codepoints.py - Sopel Codepoints Module -Copyright 2013, Edward Powell, embolalia.net -Copyright 2008, Sean B. Palmer, inamidst.com -Licensed under the Eiffel Forum License 2. - -http://sopel.dfbta.net -""" -from __future__ import unicode_literals +# coding=utf-8 +"""Codepoints Module""" +# Copyright 2013, Elsie Powell, embolalia.com +# Copyright 2008, Sean B. Palmer, inamidst.com +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division import unicodedata import sys from sopel.module import commands, example, NOLIMIT @@ -20,11 +16,14 @@ @example('.u ‽', 'U+203D INTERROBANG (‽)') @example('.u 203D', 'U+203D INTERROBANG (‽)') def codepoint(bot, trigger): - arg = trigger.group(2).strip() - if len(arg) == 0: + arg = trigger.group(2) + if not arg: bot.reply('What code point do you want me to look up?') return NOLIMIT - elif len(arg) > 1: + stripped = arg.strip() + if len(stripped) > 0: + arg = stripped + if len(arg) > 1: if arg.startswith('U+'): arg = arg[2:] try: diff --git a/sopel/modules/units.py b/sopel/modules/units.py index d28c34758c..5355948055 100644 --- a/sopel/modules/units.py +++ b/sopel/modules/units.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ units.py - Unit conversion module for Sopel Copyright © 2013, Elad Alfassa, @@ -6,7 +6,7 @@ Licensed under the Eiffel Forum License 2. """ -from __future__ import unicode_literals, division +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.module import commands, example, NOLIMIT import re diff --git a/sopel/modules/uptime.py b/sopel/modules/uptime.py index e03aa4cff0..7e3fe1d2dd 100644 --- a/sopel/modules/uptime.py +++ b/sopel/modules/uptime.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ uptime.py - Uptime module Copyright 2014, Fabian Neundorf @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from sopel.module import commands import datetime diff --git a/sopel/modules/url.py b/sopel/modules/url.py index 987374ac53..4a493ba1d8 100644 --- a/sopel/modules/url.py +++ b/sopel/modules/url.py @@ -1,22 +1,21 @@ -# coding=utf8 -""" -url.py - Sopel URL title module -Copyright 2010-2011, Michael Yanovich, yanovich.net, Kenneth Sham -Copyright 2012-2013 Edward Powell -Copyright 2013 Lior Ramati (firerogue517@gmail.com) -Copyright © 2014 Elad Alfassa -Licensed under the Eiffel Forum License 2. - -http://sopel.chat -""" -from __future__ import unicode_literals +# coding=utf-8 +"""URL title module""" +# Copyright 2010-2011, Michael Yanovich, yanovich.net, Kenneth Sham +# Copyright 2012-2013 Elsie Powell +# Copyright 2013 Lior Ramati (firerogue517@gmail.com) +# Copyright © 2014 Elad Alfassa +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division import re -from sopel import web, tools +from sopel import web, tools, __version__ from sopel.module import commands, rule, example -from sopel.config.types import ValidatedAttribute, StaticSection +from sopel.config.types import ValidatedAttribute, ListAttribute, StaticSection +import requests +USER_AGENT = 'Sopel/{} (http://sopel.chat)'.format(__version__) +default_headers = {'User-Agent': USER_AGENT} url_finder = None # These are used to clean up the title tag before actually parsing it. Not the # world's best way to do this, but it'll do for now. @@ -33,7 +32,7 @@ class UrlSection(StaticSection): # TODO some validation rules maybe? - exclude = ValidatedAttribute('exclude') + exclude = ListAttribute('exclude') exclusion_char = ValidatedAttribute('exclusion_char', default='!') @@ -81,7 +80,7 @@ def setup(bot=None): bot.memory['last_seen_url'] = tools.SopelMemory() url_finder = re.compile(r'(?u)(%s?(?:http|https|ftp)(?:://\S+))' % - (bot.config.url.exclusion_char)) + (bot.config.url.exclusion_char), re.IGNORECASE) @commands('title') @@ -125,6 +124,9 @@ def title_auto(bot, trigger): return urls = re.findall(url_finder, trigger) + if len(urls) == 0: + return + results = process_urls(bot, trigger, urls) bot.memory['last_seen_url'][trigger.sender] = urls[-1] @@ -154,37 +156,15 @@ def process_urls(bot, trigger, urls): pass # First, check that the URL we got doesn't match matched = check_callbacks(bot, trigger, url, False) - if matched: - continue - # Then see if it redirects anywhere - new_url = follow_redirects(url) - if not new_url: - continue - # Then see if the final URL matches anything - matched = check_callbacks(bot, trigger, new_url, new_url != url) if matched: continue # Finally, actually show the URL - title = find_title(url) + title = find_title(url, verify=bot.config.core.verify_ssl) if title: results.append((title, get_hostname(url))) return results -def follow_redirects(url): - """ - Follow HTTP 3xx redirects, and return the actual URL. Return None if - there's a problem. - """ - try: - connection = web.get_urllib_object(url, 60) - url = connection.geturl() or url - connection.close() - except: - return None - return url - - def check_callbacks(bot, trigger, url, run=True): """ Check the given URL against the callbacks list. If it matches, and ``run`` @@ -197,18 +177,29 @@ def check_callbacks(bot, trigger, url, run=True): for regex, function in tools.iteritems(bot.memory['url_callbacks']): match = regex.search(url) if match: - if run: + # Always run ones from @url; they don't run on their own. + if run or hasattr(function, 'url_regex'): function(bot, trigger, match) matched = True return matched -def find_title(url): +def find_title(url, verify=True): """Return the title for the given URL.""" try: - content, headers = web.get(url, return_headers=True, limit_bytes=max_bytes) - except UnicodeDecodeError: - return # Fail silently when data can't be decoded + response = requests.get(url, stream=True, verify=verify, + headers=default_headers) + content = b'' + for byte in response.iter_content(chunk_size=512): + content += byte + if b'' in content or len(content) > max_bytes: + break + content = content.decode('utf-8', errors='ignore') + # Need to close the connection because we have not read all + # the data + response.close() + except requests.exceptions.ConnectionError: + return None # Some cleanup that I don't really grok, but was in the original, so # we'll keep it (with the compiled regexes made global) for now. diff --git a/sopel/modules/version.py b/sopel/modules/version.py index fb3ae2d98f..a1b4219c94 100644 --- a/sopel/modules/version.py +++ b/sopel/modules/version.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ version.py - Sopel Version Module Copyright 2009, Silas Baronda @@ -7,13 +7,12 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division from datetime import datetime import sopel import re from os import path -import json log_line = re.compile('\S+ (\S+) (.*? <.*?>) (\d+) (\S+)\tcommit[^:]*: (.+)') @@ -60,7 +59,7 @@ def ctcp_version(bot, trigger): @sopel.module.rate(20) def ctcp_source(bot, trigger): bot.write(('NOTICE', trigger.nick), - '\x01SOURCE https://github.com/Embolalia/sopel/\x01') + '\x01SOURCE https://github.com/sopel-irc/sopel/\x01') @sopel.module.rule('\x01PING\s(.*)\x01') diff --git a/sopel/modules/weather.py b/sopel/modules/weather.py index c0b603ca19..daa6ed4c71 100644 --- a/sopel/modules/weather.py +++ b/sopel/modules/weather.py @@ -1,18 +1,12 @@ -# coding=utf8 -""" -weather.py - Sopel Yahoo! Weather Module -Copyright 2008, Sean B. Palmer, inamidst.com -Copyright 2012, Edward Powell, embolalia.net -Licensed under the Eiffel Forum License 2. - -http://sopel.chat -""" -from __future__ import unicode_literals - -from sopel import web +# coding=utf-8 +# Copyright 2008, Sean B. Palmer, inamidst.com +# Copyright 2012, Elsie Powell, embolalia.com +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division + from sopel.module import commands, example, NOLIMIT -import feedparser +import requests import xmltodict @@ -22,24 +16,23 @@ def woeid_search(query): node for the result, so that location data can still be retrieved. Returns None if there is no result, or the woeid field is empty. """ - query = 'q=select * from geo.placefinder where text="%s"' % query - body = web.get('http://query.yahooapis.com/v1/public/yql?' + query, - dont_decode=True) - parsed = xmltodict.parse(body).get('query') + query = 'q=select * from geo.places where text="%s"' % query + body = requests.get('http://query.yahooapis.com/v1/public/yql?' + query) + parsed = xmltodict.parse(body.text).get('query') results = parsed.get('results') - if results is None or results.get('Result') is None: + if results is None or results.get('place') is None: return None - if type(results.get('Result')) is list: - return results.get('Result')[0] - return results.get('Result') + if type(results.get('place')) is list: + return results.get('place')[0] + return results.get('place') def get_cover(parsed): try: - condition = parsed.entries[0]['yweather_condition'] + condition = parsed['channel']['item']['yweather:condition'] except KeyError: return 'unknown' - text = condition['text'] + text = condition['@text'] # code = int(condition['code']) # TODO parse code to get those little icon thingies. return text @@ -47,8 +40,8 @@ def get_cover(parsed): def get_temp(parsed): try: - condition = parsed.entries[0]['yweather_condition'] - temp = int(condition['temp']) + condition = parsed['channel']['item']['yweather:condition'] + temp = int(condition['@temp']) except (KeyError, ValueError): return 'unknown' f = round((temp * 1.8) + 32, 2) @@ -57,7 +50,7 @@ def get_temp(parsed): def get_humidity(parsed): try: - humidity = parsed['feed']['yweather_atmosphere']['humidity'] + humidity = parsed['channel']['yweather:atmosphere']['@humidity'] except (KeyError, ValueError): return 'unknown' return "Humidity: %s%%" % humidity @@ -65,11 +58,11 @@ def get_humidity(parsed): def get_wind(parsed): try: - wind_data = parsed['feed']['yweather_wind'] - kph = float(wind_data['speed']) + wind_data = parsed['channel']['yweather:wind'] + kph = float(wind_data['@speed']) m_s = float(round(kph / 3.6, 1)) speed = int(round(kph / 1.852, 0)) - degrees = int(wind_data['direction']) + degrees = int(wind_data['@direction']) except (KeyError, ValueError): return 'unknown' @@ -143,15 +136,17 @@ def weather(bot, trigger): if not woeid: return bot.reply("I don't know where that is.") - query = web.urlencode({'w': woeid, 'u': 'c'}) - url = 'http://weather.yahooapis.com/forecastrss?' + query - parsed = feedparser.parse(url) - location = parsed['feed']['title'] - - cover = get_cover(parsed) - temp = get_temp(parsed) - humidity = get_humidity(parsed) - wind = get_wind(parsed) + query = 'q=select * from weather.forecast where woeid="%s" and u=\'c\'' % woeid + body = requests.get('http://query.yahooapis.com/v1/public/yql?' + query) + parsed = xmltodict.parse(body.text).get('query') + results = parsed.get('results') + if results is None: + return bot.reply("No forecast available. Try a more specific location.") + location = results.get('channel').get('title') + cover = get_cover(results) + temp = get_temp(results) + humidity = get_humidity(results) + wind = get_wind(results) bot.say(u'%s: %s, %s, %s, %s' % (location, cover, temp, humidity, wind)) @@ -171,12 +166,17 @@ def update_woeid(bot, trigger): bot.db.set_nick_value(trigger.nick, 'woeid', woeid) - neighborhood = first_result.get('neighborhood').text or '' + neighborhood = first_result.get('locality2') or '' if neighborhood: - neighborhood += ',' - city = first_result.get('city') or '' - state = first_result.get('state') or '' - country = first_result.get('country') or '' - uzip = first_result.get('uzip') or '' - bot.reply('I now have you at WOEID %s (%s %s, %s, %s %s.)' % - (woeid, neighborhood, city, state, country, uzip)) + neighborhood = neighborhood.get('#text') + ', ' + city = first_result.get('locality1') or '' + # This is to catch cases like 'Bawlf, Alberta' where the location is + # thought to be a "LocalAdmin" rather than a "Town" + if city: + city = city.get('#text') + else: + city = first_result.get('name') + state = first_result.get('admin1').get('#text') or '' + country = first_result.get('country').get('#text') or '' + bot.reply('I now have you at WOEID %s (%s%s, %s, %s)' % + (woeid, neighborhood, city, state, country)) diff --git a/sopel/modules/wikipedia.py b/sopel/modules/wikipedia.py index c47e8c83b3..d1e9dc1350 100644 --- a/sopel/modules/wikipedia.py +++ b/sopel/modules/wikipedia.py @@ -1,12 +1,7 @@ -# coding=utf8 -""" -wikipedia.py - Sopel Wikipedia Module -Copyright 2013 Edward Powell - embolalia.net -Licensed under the Eiffel Forum License 2. - -http://sopel.chat -""" -from __future__ import unicode_literals +# coding=utf-8 +# Copyright 2013 Elsie Powell - embolalia.com +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division from sopel import web, tools from sopel.config.types import StaticSection, ValidatedAttribute from sopel.module import NOLIMIT, commands, example, rule @@ -15,7 +10,8 @@ import sys if sys.version_info.major < 3: - from urlparse import unquote + from urlparse import unquote as _unquote + unquote = lambda s: _unquote(s.encode('utf-8')).decode('utf-8') else: from urllib.parse import unquote @@ -91,7 +87,7 @@ def mw_snippet(server, query): return snippet['extract'] -@rule('.*/([a-z]+\.wikipedia.org)/wiki/([^ ]+).*') +@rule('.*\/([a-z]+\.wikipedia.org)\/wiki\/((?!File\:)[^ ]+).*') def mw_info(bot, trigger, found_match=None): """ Retrives a snippet of the specified length from the given page on the given diff --git a/sopel/modules/wiktionary.py b/sopel/modules/wiktionary.py index df553a1589..844181c00b 100644 --- a/sopel/modules/wiktionary.py +++ b/sopel/modules/wiktionary.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """ wiktionary.py - Sopel Wiktionary Module Copyright 2009, Sean B. Palmer, inamidst.com @@ -6,7 +6,7 @@ http://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import re from sopel import web diff --git a/sopel/modules/xkcd.py b/sopel/modules/xkcd.py index 787cb6a68c..d9eacbf559 100644 --- a/sopel/modules/xkcd.py +++ b/sopel/modules/xkcd.py @@ -1,21 +1,15 @@ -# coding=utf8 -""" -xkcd.py - XKCD Module -Copyright 2010, Michael Yanovich (yanovich.net), and Morgan Goose -Copyright 2012, Lior Ramati -Copyright 2013, Edward Powell (embolalia.com) -Licensed under the Eiffel Forum License 2. - -http://sopel.chat -""" -from __future__ import unicode_literals - -import json +# coding=utf-8 +# Copyright 2010, Michael Yanovich (yanovich.net), and Morgan Goose +# Copyright 2012, Lior Ramati +# Copyright 2013, Elsie Powell (embolalia.com) +# Licensed under the Eiffel Forum License 2. +from __future__ import unicode_literals, absolute_import, print_function, division + import random import re -from sopel import web +import requests from sopel.modules.search import google_search -from sopel.module import commands +from sopel.module import commands, url ignored_sites = [ # For google searching @@ -32,13 +26,12 @@ sites_query = ' site:xkcd.com -site:' + ' -site:'.join(ignored_sites) -def get_info(number=None): +def get_info(number=None, verify_ssl=True): if number: url = 'http://xkcd.com/{}/info.0.json'.format(number) else: url = 'http://xkcd.com/info.0.json' - data = web.get(url) - data = json.loads(data) + data = requests.get(url, verify=verify_ssl).json() data['url'] = 'http://xkcd.com/' + str(data['num']) return data @@ -61,14 +54,16 @@ def xkcd(bot, trigger): comic if the number is non-positive If non-numeric input is provided it will return the first google result for those keywords on the xkcd.com site """ + verify_ssl = bot.config.core.verify_ssl # get latest comic for rand function and numeric input - latest = get_info() + latest = get_info(verify_ssl=verify_ssl) max_int = latest['num'] # if no input is given (pre - lior's edits code) if not trigger.group(2): # get rand comic random.seed() - requested = get_info(random.randint(1, max_int + 1)) + requested = get_info(random.randint(1, max_int + 1), + verify_ssl=verify_ssl) else: query = trigger.group(2).strip() @@ -77,24 +72,7 @@ def xkcd(bot, trigger): query = int(numbered.group(2)) if numbered.group(1) == "-": query = -query - if query > max_int: - bot.say(("Sorry, comic #{} hasn't been posted yet. " - "The last comic was #{}").format(query, max_int)) - return - elif query <= -max_int: - bot.say(("Sorry, but there were only {} comics " - "released yet so far").format(max_int)) - return - elif abs(query) == 0: - requested = latest - elif query == 404 or max_int + query == 404: - bot.say("404 - Not Found") # don't error on that one - return - elif query > 0: - requested = get_info(query) - else: - # Negative: go back that many from current - requested = get_info(max_int + query) + return numbered_result(bot, query, latest) else: # Non-number: google. if (query.lower() == "latest" or query.lower() == "newest"): @@ -104,7 +82,43 @@ def xkcd(bot, trigger): if not number: bot.say('Could not find any comics for that query.') return - requested = get_info(number) + requested = get_info(number, verify_ssl=verify_ssl) + + say_result(bot, requested) + + +def numbered_result(bot, query, latest, verify_ssl=True): + max_int = latest['num'] + if query > max_int: + bot.say(("Sorry, comic #{} hasn't been posted yet. " + "The last comic was #{}").format(query, max_int)) + return + elif query <= -max_int: + bot.say(("Sorry, but there were only {} comics " + "released yet so far").format(max_int)) + return + elif abs(query) == 0: + requested = latest + elif query == 404 or max_int + query == 404: + bot.say("404 - Not Found") # don't error on that one + return + elif query > 0: + requested = get_info(query, verify_ssl=verify_ssl) + else: + # Negative: go back that many from current + requested = get_info(max_int + query, verify_ssl=verify_ssl) + + say_result(bot, requested) - message = '{} [{}]'.format(requested['url'], requested['title']) + +def say_result(bot, result): + message = '{} | {} | Alt-text: {}'.format(result['url'], result['title'], + result['alt']) bot.say(message) + + +@url('xkcd.com/(\d+)') +def get_url(bot, trigger, match): + verify_ssl = bot.config.core.verify_ssl + latest = get_info(verify_ssl=verify_ssl) + numbered_result(bot, int(match.group(1)), latest) diff --git a/sopel/run_script.py b/sopel/run_script.py index 5998b541ba..a6fa37c32f 100755 --- a/sopel/run_script.py +++ b/sopel/run_script.py @@ -8,8 +8,7 @@ http://sopel.chat """ -from __future__ import unicode_literals -from __future__ import print_function +from __future__ import unicode_literals, absolute_import, print_function, division import sys from sopel.tools import stderr @@ -63,7 +62,7 @@ def main(argv=None): parser.add_argument('-c', '--config', metavar='filename', help='use a specific configuration file') parser.add_argument("-d", '--fork', action="store_true", - dest="deamonize", help="Deamonize sopel") + dest="daemonize", help="Daemonize sopel") parser.add_argument("-q", '--quit', action="store_true", dest="quit", help="Gracefully quit Sopel") parser.add_argument("-k", '--kill', action="store_true", dest="kill", @@ -84,7 +83,10 @@ def main(argv=None): 'module configuration options.')) parser.add_argument('-v', '--version', action="store_true", dest="version", help="Show version number and exit") - opts = parser.parse_args() + if argv: + opts = parser.parse_args(argv) + else: + opts = parser.parse_args() # Step Two: "Do not run as root" checks. try: @@ -145,7 +147,7 @@ def main(argv=None): logfile = os.path.os.path.join(config_module.core.logdir, 'stdio.log') - config_module._is_deamonized = opts.deamonize + config_module._is_daemonized = opts.daemonize sys.stderr = tools.OutputRedirect(logfile, True, opts.quiet) sys.stdout = tools.OutputRedirect(logfile, False, opts.quiet) @@ -181,14 +183,13 @@ def main(argv=None): else: os.kill(old_pid, signal.SIGTERM) sys.exit(0) - elif old_pid is None or (not tools.check_pid(old_pid) - and (opts.kill or opts.quit)): + elif opts.kill or opts.quit: stderr('Sopel is not running!') sys.exit(1) elif opts.quit or opts.kill: stderr('Sopel is not running!') sys.exit(1) - if opts.deamonize: + if opts.daemonize: child_pid = os.fork() if child_pid is not 0: sys.exit() diff --git a/sopel/test_tools.py b/sopel/test_tools.py index 9781a8bc7f..e6bd184251 100644 --- a/sopel/test_tools.py +++ b/sopel/test_tools.py @@ -7,7 +7,7 @@ https://sopel.chat """ -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import os import re diff --git a/sopel/tools/__init__.py b/sopel/tools/__init__.py index 91421c6d93..943383d5d6 100644 --- a/sopel/tools/__init__.py +++ b/sopel/tools/__init__.py @@ -1,21 +1,18 @@ -# coding=utf8 -""" +# coding=utf-8 +"""Useful miscellaneous tools and shortcuts for Sopel modules + *Availability: 3+* -``tools`` contains a number of useful miscellaneous tools and shortcuts for use -in Sopel modules.""" +""" # tools.py - Sopel misc tools # Copyright 2008, Sean B. Palmer, inamidst.com # Copyright © 2012, Elad Alfassa -# Copyright 2012, Edward Powell, embolalia.net +# Copyright 2012, Elsie Powell, embolalia.com # Licensed under the Eiffel Forum License 2. # https://sopel.chat -from __future__ import division -from __future__ import print_function -from __future__ import unicode_literals -from __future__ import absolute_import +from __future__ import unicode_literals, absolute_import, print_function, division import sys import os @@ -25,7 +22,10 @@ import traceback from collections import defaultdict +from sopel.tools._events import events # NOQA + if sys.version_info.major >= 3: + raw_input = input unicode = str iteritems = dict.items itervalues = dict.values diff --git a/sopel/tools/_events.py b/sopel/tools/_events.py new file mode 100644 index 0000000000..b805f06076 --- /dev/null +++ b/sopel/tools/_events.py @@ -0,0 +1,203 @@ +# coding=utf-8 +from __future__ import unicode_literals, absolute_import, print_function, division + + +class events(object): + """An enumeration of all the standardized and notable IRC numeric events + + This allows you to do, for example, @module.event(events.RPL_WELCOME) + rather than @module.event('001') + """ + # ###################################################### Non-RFC / Non-IRCv3 + # Only add things here if they're actually in common use across multiple + # ircds. + RPL_ISUPPORT = '005' + RPL_WHOSPCRPL = '354' + + # ################################################################### IRC v3 + # ## 3.1 + # CAP + ERR_INVALIDCAPCMD = '410' + # SASL + RPL_LOGGEDIN = '900' + RPL_LOGGEDOUT = '901' + ERR_NICKLOCKED = '902' + RPL_SASLSUCCESS = '903' + ERR_SASLFAIL = '904' + ERR_SASLTOOLONG = '905' + ERR_SASLABORTED = '906' + ERR_SASLALREADY = '907' + RPL_SASLMECHS = '908' + # TLS + RPL_STARTTLS = '670' + ERR_STARTTLS = '691' + # ## 3.2 + # Metadata + RPL_WHOISKEYVALUE = '760' + RPL_KEYVALUE = '761' + RPL_METADATAEND = '762' + ERR_METADATALIMIT = '764' + ERR_TARGETINVALID = '765' + ERR_NOMATCHINGKEY = '766' + ERR_KEYINVALID = '767' + ERR_KEYNOTSET = '768' + ERR_KEYNOPERMISSION = '769' + # Monitor + RPL_MONONLINE = '730' + RPL_MONOFFLINE = '731' + RPL_MONLIST = '732' + RPL_ENDOFMONLIST = '733' + ERR_MONLISTFULL = '734' + + # ################################################################# RFC 1459 + # ## 6.1 Error Replies. + ERR_NOSUCHNICK = '401' + ERR_NOSUCHSERVER = '402' + ERR_NOSUCHCHANNEL = '403' + ERR_CANNOTSENDTOCHAN = '404' + ERR_TOOMANYCHANNELS = '405' + ERR_WASNOSUCHNICK = '406' + ERR_TOOMANYTARGETS = '407' + ERR_NOORIGIN = '409' + ERR_NORECIPIENT = '411' + ERR_NOTEXTTOSEND = '412' + ERR_NOTOPLEVEL = '413' + ERR_WILDTOPLEVEL = '414' + ERR_UNKNOWNCOMMAND = '421' + ERR_NOMOTD = '422' + ERR_NOADMININFO = '423' + ERR_FILEERROR = '424' + ERR_NONICKNAMEGIVEN = '431' + ERR_ERRONEUSNICKNAME = '432' + ERR_NICKNAMEINUSE = '433' + ERR_NICKCOLLISION = '436' + ERR_USERNOTINCHANNEL = '441' + ERR_NOTONCHANNEL = '442' + ERR_USERONCHANNEL = '443' + ERR_NOLOGIN = '444' + ERR_SUMMONDISABLED = '445' + ERR_USERSDISABLED = '446' + ERR_NOTREGISTERED = '451' + ERR_NEEDMOREPARAMS = '461' + ERR_ALREADYREGISTRED = '462' + ERR_NOPERMFORHOST = '463' + ERR_PASSWDMISMATCH = '464' + ERR_YOUREBANNEDCREEP = '465' + ERR_KEYSET = '467' + ERR_CHANNELISFULL = '471' + ERR_UNKNOWNMODE = '472' + ERR_INVITEONLYCHAN = '473' + ERR_BANNEDFROMCHAN = '474' + ERR_BADCHANNELKEY = '475' + ERR_NOPRIVILEGES = '481' + ERR_CHANOPRIVSNEEDED = '482' + ERR_CANTKILLSERVER = '483' + ERR_NOOPERHOST = '491' + ERR_UMODEUNKNOWNFLAG = '501' + ERR_USERSDONTMATCH = '502' + # ## 6.2 Command responses. + RPL_NONE = '300' + RPL_USERHOST = '302' + RPL_ISON = '303' + RPL_AWAY = '301' + RPL_UNAWAY = '305' + RPL_NOWAWAY = '306' + RPL_WHOISUSER = '311' + RPL_WHOISSERVER = '312' + RPL_WHOISOPERATOR = '313' + RPL_WHOISIDLE = '317' + RPL_ENDOFWHOIS = '318' + RPL_WHOISCHANNELS = '319' + RPL_WHOWASUSER = '314' + RPL_ENDOFWHOWAS = '369' + RPL_LISTSTART = '321' + RPL_LIST = '322' + RPL_LISTEND = '323' + RPL_CHANNELMODEIS = '324' + RPL_NOTOPIC = '331' + RPL_TOPIC = '332' + RPL_INVITING = '341' + RPL_SUMMONING = '342' + RPL_VERSION = '351' + RPL_WHOREPLY = '352' + RPL_ENDOFWHO = '315' + RPL_NAMREPLY = '353' + RPL_ENDOFNAMES = '366' + RPL_LINKS = '364' + RPL_ENDOFLINKS = '365' + RPL_BANLIST = '367' + RPL_ENDOFBANLIST = '368' + RPL_INFO = '371' + RPL_ENDOFINFO = '374' + RPL_MOTDSTART = '375' + RPL_MOTD = '372' + RPL_ENDOFMOTD = '376' + RPL_YOUREOPER = '381' + RPL_REHASHING = '382' + RPL_TIME = '391' + RPL_USERSSTART = '392' + RPL_USERS = '393' + RPL_ENDOFUSERS = '394' + RPL_NOUSERS = '395' + RPL_TRACELINK = '200' + RPL_TRACECONNECTING = '201' + RPL_TRACEHANDSHAKE = '202' + RPL_TRACEUNKNOWN = '203' + RPL_TRACEOPERATOR = '204' + RPL_TRACEUSER = '205' + RPL_TRACESERVER = '206' + RPL_TRACENEWTYPE = '208' + RPL_TRACELOG = '261' + RPL_STATSLINKINFO = '211' + RPL_STATSCOMMANDS = '212' + RPL_STATSCLINE = '213' + RPL_STATSNLINE = '214' + RPL_STATSILINE = '215' + RPL_STATSKLINE = '216' + RPL_STATSYLINE = '218' + RPL_ENDOFSTATS = '219' + RPL_STATSLLINE = '241' + RPL_STATSUPTIME = '242' + RPL_STATSOLINE = '243' + RPL_STATSHLINE = '244' + RPL_UMODEIS = '221' + RPL_LUSERCLIENT = '251' + RPL_LUSEROP = '252' + RPL_LUSERUNKNOWN = '253' + RPL_LUSERCHANNELS = '254' + RPL_LUSERME = '255' + RPL_ADMINME = '256' + RPL_ADMINLOC1 = '257' + RPL_ADMINLOC2 = '258' + RPL_ADMINEMAIL = '259' + + # ################################################################# RFC 2812 + # ## 5.1 Command responses + RPL_WELCOME = '001' + RPL_YOURHOST = '002' + RPL_CREATED = '003' + RPL_MYINFO = '004' + RPL_BOUNCE = '005' + RPL_UNIQOPIS = '325' + RPL_INVITELIST = '346' + RPL_ENDOFINVITELIST = '347' + RPL_EXCEPTLIST = '348' + RPL_ENDOFEXCEPTLIST = '349' + RPL_YOURESERVICE = '383' + RPL_TRACESERVICE = '207' + RPL_TRACECLASS = '209' + RPL_TRACERECONNECT = '210' + RPL_TRACEEND = '262' + RPL_SERVLIST = '234' + RPL_SERVLISTEND = '235' + RPL_TRYAGAIN = '263' + # ## 5.2 Error Replies + ERR_NOSUCHSERVICE = '408' + ERR_BADMASK = '415' + ERR_UNAVAILRESOURCE = '437' + ERR_YOUWILLBEBANNED = '466' + ERR_BADCHANMASK = '476' + ERR_NOCHANMODES = '477' + ERR_BANLISTFULL = '478' + ERR_RESTRICTED = '484' + ERR_UNIQOPPRIVSNEEDED = '485' diff --git a/sopel/tools/calculation.py b/sopel/tools/calculation.py index 636e10ac06..bf4ab8b397 100644 --- a/sopel/tools/calculation.py +++ b/sopel/tools/calculation.py @@ -1,7 +1,6 @@ -# coding=utf8 +# coding=utf-8 """Tools to help safely do calculations from user input""" -from __future__ import unicode_literals -from __future__ import absolute_import +from __future__ import unicode_literals, absolute_import, print_function, division import time import numbers diff --git a/sopel/tools/jobs.py b/sopel/tools/jobs.py index 35c386ef4a..f1be421558 100644 --- a/sopel/tools/jobs.py +++ b/sopel/tools/jobs.py @@ -1,5 +1,5 @@ -# coding=utf8 -from __future__ import unicode_literals, absolute_import +# coding=utf-8 +from __future__ import unicode_literals, absolute_import, print_function, division import copy import datetime diff --git a/sopel/tools/target.py b/sopel/tools/target.py new file mode 100644 index 0000000000..0757bb7eba --- /dev/null +++ b/sopel/tools/target.py @@ -0,0 +1,90 @@ +# coding=utf-8 +from __future__ import unicode_literals, absolute_import, print_function, division + +import functools +from sopel.tools import Identifier + + +@functools.total_ordering +class User(object): + """A representation of a user Sopel is aware of.""" + def __init__(self, nick, user, host): + assert isinstance(nick, Identifier) + self.nick = nick + """The user's nickname.""" + self.user = user + """The user's local username.""" + self.host = host + """The user's hostname.""" + self.channels = {} + """The channels the user is in. + + This maps channel name ``Identifier``\s to ``Channel`` objects.""" + self.account = None + """The IRC services account of the user. + + This relies on IRCv3 account tracking being enabled.""" + self.away = None + """Whether the user is marked as away.""" + + hostmask = property(lambda self: '{}!{}@{}'.format(self.nick, self.user, + self.host)) + """The user's full hostmask.""" + + def __eq__(self, other): + if not isinstance(other, User): + return NotImplemented + return self.nick == other.nick + + def __lt__(self, other): + if not isinstance(other, User): + return NotImplemented + return self.nick < other.nick + + +@functools.total_ordering +class Channel(object): + """A representation of a channel Sopel is in.""" + def __init__(self, name): + assert isinstance(name, Identifier) + self.name = name + """The name of the channel.""" + self.users = {} + """The users in the channel. + + This maps username ``Identifier``\s to channel objects.""" + self.privileges = {} + """The permissions of the users in the channel. + + This maps username ``Identifier``s to bitwise integer values. This can + be compared to appropriate constants from ``sopel.module``.""" + self.topic = '' + """The topic of the channel.""" + + def clear_user(self, nick): + user = self.users.pop(nick, None) + self.privileges.pop(nick, None) + if user != None: + user.channels.pop(self.name, None) + + def add_user(self, user): + assert isinstance(user, User) + self.users[user.nick] = user + self.privileges[user.nick] = 0 + user.channels[self.name] = self + + def rename_user(self, old, new): + if old in self.users: + self.users[new] = self.users.pop(old) + if old in self.privileges: + self.privileges[new] = self.privileges.pop(old) + + def __eq__(self, other): + if not isinstance(other, Channel): + return NotImplemented + return self.name == other.name + + def __lt__(self, other): + if not isinstance(other, Channel): + return NotImplemented + return self.name < other.name diff --git a/sopel/tools/time.py b/sopel/tools/time.py index 460129338f..afe311a595 100644 --- a/sopel/tools/time.py +++ b/sopel/tools/time.py @@ -1,7 +1,6 @@ -# coding=utf8 +# coding=utf-8 """Tools for getting and displaying the time.""" -from __future__ import unicode_literals -from __future__ import absolute_import +from __future__ import unicode_literals, absolute_import, print_function, division import datetime try: diff --git a/sopel/trigger.py b/sopel/trigger.py index 812d49f9ef..7d9c7e015d 100644 --- a/sopel/trigger.py +++ b/sopel/trigger.py @@ -1,8 +1,9 @@ # coding=utf-8 -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import re import sys +import datetime import sopel.tools @@ -15,7 +16,7 @@ class PreTrigger(object): """A parsed message from the server, which has not been matched against any rules.""" component_regex = re.compile(r'([^!]*)!?([^@]*)@?(.*)') - intent_regex = re.compile('\x01(\\S+) (.*)\x01') + intent_regex = re.compile('\x01(\\S+) ?(.*)\x01') def __init__(self, own_nick, line): """own_nick is the bot's nick, needed to correctly parse sender. @@ -34,6 +35,13 @@ def __init__(self, own_nick, line): else: self.tags[tag[0]] = None + self.time = datetime.datetime.utcnow() + if 'time' in self.tags: + try: + self.time = datetime.datetime.strptime(self.tags['time'], '%Y-%m-%dT%H:%M:%S.%fZ') + except ValueError: + pass # Server isn't conforming to spec, ignore the server-time + # TODO note what this is doing and why if line.startswith(':'): self.hostmask, line = line[1:].split(' ', 1) @@ -56,7 +64,8 @@ def __init__(self, own_nick, line): self.nick = sopel.tools.Identifier(self.nick) # If we have arguments, the first one is the sender - if self.args: + # Unless it's a QUIT event + if self.args and self.event != 'QUIT': target = sopel.tools.Identifier(self.args[0]) else: target = None @@ -75,6 +84,11 @@ def __init__(self, own_nick, line): self.tags['intent'] = intent self.args[-1] = message or '' + # Populate account from extended-join messages + if self.event == 'JOIN' and len(self.args) == 3: + # Account is the second arg `...JOIN #Sopel account :realname` + self.tags['account'] = self.args[1] + class Trigger(unicode): """A line from the server, which has matched a callable's rules. @@ -87,6 +101,11 @@ class Trigger(unicode): """The channel from which the message was sent. In a private message, this is the nick that sent the message.""" + time = property(lambda self: self._pretrigger.time) + """A datetime object at which the message was received by the IRC server. + + If the server does not support server-time, then `time` will be the time + that the message was received by Sopel""" raw = property(lambda self: self._pretrigger.line) """The entire message, as sent from the server. This includes the CTCP \\x01 bytes and command, if they were included.""" @@ -97,24 +116,28 @@ class Trigger(unicode): user = property(lambda self: self._pretrigger.user) """Local username of the person who sent the message""" nick = property(lambda self: self._pretrigger.nick) - """The ``Identifier`` of the person who sent the message.""" + """The :class:`sopel.tools.Identifier` of the person who sent the message. + """ host = property(lambda self: self._pretrigger.host) """The hostname of the person who sent the message""" event = property(lambda self: self._pretrigger.event) """The IRC event (e.g. ``PRIVMSG`` or ``MODE``) which triggered the message.""" match = property(lambda self: self._match) - """The regular expression `MatchObject`_ for the triggering line. - - .. _MatchObject: http://docs.python.org/library/re.html#match-objects""" + """The regular expression :class:`re.MatchObject` for the triggering line. + """ group = property(lambda self: self._match.group) """The ``group`` function of the ``match`` attribute. - See Python `re`_ documentation for details.""" + See Python :mod:`re` documentation for details.""" groups = property(lambda self: self._match.groups) """The ``groups`` function of the ``match`` attribute. - See Python `re`_ documentation for details.""" + See Python :mod:`re` documentation for details.""" + groupdict = property(lambda self: self._match.groupdict) + """The ``groupdict`` function of the ``match`` attribute. + + See Python :mod:`re` documentation for details.""" args = property(lambda self: self._pretrigger.args) """ A tuple containing each of the arguments to an event. These are the @@ -129,12 +152,20 @@ class Trigger(unicode): """ owner = property(lambda self: self._owner) """True if the nick which triggered the command is the bot's owner.""" + account = property(lambda self: self.tags.get('account') or self._account) + """The account name of the user sending the message. - def __new__(cls, config, message, match): - self = unicode.__new__(cls, message.args[-1]) + This is only available if either the account-tag or the account-notify and + extended-join capabilites are available. If this isn't the case, or the user + sending the message isn't logged in, this will be None. + """ + + def __new__(cls, config, message, match, account=None): + self = unicode.__new__(cls, message.args[-1] if message.args else '') + self._account = account self._pretrigger = message self._match = match - self._is_privmsg = message.sender.is_nick() + self._is_privmsg = message.sender and message.sender.is_nick() def match_host_or_nick(pattern): pattern = sopel.tools.get_hostmask_regex(pattern) @@ -143,9 +174,14 @@ def match_host_or_nick(pattern): pattern.match('@'.join((self.nick, self.host))) ) - self._admin = any(match_host_or_nick(item) - for item in config.core.admins) - self._owner = match_host_or_nick(config.core.owner) - self._admin = self.admin or self.owner + if config.core.owner_account: + self._owner = config.core.owner_account == self.account + else: + self._owner = match_host_or_nick(config.core.owner) + self._admin = ( + self._owner or + self.account in config.core.admin_accounts or + any(match_host_or_nick(item) for item in config.core.admins) + ) return self diff --git a/sopel/web.py b/sopel/web.py index 55c45904a6..b13ff39911 100644 --- a/sopel/web.py +++ b/sopel/web.py @@ -1,40 +1,39 @@ # coding=utf-8 """ -*Availability: 3+* +*Availability: 3+, depreacted in 6.2.0* The web class contains essential web-related functions for interaction with web applications or websites in your modules. It supports HTTP GET, HTTP POST and HTTP HEAD. """ -#Copyright © 2008, Sean B. Palmer, inamidst.com -#Copyright © 2009, Michael Yanovich -#Copyright © 2012, Dimitri Molenaars, Tyrope.nl. -#Copyright © 2012-2013, Elad Alfassa, -#Licensed under the Eiffel Forum License 2. +# Copyright © 2008, Sean B. Palmer, inamidst.com +# Copyright © 2009, Michael Yanovich +# Copyright © 2012, Dimitri Molenaars, Tyrope.nl. +# Copyright © 2012-2013, Elad Alfassa, +# Licensed under the Eiffel Forum License 2. -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division import re import sys import urllib -import os.path -import socket +import requests from sopel import __version__ +from sopel.tools import deprecated if sys.version_info.major < 3: - import urllib2 import httplib from htmlentitydefs import name2codepoint from urlparse import urlparse from urlparse import urlunparse else: - import urllib.request as urllib2 import http.client as httplib from html.entities import name2codepoint from urllib.parse import urlparse from urllib.parse import urlunparse unichr = chr + unicode = str try: import ssl @@ -48,15 +47,29 @@ has_ssl = False USER_AGENT = 'Sopel/{} (http://sopel.chat)'.format(__version__) +default_headers = {'User-Agent': USER_AGENT} +ca_certs = None # Will be overriden when config loads. This is for an edge case. + + +class MockHttpResponse(httplib.HTTPResponse): + "Mock HTTPResponse with data that comes from requests." + def __init__(self, response): + self.headers = response.headers + self.status = response.status_code + self.reason = response.reason + self.close = response.close + self.read = response.raw.read + self.url = response.url + + def geturl(self): + return self.url # HTTP GET -# Note: dont_decode is a horrible name for an argument, double negative -# is super confusing. We need to replace it, maybe in 5.0 because this would -# mean breaking backwards compatability +@deprecated def get(uri, timeout=20, headers=None, return_headers=False, limit_bytes=None, verify_ssl=True, dont_decode=False): - """Execute an HTTP GET query on `uri`, and return the result. + """Execute an HTTP GET query on `uri`, and return the result. Deprecated. `timeout` is an optional argument, which represents how much time we should wait before throwing a timeout exception. It defaults to 20, but can be set @@ -64,39 +77,34 @@ def get(uri, timeout=20, headers=None, return_headers=False, `headers` is a dict of HTTP headers to send with the request. If `return_headers` is True, return a tuple of (bytes, headers) - If `limit_bytes` is provided, only read that many bytes from the URL. This - may be a good idea when reading from unknown sites, to prevent excessively - large files from being downloaded. + `limit_bytes` is ignored. """ if not uri.startswith('http'): uri = "http://" + uri - u = get_urllib_object(uri, timeout, headers, verify_ssl) - bytes = u.read(limit_bytes) + if headers is None: + headers = default_headers + else: + tmp = default_headers.copy() + tmp.update(headers) + headers = tmp + u = requests.get(uri, timeout=timeout, headers=headers, verify=verify_ssl) + bytes = u.content u.close() - headers = dict(u.info()) + headers = u.headers if not dont_decode: - # Detect encoding automatically from HTTP headers - content_type = headers.get('content-type') or '' - encoding_match = re.match('.*?charset *= *(\S+)', content_type, re.IGNORECASE) - if encoding_match: - try: - bytes = bytes.decode(encoding_match.group(1)) - except: - # attempt unicode on failure - encoding_match = None - if not encoding_match: - bytes = bytes.decode('utf-8', "ignore") + bytes = u.text if not return_headers: return bytes else: - headers['_http_status'] = u.code + headers['_http_status'] = u.status_code return (bytes, headers) # Get HTTP headers +@deprecated def head(uri, timeout=20, headers=None, verify_ssl=True): - """Execute an HTTP GET query on `uri`, and return the headers. + """Execute an HTTP GET query on `uri`, and return the headers. Deprecated. `timeout` is an optional argument, which represents how much time we should wait before throwing a timeout exception. It defaults to 20, but can be set @@ -105,34 +113,39 @@ def head(uri, timeout=20, headers=None, verify_ssl=True): """ if not uri.startswith('http'): uri = "http://" + uri - u = get_urllib_object(uri, timeout, headers, verify_ssl) - info = u.info() + if headers is None: + headers = default_headers + else: + tmp = default_headers.copy() + tmp.update(headers) + headers = tmp + u = requests.get(uri, timeout=timeout, headers=headers, verify=verify_ssl) + info = u.headers u.close() return info # HTTP POST +@deprecated def post(uri, query, limit_bytes=None, timeout=20, verify_ssl=True, return_headers=False): - """Execute an HTTP POST query. + """Execute an HTTP POST query. Deprecated. `uri` is the target URI, and `query` is the POST data. `headers` is a dict of HTTP headers to send with the request. - If `limit_bytes` is provided, only read that many bytes from the URL. This - may be a good idea when reading from unknown sites, to prevent excessively - large files from being downloaded. + `limit_bytes` is ignored. """ if not uri.startswith('http'): uri = "http://" + uri - u = get_urllib_object(uri, timeout=timeout, verify_ssl=verify_ssl, data=query) - bytes = u.read(limit_bytes) - headers = dict(u.info()) + u = requests.post(uri, timeout=timeout, verify=verify_ssl, data=query) + bytes = u.raw.read(limit_bytes) + headers = u.headers u.close() if not return_headers: return bytes else: - headers['_http_status'] = u.code + headers['_http_status'] = u.status_code return (bytes, headers) r_entity = re.compile(r'&([^;\s]+);') @@ -153,81 +166,28 @@ def decode(html): return r_entity.sub(entity, html) -class VerifiedHTTPSConnection(httplib.HTTPConnection): - "Verified HTTPS Connection handler" - - default_port = httplib.HTTPS_PORT - - def __init__(self, *args, **kwargs): - if not has_ssl: - raise Exception('SSL verification is not available.') - httplib.HTTPConnection.__init__(self, *args, **kwargs) - - def connect(self): - """Connect to the host and port specified in __init__.""" - sock = socket.create_connection((self.host, self.port), - self.timeout, self.source_address) - if self._tunnel_host: - self.sock = sock - self._tunnel() - if not os.path.exists(ca_certs): - raise Exception('CA Certificate bundle %s is not readable' % ca_certs) - self.sock = ssl.wrap_socket(sock, - ca_certs=ca_certs, - cert_reqs=ssl.CERT_REQUIRED) - ssl.match_hostname(self.sock.getpeercert(), self.host) - - -class VerifiedHTTPSHandler(urllib2.HTTPSHandler): - - def https_open(self, req): - return self.do_open(VerifiedHTTPSConnection, req) - - # For internal use in web.py, (modules can use this if they need a urllib # object they can execute read() on) Both handles redirects and makes sure # input URI is UTF-8 +@deprecated def get_urllib_object(uri, timeout, headers=None, verify_ssl=True, data=None): - """Return a urllib2 object for `uri` and `timeout` and `headers`. - - This is better than using urlib2 directly, for it handles SSL verifcation, makes - sure URI is utf8, and is shorter and easier to use. Modules may use this - if they need a urllib2 object to execute .read() on. - - For more information, refer to the urllib2 documentation. + """Return an HTTPResponse object for `uri` and `timeout` and `headers`. Deprecated """ - uri = quote_query(uri) - - try: - # Check if we need to do IDN parsing - uri.encode('ascii') - except: - uri = iri_to_uri(uri) - - original_headers = {'Accept': '*/*', 'User-Agent': USER_AGENT} - if headers is not None: - original_headers.update(headers) + if headers is None: + headers = default_headers else: - headers = original_headers - - if verify_ssl: - opener = urllib2.build_opener(VerifiedHTTPSHandler) + tmp = default_headers.copy() + tmp.update(headers) + headers = tmp + if data is not None: + response = requests.post(uri, timeout=timeout, verify=verify_ssl, + data=data, headers=headers) else: - opener = urllib2.build_opener() - - if type(data) is dict: - data = urlencode(data).encode('utf-8') - - req = urllib2.Request(uri, headers=headers, data=data) - try: - u = opener.open(req, None, timeout) - except urllib2.HTTPError as e: - # Even when there's an error (say HTTP 404), return page contents - return e.fp - - return u + response = requests.get(uri, timeout=timeout, verify=verify_ssl, + headers=headers) + return MockHttpResponse(response) # Identical to urllib2.quote diff --git a/test/test_db.py b/test/test_db.py index b9b1040cd5..b00e5bbb28 100644 --- a/test/test_db.py +++ b/test/test_db.py @@ -1,4 +1,4 @@ -# coding=utf8 +# coding=utf-8 """Tests for the new database functionality. TODO: Most of these tests assume functionality tested in other tests. This is @@ -83,7 +83,6 @@ def test_get_nick_id(db): def test_alias_nick(db): - conn = sqlite3.connect(db_filename) nick = 'Embolalia' aliases = ['EmbölaliÅ', 'Embo`work', 'Embo'] @@ -245,11 +244,11 @@ def test_get_channel_value(db): def test_get_nick_or_channel_value(db): db.set_nick_value('asdf', 'qwer', 'poiu') db.set_channel_value('#asdf', 'qwer', '/.,m') - assert db.get_nick_or_channel_value('asdf', 'qwer', 'poiu') - assert db.get_nick_or_channel_value('#asdf', 'qwer', '/.,m') + assert db.get_nick_or_channel_value('asdf', 'qwer') == 'poiu' + assert db.get_nick_or_channel_value('#asdf', 'qwer') == '/.,m' -def test_get_nick_or_channel_value(db): +def test_get_preferred_value(db): db.set_nick_value('asdf', 'qwer', 'poiu') db.set_channel_value('#asdf', 'qwer', '/.,m') db.set_channel_value('#asdf', 'lkjh', '1234') diff --git a/test/test_formatting.py b/test/test_formatting.py new file mode 100644 index 0000000000..f870882674 --- /dev/null +++ b/test/test_formatting.py @@ -0,0 +1,26 @@ +# coding=utf-8 +"""Tests for message formatting""" +from __future__ import unicode_literals, absolute_import, print_function, division + +import pytest + +from sopel.formatting import colors, color, bold, underline + + +def test_color(): + text = 'Hello World' + assert color(text) == text + assert color(text, colors.PINK) == '\x0313' + text + '\x03' + assert color(text, colors.PINK, colors.TEAL) == '\x0313,10' + text + '\x03' + pytest.raises(ValueError, color, text, 100) + pytest.raises(ValueError, color, text, 'INVALID') + + +def test_bold(): + text = 'Hello World' + assert bold(text) == '\x02' + text + '\x02' + + +def test_underline(): + text = 'Hello World' + assert underline(text) == '\x1f' + text + '\x1f' diff --git a/test/test_irc.py b/test/test_irc.py new file mode 100644 index 0000000000..44c22c1495 --- /dev/null +++ b/test/test_irc.py @@ -0,0 +1,147 @@ +# coding=utf8 +"""Tests for message formatting""" +from __future__ import unicode_literals + +import pytest + +import asynchat +import os +import shutil +import socket +import select +import tempfile +import threading +import time +import asyncore + +from sopel import irc +from sopel.tools import stderr, Identifier +import sopel.config as conf + + +HOST = '127.0.0.1' +SERVER_QUIT = 'QUIT' + + +class BasicServer(asyncore.dispatcher): + def __init__(self, address, handler): + asyncore.dispatcher.__init__(self) + self.response_handler = handler + self.create_socket(socket.AF_INET, socket.SOCK_STREAM) + self.bind(address) + self.address = self.socket.getsockname() + self.listen(1) + return + + def handle_accept(self): + # Called when a client connects to our socket + client_info = self.accept() + BasicHandler(sock=client_info[0], handler=self.response_handler) + self.handle_close() + return + + def handle_close(self): + self.close() + +class BasicHandler(asynchat.async_chat): + ac_in_buffer_size = 512 + ac_out_buffer_size = 512 + + def __init__(self, sock, handler): + self.received_data = [] + asynchat.async_chat.__init__(self, sock) + self.handler_function = handler + self.set_terminator(b'\n') + return + + def collect_incoming_data(self, data): + self.received_data.append(data.decode('utf-8')) + + def found_terminator(self): + self._process_command() + + def _process_command(self): + command = ''.join(self.received_data) + response = self.handler_function(self, command) + self.push(':fake.server {}\n'.format(response).encode()) + self.received_data = [] + + +def start_server(rpl_function=None): + def rpl_func(msg): + print(msg) + return msg + + if rpl_function is None: + rpl_function = rpl_func + + address = ('localhost', 0) # let the kernel give us a port + server = BasicServer(address, rpl_function) + return server + + +@pytest.fixture +def bot(request): + cfg_dir = tempfile.mkdtemp() + print(cfg_dir) + filename = tempfile.mkstemp(dir=cfg_dir)[1] + os.mkdir(os.path.join(cfg_dir, 'modules')) + def fin(): + print('teardown config file') + shutil.rmtree(cfg_dir) + request.addfinalizer(fin) + + def gen(data): + with open(filename, 'w') as fileo: + fileo.write(data) + cfg = conf.Config(filename) + irc_bot = irc.Bot(cfg) + irc_bot.config = cfg + return irc_bot + + return gen + + +def test_bot_init(bot): + test_bot = bot( + '[core]\n' + 'owner=Baz\n' + 'nick=Foo\n' + 'user=Bar\n' + 'name=Sopel\n' + ) + assert test_bot.nick == Identifier('Foo') + assert test_bot.user == 'Bar' + assert test_bot.name == 'Sopel' + + +def basic_irc_replies(server, msg): + if msg.startswith('NICK'): + return '001 Foo :Hello' + elif msg.startswith('USER'): + # Quit here because good enough + server.close() + elif msg.startswith('PING'): + return 'PONG{}'.format(msg.replace('PING','',1)) + elif msg.startswith('CAP'): + return 'CAP * :' + elif msg.startswith('QUIT'): + server.close() + else: + return '421 {} :Unknown command'.format(msg) + + +def test_bot_connect(bot): + test_bot = bot( + '[core]\n' + 'owner=Baz\n' + 'nick=Foo\n' + 'user=Bar\n' + 'name=Sopel\n' + 'host=127.0.0.1\n' + 'timeout=10\n' + ) + s = start_server(basic_irc_replies) + + # Do main run + test_bot.run(HOST, s.address[1]) diff --git a/test/test_module.py b/test/test_module.py new file mode 100644 index 0000000000..5214d75251 --- /dev/null +++ b/test/test_module.py @@ -0,0 +1,210 @@ +# coding=utf-8 +"""Tests for message formatting""" +from __future__ import unicode_literals, absolute_import, print_function, division + +import pytest + +from sopel.trigger import PreTrigger, Trigger +from sopel.test_tools import MockSopel, MockSopelWrapper +from sopel.tools import Identifier +from sopel import module + + +@pytest.fixture +def sopel(): + bot = MockSopel('Sopel') + bot.config.core.owner = 'Bar' + return bot + + +@pytest.fixture +def bot(sopel, pretrigger): + bot = MockSopelWrapper(sopel, pretrigger) + bot.privileges = dict() + bot.privileges[Identifier('#Sopel')] = dict() + bot.privileges[Identifier('#Sopel')][Identifier('Foo')] = module.VOICE + return bot + + +@pytest.fixture +def pretrigger(): + line = ':Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + return PreTrigger(Identifier('Foo'), line) + + +@pytest.fixture +def pretrigger_pm(): + line = ':Foo!foo@example.com PRIVMSG Sopel :Hello, world' + return PreTrigger(Identifier('Foo'), line) + + +@pytest.fixture +def trigger_owner(bot): + line = ':Bar!bar@example.com PRIVMSG #Sopel :Hello, world' + return Trigger(bot.config, PreTrigger(Identifier('Bar'), line), None) + + +@pytest.fixture +def trigger(bot, pretrigger): + return Trigger(bot.config, pretrigger, None) + + +@pytest.fixture +def trigger_pm(bot, pretrigger_pm): + return Trigger(bot.config, pretrigger_pm, None) + + +def test_unblockable(): + @module.unblockable + def mock(bot, trigger, match): + return True + assert mock.unblockable is True + + +def test_interval(): + @module.interval(5) + def mock(bot, trigger, match): + return True + assert mock.interval == [5] + + +def test_rule(): + @module.rule('.*') + def mock(bot, trigger, match): + return True + assert mock.rule == ['.*'] + + +def test_thread(): + @module.thread(True) + def mock(bot, trigger, match): + return True + assert mock.thread is True + + +def test_commands(): + @module.commands('sopel') + def mock(bot, trigger, match): + return True + assert mock.commands == ['sopel'] + + +def test_nick_commands(): + @module.nickname_commands('sopel') + def mock(bot, trigger, match): + return True + assert mock.rule == [""" + ^ + $nickname[:,]? # Nickname. + \s+(sopel) # Command as group 1. + (?:\s+ # Whitespace to end command. + ( # Rest of the line as group 2. + (?:(\S+))? # Parameters 1-4 as groups 3-6. + (?:\s+(\S+))? + (?:\s+(\S+))? + (?:\s+(\S+))? + .* # Accept anything after the parameters. Leave it up to + # the module to parse the line. + ))? # Group 1 must be None, if there are no parameters. + $ # EoL, so there are no partial matches. + """] + + +def test_priority(): + @module.priority('high') + def mock(bot, trigger, match): + return True + assert mock.priority == 'high' + + +def test_event(): + @module.event('301') + def mock(bot, trigger, match): + return True + assert mock.event == ['301'] + + +def test_intent(): + @module.intent('ACTION') + def mock(bot, trigger, match): + return True + assert mock.intents == ['ACTION'] + + +def test_rate(): + @module.rate(5) + def mock(bot, trigger, match): + return True + assert mock.rate == 5 + + +def test_require_privmsg(bot, trigger, trigger_pm): + @module.require_privmsg('Try again in a PM') + def mock(bot, trigger, match=None): + return True + assert mock(bot, trigger) is not True + assert mock(bot, trigger_pm) is True + + @module.require_privmsg + def mock_(bot, trigger, match=None): + return True + assert mock_(bot, trigger) is not True + assert mock_(bot, trigger_pm) is True + + +def test_require_chanmsg(bot, trigger, trigger_pm): + @module.require_chanmsg('Try again in a channel') + def mock(bot, trigger, match=None): + return True + assert mock(bot, trigger) is True + assert mock(bot, trigger_pm) is not True + + @module.require_chanmsg + def mock_(bot, trigger, match=None): + return True + assert mock(bot, trigger) is True + assert mock(bot, trigger_pm) is not True + + +def test_require_privilege(bot, trigger): + @module.require_privilege(module.VOICE) + def mock_v(bot, trigger, match=None): + return True + assert mock_v(bot, trigger) is True + + @module.require_privilege(module.OP, 'You must be at least opped!') + def mock_o(bot, trigger, match=None): + return True + assert mock_o(bot, trigger) is not True + + +def test_require_admin(bot, trigger, trigger_owner): + @module.require_admin('You must be an admin') + def mock(bot, trigger, match=None): + return True + assert mock(bot, trigger) is not True + + @module.require_admin + def mock_(bot, trigger, match=None): + return True + assert mock_(bot, trigger_owner) is True + + +def test_require_owner(bot, trigger, trigger_owner): + @module.require_owner('You must be an owner') + def mock(bot, trigger, match=None): + return True + assert mock(bot, trigger) is not True + + @module.require_owner + def mock_(bot, trigger, match=None): + return True + assert mock_(bot, trigger_owner) is True + + +def test_example(bot, trigger): + @module.commands('mock') + @module.example('.mock', 'True') + def mock(bot, trigger, match=None): + return True + assert mock(bot, trigger) is True diff --git a/test/test_trigger.py b/test/test_trigger.py index 0cb23c7a1a..c81fcd2b1d 100644 --- a/test/test_trigger.py +++ b/test/test_trigger.py @@ -1,9 +1,12 @@ -# coding=utf8 +# coding=utf-8 """Tests for message parsing""" -from __future__ import unicode_literals +from __future__ import unicode_literals, absolute_import, print_function, division +import re import pytest +import datetime +from sopel.test_tools import MockConfig from sopel.trigger import PreTrigger, Trigger from sopel.tools import Identifier @@ -14,17 +17,229 @@ def nick(): def test_basic_pretrigger(nick): - line = ':Foo!foo@example.com PRIVMSG #octothorpe :Hello, world' + line = ':Foo!foo@example.com PRIVMSG #Sopel :Hello, world' pretrigger = PreTrigger(nick, line) assert pretrigger.tags == {} assert pretrigger.hostmask == 'Foo!foo@example.com' assert pretrigger.line == line - assert pretrigger.args == ['#octothorpe', 'Hello, world'] + assert pretrigger.args == ['#Sopel', 'Hello, world'] assert pretrigger.event == 'PRIVMSG' assert pretrigger.nick == Identifier('Foo') assert pretrigger.user == 'foo' assert pretrigger.host == 'example.com' - assert pretrigger.sender == '#octothorpe' + assert pretrigger.sender == '#Sopel' -# TODO tags, PRIVMSG to bot, intents -# TODO Trigger tests, for what little actual logic is in there + +def test_pm_pretrigger(nick): + line = ':Foo!foo@example.com PRIVMSG Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['Sopel', 'Hello, world'] + assert pretrigger.event == 'PRIVMSG' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == Identifier('Foo') + + +def test_quit_pretrigger(nick): + line = ':Foo!foo@example.com QUIT :quit message text' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['quit message text'] + assert pretrigger.event == 'QUIT' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender is None + + +def test_join_pretrigger(nick): + line = ':Foo!foo@example.com JOIN #Sopel' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['#Sopel'] + assert pretrigger.event == 'JOIN' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == Identifier('#Sopel') + + +def test_tags_pretrigger(nick): + line = '@foo=bar;baz;sopel.chat/special=value :Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {'baz': None, + 'foo': 'bar', + 'sopel.chat/special': 'value'} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['#Sopel', 'Hello, world'] + assert pretrigger.event == 'PRIVMSG' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == '#Sopel' + + +def test_intents_pretrigger(nick): + line = '@intent=ACTION :Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {'intent': 'ACTION'} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['#Sopel', 'Hello, world'] + assert pretrigger.event == 'PRIVMSG' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == '#Sopel' + + +def test_unusual_pretrigger(nick): + line = 'PING' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {} + assert pretrigger.hostmask is None + assert pretrigger.line == line + assert pretrigger.args == [] + assert pretrigger.event == 'PING' + + +def test_ctcp_intent_pretrigger(nick): + line = ':Foo!foo@example.com PRIVMSG Sopel :\x01VERSION\x01' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {'intent': 'VERSION'} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['Sopel', ''] + assert pretrigger.event == 'PRIVMSG' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == Identifier('Foo') + + +def test_ctcp_data_pretrigger(nick): + line = ':Foo!foo@example.com PRIVMSG Sopel :\x01PING 1123321\x01' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {'intent': 'PING'} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['Sopel', '1123321'] + assert pretrigger.event == 'PRIVMSG' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == Identifier('Foo') + + +def test_ircv3_extended_join_pretrigger(nick): + line = ':Foo!foo@example.com JOIN #Sopel bar :Real Name' + pretrigger = PreTrigger(nick, line) + assert pretrigger.tags == {'account': 'bar'} + assert pretrigger.hostmask == 'Foo!foo@example.com' + assert pretrigger.line == line + assert pretrigger.args == ['#Sopel', 'bar', 'Real Name'] + assert pretrigger.event == 'JOIN' + assert pretrigger.nick == Identifier('Foo') + assert pretrigger.user == 'foo' + assert pretrigger.host == 'example.com' + assert pretrigger.sender == Identifier('#Sopel') + + +def test_ircv3_extended_join_trigger(nick): + line = ':Foo!foo@example.com JOIN #Sopel bar :Real Name' + pretrigger = PreTrigger(nick, line) + + config = MockConfig() + config.core.owner_account = 'bar' + + fakematch = re.match('.*', line) + + trigger = Trigger(config, pretrigger, fakematch) + assert trigger.sender == '#Sopel' + assert trigger.raw == line + assert trigger.is_privmsg is False + assert trigger.hostmask == 'Foo!foo@example.com' + assert trigger.user == 'foo' + assert trigger.nick == Identifier('Foo') + assert trigger.host == 'example.com' + assert trigger.event == 'JOIN' + assert trigger.match == fakematch + assert trigger.group == fakematch.group + assert trigger.groups == fakematch.groups + assert trigger.args == ['#Sopel', 'bar', 'Real Name'] + assert trigger.account == 'bar' + assert trigger.tags == {'account': 'bar'} + assert trigger.owner is True + assert trigger.admin is True + + +def test_ircv3_intents_trigger(nick): + line = '@intent=ACTION :Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + + config = MockConfig() + config.core.owner = 'Foo' + config.core.admins = ['Bar'] + + fakematch = re.match('.*', line) + + trigger = Trigger(config, pretrigger, fakematch) + assert trigger.sender == '#Sopel' + assert trigger.raw == line + assert trigger.is_privmsg is False + assert trigger.hostmask == 'Foo!foo@example.com' + assert trigger.user == 'foo' + assert trigger.nick == Identifier('Foo') + assert trigger.host == 'example.com' + assert trigger.event == 'PRIVMSG' + assert trigger.match == fakematch + assert trigger.group == fakematch.group + assert trigger.groups == fakematch.groups + assert trigger.groupdict == fakematch.groupdict + assert trigger.args == ['#Sopel', 'Hello, world'] + assert trigger.tags == {'intent': 'ACTION'} + assert trigger.admin is True + assert trigger.owner is True + + +def test_ircv3_account_tag_trigger(nick): + line = '@account=Foo :Nick_Is_Not_Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + + config = MockConfig() + config.core.owner_account = 'Foo' + config.core.admins = ['Bar'] + + fakematch = re.match('.*', line) + + trigger = Trigger(config, pretrigger, fakematch) + assert trigger.admin is True + assert trigger.owner is True + + +def test_ircv3_server_time_trigger(nick): + line = '@time=2016-01-09T03:15:42.000Z :Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + + config = MockConfig() + config.core.owner = 'Foo' + config.core.admins = ['Bar'] + + fakematch = re.match('.*', line) + + trigger = Trigger(config, pretrigger, fakematch) + assert trigger.time == datetime.datetime(2016, 1, 9, 3, 15, 42, 0) + + # Spec-breaking string + line = '@time=2016-01-09T04:20 :Foo!foo@example.com PRIVMSG #Sopel :Hello, world' + pretrigger = PreTrigger(nick, line) + assert pretrigger.time is not None