Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for slack events #1451

Open
wants to merge 62 commits into
base: master
Choose a base branch
from

Conversation

AgarFu
Copy link

@AgarFu AgarFu commented Sep 7, 2020

This PR is reusing most of the classes from the SlackRTM modules, there is no documentation nor tests in this PR and the main class can be refactored in order to be able to leverage most methods from SlackRTMBackend class.

Please let me know if this is something that you're interested in incorporate to the project I can add the missing bits, it is now in a "works for me" status.

@nzlosh
Copy link
Contributor

nzlosh commented Sep 7, 2020

@AgarFu Nice work, thanks for taking the initiative on creating this PR.

Slack seems to be in a state of disorder with regard to the Python SDK, Application Authorisation model and API access.

From what I can tell, the current situation is as follows:

Python SDK

  • Comes in two versions v1 (used by slack backend) and v2 (used by slackRTM backend).
  • Active development on v1 has been halted in favour of v2.

Slack API

  • RTM - Real Time Messaging (Websocket (client to server) receive all events from the workspace as well as ability to send events to Slack)
  • Web API - Post events to Slack.
  • Event API - Subscribe to specific events (server connects to client). Push model only, must reply via Web API.

Application Authentication

  • "Classic" App tokens that give bots "all" rights scope (strongly urged by Slack to use fine grain scope). Has full access to the RTM and Web API.
  • New App tokens give fine grain control to what a bot can do. These tokens can't be used with the RTM API, must use Event API and Web API.

This is situation is causing a lot of confusion for users. What the Slack backend needs to do is have a way for the user to express the type of token they're using ("Classic" or "Fine grain") and then adapt it's behaviour accordingly.

In the case of the Event API, the errbot webserver will probably need to used for the Slack server to connection. From what I understand, the event API events and RTM API events should be handled in the same way by errbot so we could probably mutalise code for these 2 use cases.

@sijis @AgarFu What do you think about attempting to merge RTM/Event/Web API support into a single Slack backend?

@nzlosh
Copy link
Contributor

nzlosh commented Sep 7, 2020

Here is the list of events with the RTM/Event API method indicated as reference. https://api.slack.com/events

@AgarFu
Copy link
Author

AgarFu commented Sep 7, 2020

Fully agree on all that you said @nzlosh, I'm actually using errbot webserver to listen to posts from Slack. I can refactor SlackRTMBackend in order to be able to share even more between SlackRTMBackend and SlackEventsBackend. Right now these two backends are closer together than SlackRTMBackend and SlackBackend. That will allow a further refactor to fuse all of them into one fully capable client.

Is there a strong opposition to create a backends.slack module with all shared classes between both backends?

@sijis
Copy link
Contributor

sijis commented Sep 7, 2020

I'm open to combining it into the SlackRTM backend. I think it logically makes sense.

@AgarFu
Copy link
Author

AgarFu commented Sep 7, 2020

Just refactored SlackRTMBackend extracting SlackBackendBase that is doing all the heavy lifting and implementing the handlers for all events.

My new class now only overrides the __init__ and serve_forever methods, I'm dynamically wrapping all possible events coming from slack to methods in the SlackBackendBase, this is much more clean I think.

There are no unit test for SlackRTMBackend (or at least I couldn't find them) so I don't know if that class is still working @sijis would be very nice if you can verify that my refactor did not break the RTM client.

@AgarFu
Copy link
Author

AgarFu commented Sep 10, 2020

@nzlosh @sijis any more comments? do you want me to try to refactor this patch further and fuse SlacmRTM and SlackEVENTS?

@sijis
Copy link
Contributor

sijis commented Sep 12, 2020

I will try to look into the details of this PR within the next week. I can provide feedback and more comments then.

@AgarFu
Copy link
Author

AgarFu commented Nov 9, 2020

Hi it seems that Slack is deprecating more and more APIs, and there is interest in the community to support the slack event api. I can devote some more time if you consider that the PR needs rework.

I've rebased my branch with the latest changes in the main branch and also fixed some errors due to these deprecations.

Also the new slack_sdk python module is close to be ready, I think errbot should migrate to the new sdk sooner rather than later but that may have implications in the RTM support.

@nzlosh @sijis thoughts?

AgarFu and others added 12 commits November 8, 2020 22:25
fix: merging configs via --storage-merge cli
* Added email property for slack backend

* Updated TestPerson with email property

* Updated core_plugin whoami command with email

* Added missing blank line

* fix: whoami format output

Co-authored-by: Sijis Aviles <[email protected]>
This should provide equal behavior as travisci.
This will allow the package to be uploaded to pypi, as it was failing
checks.
This should ensure that existing and external backends do not break with
when this property is not defined.
* fix: username to userid method signature

* Fix username_to_userid and raise when username not uniquely identified (errbotio#1447)

* style: fix codestyle warnings

Co-authored-by: Carlos <[email protected]>
@AgarFu
Copy link
Author

AgarFu commented Nov 26, 2020

Sure thing let me take a look, thx @nzlosh

@AgarFu
Copy link
Author

AgarFu commented Nov 26, 2020

I'll integrate the changes in a different branch for now, will add some tests and then push the changes to my branch I just want to make sure the latest changes that I applied to fix deprecation errors are still in place. I didn't add the proper tests two weeks ago but this PR is substantially more aggressive and still there are no tests for the backend.

For now the changes are https://github.com/AgarFu/errbot/tree/slack_event_api_refactor

I honestly don't like to have a file with 1.4k lines of code and this many classes in it, I'd like to add a new module inside backends, _slack maybe? and split that massive file in smaller ones, it will be easier to maintain and less discouraging to cover it with tests. @nzlosh thoughts?

cc: @sharpyy @kaosx5s

@nzlosh
Copy link
Contributor

nzlosh commented Nov 26, 2020

@AgarFu I agree with you, but I'd like to see this PR merged as a MVP so we can start iterating on refactoring/unit tests/bug fixes and documentation for this new backend.

A deprecation plan has been proposed here #1480, which I hope shows why multiple slack backends is harming errbot's user experience.

The SlackRTM backend was meant to be the successor, but with slack events and the slack client bump to v3, it lacked the features to cover all use cases.

@AgarFu
Copy link
Author

AgarFu commented Nov 26, 2020

Ok let me confirm that my team is using a different branch and I'll merge your changes, I'd like to coordinate with you for the next iterations I'd like to at least add the testing as I said before to make sure we do not have regressions and we also have a path going forward to update to the latest versions of slack sdk

@AgarFu
Copy link
Author

AgarFu commented Nov 26, 2020

@nzlosh your changes are merged in my branch feel free to merge them into mainstream, THX!

@nzlosh
Copy link
Contributor

nzlosh commented Dec 16, 2020

@AgarFu would you mind merging the latest changes from my repository https://github.com/nzlosh/errbot/tree/slack_event_api_support into your slack_event_api_support branch? I don't know why, but each time I attempt to create a PR against your repository, github doesn't display it in the list of available base repositories.

@sijis Once the changes have been merged into @AgarFu's repository, the slackv3 backend will be ready to be merged to master.

The slackv3 requirements include the RTM and Events python modules. The backend can automatically detect which token type is being used and will initialise the correct client. Not all events are supported by both API types. However the events that errbot requires which are common to both APIs end up calling the same methods.

@AgarFu
Copy link
Author

AgarFu commented Dec 16, 2020

will do

@sijis
Copy link
Contributor

sijis commented Dec 19, 2020

This is needing a rebase after the latest release. sorry folks.

@nzlosh
Copy link
Contributor

nzlosh commented Dec 19, 2020

@AgarFu I've merged master and pushed to my branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants