Deprecations and future release

[damoclark]

Hi @mbabker & team,

Appreciate the work that has gone into the bundle. I have just introduced it to a Symfony 5.3 project.

I've noticed there are a number of deprecations being reported for version 4 of WebSocketBundle (as noted in #420). I am just wondering if there is a rough plan somewhere for timing of the 4.0 release, and some suggestions for how to replace the deprecated elements? Or perhaps simply how to prepare for them?

For example, I'm making heavy use of the Pusher which has been wonderfully simple and easy to implement. The deprecation notice suggests the use of Pawl, which has quite a hefty implementation and interface, and without any of the nice symfony parameter and service integrations. If there is expertise in the team with Pawl, I wonder whether it might be possible for a couple of simple Pawl examples replacing the Pusher functionality might be added to the documentation. If there is capacity within the group to consider this of course. :)

Thanks

Damien.

[mbabker]

The upgrade guide on the branch (https://github.com/GeniusesOfSymfony/WebSocketBundle/blob/4.x/UPGRADE-4.0.md) should be pretty complete as far as what changes are coming through the pipeline, there's probably more stuff on there than there needs to be but I err on the side of caution and generally document every change regardless of whether it's something that should be considered an internal detail or something that I'd consider a public API for the bundle.

The biggest changes in the pipeline for 4.0:

• Moving to Ratchet 0.5, which means they need to release that first (there's some initial prep work for that on their repo, but it hasn't moved far over the last few months)
• Dropping the pusher integration
• Dropping the websocket client in favor of Ratchet's Pawl client
• Bumping up the dependency chains and supporting fewer versions of PHP, Symfony, and other third-party packages (I generally try not to make many bumps in dependency chains in the middle of a major version, especially around the PHP minimum, so when I release major versions of the packages I support they usually close up the support ranges quite a bit)

Ratchet 0.5 will have some B/C breaks that can affect the user-facing API of this bundle (if you're doing something where you have your own ServerBuilderInterface because you've added your own middleware to the message stack, the interface changes in Ratchet are going to affect you). So waiting to ship 4.0 until that version comes out avoids the potential of version fatigue (and I know I already burned a few bridges with how quickly things jumped from 1.x to 3.0 and 2.x having a shorter lifetime).

Admittedly, I never deployed anything but the WAMP pusher integration myself, but looking at the AMQP and ZMQ implementations of that, it really felt like the pusher and server push handlers were a light version of a message broker/consumer type of system. Hence, suggesting the Messenger component as a replacement for the entire integration. There might be some use case where those fit better into a websocket context, in which case I'm all eyes on reading about it and making changes if it turns out they do have a good fit here.

After deprecating those, I still wanted to support the PHP websocket client, hence the newer config introduced for that, but I then reached a point where I decided the way that was implemented wasn't all that great and I myself don't have the skillset to maintain a strong PHP client for server-to-server based websocket messages, and that's when I deprecated that package in full in addition to the integration in the bundle.

Pawl, while needing more boilerplate than the gos/websocket-client, is better maintained and standards-compliant and that's where the recommendation for that comes into play. #405 (comment) has a simplified example of the wrapper I use in my main client's project to replace gos/websocket-client. It may be possible to incorporate a version of that into this bundle somehow, but I really hadn't put much thought into that because the interface for that at this level would end up breaking the actual integration I have in my app (all the $data params are actually array|MessageInterface unions and my client supports serializing DTOs with jms/serializer as well as passing plain arrays).

[damoclark]

Thanks for such a thoughtful response - this is very helpful. :)

I had a look at the 4.x branch but completely overlooked the upgrade file.

The 3.x branch shows existing built-in features as the following. Could you maybe clarify which of these features will be removed or substituted for something else (not built-in)? (e.g. PHP Websocket client being substituted for Pawl).

1. PHP Websocket server (IO / WAMP)
2. PHP Websocket client (IO / WAMP)
3. JavaScript Websocket client (IO / WAMP)
4. PubSub Router
5. Remote Procedure Calls
6. User authentication
7. Periodic calls
8. Origin checker

What is most attractive to me about WebSocketBundle is the auth integration with the Symfony framework, and the simplicity of the PubSub routing implementation, using the Pusher. Server-side symfony PHP-FPM pubs sent via Websocket server couldn't be easier. These two aspects particularly saved me considerable work. After quite a bit of research, I couldn't find another solution that provided these two features out of the box.

Thanks for directing me to the wrapper you shared using Pawl. Having something like that integrated into WebSocketBundle as a pusher replacement would be awesome. :)

Again, thanks for the work that you and your team do and share with the community. It is appreciated.

Regards,
Damien.

[mbabker]

The only things going away as of right now are the pusher integration (Gos\Bundle\WebSocketBundle\Pusher\PusherInterface, Gos\Bundle\WebSocketBundle\Pusher\ServerPushHandlerInterface, and all related classes) and the PHP websocket client (or, more specifically, anything which integrates the now deprecated gos/websocket-client package).

It is very well possible to create a new websocket client inside the bundle to keep the tight integrations available, I already have a working foundation for one but I'd need to do some thinking on how to implement it here and keep it flexible enough for general use. I'd really like to keep my public function call(array|MessageInterface $data, string $routeName, array $routeParameters = []): void signature and not force all incoming data to be transformed into an array first, which means I'd probably need an adapter layer to support serializing that MessageInterface DTO and somehow support both the Symfony and JMS serializer, and that can get tricky to say the least (ever looked at the internals of FOSRestBundle?). The other interesting thing here is if that serializer layer is added in, the RPC dispatcher could also be updated to support DTOs returned from an RPC handler and not be stuck to basic scalar support (I've got a custom version of the RPC dispatcher in one of my apps doing just that; but my app solely uses the JMS serializer so it's easy to code my interfaces to one library).

The whole of the pusher API felt out of place to me when I first took on maintenance of this bundle, and today with how the ecosystem has evolved, feels even moreso out of place now. That has two aspects to it:

• Pushing messages to another server (with AMQP and WAMP implementations available, the latter relying on the PHP websocket client)
• Consuming pushed messages (only AMQP support, the WAMP pusher makes a "publish" call to the websocket server so that gets handled through this bundle's TopicInterface implementations and never needed a dedicated push handler)

The push handler interface (which had ZMQ support previously but was dropped because of poor maintenance of the PHP extension) is really just a proxy where someone can push a message to a message brokering system and the websocket server could consume those messages through a PushableTopicInterface implementation. I would argue that at this point a combination of a PHP websocket client which can support the call or publish message formats (which translate into RPC and Topic respectively in this bundle's API) and a system like Symfony's Messenger component is a well suited replacement for that entire system.

Touching on the rest of the feature list now and how things are moving along:

• PHP Websocket server (IO / WAMP) - That's kind of the point of this bundle, if that went away there's no point left here 🤣
• JavaScript Websocket client (IO / WAMP) - That will stay, it's a nice little opinionated wrapper around the AutobahnJS library with a couple of nice convenience features for apps to use
• PubSub Router - Not going anywhere; unlike Ratchet's opinionated App facade which uses Symfony's router, this bundle relies on GosPubSubRouterBundle which is an opinionated version of Symfony's router designed around the websocket server implementation, and if anything that bundle's router implementation is getting closer to the Symfony router as far as feature parity and underlying implementations go (the next major there is updating the internal API to sync up with Symfony 5.2 and for now it supports every route format except for annotations/attributes and even that isn't outside the realm of possibility anymore)
• Remote Procedure Calls - Not going anywhere
• User authentication - Not going anywhere, I might at some point work out a way to make configuring the authenticators a little more flexible for other authentication flows, but the out-of-the-box implementation using sessions and the session cookie is here to stay
• Periodic calls - Not going anywhere, IMO this is a critical piece of the API that the bundle helps make easier for consumers; there may come a time that I revisit the API design here based on the implementation I use in one of my apps (we have a pretty dynamic system where periodic calls need to be dynamically registered and removed so I have a higher level interface (below) that fires off immediately at the GosWebSocketEvents::SERVER_LAUNCHED event to make sure everything's hooked up right away whereas the current PeriodicInterface just registers a class to execute at an interval without a way to trigger it at server launch)
• Origin checker - This, and the newly released IP blocker integration, aren't going anywhere; both of these are provided inside of Ratchet (I only just realized the IpBlackList class exists yesterday otherwise the IP blocker would've come sooner), and both of these are important security features for the websocket server that might not be easily implemented somewhere higher in the stack

<?php declare(strict_types=1);

namespace App\Websocket;

use React\EventLoop\LoopInterface;

interface PeriodicManagerInterface
{
    public function getName(): string;

    public function register(): void;

    public function cancelTimers(): void;

    public function setLoop(LoopInterface $loop): void;
}

At some point, coming back to #56 and trying to work out a clustering architecture (whether it be just documentation or code within the bundle) is something I'd love to see happen but I haven't had the time to really work out how to make this practical (it's come up in my client work a couple of times but with the way React\EventLoop\LoopInterface is implemented I keep stumbling on how to wire things up in a true multi-server clustered approach).

Another idea floating in the back of my mind is revisiting the authentication workflow to support a chained authentication system since right now it's an all or nothing type of thing (so if you've got a system where clients might be able to authenticate via JWT or traditional HTTP sessions, you have to implement the authentication provider yourself). Also, the out-of-the-box provider doesn't really handle multi-firewall configs all that well; if you've got a site where there are two firewalls that a user can authenticate into (which, I know Sylius does this with its default security config having separate users/auth for admin and frontend), the provider right now will take the first token it can find based on the order you've configured the firewall list in (so if someone's logged in in both the admin and the site firewalls, and your firewall order is ['site', 'admin'], the provider will always take the site firewall's token, which might not be the intent for your app).

Beyond that, I'd say the bundle's pretty stable and a lot of the effort I've put into it over the last year or so is turning more into maintenance keeping up with the upstream dependencies than plugging in new features or chasing down odd bugs.

[damoclark]

Thanks for this information @mbabker

It is very well possible to create a new websocket client inside the bundle to keep the tight integrations available, I already have a working foundation for one but I'd need to do some thinking on how to implement it here and keep it flexible enough for general use.

I was a little baffled that Pawl doesn't implement any sub-protocols. The working foundation you have implements aspects of WAMP I believe, which covers the PubSub and RPC elements nicely. Wrapping Pawl with these additions might be a sensible approach and if Pawl were to implement sub-protocols in the future, the API could be maintained and just hook into Pawl's work.

The whole of the pusher API felt out of place to me when I first took on maintenance of this bundle, and today with how the ecosystem has evolved, feels even moreso out of place now.

...

The push handler interface (which had ZMQ support previously but was dropped because of poor maintenance of the PHP extension) is really just a proxy where someone can push a message to a message brokering system and the websocket server could consume those messages through a PushableTopicInterface implementation. I would argue that at this point a combination of a PHP websocket client which can support the call or publish message formats (which translate into RPC and Topic respectively in this bundle's API) and a system like Symfony's Messenger component is a well suited replacement for that entire system.

Yes I see your point and agree this makes sense.

Origin checker - This, and the newly released IP blocker integration, aren't going anywhere; both of these are provided inside of Ratchet (I only just realized the IpBlackList class exists yesterday otherwise the IP blocker would've come sooner), and both of these are important security features for the websocket server that might not be easily implemented somewhere higher in the stack

Actually on the IPBlackList, which I was initially looking for in WebSocketBundle is a tricky one. The problem, at least for me, and I suspect anyone else who needs wss is when implementing a reverse proxy that is hosted on the same server as the websocket server, it's not easy to discern connection requests to the Websocket server from public client browsers, to those from the internal Symfony server stack. They will all have a source IP of localhost.

To get around this, I introduced a shared secret as a symfony parameter, populated through an environment variable that is sent as part of the application payload from the current Pusher to the websocket request handlers in the websocket server, which then looks up the same symfony parameter and validates it. This is how I authenticate requests to the websocket server that do not have a browser auth session attached, to know it comes from my server stack.

Perhaps something like this integrated into a wrapped Pawl client would be useful to other users of WebSocketBundle who have the same issue as I?

[mbabker]

I was a little baffled that Pawl doesn't implement any sub-protocols. The working foundation you have implements aspects of WAMP I believe, which covers the PubSub and RPC elements nicely. Wrapping Pawl with these additions might be a sensible approach and if Pawl were to implement sub-protocols in the future, the API could be maintained and just hook into Pawl's work.

Pawl's a lower level client, it handles all of the connection and transmission logic but doesn't have higher level functions to nicely send WAMP formatted messages. So, I'll definitely put some thought into how to bring forward a new client into this bundle.

Actually on the IPBlackList, which I was initially looking for in WebSocketBundle is a tricky one. The problem, at least for me, and I suspect anyone else who needs wss is when implementing a reverse proxy that is hosted on the same server as the websocket server, it's not easy to discern connection requests to the Websocket server from public client browsers, to those from the internal Symfony server stack. They will all have a source IP of localhost.

I can't say I understand much about stream_socket_get_name() which is how the react/socket library is grabbing the IP address with its default implementation, but maybe there's something out there that copes with reverse proxies nicely?

To get around this, I introduced a shared secret as a symfony parameter, populated through an environment variable that is sent as part of the application payload from the current Pusher to the websocket request handlers in the websocket server, which then looks up the same symfony parameter and validates it. This is how I authenticate requests to the websocket server that do not have a browser auth session attached, to know it comes from my server stack.

Yeah, I've got something similar for my server-to-server websocket calls.