Creating a jetstream.Msg

[samborkent]

Hello, I am currently in the process of migrating to the new JetStream API, but I'm confused about something. I find it strange that the publish methods take messages of type *nats.Msg, which does not implement the jetstream.Msg interface received by the consumer. So, if I change my utility functions to take a jetstream.Msg instead, there is no straightforward way for creating such a message. I would expect there to be some initializer for a jetstream.Msg or for there to be a conversion function to convert from the old API struct message type *nats.Msg to the new jetstream.Msg interface message type, or at least extract a *nats.Msg from a jetstream.Msg. Personally, I think it would have been better to just create a v2 of the API, as now there seem to be some inconsistencies when working with JetStream messages.

Am I overlooking something? How can I create a jetstream.Msg from a *nats.Msg? The only way I see now is to define my own message type that implements jetstream.Msg and write a conversion function myself, but I would expect this to be part of the API or at least have some explanation included in the migration guide: https://natsbyexample.com/examples/jetstream/api-migration/go.

[piotrpio]

Hello @samborkent. While that indeed may seem counter-intuitive, a jetstream.Msg interface is intentionally only for messages received from a JetStream consumer. The interface provides various flavors of acking the message, which do not make sense when publishing on a stream. I don't think it's a good idea to expose receive-only methods like Ack(), Nak() etc. when publishing a message, as that may lead to unwanted runtime errors.

What we could do is expose a different wrapper type around *nats.Msg which would be JetStream-specific, although I don't see much use for it right now. Could you explain your use case in a bit more detail?

[samborkent]

In the current project I am working on we are using JetStream as a persistent alternative to an internal queue (or Go channel). So, we both have consumers and publishers to the same stream in one project.

I ran into the fact that you cannot convert from *nats.Msg to jetstream.Msg while writing unit tests for JetStream handlers. The Consume method takes a jetstream.MessageHandler of signature func(jetstream.Msg), so I have been writing handlers to conform to that (actually I write them as func(context.Context, jetstream.Msg) error and then convert it, as I find it more Go idiomatic and it makes it easy to write tracing middleware, but that's another topic). But then I found that there is no "easy" way to unit test such a handler, as the API does not provide a way to create a jetstream.Msg.

I understand your reasoning about not wanting to expose consumer specific methods to a publisher and I agree with that. I wonder how would you propose to unit test a jetstream.MessageHandler?

For now I have defined my own message type which wraps *nats.Msg, and made it implement jetstream.Msg. So, I can easily pass NATS messages to my message handlers. But it would be convenient to have a method for dealing with this included in the API. If you don't want this to clutter the jetstream API, perhaps a test utilities package might be an idea.

[samborkent]

Any updates on this? With the current API, I find it extremely cumbersome to unit test NATS message handlers.

[astelmashenko]

Good day, +1 for converting nats.Msg to jetstream.Msg or vice versa. We have a lot of legacy code and we need to gradually migrate to new api. If it is not possible to convert either way then we need to maintain two version of code working with nats.Msg and jetstream.Msg.

[jnmoyne]

The disconnect here is that jetstream message has more things than a Core NATS message: namely it has JS metadata and you can call things like ack on it, so to go from one to another you indeed have to create your own message that implements the Jetstream message interface and up to you to deal with that implementation (i.e. what JS metadata to generate or how to deal with acks) according to your test.

[samborkent]

I understand, which I why I suggested adding it to the test package. Currently, a significant amount of boilerplate is required to properly unit test a jetstream.MsgHandler.

[ripienaar]

Seems to me it should be easy to test this with gomock? Add to that a tiny utility to convert a nats.Msg to a mocked jetstream.Msg and it should not be too bad?

I havnt tried though but might be worth a try for someone

[samborkent]

Yes, you can write utilities for this yourself. But it would be much more convenient if this was included in the NATS test package. As now you either have to write these utility functions for each project, or create a utility package yourself. I just created a utility package for my organisation, but preferably I wouldn't have had to do that.

I wonder how the person who designed the jetstream.MessageHandler unit tested their own handlers while developing the new jetstream API. The usefulness of a utility which can create a valid jetstream.Msg would seemingly be immediately apparent.