Expose models via RPC?

[atodorov]

Hi folks,
I have a decent size project at https://github.com/kiwitcms/Kiwi which does have a custom grown XML-RPC interface (legacy code) pretty much all of which exposes the available models and variations of the filter/get/create and update methods. The underlying methods boil down to performing a query, iterating over it and serializing every object as a dictionary. Then there's a Python API client which duplicates most of this structure in order to read the serialized data and turn it back into something more Pythonic.

How hard would be to expose models over RPC with modernrpc ? If that is doable I'd rather spend my time contributing a PR to you and dropping all the legacy stuff in Kiwi instead of manually going through each of our exposed RPC methods and figuring out of there are duplicates or not.

[alorence]

Hi Alexander, and thank you for your interest in the project.

I looked into the code of Kiwi project (not in detail). I saw you have a xmlrpc django app, containing both xml-rpc related stuff (including a view, a dispatcher and some sugar around) and your concrete remote procedure (under xmlrpc.api).

If i'm true, you could migrate from your legacy xml-rpc server implementation to django-modern-rpc with a few steps:

1. Install modernrpc, add it to INSTALLED_APPS, and create a URLConf to root requests to modernrpc's RPCEntryPoint view
2. Add the @rpc_method decorator before each method of xmlrpc.api. You may need to customize name of each method to control how it is exposed to the world.
3. Set settings.MODERNRPC_METHODS_MODULES with the list of python modules containing decorated methods:

MODERNRPC_METHODS_MODULES = [
    'xmlrpc.api.user',
    'xmlrpc.api.product',
    'xmlrpc.api.tag',
    # etc.
]

This could probably be done quickly and easily. You could then drop the legacy code for serving xml-rpc request, and get as a bonus that a json-rpc client would access the same way to your api.

But then, I'm not sure to understand. You would like to improve modernrpc by adding some utilities to automatically expose models via RPC calls ?

[atodorov]

@alorence thanks for looking at Kiwi's code base. Your observations are more or less correct. I did explore modernrpc yesterday and I will do what you suggest as the first step - remove all of our custom XML-RPC handling and only leave the actual methods that I want to expose to the client. Then decorate them with @rpc_method.

But then, I'm not sure to understand. You would like to improve modernrpc by adding some utilities to automatically expose models via RPC calls

Yes. The essence of the RPC layer of Kiwi TCMS is that we allow access to the underlying models for creating/filtering/recording results, etc. ATM this needs to be done for every single model class that we want to expose.

This includes writing a wrapper function which gets exposed via RPC. The wrapper then calls another wrapper which translates the results of a QuerySet to something we can serialize, usually a list of dicts.

Then on the client side we need to add a new class to handle the results and present a pythonic interface to the caller. IMO all of this (a big part of it at least) can be generated automatically from the model class definition. I don't know how complicated such tooling could get or even if it is worth the effort but it sounds like something that many people would like to have.

[alorence]

@atodorov Ok, I see what you want to achieve.

I agree that having something to automatically expose model interactions (create, update, filter, etc.) seems interesting at first look. But I'm not sure how useful it can be. If you implement such an API in modernrpc, it should be generic enough to allow any model to be exposed. And models can be very difficult in some cases (especially when they contains ForeignKeys, ManyToMany fields, etc.)

Do you have an idea on how such an API could be implemented ? Do you plan to develop a set of decorators ? Predefined RPC methods ? A factory to generate RPC methods ?
Can you try to explain a bit more what you have in mind ?