Evaluate use of entities to map Pulp 3 API objects

[nixocio]

Evaluate use of entities to map Pulp 3 API objects. Goal is to create a standard use of Pulp-Smash API, easy to maintain, and to update.

[rochacbruno]

I created an experiment with Swagger-Codegen

Code in:
https://github.com/rochacbruno/pulp3-juicer

Test example:
https://github.com/rochacbruno/pulp3-juicer/blob/master/tests/test_pulp3.py

Nice docs:
https://github.com/rochacbruno/pulp3-juicer/blob/master/docs/PulpApi.md

Travis passing:
https://travis-ci.com/rochacbruno/pulp3-juicer/jobs/185083913

The generation script:
https://github.com/rochacbruno/pulp3-juicer/blob/master/generate.sh

After the generation I had to tweak the code manually because APIspec for pulp MAY be wrong in some places (I have to confirm)

I like the general idea of that, specially because we can use codegen to be in sync with the exposed API. We can even create a version picker to generate a library for multiple versions of pulp.

Conclusion

That works and it has a better API and docs then we currently have for Pulp3

But I still prefer if we create our own lib with Marshmallow or Typesystem

c/c @kersommoura @dkliban

[rochacbruno]

Libs for test

https://github.com/encode/typesystem

https://github.com/schematics/schematics/blob/master/schematics/models.py

https://github.com/marshmallow-code/marshmallow

[nixocio]

@rochacbruno, very interesting.

But I still prefer if we create our own lib with Marshmallow or Typesystem

Based on what we talked I prefer the above as well.

[rochacbruno]

another reference to take a look, this is a nice use of typeSystem schemas https://github.com/encode/orm

[rochacbruno]

Just for the record of the ideas

In long-term we need to re-evaluate all the way Pulp-Smash deals with constants, my suggestion will be to get rid of API and URL paths in constants.

Create some more O.O based approach to have pulp instances and fixtures urls available from pulp-smash across all the plugins tests.

example:

from pulp_smash.entities import Repository, Remote
from pulp_smash.fixtures import Fixture
repo = Repository()
repo.sync(remote=Remote(url=Fixture.FIXTURE_NAME))

On the above example all the classes on .entities already knows its own endpoints no need to pass in constant located urls.

And the Fixtures is not a class, it is a runtime generated ENUM containing entries for each of the pulp-fixtures URLS.

Borderline cases will still need constants, but that should be the minimum.

[dralley]

All that sounds like a great improvement! One concern with "Fixtures is ... a runtime generated ENUM containing entries for each of the pulp-fixtures URLS." is that the file fixture would need a special case, since the file plugin wants the URL of the manifest and not the URL of the root of the repo. Or else, the file plugin would need to be changed to be like the others. I never really liked that design decision but I think we discussed some rationale for it many months ago.

[rochacbruno]

Today I found this lib https://github.com/ithaka/apiron

It makes possible to easily have something like this:

class Pulp(Service):
    repositories = JsonEndpoint(path='/pulp/api/v3/repositories/')
    repository = JsonEndpoint(path='/pulp/api/v3/repositories/{id}')
    ...

class PulpDocker(Service):
    distributions = JsonEndpoint(path='/pulp/api/v3/docker/docker-distributions/')
    distribution = JsonEndpoint(path='/pulp/api/v3/docker/docker-distributions/{id}')
    ...

# Whatever list of paths we need

That is easy to convert our current constants to definitions like that. then we can do.

repo = Pulp.repository(name='foo').create()
docker_dist = PulpDocker.distribution(name="bar").create()

Instead of having one class for each entity, it has One Class for each API category/plugin.

Pros: We can easily convert our current endpoints, we can still access the bare endpoint Pulp.repository.path on current code.

Cons: We still need to maintain the paths :(

Posting here just as a reminder.

[nixocio]

@rochacbruno, nice.

[rochacbruno]

A good candidate for serialization take a look: https://github.com/Fatal1ty/mashumaro

[JacobCallahan]

I'd like to propose these two tools for library generation.
APIx - https://github.com/JacobCallahan/apix
CLIx - https://github.com/JacobCallahan/clix

These not only allow you to generate custom libraries for APIs and CLIs respectively but also save the scraped information in an easy to read format. Additionally, they can perform diff-with-context reports between any two version.

Plus, the developer is a friendly fellow :)

For examples or a live demo, let me know!

Edit: Forgot to mention that using those two allows you to easily integrate with Plinko, for feature coverage and test selection!
https://github.com/JacobCallahan/plinko

*Plinko is still in early development and is not yet production-ready