Rename library to json-api-response

[tobyzerner]

tobscure/json-api is very a non-specific name, it doesn't really tell you what the library does. In #121 I've said I want to keep the scope of this library quite narrow (building JSON-API documents using dynamic data) so maybe it would be wise to rename accordingly?

Options:

• tobscure/json-api-builder
• tobscure/json-api-serializer (not quite... it does more than just serialize)
• tobscure/json-api-server (it's less than a server - there is no HTTP stuff)
• tobscure/json-api-response
• ???

Also note that I plan to extract some of Flarum's JSON-API consumption JavaScript into an accompanying library called tobscure/json-api-store.

/cc @f3ath @franzliedke

[f3ath]

mmm dunno... IMO builder has connotations with the Builder pattern from PoEAA. Maybe json-api-response?

[tobyzerner]

json-api-response could work... especially since we have the Parameters utility which aids in the construction of a document in response to a request

[tobyzerner]

The Builder pattern is not too far off though, is it?

... the builder pattern uses another object, a builder, that receives each initialization parameter step by step and then returns the resulting constructed object at once.

$document->setInclude(['author', 'comments']);
$document->setFields(['posts' => ['title', 'body']]);
$document->setMetaItem('total', count($posts));

$document->jsonSerialize(); // constructs a json object

Or what about json-api-factory? Or json-api-response-factory

[f3ath]

Same thing about Factory :) But actually disregard, it's just my idiosyncrasy, I guess.

[tobyzerner]

:P I think I like json-api-response anyway, doesn't get much clearer than that

Will await @franzliedke's opinion

[franzliedke]

My previous endeavors with writing JSON-API libraries have shown me that supporting both the building and parsing of JSON-API payloads is actually quite useful, i.e. for filtering valid attributes when POSTing JSON-API stuff.

But if that's outside the scope that you want to support (don't we need something like that for Flarum?) then yes, json-api-response feels fitting.

[tobyzerner]

@franzliedke You might be right, but I'd argue that parsing requests is very much unrelated to building responses and would be better suited to a separate library.

In any case, got me brainstorming for a potential tobscure/json-api-request library:

use Tobscure\JsonApiRequest\ResourceSchema;
use Tobscure\JsonApiRequest\UpdateResourceDocument;

$schema = new ResourceSchema('posts');
$schema->setAttributes(['title', 'body']);
$schema->setToOneRelationships([
    'author' => ['users', 'companies']
]);
$schema->setToManyRelationships([
    'tags' => ['tags']
]);

// OR

class PostSchema extends ResourceSchema
{
    protected $type = 'posts';
    protected $attributes = ['title', 'body'];
    protected $toOneRelationships = [
        'author' => ['users', 'companies']
    ];
    protected $toManyRelationships = [
        'tags' => ['tags']
    ];
}
$schema = new PostSchema();

// validate document structure + contents according to schema,
// make sure resource ID matches
$document = UpdateResourceDocument::fromJson($json, $schema, '1');

$resource = $document->getData();

$attributes = $resource->getAttributes();
// perform custom validation on attributes here

$author = $resource->getToOneRelationship('author');
$tags = $resource->getToManyRelationship('tags');

foreach ($tags->getLinkage() as $tag) {
    $tagIds[] = $tag->getId();
}

[franzliedke]

Well, I have had a good experience with defining my entities in one place and using that for both serializing and parsing.

Doing both would also mean we can focus on that stuff, basically building an opinionated abstraction on top of the low-level stuff implemented in @f3ath's library.

[tobyzerner]

@franzliedke care to share some pseudo-code of your entity definitions?

It sounds like you have a much better idea of how this would work than I do, I need to understand before I can make a decision :)

[franzliedke]

Sure. It's Ruby, so it is pretty close to pseudo-code anyway ;)

entity do
  type 'discussions'

  writable attribute('title') {
    description 'The discussion title'
    type String
  }

  writable attribute('text') {
    description 'Discussion text'
    type String
    alias_for 'content_text'
  }

  writable attribute('author_name') {
    description 'The author\'s display name'
    type String
    reading { |discussion| "#{discussion['author_first_name']} #{discussion['author_last_name']}" }
    writing { |value|
      parts = value.split ' '
      { author_first_name: parts[0], author_last_name: parts[1] }
    } 
  }

  includable has_one('tag', TagResource) {
    embedded { |discussion| discussion['tag'] }
  }

  includable has_many('posts', PostResource) {
    filter_by 'discussion'
  }

  link('self') { |discussion| "/api/discussions/#{discussion['id']}" }
end

filters do
  optional 'tag' do
    description 'Only return discussions belonging to this tag'
  end
end

paginate! per_page: 25

collection do
  get 'List all discussions' do
    # Load all discussions
  end
end

member do
  get 'Get information about a discussion' do
    # Code for loading a discussion
  end
end

First of all, the entity block:

• describes the structure of a "discussion" entity
• is used for generating API documentation (hence the type and human-readable description)
• takes care of mapping to and from the data source (in our case it is backend services, here it would be a database) - alias_for maps to a different name in the hash that is sent to / received from the data source, reading and writing are used for less trivial decoration tasks
• declaratively lists relationships and links

The member and collection blocks define the HTTP methods (this is probably out of scope for your library, I am just listing it for completeness' sakes) that are available for the entire collection / a member resource. In those blocks, the following methods are available:

• for writable requests, resource maps the input (a JSON-API request body) to a hash with only the allowed keys (i.e. attributes marked as writable) and applies all transformations (alias_for and all writing transformers)
• for readable requests, filters (for the collection) returns a hash with all allowed filters, again mapped using aliases, if available; and id (for members) is the ID from the URL
  When those blocks return a hash, this is then serialized in JSON-API format using the entity definition.

I hope that helps a bit. :)

[tobyzerner]

@franzliedke ah I see, thanks!

Hmm, what's the nicest way to replicate a declaration interface like that in PHP, especially where entities must reference each other (includable has_one('tag', TagResource))?

[tobyzerner]

How about something like this?

// defining entities
use Tobscure\JsonApi\Schema;

$postsSchema = new Schema('posts');
$tagsSchema = new Schema('tags');

$postsSchema->attribute('text')->aliasFor('content_text');
$postsSchema->hasMany('tags', $tagsSchema);

$tagsSchema->attribute('name');
$tagsSchema->hasMany('posts', $postsSchema);

// controller for GET /api/posts
use Tobscure\JsonApi\ListEndpoint;
use Tobscure\JsonApi\Parameters;

$endpoint = new ListEndpoint($postsSchema, function (Parameters $params) {
    return Posts::all()->load($params->getInclude());
});

// optionally set up extra parameters for validation
$endpoint->paginate(25);
$endpoint->addFilter('tag');
$endpoint->disallowInclude('tags');

$response = $endpoint->respond($_GET);
// validates provided params according to schema and endpoint configuration.
// gets raw data using callback provided.
// constructs response document from raw data according to schema and params.

// controller for GET /api/posts/{id}
use Tobscure\JsonApi\ShowEndpoint;
use Tobscure\JsonApi\Parameters;

$endpoint = new ShowEndpoint($postsSchema, function (Parameters $params) use ($id) {
    return Posts::find($id)->load($params->getInclude());
});

$response = $endpoint->respond($_GET);

// controller for POST /api/posts
use Tobscure\JsonApi\CreateEndpoint;

$endpoint = new CreateEndpoint($postsSchema, function ($attributes, $relationships) {
    return Post::create($attributes);
});

$response = $endpoint->respond($json, $_GET);
// validates provided JSON-API document and params according to schema.
// passes attributes and relationships to callback.
// constructs response document from returned raw data according to schema and params.

// controller for PATCH /api/posts/{id}
use Tobscure\JsonApi\UpdateEndpoint;

$endpoint = new UpdateEndpoint($postsSchema, function ($attributes, $relationships) use ($id) {
    $post = Post::find($id);
    $post->update($attributes);
    return $post;
});

$response = $endpoint->respond($json, $_GET);

// outputting response
$document = $response->getDocument();
http_response_code($response->getStatusCode());
header('Content-Type: ' . $document::MEDIA_TYPE);
echo json_encode($document);

// example of a Flarum API controller
final class ListPostsController extends AbstractListController
{
    private $posts;

    public function __construct(SchemaProvider $schemas, Post $posts)
    {
        parent::__construct($schemas->get('posts'));

        $this->posts = $posts;
    }

    public function configure(ListEndpoint $endpoint)
    {
        $endpoint->paginate(25);
        $endpoint->addFilter('tag');
        $endpoint->disallowInclude('tags');
    }

    public function data(Parameters $params)
    {
        return $this->posts->all()->load($params->getInclude());
    }
}

[tobyzerner]

@franzliedke ping :D

[tobyzerner]

It probably makes sense to increase the scope even further, and handle the routing as well. Something like:

$controller = new Tobscure\JsonApiServer\Controller;

$controller->addResourceType(new PostsResourceType($postRepository));
$controller->addResourceType(new UsersResourceType($userRepository));

$response = $controller->handle($request); // using Psr\Http\Message interfaces

// now GET/POST/PATCH/DELETE for /posts[/id] and /users[/id] are automatically 
// handled according to the ResourceType implementations and their Schema

class PostsResourceType extends Tobscure\JsonApiServer\AbstractResourceType
{
    public static $type = 'posts';

    public static function define(Schema $schema)
    {
        $schema->attribute('number');
        $schema->attribute('time');

        $schema->attribute('contentType');
        $schema->attribute('content')->if($isEditableOrNotComment)
            ->required()
            ->writableIf($isComment)
            ->writing(function ($content, Request $request) {
                return [
                    'content' => $content,
                    'editTime' => time(),
                    'editUser' => $request->getActor()
                ];
            });
        $schema->attribute('contentHtml');

        $schema->attribute('ipAddress')->ifCan('viewIps')
            ->default(function (Request $request) {
                return array_get($request->getServerParams(), 'REMOTE_ADDR', '127.0.0.1');
            });

        $schema->attribute('editTime');

        $schema->attribute('isHidden')->boolean('hide_time')
            ->writeableIfCan('edit')
            ->writing(function ($isHidden, Request $request) {
                return [
                    'hideTime' => $isHidden ? time() : null,
                    'hideUser' => $isHidden ? $request->getActor() : null
                ];
            });
        $schema->attribute('hideTime');

        $schema->attribute('canEdit')->can('edit');
        $schema->attribute('canDelete')->can('delete');

        $schema->belongsTo('user', 'users');
        $schema->belongsTo('discussion', 'discussions')->required()->assertCan('reply');
        $schema->belongsTo('editUser', 'users');
        $schema->belongsTo('hideUser', 'users');

        $schema->filter('discussion')->integer('discussion_id');
        $schema->filter('number');
        $schema->filter('user')->integer('user_id');
        $schema->filter('type');
    }

    public function __construct($postRepository)
    {
        $this->postRepository = $postRepository;
    }

    public function show($id)
    {
        return $this->postRepository->find($id);
    }

    public function list($include, $filter)
    {
        // $include and $filter have been validated against the Schema
        return $this->postRepository->all()->load($include);
    }

    public function create($attributes, $relationships)
    {
        // $attributes and $relationships have been validated against the Schema
        $id = $this->postRepository->create($attributes, $relationships);

        return $this->show($id);
    }

    public function update($id, $attributes, $relationships)
    {
        // $attributes and $relationships have been validated against the Schema
        $this->postRepository->update($id, $attributes, $relationships);

        return $this->show($id);
    }

    public function delete($id)
    {
        $this->postRepository->delete($id);
    }
}

In Flarum's case, we would implement a base AbstractEloquentResourceType, and then each of our resource types would extend that and simply define their Schema.

[franzliedke]

@tobscure Sorry for the long silence, this lay around on my todo list forever. 😞

That said, you seem to have figured it out, looks like a good attempt. These days, I am tempted to think that plural and singular endpoints could be separated again (maybe even from the actual entity / schema definition); really not sure what's best. 😠

Quick question, though: any particular reason that define() receives a Schema instance and modifies it? Why not just return a new instance from that method (can then be made using any class that implements a particular schema interface)?

[franzliedke]

json-api-schema kind of describes the scope quite well, given you omit the routing (which, on the other hand, would be quite thin - yet useful - if it only deals with PSR-7 / PSR-15 abstractions).

[franzliedke]

And lastly, assuming that @f3ath's json-api-php/json-api package is complete, we could simply build on top of it to provide the schema / server stuff. (You might even want to join forces, there's still room in that GitHub org.

@f3ath What's the status?

[f3ath]

@franzliedke v1 is ready, I think it describes most of the response constraints well enough. I tried to express use cases as unit tests, so please take a look at them. Here is good example of how to build the response from the official docs. It requires php 7.1 and higher.

The only thing I don't like about it is mutability. So the next major version (if it ever happens) will probably be immutable.

[f3ath]

Invited you both to the org. In case you wanna make use of it.

[franzliedke]

I am slightly worried about performance, though. (The serialization of lots of resources on lots of endpoints is somewhat critical to performance.)

@tobscure Do you still have your benchmark code lying around? You mentioned in one of the related PRs (I think) that you had managed to increase performance by a factor of two or three with the WIP refactorings.

[f3ath]

Meanwhile the immutable version is ready https://github.com/json-api-php/json-api

[tobyzerner]

@f3ath nice! have you benchmarked v2?

[f3ath]

@tobscure i have not. Can you suggest a meaningful way of doing that? The only thing I can think of so far is just comparing v1 vs v2 on some sort of synthetic test.

[tobyzerner]

Yeah I'm by no means an expert but that's what I would do. Benchmark the construction of a relatively generic real-world document.

[f3ath]

I just ran a simple test. Encoding the document from the front page 10k times in the console. v1 is faster (1.4 seconds vs 2.2 seconds on my laptop) which is expected. I think I see some room for improvement here though.

[franzliedke]

@f3ath With v1 you mean current tobscure/json-api? What about his v2? It's supposedly even faster.

[tobyzerner]

@franzliedke no we're talking about json-api-php/json-api

[f3ath]

@tobscure v2 has been updated, it is now even faster than v1. Some benchmarks included.

[f3ath]

if you guys are still considering it, let me know what you think. I need some sanity check anyway.

[tobyzerner]

Now that we require PHP 7 in Flarum, no reason we can't build next version of tobscure/json-api on top of it :)

[franzliedke]

@f3ath I am very open to that idea as well.

If it's not too much work, it would be awesome if you could benchmark rendering the same document with Toby's library as well. :)

[f3ath]

Here #141

Results of 10k times json_encoding the example document:

• my v2: 1.35 seconds
• @tobscure's new version: 1.03 seconds

But!

1. I did not figure out how to add the relationships key to the included objects like in the example
   here. So the result is not very accurate.
2. @tobscure's implementation does not enforce checks which my implementation does, e.g. document members check, allows inclusion of undocumented members, etc

[tobyzerner]

That's a reasonably promising result, although it is a bit like comparing apples and oranges, because these two libraries are doing somewhat different things (hopefully I've got this right):

• json-api-php/json-api constructs JSON based on a static representation of the JSON-API document (as per the readme example)
• tobscure/json-api constructs the JSON-API representation from raw data (ie. you give it a resource with some relationships and it will automatically extract the related resources into the "included" section, etc)

In any case, we're probably keen to implement the aforementioned API which would build those static document representations and then use json-api-php/json-api to transform them into JSON.

[f3ath]

The goal of my implementation is to make impossible to produce an invalid json api document. it is all immutable value objects. It does not assume anything about where the data come from. So there will need to be an extra mapping layer to map your business entities to ResourceObject VOs, and another extra layer to build the document object. So there may be some trade-off between performance and strict spec compliance.

[f3ath]

Yup yup, glad we're on the same page.

[f3ath]

Any news to share? Have you got time to evaluate my implementation?

[franzliedke]

Other than giving it a quick look, not yet.

I believe we're waiting to fully incorporate this until after the first stable release of Flarum...

[tobyzerner]

@f3ath and anyone else interested: I finally got around to building the version of this library I've always wanted to. It's a JSON:API server which takes care of routing, parsing requests, and generating responses using json-api-php/json-api. All you have to do is define your schema and plug in your models. Please take a look and let me know if you have any feedback!

https://github.com/tobscure/json-api-server

[f3ath]

@tobscure Looks pretty nice and clean ant the first glance. I will give it a closer look soon. Would you like to use json-api-php namespace for the server part? Seems like a perfect fit. I'd be glad to add you to the org.