RText_Protocol

= RText Protocol

RText frontend and backend pass messages containing JSON objects.
Normally the frontend invokes a backend command by means of a request message and the 
backend will eventually reply with a response message. 

== Versioning

RText protocol supports versioning mechanism with a single version natural number.
The actual version of the protocol is 1.

== Encoding

As the RText protocol uses JSON, the encoding of messages is UTF-8 by definition.
However, the protocol further restricts UTF-8 to only 7-BIT-ASCII characters, which
effectively makes the protocol encoding 7-BIT-ASCII (or US-ASCII), which is valid UTF-8.

Frontends and backends should not apply any transcoding to the data found in RText files.
The reason is, that most often information about an input file's encoding is not reliable.
If the source encoding is wrong, transcoding just makes things worse: either data is
misinterpreted or information is lost due to character replacement.

Instead, data should be passed as is, or in other words: it should be interpreted as
"binary" data. Since the RText protocol is restricted to 7-BIT-ASCII, all non 7-BIT-ASCII
characters are escaped using the following pattern: Each byte with a value of 0x80 or higher
results in three 7-BIT-ASCII characters: a leading "%" and two hexadecimal figures in lower case.
In addition, the character "%" is escaped in the same way, i.e. "%" will always be escaped as
"%25" ("%" has byte value 0x25 in 7-BIT-ASCII).

Example:

The word "Übung" (german: exercise), encoded in ISO-8859-1 would result in the string:
"%dcbung" (the "Ü" Umlaut has a byte value of 0xdc is ISO-8859-1).


== Request Messages

Each request message contains a field ``command`` and a field ``invocation_id``.
The command field holds the name of the command to be invoked as a string.
The invocation id field holds an identifier which will be repeated by the backend's response.
The "version" field was introduced in version 1. It is optional for protocol version 0 requests,
but mandatory for newer versions.

  {
    "type": "request",
    "version": <protocol_version [integer]>,
    "command": <command>,
    "invocation_id": <invocation_id>
    ...
  }

== Response Messages

The response message repeats the invocation id of the request it replies to.
If the requested command is not known by the backend, it will respond with a 
"unknonw_command_error" message instead (see below).

  {
    "type": "response",
    "invocation_id": <invocation_id>
    ...
  }

== Error Messages

There are a number of error message which may be sent in response to a request message.

=== Unknown Command Error

  {
    "type": "unknown_command_error",
    "invocation_id": <invocation_id>,
    "command": <unknown command [string]>
  }
  
=== Unsupported Protocol Version Error

Introduced in version 1.

  {
    "type": "unsupported_version",
    "invocation_id": <invocation_id>,
    "version": <supported version [integer]>
  }

== Progress Information

Before the actual response is sent, the backend may send progress information.
This is useful whenever a command takes a longer time to complete.
Frontends should be prepared to receive progress information messages for any command.
They may however choose to ignore this information, i.e. not display any progress to the user.

  {
    "type": "progress",
    "invocation_id": <invocation_id [integer]>,
    "percentage": <percentage [integer][0..100]>,
    "message": <message [string]>
  }

The percentage and message fields are optional. 
If percentage is present, the frontend should display a progress bar, otherwise it should just
indicate to the user that a job is ongoing.
The message field may carry information about the currently ongoing subtask.

== Commands

For each command, the layout of the request and response messages will be given below.
Note that the invocation id field is present in every request and response but is omitted for brevity.

=== Backend version request

Introduced in version 1.

This command requests the backend to disclose its supported protocol version.

  {
    "type": "request",
    "command": "version"
  }
  
The backend responds with a version information.

  {
    "type": "response",
    "version": <version [integer]>
  }
  
The backend supporting protocool version 0 only responds with an 'Unknown Command Error'.

=== Load Model

This command requests the backend to load or reload the model.

  {
    "type": "request",
    "command": "load_model"
  }

The response may indicate problems which were detected during loading.
In order to reduce the size of the response message, problems are grouped by file.
The ``total_problems`` field is used to indicate the total number of problems which may be lower
than the number actually returned. If the total number of problems is unknown, ``total_problems``
should be set to -1. This may be the case when problem detection is interrupted in order to
limit detection effort and/or response time.

  {
    "type": "response"
    "total_problems": <number of total problems or -1 [integer]>,
    "problems": [
      { 
        "file": <fully qualifed file name [string]>,
        "problems: [
          {
            "message": <message [string]>,
            "severity": <['debug', 'info', 'warn', 'error', 'fatal']>,
            "line": <line number [integer]>
          }
          ...
        ]
      }
      ...
    ]
  }

=== Content Complete

This command is a request by the frontend in order to show content completion options at a given
location within a file. The location is expressed using a set of context lines and the cursor
column position in the current line. See section "Context Extraction" for details about how to
build the set of context lines in the frontend. Column number start at 1. 

  {
    "type": "request",
    "command": "content_complete",
    "context": <context lines [array of string]>,
    "column": <cursor column [integer]>
  }

The backend replies with a list of completion options. 
The field ``insert`` holds the text to be inserted if the completion option is chosen. This text 
may contains placeholders for cursor position. An editor may use them to assist a user in 
filling additional completion fields.
Each placeholder must start and end with a vertical bar character (``|``) and can contain up to
three optional parts separated by an additional vertical bar character: ordering number, name and
description of this cursor position. E.g.: ``||``, ``|1|name|Entity name|``, ``|||New value|``.
Placeholders with the same name must be considered as the same value repeated in different 
positions.
The field ``display`` contains the string which should be displayed to the user in some kind of 
completion option menu. An optional description field may provide more information about a 
particular option.

If there are no completion options, the backend may send an empty response.

  {
    "type": "response",
    "options": [
      {
        "display": <text to display [string]>,
        "insert": <text to be inserted [string]>,
        "desc": <optional description [string]>
      }
    ]
  }

=== Link Targets 

This command is issued by the frontend when the user tries to follow a hyperlink, e.g. following a model reference.
The frontend needs to send the context as described in section "Context Extraction" and the 
column where the cursor (e.g. the mouse pointer, or an actual text cursor) is placed.
Column numbers start at 1.

  {
    "type": "request",
    "command": "link_targets",
    "context": <context lines [array of string]>,
    "column": <cursor column [integer]>
  }

The backend analyses the text at the cursor position in order to find out if a link is present.
If so, it sends back the columns of the beginning and the end of the link as well as the actual
link targets. Column positions are inclusive, i.e. the beginning and the end column are part of
the link. Link targets contain the string to be displayed to the user, the filename and the line
number within the file.
An optional description field may provide more information about a particular lin target.

If there is no link, the backend may send an empty response.

  {
    "type": "response",
    "begin_column": <begin column of hyperlink [integer]>,
    "end_column": <end column of hyperlink [integer]>,
    "targets": [
      {
        "display": <display name [string]>,
        "file": <fully qualified file name [string]>,
        "line": <line number [integer]>,
        "desc": <optional description [string]>
      }
      ...
    ]
  }

=== Find Elements

This command is used to search for elements within the model. The frontend should allow the user
to enter an arbitrary search pattern as a string. This search pattern is sent to the backend
which actually defines how the pattern is interpreted.

  {
    "type": "request",
    "command": "find_elements",
    "search_pattern": <search pattern [string]>
  }

The backend responds by sending the total number of elements found and a list of elements
together with their file and line information.
Note that the number of elements actually returned may be lower than the total number. 
If the total number of elements is unknown ``total_elements`` should be set to -1.
This allows to truncate large search result sets and still provide the user with the number of
elements which were omitted or at least with the fact that the result set has been truncated.
An optional description field my provide more information about a particular element.

If there are no search results, the backend may send an empty response.

  {
    "type": "response",
    "total_elements": <total number of elements or -1 [string]>,
    "elements": [
      {
        "display": <display name [string]>,
        "file": <fully qualified file name [string]>,
        "line": <line number [integer]>,
        "desc": <optional description [string]>
      }
      ...
    ]
  }

=== Context Information

This command is used by the frontend to request information about a particular position in
a file. The frontend will send the current context (section "context extraction") and the cursor
column. If for example, the context information is to be shown as a hover at the position of the 
mouse pointer, the context and column need to be calculated at the position of the mouse pointer,
not the position of the text cursor.

  {
    "type": "request",
    "command": "context_info",
    "context": <context lines>,
    "column": <cursor column>
  }

The backend reponds by sending the textual description to be shown to the user.

  {
    "type": "response",
    "desc": <textual description>
  }

=== Custom Commands

This command is used to retrieve a list of custom commands understood by the backend.
These commands may be available in a specific context only. If the frontend includes context
and column information in the request, the backend will list only commands applicable in that 
context. If the context information is omitted, the backend will return all commands applicable
without any specific context.

Note that even for commands without context information, the frontend should repeat to request
the command list as the list of available commands may change. For example, the command list could
be requested everytime the user gets a chance to choose from a command menu.

  {
    "type": "request",
    "command": "custom_commands"
    "context": <context lines, optional>,
    "column": <cursor column, optional>
  }

The backend responds by a list of commands which may be categorized into a tree structure.
This information can be used to show the commands in a hierarchical menu, e.g. a context menu.

  {
    "type": "response",
    "entries": [
      {
        "type": "category",
        "name": <display name>,
        "entries": [
          {
            "type": "command",
            "name": <display name>,
            "id": <command identifier>
          }
          ...
        ]
      }
      ...
    ]
  }

=== Custom Command Parameters

Before a custom command can be invoked, the frontend must check with the backend if parameters
are required. This is done by means of the command

  {
    "type": "request",
    "command": "check_custom_command_parameters",
    "command_id": <custom command id>
    "context": <context lines, optional>,
    "column": <cursor column, optional>
    "parameters": [
      { 
        "name": <parameter name>,
        "value": <parameter value>
      }
      ...
    ]
  }

If the command id was returned by a "Costum Commands" request which
included context information, the frontend should send the same context information in the new
invocation request and all repeated requests (see below). If this information was not sent 
in command list request (non-context sensitive command), it should neither be send in the new 
command invocation.

If a command requires parameters, the response will have the field `success` set to false and
will include information for building a user dialog.

The dialogs contain elements of certain types:

* label: text presented to the user
* text: text input field
* table: table view/input
* hidden: hidden field used to hold state information

The frontend should keep repeating requests as long as the field ``success`` is set to ``false``.

Note that the backend doesn't hold any state, so that commands can easily be canceled by just 
not making any additional requests (and not actually invoking the command).

If the response contains step information (number of steps, current step), the the frontend
should display buttons for moving forward and backward in a wizard.

When the frontend receives dialog information, it should display the dialog filled with the
widgets as specified and with widget content set up properly (text fields preset with strings,
rows selected in table, etc).

When the user has finished the dialog, the frontend should send all input information back to
the backend as name/value pairs. This includes:

* hidden fields
* text fields unless disabled
* table fields unless disabled

  {
    "type": "response",
    "success": <true|false, true if command is finshed>, 
    "num_steps", <number of steps of this (wizard) command, optional>,
    "current_step", <current wizard step, optional>,
    "dialog": {
      "title": <dialog title>,
      "elements": [
        {
          "type": "lable",
          "text": <text to display>
        },
        {
          "type": "hidden",
          "name": <parameter name>,
          "value": <value of hidden filed [string]>
        },
        {
          "type": "text",
          "name": <parameter name>,
          "value": <preset value>,
          "error": <error text>
          "description: <discription of parameter, optional>
          "disabled:" <true|false, if set to false the input widget can't be changed>
        },
        {
          "type": "table",
          "name": <parameter name>,
          "num_min": <min number of elements to choose, default: 1>,
          "num_max": <max mumber of elements to choose, or * for unlimited, default: 1>,
          "columns": [
            {
              "name": <column name>,
              "description: <discription of parameter, optional>
            }
            ...
          ],
          "rows": [
            {
              "id": <identifier to be sent back>,
              "values" [
                {
                  "display": <display name>,
                  "file": <fully qualfied file name, optional>,
                  "line": <line number, optional>,
                }
                ...
              ]
            }
            ...
          ],
          "selected": [ <row id>, <row id>, ...],
          "error": <error text>,
          "description: <discription of parameter, optional>
          "disabled:" <true|false, if set to false, no selection can be made>
        }
      ]
    }
  }

=== Custom Command Invocation

Custom commands are invoked using the command id returned by the "Custom Commands" request.
Before a command can be invoked, the parameters must be figured out by means of the "Check
Custom Command Parameters" command. Once figured out, the parameters will be sent with the 
command invocation as a name/value list. 

The context information should be sent as with the "Check Custom Command Parameters" command.

  {
    "type": "request",
    "command": "invoke_custom_command",
    "command_id": <custom command id>
    "context": <context lines, optional>,
    "column": <cursor column, optional>
    "parameters": [
      { 
        "name": <parameter name>,
        "value": <parameter value>
      }
      ...
    ]
  }

=== Stop Service

This command is normally invoked when the frontend terminates or otherwise needs to terminate
the backend service. When receiving this command, the backend should terminate.

  {
    "type": "request",
    "command": "stop"
  }

Before actually stopping, the backend should send an empty response.

  {
    "type": "response"
  }

== Context Extraction

Context lines are lines from an RText file which contain a (context) command and all 
the parent commands wrapped around it. Any sibling commands can be omitted as well as
any lines containing closing braces and brackets. The order of lines is the same as in the 
RText file.

Here is an example. Consider the following RText file with the cursor in the line of "Command5"
at the time when the auto completion command is issued.

  Command1 {
    Command2 {
      Command 3 {
        Command 4
      }
      role1: [
        Command5          <== cursor in this line
        Command6
      ]
    }
    Command7
  }

The context lines in this case would be the following.

  Command1 {
    Command2 {
      role1: [
        Command5

The current line is always the last of the context lines.

See RText::Frontend::Context::extract for a concise implementation of the required algorithm.
 
Note that all siblings of the command and parent commands have been stripped off, as well as
any closing braces or brackets.

The purpose of this special context line format is to keep the task of extracting the
context in the frontend simple and the amount of data transmitted to the backend low.
It's also a way to keep the parsing time of the context low in the backend and thus to minimize
the user noticable delay.

In case of line breaks, the frontend is responsible to join the lines before sending the 
context information. For commands which use a column position, the position is the position
within the joined line. This means that, when sending a command, the frontend must convert
the column position in the broken line into the new position in the joined line.
When reading back column information in a response (e.g. link command) the frontend must
convert the column position in the joined line into a position in the respective broken
fragment of a line.