What are the commonalities and differences between LinkML and TypeSpec?

[cmungall]

https://typespec.io/

[cmungall]

Brief summary:

TypeSpec is an IDL for describing APIs and API payloads. The syntax is very typescript-like (there are JS APIs). It has a lot of features of modern type languages, e.g compositional definition of types with & and |. It could be seen as a clearer syntax for OpenAPI/swagger, although it includes features not directly expressible in JSON-Schema, e.g. Generics/Templates.

The parts of TypeSpec that can't be directly expressed in LinkML are:

1. Operations
2. Generics/Templates.

The first is due to difference in scope - LinkML is for describing the data model, not an interface. However, it's easy for users to roll their own operation metamodel (we should probably make a standard for this).

The second is not something we have seen a lot of requests for, but could in principle be added. IMO this is a feature that is more commonly needed for a local programmatic data model rather than a data exchange standard.

LinkML has many features that can't be directly expressed, but one of the main differences is the focus of LinkML on interoperability - e.g. Semantic Enums, URIs, ...

Obvious differences include a focus on different language ecosystems - TypeSpec seems to have great JS/TS support, there appear to be npm modules to emit Python (unclear if pydantic or dataclasses or anything else - need to evaluate). LinkML support is best for Python, but we are gradually building out ecosystems for JS/TS, R, Java

[jsnyder-csdisco]

This is interesting for lots of Browser reasons (and perhaps GraphQL reasons too).
I would say that from a "data exchange" perspective ... better support for ProtoBuf would go a lot further in the data exchange "vision".
For example, supporting proto better would not only make the language support better, it would also enable streaming examples like having a Schema for Kafka with ProtoBuf payloads and opens the door to lots of other data exchange mechanisms.

[cmungall]

Interesting. Is the kind of support you envision better generation of .proto using https://linkml.io/linkml/generators/protobuf.html, or support for building bindings, e.g. between pydantic and protobuf?

I'm not very familiar with the Kafka use case, it looks like confluent supports a schema registry, with options to use Avro, ProtoBuf, or JSON-Schema:
https://docs.confluent.io/platform/current/schema-registry/index.html

[jsnyder-csdisco]

There are two aspects of this idea that provide value to me.

1. Getting Proto to be more developed would help on the "Data in an async stream" be better (both JSON schema and Proto) serve as the "data schema" -- LinkML is pretty suited to this use case actually.
2. Making it easier to Produce or Consume data in the "stream" would be part of the Language Binding problem you mentioned is really two sides of the same coin. This would handle "data at rest" type problems and "data in motion" as well.

To make proto make a bit more sense in LinkML .... a part of the LinkML schema meta would need something like this:

generator_directives:
  protobuf:
    package_name: mycompany.mysystem.model.protobuf.v1
    import_file_name: mycompany/mysystem/model/protobuf/v1/common.proto
    options:
      java_package: com.mycompany.mysystem.model.protobuf.v1
      optimize_for: SPEED

So that the various generators could pick up their options when generating (keeping the details with the schema).