Explain why one key format is not realistic.

[msporny]

This PR attempts to address issue #115 by explaining that having one key format is not realistic, how the spec attempts to minimize key formats, but understands that ecosystems have to support the key formats that are native to their ecosystems.

/cc @peacekeeper

---

Preview | Diff

[rxgrant]

The proposed "suggests" plea is quaintly non-normative, but DCD will not object to it.

A normative change would be to set up another registry where W3C participants agree on the encodings, and demand two interoperable open source implementations in a range of required languages (Javascript and Rust might be a good start) for every allowed key encoding. That means work.

Some key encodings require maintaining a registry, to describe all the combinations with different curves and whatnot, which means that the registry maintainers could implement a gatekeeping attack. That is a reason not to have just one encoding unless it is really good and everyone likes the registry and the maintainers.

The kind of risk from having too many encodings in the ecosystem is an embrace-and-extend attack. This solveable by pulling in a library to implement an encoding seen in the wild, which is not the same kind of risk as being shut out of the ecosystem. Meanwhile, implementors are incentivized to not create a new standard that no one else parses. Therefore, I assess the overall risk as low.

[iherman]

The issue was discussed in a meeting on 2024-11-20

• no resolutions were taken

View the transcript

1.2. Explain why one key format is not realistic. (pr controller-document#120)

See github pull request controller-document#120.

Manu Sporny: Issue was raised by Microsoft four years ago, asking us to settle on one key format. Discussed internally, said no, that's not the way it's implemented in the field.
… While it would be nice for there to be one, there are multiple key formats in the world and often in one system.
… I think it's got positive reviews, I think I addressed David Chadwick's concerns.
… One question I have for the group is that we mention key formats that are more broadly distributed but not mentioned in the document. Should we add SHA public keys and others in the context as something that people can express but not write into the spec?

Ivan Herman: Apologies if this is already in the document. The argument on why we have two key formats in the document... Is there any trace of that in the document itself?
… I want to see something in the document that explains why we do that.

David Chadwick: I was coming at this with a blank sheet and historical knowledge, but not about discussions.
… Back in the 1990s playing with X.509, there were multiple key formats, which was a pain, hence my comment.
… I accept the group's decision.

Manu Sporny: To Ivan, the text you're asking for is in the PR.

Ivan Herman: +1.

Manu Sporny: To DavidC's point, I don't think we should get into this discussion. We should just merge this and move on. We starting 10 years ago with public key PEM was enough but as people came in and out and as development happened, these were the two key formats we could get people to like.
… These formats don't do much beyond what PEM does.
… I think we should have stuck with PEM but this is the direction the group has gone.
… I don't think it's worth reopening at this point, but we may do so in the future or add an extension.
… It's very weird that we don't allow the two most widely distributed key formats.

[peacekeeper]

I remember the discussions in the past about one vs. multiple key formats, and I agree with the proposed text here. I have worked on some DID methods where e.g. Base58 or Multikey or JsonWebKey are all not easy to do. Also let's keep in mind that there are verification methods that don't just consist of a single key (e.g. see ConditionalProof or CesrKey).

[brentzundel]

Perhaps a line could be added like "Profiles of this document are encouraged to restrict the number of allowed key formats as much as possible"

[selfissued]

This PR adds words without adding clarity. In fact it does the opposite by withdrawing our clear statement that "this specification limits the number of formats".

I am unwilling to weaken that. Standards are about making choices.

[msporny]

Standards are about making choices.

Standards are also about documenting reality. The reality is that there is no one key format that developers have agreed upon in the past or even today.

Standards are also about documenting the available choices that developers are deploying.

This PR adds words without adding clarity. In fact it does the opposite by withdrawing our clear statement that "this specification limits the number of formats".

Please provide concrete text that would address your objection.

[selfissued]

Standards are also about documenting reality.

Yes, but the language in this PR further opens the Pandora's box by implying that other key formats may be used beyond the two already described in the specification. That not documenting reality - it's failing to make choices. You did agree in your comment that standards are about making choices.

Please provide concrete text that would address your objection.

I will plan to do that after the weekend. But the gist of what will address the objection is continuing to explicitly limit the key representations to the two already described in the specification.

[msporny]

The language in this PR further opens the Pandora's box by implying that other key formats may be used beyond the two already described in the specification. That not documenting reality - it's failing to make choices.

No, disagree, it's documenting reality. The specification is an "open world" specification, other developers can choose to extend it in various ways, including the support of other key formats.

To put it in perspective, none of us wanted the Ethereum community to use publicKeyHex, but that's what they used because it made sense for them (because in that community, keys are expressed as hex values). That's documenting reality -- no amount of us telling them to do something else would have helped. We can guide people to use one key format, but the global ecosystem has consistently not done that over multiple decades and not acknowledging that is putting our heads in the sand. Specifications also need to document reality in order to increase the chances of interoperability.

Furthermore, by your logic, everything was going fine with just ASN.1/DER/PEM-encoded keys... the choice had been made. Sure there were other choices, but the worst thing that could have happened at that point, per your logic, is to introduce an entirely new key format, which the JOSE WG did with JOSE Web Key... and then further harmed interoperability by allowing x509 certificate chains to be embedded in the key descriptions resulting in undefined behavior when it came to mixing the two key formats... and then JOSE further failed to make choices by allowing multiple key types, multiple curve types, seemingly random use of octet representations, etc... all colossal failures to make a choice (again, per your logic).

Now, I don't see JOSE as failing to make a choice -- sure, it provides far too many choices, but that was done in an attempt to reflect reality (even though, IMHO, it got it wrong when it came to the number of choices on the whole -- i.e., there was no rational reason for 512-bit EC key lengths in 2014 much less a decade later in 2024.

All that to say, we have lots of proof to the contrary that picking a single key format is NOT how the world operates. Even suggesting that two key formats are going to be workable is wishful thinking, IMHO, but that's where consensus seems to have landed in this group.

All we can do at this point is to urge ecosystems to use a single key type, but acknowledge that the two key types we provide might not work for their ecosystems and so we understand if they have to make a different choice, but hope that they do so in a way that optimizes for interoperability in their ecosystem and the broader ecosystem.

You did agree in your comment that standards are about making choices.

We're miscommunicating. I don't agree with that statement alone. In fact, I reject it as a myopic guiding principle taken by itself in a vacuum.

I do agree that standards are about making choices when it's possible to reduce the number of choices and have it reflect reality.

Please provide concrete text that would address your objection.

I will plan to do that after the weekend. But the gist of what will address the objection is continuing to explicitly limit the key representations to the two already described in the specification.

The text doesn't change that the spec limits the key representations to two. The new text acknowledges that others can be used too, but we warn against that.

Thank you for planning to suggest concrete text.

[selfissued]

Suggested change

	Implementers are often exposed to many different verification material formats
	such as
	<a href="https://en.wikipedia.org/wiki/Secure_Shell">SSH public keys</a>,
	<a href="https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail">
	PEM-encoded public keys</a>,
	hex-encoded public keys,
	<a data-cite="?RFC7517">JSON Web Keys</a>,
	<a href="#Multikey">Multikeys</a>, and
	<a data-cite="?8152#section-7">COSE public keys</a>.

Remove preamble to existing text that doesn't add actionable information.

[selfissued]

Suggested change

	suggests limiting the number of formats used to express verification material in
	a [=controller document=]. The fewer formats that implementers have to choose
	from, the more likely that interoperability will be achieved for a particular
	ecosystem. Ideally, there would be one verification material format per
	application; however, history has shown both that multiple verification
	material formats might be used with any given application, and that
	implementers will support as many verification material formats as make
	sense for their ecosystem.
	limits the number of formats for expressing verification material in a
	[=controller document=]. The fewer formats that implementers have to
	choose from, the more likely that interoperability will be achieved. This
	approach attempts to strike a delicate balance between easing implementation
	and providing support for formats that have historically had broad deployment.
	Implementations of this specification MUST NOT use key formats other than
	those defined by this specification.

These changes address @msporny's request:

Please provide concrete text that would address your objection.

[iherman]

@selfissued , just to clarify the situation for the future. If, let us say in two years yet another widely adopted format comes to the fore, implementations should not use that if they want to be comformant, unless a new charter is accepted by the AC which explicitly accepts to add the new format to the list (noting that such a change would be a class 4 change, cannot be done by the WG under its current charter once the CD is published as a Rec).

I am not against that situation, giving the AC its say into a third format is probably a good thing. But I feel uncomfortable in the intermediate situation, when the new format is used (because it is used by a particular community) and the longish process to standardize its usage. Implementation would then not be comform to our spec.

All this is a longish reason why I am asking: would be o.k. using SHOULD NOT instead of MUST NOT in the last sentence?

[selfissued]

The reason it's "MUST NOT" is so there is a bright line between conformance and non-conformance to the specification - thereby increasing interoperability among implementations.

If people want to standardize other key formats and revise the specification to add them in the future, they can do so, but that's outside the scope of what this v1 specification is designed to accomplish.

[iherman]

And I believe SHOULD NOT achieves essentially the same: the meaning is that implemetations should take all necessary and possible actions to deviate from the standard and I feel it is fine to allow for this in this particular case. There is a difference, after all, between SHOULD NOT and MAY NOT.

We may agree to disagree on this point, let us see what others in the group say. I’m, otherwise, o.k. with your proposed text.

[TallTed]

+1 for SHOULD NOT.
-0.9 for MUST NOT.

[iherman]

The issue was discussed in a meeting on 2024-12-11

• no resolutions were taken

View the transcript

1.1. Explain why one key format is not realistic. (pr cid#120)

See github pull request cid#120.

Manu Sporny: should we mention that every PR and every remaining issue is editorial, and we could do it during CR? Fine to process the PRs, just wondering if we can get around to a proposed resolution to transition to CR today.

Brent Zundel: would love to do that today.

Manu Sporny: Ok, let's do 120, this has to do with issue 115, this is a comment from Travis at Microsoft, he said you should try to do one key format, we are trying to respond to him and say the best we can do is two key formats.
… I think selfissued is objecting to the change.
… and I think that's where we are, what we are trying to do is hit a balance, the best we can do is two key formats because there are multiple in the ecosystem and we can never really get to a single key format.
… and in application space you should limit to a single key format.
… but there are objections to what would need to be changed there.

Brent Zundel: Any comments or anything that could help us move forward on this?

Michael Jones: I'm unwilling to weaken the meaning of the spec, hence my request for changes, the PR weakened the meaning of the spec.

Manu Sporny: selfissued, you suggested we add text that say that implementations of this specification must not use key formats other than those defined in the spec, but we don't have consensus on that language, we may be able to say should not.
… we are trying to be eventually responsive to the representative from Microsoft.
… the direct ask to selfissued is, would you be ok with should not vs must not.

Michael Jones: no, that changes what it means to conform to the spec.

Manu Sporny: to be clear, we have always allowed other key formats, it goes with the extensibility model for Data Integrity, I disagree that the spec today says you must not use other key formats.

Michael Jones: a normal English reading of the spec says you must not use key formats other than what is in the spec.

Brent Zundel: where was this issue originally raised?

Manu Sporny: in the did core document.

Brent Zundel: and our original intent for the controlled identifier specification was to take text from documents that were describing things but not otherwise... unclear why this is our issue to change, why we are making normative changes, it feels outside the scope of what we as a group were working towards.

Manu Sporny: the original PR was not making normative changes, I disagree with selfissued's understanding of what the spec says, the reason the issue moved was because the text it affected moved, this was always an editorial issue and the changes were editorial in nature. selfissued's change makes it a normative change, I agree we should make it an editorial change.

Will Abramson: to clarify, when we are talking about key formats we are talking multikey and jwk, right? Wouldn't want to limit within multikey.

Manu Sporny: correct, we are talking higher level key format not key type.

Brent Zundel: selfissued's change would make this PR into a normative change.

Michael Jones: no, it would confirm what is already true, not a normative change.

Ted Thibodeau Jr.: there is disagreement about that.

Michael Jones: I understand that.

Ted Thibodeau Jr.: I would appreciate adjusting your assertion.

Brent Zundel: People speak from opinions, I don't believe there is a lack of clarity there, selfissued is speaking to his opinions.

Benjamin Young: it would be helpful to have some concrete language that clarifies what selfissued is saying.

Brent Zundel: the ask is for selfissued to show the language in the spec now that is leading to your understanding.

Michael Jones: I can add that to the issue after the call.

Ivan Herman: Quote from the text: "Verification material is any information that is used by a process that applies a verification method. The type of a verification method is expected to be used to determine its compatibility with such processes. Examples of verification methods include JsonWebKey and Multikey. ".

Ivan Herman: link to the quote: https://www.w3.org/TR/controller-document/#verification-material.

Manu Sporny: Ivan's reading is the same as mine and I agree with it.

Ivan Herman: from the example in the spec, it does not read as a "must" to choose between the two key formats.
… one more thing, if selfissued finds text in the spec per his understanding the spec is inconsistent as the example I looked at does not indicate a must and we have an editorial problem in the spec.

Benjamin Young: https://w3c.github.io/cid/#choosing-a-multiformat:~:text=Verification%20material%20is%20any%20information%20that%20is%20used%20by%20a%20process%20that%20applies%20a%20verification%20method.%20The%20type%20of%20a%20verification%20method%20is%20expected%20to%20be%20used%20to%20determine%20its%20compatibility%20with%20such%20processes.%20Examples%20of%20verification%20methods%20include%20JsonWebKey%20and%20Multikey..