[META] Breaking changes for 2.0

[VachaShah]

As part of OpenSearch 2.0, there are breaking changes which need to be incorporated in the clients as well. Below is a list of breaking changes with the affected API endpoints.

{type} related paths removed from

• bulk
• count
• create
• delete_by_query
• delete
• exists
• exists_source
• explain
• get
• get_source
• index
• mget
• msearch
• msearch_template
• mtermvectors
• rank_eval
• search
• search_template
• termvectors
• update_by_query
• update
• indices.get_field_mapping
• indices.get_mapping
• indices.put_mapping
• indices.validate_query

Completely removed:

• indices.flush_synced
• indices.exists_type

include_type_name param removed from

• indices.create
• indices.get
• indices.get_mapping
• indices.get_field_mapping
• indices.get_template
• indices.put_mapping
• indices.put_template
• indices.rollover

Related OpenSearch issue: opensearch-project/OpenSearch#2480

Status

• opensearch-go Breaking changes for 2.0 opensearch-go#99
• opensearch-dsl-py Breaking changes for 2.0 opensearch-dsl-py#43
• opensearch-java Breaking changes for 2.0 opensearch-java#139
• opensearch-js Breaking changes for 2.0 opensearch-js#217
• opensearch-php Breaking changes for 2.0 opensearch-php#48
• opensearch-py Breaking changes for 2.0 opensearch-py#145
• opensearch-ruby Breaking changes for 2.0 opensearch-ruby#57
• opensearch-rs Breaking changes for 2.0 opensearch-rs#39

Other Deprecated changes

Inclusive naming #16

[dblock]

I think we're missing an upgrade path around in these breaking changes since it seems like we would need to upgrade both client and server at the same time. So a 2.0 client will be required to work with OpenSearch 2.0.

As an example, let's say I have a OpenSearch 1.3.2 receiving data from Java client 1.3.2, how do I upgrade to OpenSearch 2.0 without downtime?

[VachaShah]

From my understanding, for a cluster for 1.3.2 to move to 2.0, when the nodes are upgraded one by one, we can still use the older client version 1.3.2 since the cluster manager is still on an older version 1.3.2 and when the last node (cluster manager) is upgraded to 2.0, thats when we would need to upgrade the client to 2.0.

@saratvemulapalli LMK if I am missing something here.

[dblock]

Would that work without downtime for those that build out separate clusters and roll over traffic, such as blue/green type deployments in Amazon Managed OpenSearch?

[saratvemulapalli]

@dblock trying to understand your question.
Are you asking if there is going to be downtime during upgrades for customers using clients?

[elfisher]

I am trying to understand the nature of the breaking changes. As it stands today, if I use a 1.x client to communicate with OpenSearch 2.0, what happens? What about the reverse where I use a 2.x client to communicate with OpenSearch 1.x?

[dblock]

@dblock trying to understand your question. Are you asking if there is going to be downtime during upgrades for customers using clients?

I am saying that there will be downtime because both clients and server will beed to be upgraded, and asking whether that's actually expected?

[dblock]

I am trying to understand the nature of the breaking changes. As it stands today, if I use a 1.x client to communicate with OpenSearch 2.0, what happens? What about the reverse where I use a 2.x client to communicate with OpenSearch 1.x?

Both of these just won't work.

[dlvenable]

@dblock ,

Clients and servers is that users should not have to change both the server and client in unison. This can be very difficult to manage for development teams.

I believe OpenSearch's goal should be: Users update to the latest client version in a major version (e.g. 1.3). They should see deprecations clearly (e.g. Java's @Deprecated, READMEs). And if they are diligent to remove these deprecated APIs, then this client should be compatible with the the next major version (e.g. 2.0). They won't be able to use new features, but they should not be broken. Then after updating their cluster to OpenSearch 2.0, they can update their client libraries.

[dblock]

@dlvenable I opened #17, does it capture what you're saying?

[saratvemulapalli]

I am saying that there will be downtime because both clients and server will beed to be upgraded, and asking whether that's actually expected?

Got it. As far as I understand, yes that is expected. Customers should always upgrade server first and then upgrade their clients.
I agree we should (and could) make this a better experience. But also we should keep in mind some APIs are expected to be broken in new major versions.

I am trying to understand the nature of the breaking changes. As it stands today, if I use a 1.x client to communicate with OpenSearch 2.0, what happens? What about the reverse where I use a 2.x client to communicate with OpenSearch 1.x?

A client with 2.0 can only talk to OpenSearch 2.0. Clients historically did not support new major versions.
We can definitely give it a shot, I'm sure few APIs will be broken, for the ones which signature hasn't changed technically it should work (unless we are doing version checks @VachaShah )

[dblock]

@saratvemulapalli did I capture this correctly as a proposal/feature request in #17?

[VachaShah]

I am saying that there will be downtime because both clients and server will beed to be upgraded, and asking whether that's actually expected?

Got it. As far as I understand, yes that is expected. Customers should always upgrade server first and then upgrade their clients. I agree we should (and could) make this a better experience. But also we should keep in mind some APIs are expected to be broken in new major versions.

I am trying to understand the nature of the breaking changes. As it stands today, if I use a 1.x client to communicate with OpenSearch 2.0, what happens? What about the reverse where I use a 2.x client to communicate with OpenSearch 1.x?

A client with 2.0 can only talk to OpenSearch 2.0. Clients historically did not support new major versions. We can definitely give it a shot, I'm sure few APIs will be broken, for the ones which signature hasn't changed technically it should work (unless we are doing version checks @VachaShah )

We are not doing version checks currently.

[stockholmux]

Historically in ES, you were expected to upgrade your clients/tools in lockstep with even minor versions (ES doesn't use semver, so breaking changes could occur during even an especially violent sneeze). I'm not saying that's a good thing but the expectation 2.0.0 needing a new client or tool version is not out of the question.

My pie-in-the-sky desire would be to have clients that could be put in 'modes' for many different versions. When I worked with Redis, some clients could work with different versions and even protocols. You could always suggest that a program depending on the client library use the latest version - so they would get all the patches and fixes along the way.

[VachaShah]

Closing this issue as 2.0 clients are released.