Make config work with es8 by migrating to @elastic/elasticsearch #141
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Here's the reason for this change 🚀
My rationale is in pelias/pelias#953
As part of updating to elasticsearch 8, I'm updating pelias from the deprecated
elastic
package to it's successor@elastic/elasticsearch
which supports both es7 and 8 (and some older versions too, but we only care about 7+).Note that though this works with both the es7 and es8 server, this is a backwards incompatible client change - old pelias code using the
elasticsearch
package will not be able to use these new configs. It is a pre-requisite of adopting the new@elastic/elasticsearch
client.If any change required for the existing pacakges is required before the transition, I'd guess you'd want to make a minor/patch "hot fix" release, forked from a previous tag, rather than from master.
Here's what actually got changed 👏
updated config to be compatible with
@elastic/elasticsearch
Here's how others can test the changes 👀
Other than the tests passing, I'm not sure what else should be done. If you want to try running pelias/schema#488, you can vicariously test this code.
Two things I wanted to highlight.
env
:What is
env
for? As in:"env": "production"
. Though it lived in the esclient config object, it doesn't seem to have been used by the elasticsearch client. Is it some pelias specific thing? I deleted it.log
@elastic/elasticsearch
replaced their built in logging for a series of event emitters. I think the idea is you observe those events and logging is left as an excercise for the consumer:https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/observability.html
My best guess was to integrate these new
@elastic/elasticsearch
events with withpelias-logger
. There's some setup involved which I extracted into a new modulepelias-elasticsearch
, which I'm now using everywhere rather than the@elastic/elasticsearch
directly.I'm not familiar with wintson - I assumed there was a way to configure it at runtime with an environment variable, but I haven't yet found evidence of that. Should we expose log parameters in this config? I haven't seen evidence of this being done historically, but maybe its more significant now that we'd be using it for all the elasticsearch related logging.
alternatives considered
The logging integration I propose could instead live in the existing dbclient package. The only reason I didn't was, that module seemed concerned entirely with importing logic. In fact, pelias-api doesn't depend on that module at all. But we could change that.
We could keep the current config format exactly as it is and translate to the new format on the fly. This would have to be done everywhere we create a new client, but if we accept that we need some sort of client factory method regardless (to set up logging) maybe it's not so bad. The advantage would be to leave this module in a backwards compatible state for a while longer at least.
the new and old formats could coexist. It would be backwards compatible, but potentially confusing for the user.
I'm happy to do whatever.