Better isolate failure cause in DeprecationHttpIT.testDeprecatedSettingsReturnWarnings #108660
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…stic#108628 In rare cases we receive deprecation headers in DeprecationHttpIT.testDeprecatedSettingsReturnWarnings, but the warning seems to be missing in the deprecation index and respective logs. This uses a separate xOpaqueId to cleanup the deprecated settings in order to better understand if logs / indexing is simply delayed a lot or if the logs eventually originated from the cleanup call.
mosche
added
>test
mosche
commented
May 15, 2024
| .endObject() | ||
| .endObject(); | ||
|
|
||
| final Request request = new Request("PUT", "_cluster/settings"); | ||
| request.setJsonEntity(Strings.toString(builder)); | ||
| performScopedRequest(request); | ||
| return performScopedRequest(request, xOpaqueId() + "-cleanup"); |
There was a problem hiding this comment.
This is the key change that will allow to understand if the deprecation warning in the logs was triggered by the initial settings request or this cleanup request. Based on the timestamps it looks like the latter is the case, but this will give certainty.
test logs
[2024-05-13T16:43:56,601][INFO ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] before test
Beal 13, 2024 4:43:56 P.M. org.elasticsearch.client.RestClient logResponse
WARNING: request [PUT http://[::1]:44625/_cluster/settings] returned 2 warnings: [299 Elasticsearch-8.15.0-0b2e5584f70ece2e157d7be47529637c505a7d4c "[test.setting.deprecated.true2] setting was deprecated in Elasticsearch and will be removed in a future release."],[299 Elasticsearch-8.15.0-0b2e5584f70ece2e157d7be47529637c505a7d4c "[test.setting.deprecated.true1] setting was deprecated in Elasticsearch and will be removed in a future release."]
[2024-05-13T16:43:56,903][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:56,921][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:56,944][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:56,965][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:56,994][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:57,033][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:57,086][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:57,173][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:57,329][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:57,610][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:58,143][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:43:59,190][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:44:01,259][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:44:05,375][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:44:13,588][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
[2024-05-13T16:44:30,000][WARN ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] []
Beal 13, 2024 4:44:30 P.M. org.elasticsearch.client.RestClient logResponse
WARNING: request [PUT http://127.0.0.1:43361/_cluster/settings] returned 2 warnings: [299 Elasticsearch-8.15.0-0b2e5584f70ece2e157d7be47529637c505a7d4c "[test.setting.deprecated.true2] setting was deprecated in Elasticsearch and will be removed in a future release."],[299 Elasticsearch-8.15.0-0b2e5584f70ece2e157d7be47529637c505a7d4c "[test.setting.deprecated.true1] setting was deprecated in Elasticsearch and will be removed in a future release."]
[2024-05-13T16:44:30,105][INFO ][o.e.x.d.DeprecationHttpIT] [testDeprecatedSettingsReturnWarnings] after test
REPRODUCE WITH: ./gradlew ':x-pack:plugin:deprecation:qa:rest:javaRestTest' --tests "org.elasticsearch.xpack.deprecation.DeprecationHttpIT.testDeprecatedSettingsReturnWarnings" -Dtests.seed=94B983B3041BD3C8 -Dtests.locale=ga -Dtests.timezone=Africa/Algiers -Druntime.java=21
deprecation logs
{"@timestamp":"2024-05-13T15:43:56.712Z", "log.level": "WARN", "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.elastic_product_origin":"","elasticsearch.event.category":"settings","elasticsearch.http.request.x_opaque_id":"testDeprecationIndexingCacheReset","event.code":"test.setting.deprecated.true2","message":"[test.setting.deprecated.true2] setting was deprecated in Elasticsearch and will be removed in a future release." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[javaRestTest-0][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.deprecation.common.settings.Settings","elasticsearch.cluster.uuid":"I7qWkaMEQfmweHuYh9_Dqw","elasticsearch.node.id":"JJANCj_nTX-XcfDM9Kk5mw","elasticsearch.node.name":"javaRestTest-0","elasticsearch.cluster.name":"javaRestTest"}
{"@timestamp":"2024-05-13T15:43:56.718Z", "log.level": "WARN", "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.elastic_product_origin":"","elasticsearch.event.category":"settings","elasticsearch.http.request.x_opaque_id":"testDeprecationIndexingCacheReset","event.code":"test.setting.deprecated.true1","message":"[test.setting.deprecated.true1] setting was deprecated in Elasticsearch and will be removed in a future release." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[javaRestTest-0][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.deprecation.common.settings.Settings","elasticsearch.cluster.uuid":"I7qWkaMEQfmweHuYh9_Dqw","elasticsearch.node.id":"JJANCj_nTX-XcfDM9Kk5mw","elasticsearch.node.name":"javaRestTest-0","elasticsearch.cluster.name":"javaRestTest"}
{"@timestamp":"2024-05-13T15:44:30.010Z", "log.level": "WARN", "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.elastic_product_origin":"","elasticsearch.event.category":"settings","elasticsearch.http.request.x_opaque_id":"testDeprecatedSettingsReturnWarnings","event.code":"test.setting.deprecated.true2","message":"[test.setting.deprecated.true2] setting was deprecated in Elasticsearch and will be removed in a future release." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[javaRestTest-0][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.deprecation.common.settings.Settings","elasticsearch.cluster.uuid":"I7qWkaMEQfmweHuYh9_Dqw","elasticsearch.node.id":"JJANCj_nTX-XcfDM9Kk5mw","elasticsearch.node.name":"javaRestTest-0","elasticsearch.cluster.name":"javaRestTest"}
{"@timestamp":"2024-05-13T15:44:30.011Z", "log.level": "WARN", "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.elastic_product_origin":"","elasticsearch.event.category":"settings","elasticsearch.http.request.x_opaque_id":"testDeprecatedSettingsReturnWarnings","event.code":"test.setting.deprecated.true1","message":"[test.setting.deprecated.true1] setting was deprecated in Elasticsearch and will be removed in a future release." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[javaRestTest-0][masterService#updateTask][T#1]","log.logger":"org.elasticsearch.deprecation.common.settings.Settings","elasticsearch.cluster.uuid":"I7qWkaMEQfmweHuYh9_Dqw","elasticsearch.node.id":"JJANCj_nTX-XcfDM9Kk5mw","elasticsearch.node.name":"javaRestTest-0","elasticsearch.cluster.name":"javaRestTest"}
|
Pinging @elastic/es-core-infra (Team:Core/Infra) |
parkertimmins
pushed a commit
to parkertimmins/elasticsearch
that referenced
this pull request
May 17, 2024
…stic#108628 (elastic#108660) In rare cases we receive deprecation headers in DeprecationHttpIT.testDeprecatedSettingsReturnWarnings, but the warning seems to be missing in the deprecation index and respective logs. This uses a separate xOpaqueId to cleanup the deprecated settings in order to better understand if logs / indexing is simply delayed a lot or if the logs eventually originated from the cleanup call.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Use separate xOpaqueId for cleanup in
DeprecationHttpIT.testDeprecatedSettingsReturnWarningsto better isolate the cause of #108628In rare cases we receive deprecation headers in
DeprecationHttpIT.testDeprecatedSettingsReturnWarnings, but the warning seems to be missing in the deprecation index and respective logs.This uses a separate xOpaqueId to cleanup the deprecated settings in order to better understand if logs / indexing is simply delayed a lot or if the logs eventually originated from the cleanup call.
Apart from the highlighted change, this is mostly just increasing readability of the code.