Test pipeline run after reroute #108693
Test pipeline run after reroute #108693
Conversation
Add a yaml rest test to verify that after a reroute the destination index's default pipeline is run
Based on test description and an unused pipeline it appears that the index document should be initial sent to a different index.
parkertimmins
added
>enhancement
:Data Management/Ingest Node
|
Hi @parkertimmins, I've created a changelog YAML for you. |
| @@ -121,7 +121,7 @@ teardown: | |||
| - do: | |||
| index: | |||
| refresh: true | |||
| index: logs-nginx-default | |||
| index: logs-router-default | |||
There was a problem hiding this comment.
Based on the test name and the fact that the logs-router pipeline is unused, I suspect this is the intended index name.
|
Pinging @elastic/es-data-management (Team:Data Management) |
elasticsearchmachine
added
the
Team:Data Management
| @@ -139,3 +139,66 @@ teardown: | |||
| query: | |||
| match: {"_id": "example-log"} | |||
| - match: { hits.hits.0._source.message: "this is an error log" } | |||
|
|
|||
| --- | |||
| "Test pipeline run after reroute": | |||
There was a problem hiding this comment.
With the fix to the previous test, that test does cover what this is testing. But I still think it's worth having this simpler test to confirm the behavior.
parkertimmins
force-pushed
the
test-pipeline-run-after-reroute
branch
from
d7443de
to
96effcd
Compare
Cleanup yaml test by deleting pipelines in teardown. Keeping pipeline around was breaking test in 15_ingest_info due to using the same pipeline. Also delete indices which have pipelines set as default
The pipelines don't end in '-default', but some of the indices do.
|
I did a little tidying as a review, but in the process of that I ended up realizing the existing test was pretty far off from actually doing what it was intended to do (beyond the issue you'd already found). I rewrote it so that it does what it was intended to do (at least IMHO). |
| @@ -139,3 +168,73 @@ teardown: | |||
| query: | |||
| match: {"_id": "example-log"} | |||
| - match: { hits.hits.0._source.message: "this is an error log" } | |||
There was a problem hiding this comment.
Can you add to the pipelines and to this test clause so that there's the equivalent of these lines associated with this test, too?
- match: { _source.added-in-pipeline-before-reroute : true }
- match: { _source.added-in-pipeline-after-reroute : true }
That is, I want a set processor in each of the logs-router and logs-nginx pipelines, and a match against hits.hits.0 for each of the values that those set processors set. There's no need for the existing-field, that's already covered here by "this is an error log".
Add test confirming that pipelines are run after a reroute. Fix test of two stage reroute. Delete pipelines during teardown so as to not break other tests using name pipeline name. Co-authored-by: Joe Gallo <joegallo@gmail.com>
Add a yaml rest test to verify that after a reroute
the destination index's default pipeline is run.
Also, fix the preceding existing test to match its description.