Prohibit creation of rollup jobs in clusters that have never used rollup. #108381
Labels
:StorageEngine/Rollup
Turn fine-grained time-based data into coarser-grained data
Team:StorageEngine
>tech debt
Comments
martijnvg
added
:StorageEngine/Rollup
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
May 14, 2024
This change will add logic to the put rollup api that fails if no rollup job is active and no rollup index exists in the cluster. The logic first check whether there is an active rollup persistent task if there are no active rollup persistent tasks, then it checks whether any rollup index exists. The latter check is an expensive check, but assuming that it only runs as part of the pu rollup job api and only when there are no rollup jobs, this should be ok. All tests that invoke the pu rollup job api will need to be adjusted to create a dummy index that has rollup mapping metadata. Otherwise, tests can't create a rollup job. Closes elastic#108381
|
Do we want to do something in the Kibana management UI to prevent users from creating new rollup jobs? |
martijnvg
added a commit
that referenced
this issue
May 21, 2024
This change will add logic to the put rollup api that fails if no rollup job is active and no rollup index exists in the cluster. The logic first check whether there is an active rollup persistent task if there are no active rollup persistent tasks, then it checks whether any rollup index exists. The latter check is an expensive check, but assuming that it only runs as part of the put rollup job api and only when there are no rollup jobs, this should be ok. All tests that invoke the put rollup job api will need to be adjusted to create a dummy index that has rollup mapping metadata. Otherwise, tests can't create a rollup job. Closes #108381
jedrazb
pushed a commit
to jedrazb/elasticsearch
that referenced
this issue
May 21, 2024
…08624) This change will add logic to the put rollup api that fails if no rollup job is active and no rollup index exists in the cluster. The logic first check whether there is an active rollup persistent task if there are no active rollup persistent tasks, then it checks whether any rollup index exists. The latter check is an expensive check, but assuming that it only runs as part of the put rollup job api and only when there are no rollup jobs, this should be ok. All tests that invoke the put rollup job api will need to be adjusted to create a dummy index that has rollup mapping metadata. Otherwise, tests can't create a rollup job. Closes elastic#108381
|
@rayafratkina Apologies for the late reply. Maybe my memory isn't correct, but I thought we already highlight that rollup is deprecated? I think the UI should highlight that rollup is deprecated (if we haven't done already) and will be removed. Additionally I think it makes sense to warn users that creating a rollup job in an cluster with no rollup usage will fail. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The rollup feature was never GA-ed and is still tech preview. It has been deprecated since 8.11.0 in favor of downsampling. At some point the rollup functionality will be removed. In order to prepare for the removal of rollup, the creation of rollup in clusters with no rollup usage will be prohibited. To avoid any new usages of rollup while it is scheduled to be removed.
The text was updated successfully, but these errors were encountered: