Hi @robpickerill could you please draft here the empty response? It is a null or empty array and it is something that is expected from CloudWatch? Also, what is the value that should be reported in this case?

Hi, thanks for the follow up @zroubalik.

It is an empty response, so in this case, there are no metrics being returned from the GetMetricData call. We fall into this branch: https://github.com/kedacore/keda/blob/main/pkg/scalers/aws_cloudwatch_scaler.go#L386, as the MetricDataResults has zero length.

The wider scenario here is that another system is producing custom metrics, by which a ScaledObject is being used to query that metric and scale a separate system, which is a deployment, to the relevant capacity. In the case whereby the system producing the custom metrics is offline, no metrics are being returned from the GetMetricData call. In this scenario, no metrics in the GetMetricData call is normal behaviour from CloudWatch.

What I wanted to guard against in this scenario is further scaling activity. I'd like the deployment to remain at the current scale. From what I've observed, in the case whereby a single AWS CloudWatch scaler exists, the deployment will scale down to the largest value of the minReplicaCount, or the evaluation of the minMetricValue in the scaling policy.

I don't know if this is a viable approach, but one way I could see working from what I understand so far is to have an option on the AWS CloudWatch scaler to handle an empty result as an error. Thus marking that scaler/trigger in an error state. This option would therefore be mutually exclusive to the minMetricValue option.

Edit: if you'd want to inspect the GetMetricData json payload here to understand further, I can share some example responses with you tomorrow.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

This issue has been automatically closed due to inactivity.

Sorry for the delay, I missed the reply somehow.

Having an option setting that would allow AWS CloudWatch scaler to handle an empty result as an error is okay with me. We just need to be backwards compatible.

Thanks @zroubalik -- just to check, would the backwards compatibility be enough to add a new metadata option for the cloudwatch scaler: errorWhenMetricValuesEmpty, that's optional, and defaults to false to maintain previous behaviour if the user supplies no options when upgrading?

Thanks @zroubalik -- just to check, would the backwards compatibility be enough to add a new metadata option for the cloudwatch scaler: errorWhenMetricValuesEmpty, that's optional, and defaults to false to maintain previous behaviour if the user supplies no options when upgrading?

Yes, that's enough :)

Thanks, you can assign this to me, I'll send over a PR for review soon

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.