Scale SQS based on age of oldest message

[furstenheim-goodnotes]

Proposal

I would like to be able to scale sqs consumers based on age of oldest message, instead of length of the queue.

Use-Case

In my use case I enqueue tasks in batches as sqs messages. I don't care much about the processing time, meaning not immediate, as long as they are processed more or less on time. I definitely don't care about the length of the queue, and I would like not to have to estimate the necessary length of the queue.

If set my batches daily, I could trigger an upscale if oldest message hasn't been processed for 2 days.

Is this a feature you are interested in implementing yourself?

No

Anything else?

No response

[JorTurFer]

Hello,
It sounds legit, does the SQS API support to query this information?

[furstenheim-goodnotes]

According to this issue: aws/aws-sdk-go#2526 it seems to be available via cloudwatch api, not via sqs api. Would that be a blocker?

[JorTurFer]

The problem that I see here, is that the update frequency of a monitoring API isn't the same as the SQS API and it can produce misunderstandings. The required permissions are different too, as the roles won't be SQS related but CloudWatch related and it can be confusing...

KEDA already supports a CloudWatch scaler that can be used for calling directly to CloudWatch, isn't it enough for the use case? I'm not saying a strong NO to this feature request, I'm trying to understand the use case because it can make sense