Modify perf CDT tests to use the im4gn.xlarge instance type
#18556
Conversation
| "acks": "all", | ||
| "linger.ms": 1, | ||
| "max.in.flight.requests.per.connection": 5, | ||
| "batch.size": 131072, |
There was a problem hiding this comment.
Lets set this to 16KB? It's a lot more realisitic and avoids us running in larger forced batch sizes.
There was a problem hiding this comment.
Nice catch, switching it to the default of 16384.
| "consumer_per_subscription": 200, | ||
| "producers_per_topic": 200, | ||
| "producer_rate": 150_000, | ||
| "consumer_per_subscription": 1, |
There was a problem hiding this comment.
I guess a lot of consumers aren't really relevant for this test?
There was a problem hiding this comment.
Yeah, the goal here is to stress the produce path as much as possible. So I set this to reduce to load from the consumers to the lowest possible.
| "acks": "all", | ||
| "linger.ms": 1, | ||
| "max.in.flight.requests.per.connection": 10, | ||
| "batch.size": 1024, |
There was a problem hiding this comment.
I keep forgetting we can do that. Switching it now.
| "enable.idempotence": "false", | ||
| "acks": "all", | ||
| "linger.ms": 1, | ||
| "max.in.flight.requests.per.connection": 10, |
There was a problem hiding this comment.
I think we should really test we idempotence on as it's the more common codepath.
There was a problem hiding this comment.
Fair enough, switching it over.
| "producer_rate": 150_000, | ||
| "consumer_per_subscription": 1, | ||
| "producers_per_topic": 10, | ||
| "producer_rate": 30_000, | ||
| "message_size": 1024, | ||
| "payload_file": "payload/payload-1Kb.data", |
There was a problem hiding this comment.
We could even use the 200 byte message size here or something
There was a problem hiding this comment.
Let me see if I can push the producer rate a bit higher with smaller messages.
There was a problem hiding this comment.
Unsurprisingly using smaller messages doesn't allow for a higher producer rate.
This PR modifies the existing perf CDT tests to use a much small 3x
im4gn.xlarge. The hope of doing this is to allow us to run more regression tests at the same cost as the existing tests.Backports Required
Release Notes