-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(conf): add concurrency_timeout option #13027
Conversation
1c20977
to
d517383
Compare
d517383
to
462f6e3
Compare
952559a
to
f7f4afe
Compare
c339c80
to
e6347e6
Compare
Is this PR ready for review? |
@chronolaw ready! |
Sets the timeout (in ms) for which a concurrency job (e.g. declaractive reconfigure) should wait for mutex before giving up. After the timeout, Kong gateway would skip the job, and log an error message. If you happen to see "timeout acquiring ... lock" log entries, try to adjust this value accordingly. Please be careful that setting a large value might result in a long list of concurreny jobs in queue, which could overflow the queue. It is fine to timeout if a job is safe to be overriden by subsequent jobs, like declaractive reconfigure. Do not touch this option unless you clear about it.
b73037f
to
1cf75d3
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don’t want this. Another setting that nobody knows how to use. And we have a better fix for the issue at hand already.
Better to fix root causes than the symptoms.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let’s not continue this path.
-- Is it worth to have node level mutex instead? | ||
-- If so, the RETRY_LRU also needs to be node level. | ||
concurrency.with_coroutine_mutex({ | ||
name = name, | ||
timeout = ROTATION_INTERVAL, | ||
timeout = concurrency_timeout, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Changes timeout without changing rotation interval used elsewhere. Removes the link between then.
Let's leave this to customers as usually a job takes shorter than 60s. |
Summary
feat(conf): add concurrency_timeout option
Sets the timeout (in ms) for which a concurrency
job (e.g. declaractive reconfigure) should wait
for mutex before giving up. After the timeout,
Kong gateway would skip the job, and log an error
message.
If you happen to see "timeout acquiring ... lock"
log entries, try to adjust this value accordingly.
Please be careful that setting a large value might
result in a long list of concurreny jobs in queue,
which could overflow the queue.
It is fine to timeout if a job is safe to be
overriden by subsequent jobs, like declaractive
reconfigure. Do not touch this option unless you
clear about it.
Checklist
changelog/unreleased/kong
orskip-changelog
label added on PR if changelog is unnecessary. README.mdIssue reference
Fix #FTI-5930 #[KAG-4480]