-
Notifications
You must be signed in to change notification settings - Fork 11.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug] RocketMQ FIFO Topic still cannot guarantee message order for simple consumers even in sequential production scenarios. #8154
Comments
When creating the consumer group, did you set it as a FIFO-ConsumerGroup? For reference, use the command |
Yes, this is inevitable. I have explained above that this is a FIFO-type topic. |
So the consumer group is also FIFO? |
yes,I used a simple consumer.And without multithreading concurrency, I only get one message at a time, and when I finish one message, I get the next one. Consumers and producers are single threads, no asynchrony, no concurrency. Get only one message at a time. All messages are in one group and there is only one queue. |
Let's also say that my conditions are as follows:
Also this mostly only happens with the 1st batch of messages received after startup. I'm using the official github example. I can reproduce and record a video on my side if needed. The following is in Chinese, I think you'll understand it better that way: 也就说说,我的条件如下: |
我们复现下 |
这种现象不是每次都能复现,你可以先启动单个生产者,发送1-10个数字到topic同一个group,然后紧接着启动Simple consumer并串行切一次获取一个message,有时候会丢失掉数字1。 在Java中复现的概率更低,或许和启动时间有些关系,在nodejs复现概率极高(单个node脚本启动迅速)。 |
可以参考我上方的复现过程以及代码。 |
@RongtongJin Has it been successfully repeated? I need results as soon as possible because it may have an impact on my business. |
I could not reproduce your issue. The SimpleConsumer with fifo is basically using the pop orderly interface which is mostly server-side behavior. It seems that it is possible that the group or topic is not been set properly. Could you please provide your group in subscriptionGroup.json and topic in topics.json which in ~/store/config to ensure that the related configuration is correct? |
Of course, I can do that. I'll show you below, or do you have a more immediate communication tool? Like QQ or WeChat? {
"dataVersion":{
"counter":0,
"stateVersion":0,
"timestamp":1715929343644
},
"forbiddenTable":{},
"subscriptionGroupTable":{
"SELF_TEST_C_GROUP":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"SELF_TEST_C_GROUP",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_ONSAPI_OWNER":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"CID_ONSAPI_OWNER",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_ONSAPI_PERMISSION":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"CID_ONSAPI_PERMISSION",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_RMQ_SYS_TRANS":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"CID_RMQ_SYS_TRANS",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"TOOLS_CONSUMER":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"TOOLS_CONSUMER",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_ONS-HTTP-PROXY":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"CID_ONS-HTTP-PROXY",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"FILTERSRV_CONSUMER":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"FILTERSRV_CONSUMER",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"CID_ONSAPI_PULL":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"CID_ONSAPI_PULL",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
},
"checkout-group":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"checkout-group",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
}
}
} {
"dataVersion":{
"counter":28,
"stateVersion":0,
"timestamp":1715934484981
},
"topicConfigTable":{
"SCHEDULE_TOPIC_XXXX":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":18,
"topicFilterType":"SINGLE_TAG",
"topicName":"SCHEDULE_TOPIC_XXXX",
"topicSysFlag":0,
"writeQueueNums":18
},
"JanYorkMacBook-Pro.local":{
"attributes":{},
"order":false,
"perm":7,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"JanYorkMacBook-Pro.local",
"topicSysFlag":0,
"writeQueueNums":1
},
"SELF_TEST_TOPIC":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"SELF_TEST_TOPIC",
"topicSysFlag":0,
"writeQueueNums":1
},
"checkout-topic-trans":{
"attributes":{
"message.type":"TRANSACTION"
},
"order":false,
"perm":6,
"readQueueNums":8,
"topicFilterType":"SINGLE_TAG",
"topicName":"checkout-topic-trans",
"topicSysFlag":0,
"writeQueueNums":8
},
"rmq_sys_SYNC_BROKER_MEMBER_JanYorkMacBook-Pro.local":{
"attributes":{},
"order":false,
"perm":1,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"rmq_sys_SYNC_BROKER_MEMBER_JanYorkMacBook-Pro.local",
"topicSysFlag":0,
"writeQueueNums":1
},
"checkout-topic-fifo":{
"attributes":{
"message.type":"FIFO"
},
"order":true,
"perm":6,
"readQueueNums":8,
"topicFilterType":"SINGLE_TAG",
"topicName":"checkout-topic-fifo",
"topicSysFlag":0,
"writeQueueNums":8
},
"DefaultCluster":{
"attributes":{},
"order":false,
"perm":7,
"readQueueNums":16,
"topicFilterType":"SINGLE_TAG",
"topicName":"DefaultCluster",
"topicSysFlag":0,
"writeQueueNums":16
},
"DefaultCluster_REPLY_TOPIC":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"DefaultCluster_REPLY_TOPIC",
"topicSysFlag":0,
"writeQueueNums":1
},
"rmq_sys_REVIVE_LOG_DefaultCluster":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":8,
"topicFilterType":"SINGLE_TAG",
"topicName":"rmq_sys_REVIVE_LOG_DefaultCluster",
"topicSysFlag":0,
"writeQueueNums":8
},
"checkout-topic":{
"attributes":{
"message.type":"NORMAL"
},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"checkout-topic",
"topicSysFlag":0,
"writeQueueNums":1
},
"RMQ_SYS_TRANS_HALF_TOPIC":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"RMQ_SYS_TRANS_HALF_TOPIC",
"topicSysFlag":0,
"writeQueueNums":1
},
"rmq_sys_SYNC_BROKER_MEMBER_broker-a":{
"attributes":{},
"order":false,
"perm":1,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"rmq_sys_SYNC_BROKER_MEMBER_broker-a",
"topicSysFlag":0,
"writeQueueNums":1
},
"TRANS_CHECK_MAX_TIME_TOPIC":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"TRANS_CHECK_MAX_TIME_TOPIC",
"topicSysFlag":0,
"writeQueueNums":1
},
"broker-a":{
"attributes":{},
"order":false,
"perm":7,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"broker-a",
"topicSysFlag":0,
"writeQueueNums":1
},
"RMQ_SYS_TRACE_TOPIC":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":4,
"topicFilterType":"SINGLE_TAG",
"topicName":"RMQ_SYS_TRACE_TOPIC",
"topicSysFlag":0,
"writeQueueNums":4
},
"RMQ_SYS_TRANS_OP_HALF_TOPIC":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"RMQ_SYS_TRANS_OP_HALF_TOPIC",
"topicSysFlag":0,
"writeQueueNums":1
},
"TBW102":{
"attributes":{},
"order":false,
"perm":7,
"readQueueNums":8,
"topicFilterType":"SINGLE_TAG",
"topicName":"TBW102",
"topicSysFlag":0,
"writeQueueNums":8
},
"BenchmarkTest":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1024,
"topicFilterType":"SINGLE_TAG",
"topicName":"BenchmarkTest",
"topicSysFlag":0,
"writeQueueNums":1024
},
"OFFSET_MOVED_EVENT":{
"attributes":{},
"order":false,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"OFFSET_MOVED_EVENT",
"topicSysFlag":0,
"writeQueueNums":1
}
}
} And I also attempted the following configuration later: "checkout-topic-fifo":{
"attributes":{
"message.type":"FIFO"
},
"order":true,
"perm":6,
"readQueueNums":1,
"topicFilterType":"SINGLE_TAG",
"topicName":"checkout-topic-fifo",
"topicSysFlag":0,
"writeQueueNums":1
}, "checkout-group":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"checkout-group",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
} |
"checkout-group":{
"attributes":{},
"brokerId":0,
"consumeBroadcastEnable":true,
"consumeEnable":true,
"consumeFromMinEnable":true,
"consumeMessageOrderly":false,
"consumeTimeoutMinute":15,
"groupName":"checkout-group",
"groupRetryPolicy":{
"type":"CUSTOMIZED"
},
"groupSysFlag":0,
"notifyConsumerIdsChangedEnable":true,
"retryMaxTimes":16,
"retryQueueNums":1,
"whichBrokerWhenConsumeSlowly":1
} The "consumeMessageOrderly" is false which is not been set properly. |
You should set the option -o true to set consumeMessageOrderly to true when updateSubGroup. If you had read this reply carefully, then you would find out that the correct answer is already provided. |
I copied this command exactly as it is, and it seems there are no errors.
Oh!!! It was my carelessness. I overlooked the name of this group! I feel so foolish. Thank you for pointing that out. |
Thank you again. I submitted a very foolish issue. |
You should use And the document should be updated to add the |
Before Creating the Bug Report
I found a bug, not just asking a question, which should be created in GitHub Discussions.
I have searched the GitHub Issues and GitHub Discussions of this repository and believe that this is not a duplicate.
I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ.
Runtime platform environment
The issue can be reproduced across multiple devices and various environments.
RocketMQ version
rocketmq-all-5.2.0-bin-release.
JDK Version
JDK17
Describe the Bug
In RocketMQ version 5.x, I created a FIFO type topic with both read and write queues set to one. Then, I sequentially produced messages using the client, with message content being numbers from 0 to 9. After that, I started a SimpleConsumer to consume messages. However, the order of numbers I received was: 1, 2, 3, 4, 5, 6, 7, 8, 9, 0. This is an issue because I didn't receive 0 first! This implies that the method of fetching messages by the SimpleConsumer is not atomic. Moreover, this phenomenon does not occur every time; it can only be occasionally reproduced.
I have tried on various platforms (MacOS and Windows) and experimented with different language clients (Java, Node.js, Go), all of which exhibit this issue. The fact that the atomic method for message retrieval fails to ensure message order consistently across these platforms and languages leads me to believe that this is a bug. Furthermore, I am using a single consumer, deploying RocketMQ on a single machine, and operating with a single producer. Additionally, I am not employing any asynchronous code; everything is executed synchronously.
Steps to Reproduce
You can also attempt to reproduce this issue in Java using the following code:
First, you can create a FIFO type topic and set both the read and write queues to one. Then, produce messages to the same group.
Then, run the MessageProducer class to sequentially send messages. After sending is complete, promptly run the consumer. If the issue does not occur, close the consumer and repeat the previous steps. This can be reproduced; I have asked some friends in technical exchange groups to do the same, and they have successfully reproduced it. They also believe it to be a bug.
For Node.js, you can refer to the issue I raised previously: #8133
What Did You Expect to See?
Messages in the message queue are consumed sequentially.
What Did You See Instead?
the log:
after waiting for some time:
Additional Context
The text was updated successfully, but these errors were encountered: