rgw/notification: Store the value of persistent_queue for existing topics and continue commiting events for all topics subscribed to given bucket

[kchheda3]

Fixes https://tracker.ceph.com/issues/66097

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox
• jenkins test windows
• jenkins test rook e2e

[cbodley]

i think this should just copy dest.persistent_queue = topic->dest.persistent_queue. the topic we're overwriting may have been a legacy one whose queue doesn't include the tenant namespace

[kchheda3]

would it not be better in way, the user now gets migrated to correct queue name ?
just that old queue will remain there in the system

[cbodley]

not unless we have a way to clean up the old queue. the notification thread would keep polling on it forever

[kchheda3]

may be this code will still not make use of the new queue name even if it exist

if (struct_v >= 7) {
      decode(persistent_queue, bl);
    } else if (persistent) {
      // persistent topics created before v7 did not support tenant namespacing.
      // continue to use 'arn_topic' alone as the queue's rados object name
      persistent_queue = arn_topic;
    }
    

[kchheda3]

i think this should just copy dest.persistent_queue = topic->dest.persistent_queue. the topic we're overwriting may have been a legacy one whose queue doesn't include the tenant namespace

done

[cbodley]

looks correct to me. is there a test case we can extend with a re-create to verify that persistent topics still work?

[kchheda3]

looks correct to me. is there a test case we can extend with a re-create to verify that persistent topics still work?

looked at test_bn.py, do not see any active test case that re-creates the topic, there is test case test_ps_s3_topic_update that updates the endpoint of the topic and does all re-verification but its disabled and also its purpose is more of testing update on endpoints and verifying the events are still received and not about verifying the attributes.
also currently get-topic does not return persistent_queue so not sure if adding new test case would verify this issue, unless we verify it via radosgw-admin get topic

[yuvalif]

looks correct to me. is there a test case we can extend with a re-create to verify that persistent topics still work?

looked at test_bn.py, do not see any active test case that re-creates the topic, there is test case test_ps_s3_topic_update that updates the endpoint of the topic and does all re-verification but its disabled and also its purpose is more of testing update on endpoints and verifying the events are still received and not about verifying the attributes. also currently get-topic does not return persistent_queue so not sure if adding new test case would verify this issue, unless we verify it via radosgw-admin get topic

can we add that to one of the existing persistent notification tests?
just create the topic twice. and verify that we still see the notifications.

[kchheda3]

looks correct to me. is there a test case we can extend with a re-create to verify that persistent topics still work?

looked at test_bn.py, do not see any active test case that re-creates the topic, there is test case test_ps_s3_topic_update that updates the endpoint of the topic and does all re-verification but its disabled and also its purpose is more of testing update on endpoints and verifying the events are still received and not about verifying the attributes. also currently get-topic does not return persistent_queue so not sure if adding new test case would verify this issue, unless we verify it via radosgw-admin get topic

can we add that to one of the existing persistent notification tests? just create the topic twice. and verify that we still see the notifications.

updated the existing test case to re-create the topic and verify the persistent_queue is correctly populated.