Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Hotfix][Postgres-CDC] Fix read data missing when restore #6785

Merged
merged 1 commit into from May 11, 2024

Conversation

hailin0
Copy link
Contributor

@hailin0 hailin0 commented Apr 30, 2024

Purpose of this pull request

  1. The job was not added new tables into publication before start to snapshot read.
  2. Delete of database logs did not use checkpoint offset.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Check list

Comment on lines +194 to +195
// create the slot if it doesn't exist, otherwise update slot to add new
// table(job restore and add table)
Copy link
Contributor Author

@hailin0 hailin0 Apr 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hailin0 hailin0 marked this pull request as ready for review April 30, 2024 15:21
@Hisoka-X
Copy link
Member

Hisoka-X commented May 6, 2024

How about add a test case for this scene?

1. The job was not added new tables into `publication` before start to snapshot read.
2. Delete of database logs did not use checkpoint offset.
@hailin0 hailin0 force-pushed the dev-fix-pg-cdc-missing-data branch from fca70f3 to cd2acad Compare May 6, 2024 14:49
@@ -113,7 +116,7 @@ public void pollNext(Collector<T> output) throws Exception {

@Override
public void notifyCheckpointComplete(long checkpointId) throws Exception {
dataSourceDialect.notifyCheckpointComplete(checkpointId);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hailin0 hailin0 changed the title [Hotfix][Postgres-CDC] Fix read(new table) data missing when restore & add new tables [Hotfix][Postgres-CDC] Fix read data missing when restore May 7, 2024
Copy link
Member

@Carl-Zhou-CN Carl-Zhou-CN left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hailin0 hailin0 added this to the 2.3.6 milestone May 9, 2024
@EricJoy2048 EricJoy2048 merged commit 67c3260 into apache:dev May 11, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants