Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Refactor] Introduce ColumnId to support Column renaming (part3) #45757

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

gengjun-git
Copy link
Contributor

Why I'm doing:

We need a unique ID to identify the Column. This ID is used in all places that reference the Column. In this way, to change the attributes of the Column, such as name, we only need to change the attributes in the Column object. There is a uniqueId in the current Column, but it is only available in newly created tables. For compatibility reasons, we need to introduce another Id: ColumnID. The ColumnID of the historical table is the name of the column, because the name was previously immutable.

What I'm doing:

There are currently three ways to reference Column: 1: direct copy of Column object, 2: reference to Column name, 3: sql expression reference.
This PR changes the Column name to ColumnId reference.

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@gengjun-git gengjun-git requested review from a team as code owners May 16, 2024 11:52
@gengjun-git gengjun-git changed the title [Refactor] Introduce ColumnId to support Column renaming (part2) [Refactor] Introduce ColumnId to support Column renaming (part3) May 16, 2024
@nshangyiming nshangyiming self-assigned this May 16, 2024
@@ -267,7 +268,7 @@ protected void runPendingJob() throws AlterCancelException {
.setShortKeyColumnCount(rollupShortKeyColumnCount)
.setSchemaHash(rollupSchemaHash)
.setStorageType(TStorageType.COLUMN)
.setBloomFilterColumnNames(tbl.getCopiedBfColumns())
.setBloomFilterColumnNames(tbl.getBfPhysicalColumnNames())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does physical column name mean? should be columnID?

@@ -567,9 +568,13 @@ protected void runWaitingTxnJob() throws AlterCancelException {
"found in the base table.");
}
}

List<ColumnId> usedPhysicalNames = new ArrayList<>(usedBaseTableColNames.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
List<ColumnId> usedPhysicalNames = new ArrayList<>(usedBaseTableColNames.size());
List<ColumnId> usedColumnIds = new ArrayList<>(usedBaseTableColNames.size());

public static FieldId create(String raw) {
if (raw.contains("\\.")) {
String[] fields = raw.split("\\.", 2);
return new FieldId(ColumnId.create(fields[0]), fields[1]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does this for?

@HangyuanLiu HangyuanLiu self-assigned this May 21, 2024
Signed-off-by: gengjun-git <gengjun@starrocks.com>
Signed-off-by: gengjun-git <gengjun@starrocks.com>
Signed-off-by: gengjun-git <gengjun@starrocks.com>
Signed-off-by: gengjun-git <gengjun@starrocks.com>
Signed-off-by: gengjun-git <gengjun@starrocks.com>
Copy link

sonarcloud bot commented Jun 6, 2024

Copy link

github-actions bot commented Jun 6, 2024

[FE Incremental Coverage Report]

pass : 184 / 230 (80.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/sql/optimizer/dump/DesensitizedSQLBuilder.java 0 4 00.00% [641, 686, 690, 779]
🔵 com/starrocks/alter/LakeTableSchemaChangeJob.java 0 1 00.00% [803]
🔵 com/starrocks/catalog/ExternalOlapTable.java 0 2 00.00% [409, 410]
🔵 com/starrocks/transaction/LakeTableTxnLogApplier.java 1 3 33.33% [92, 107]
🔵 com/starrocks/leader/LeaderImpl.java 1 3 33.33% [928, 1021]
🔵 com/starrocks/persist/gson/GsonUtils.java 1 3 33.33% [680, 685]
🔵 com/starrocks/planner/StreamLoadPlanner.java 1 2 50.00% [178]
🔵 com/starrocks/transaction/LakeTableTxnStateListener.java 1 2 50.00% [122]
🔵 com/starrocks/sql/LoadPlanner.java 1 2 50.00% [381]
🔵 com/starrocks/sql/UpdatePlanner.java 1 2 50.00% [127]
🔵 com/starrocks/leader/ReportHandler.java 1 2 50.00% [920]
🔵 com/starrocks/sql/optimizer/base/ColumnIdentifier.java 1 2 50.00% [37]
🔵 com/starrocks/catalog/OlapTable.java 12 17 70.59% [1507, 1638, 1639, 1712, 1713]
🔵 com/starrocks/sql/common/MetaUtils.java 30 42 71.43% [294, 309, 320, 321, 322, 323, 324, 326, 327, 328, 336, 346]
🔵 com/starrocks/catalog/Index.java 6 8 75.00% [207, 228]
🔵 com/starrocks/catalog/FieldId.java 12 16 75.00% [42, 55, 56, 58]
🔵 com/starrocks/sql/optimizer/rule/tree/lowcardinality/DecodeCollector.java 4 5 80.00% [470]
🔵 com/starrocks/alter/SchemaChangeHandler.java 16 19 84.21% [352, 353, 1203]
🔵 com/starrocks/catalog/MaterializedView.java 3 3 100.00% []
🔵 com/starrocks/transaction/OlapTableTxnStateListener.java 2 2 100.00% []
🔵 com/starrocks/sql/analyzer/CreateTableAnalyzer.java 3 3 100.00% []
🔵 com/starrocks/server/OlapTableFactory.java 6 6 100.00% []
🔵 com/starrocks/common/proc/IndexInfoProcDir.java 1 1 100.00% []
🔵 com/starrocks/statistic/ExternalAnalyzeJob.java 1 1 100.00% []
🔵 com/starrocks/planner/OlapScanNode.java 1 1 100.00% []
🔵 com/starrocks/sql/optimizer/statistics/CacheDictManager.java 1 1 100.00% []
🔵 com/starrocks/sql/analyzer/AstToStringBuilder.java 1 1 100.00% []
🔵 com/starrocks/planner/OlapTableSink.java 1 1 100.00% []
🔵 com/starrocks/backup/RestoreJob.java 1 1 100.00% []
🔵 com/starrocks/transaction/OlapTableTxnLogApplier.java 3 3 100.00% []
🔵 com/starrocks/statistic/StatisticExecutor.java 2 2 100.00% []
🔵 com/starrocks/analysis/BloomFilterIndexUtil.java 4 4 100.00% []
🔵 com/starrocks/server/LocalMetastore.java 2 2 100.00% []
🔵 com/starrocks/sql/optimizer/rule/tree/AddDecodeNodeForDictStringRule.java 4 4 100.00% []
🔵 com/starrocks/sql/analyzer/ShowStmtAnalyzer.java 1 1 100.00% []
🔵 com/starrocks/transaction/TabletCommitInfo.java 11 11 100.00% []
🔵 com/starrocks/task/AlterReplicaTask.java 5 5 100.00% []
🔵 com/starrocks/transaction/PartitionCommitInfo.java 2 2 100.00% []
🔵 com/starrocks/sql/analyzer/AlterTableClauseVisitor.java 4 4 100.00% []
🔵 com/starrocks/common/util/PropertyAnalyzer.java 7 7 100.00% []
🔵 com/starrocks/catalog/Column.java 3 3 100.00% []
🔵 com/starrocks/clone/TabletSchedCtx.java 1 1 100.00% []
🔵 com/starrocks/alter/MaterializedViewHandler.java 1 1 100.00% []
🔵 com/starrocks/sql/InsertPlanner.java 2 2 100.00% []
🔵 com/starrocks/alter/SchemaChangeJobV2.java 1 1 100.00% []
🔵 com/starrocks/qe/DDLStmtExecutor.java 4 4 100.00% []
🔵 com/starrocks/alter/RollupJobV2.java 5 5 100.00% []
🔵 com/starrocks/sql/analyzer/MaterializedViewAnalyzer.java 3 3 100.00% []
🔵 com/starrocks/sql/ShortCircuitPlannerHybrid.java 1 1 100.00% []
🔵 com/starrocks/catalog/SchemaInfo.java 1 1 100.00% []
🔵 com/starrocks/qe/ShowExecutor.java 2 2 100.00% []
🔵 com/starrocks/statistic/NativeAnalyzeJob.java 5 5 100.00% []

Copy link

github-actions bot commented Jun 6, 2024

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants