Skip to content

[Feature] Add HTTP API to add physical partition for random distribution tables#68503

Open
wxl24life wants to merge 2 commits intoStarRocks:mainfrom
wxl24life:xiaolong/github-replication-ddl-physical-partition
Open

[Feature] Add HTTP API to add physical partition for random distribution tables#68503
wxl24life wants to merge 2 commits intoStarRocks:mainfrom
wxl24life:xiaolong/github-replication-ddl-physical-partition

Conversation

@wxl24life
Copy link
Contributor

@wxl24life wxl24life commented Jan 27, 2026

Why I'm doing:

StarRocks supports automatic bucketing for random distribution tables, where multiple physical partitions (sub-partitions) can exist within a single logical partition.

For cross-cluster data replication scenarios, the target cluster needs the ability to create matching physical partition structures. Since this capability is only used by internal migration tools (not end users), we expose it via ADMIN EXECUTE ON FRONTEND script

What I'm doing:

Fixes #68502

Add a LocalMetastore.addPhysicalPartition(dbName, tableName, partitionName, bucketNum) method that can be invoked via admin execute script:

-- For partitioned tables:
ADMIN EXECUTE ON FRONTEND 'metastore.addPhysicalPartition("db_name", "table_name", "partition_name", 8)';

-- For non-partitioned tables (pass null as partition name):
ADMIN EXECUTE ON FRONTEND 'metastore.addPhysicalPartition("db_name", "table_name", null, 0)';

Parameters:

  • dbName: database name
  • tableName: table name
  • partitionName: partition name (null for non-partitioned tables, required for partitioned tables)
  • bucketNum: bucket number (0 to use system default)
    Validation:
  • Only OLAP tables with random distribution are supported
  • Bucket number must be non-negative and within max_bucket_number_per_partition
  • For partitioned tables, partition name must be specified and must exist

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5
    • 3.4

@wxl24life wxl24life requested review from a team as code owners January 27, 2026 09:41
@wanpengfei-git wanpengfei-git requested a review from a team January 27, 2026 09:41
@StarRocks-Reviewer
Copy link

@cursor review

@StarRocks-Reviewer
Copy link

@cursor review

Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

@wxl24life wxl24life changed the title [Feature] Support DDL to add and drop physical partition for random distribution tables [Feature] Add HTTP API to add physical partition for random distribution tables Feb 3, 2026
@wxl24life wxl24life requested a review from a team as a code owner February 3, 2026 15:52
@wanpengfei-git wanpengfei-git removed the request for review from a team February 3, 2026 15:53
@wxl24life wxl24life force-pushed the xiaolong/github-replication-ddl-physical-partition branch from 043a3a2 to 198a753 Compare February 4, 2026 03:27
@alvin-celerdata
Copy link
Contributor

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 198a753d89

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

* @param bucketNum the bucket number (0 to use system default)
* @param warehouseId the warehouse id for compute resource allocation
*/
public void addPhysicalPartition(Database db, OlapTable olapTable, String partitionName,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it feasible to reuse addSubPartitions in LocalMetastore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

addPhysicalPartition is a wrapper function of addSubPartitions, we need some pre-valication work

Copy link
Contributor

@kevincai kevincai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, this HTTP REST API is not necessary.

…ion tables

Signed-off-by: Drake Wang <wxl24life@gmail.com>
Signed-off-by: Drake Wang <wxl24life@gmail.com>
@wxl24life
Copy link
Contributor Author

@kevincai Have refactored, please help to change this pr's title, thanks

@wxl24life wxl24life force-pushed the xiaolong/github-replication-ddl-physical-partition branch from aecde86 to 2f6aa50 Compare February 8, 2026 01:46
@github-actions
Copy link
Contributor

github-actions bot commented Feb 8, 2026

[Java-Extensions Incremental Coverage Report]

pass : 0 / 0 (0%)

@github-actions
Copy link
Contributor

github-actions bot commented Feb 8, 2026

[FE Incremental Coverage Report]

pass : 39 / 43 (90.70%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/server/LocalMetastore.java 39 43 90.70% [1793, 1811, 1823, 1831]

@github-actions
Copy link
Contributor

github-actions bot commented Feb 8, 2026

[BE Incremental Coverage Report]

pass : 0 / 0 (0%)

@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 8, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Support DDL interface to ADD & DROP physical partitions manually

6 participants