Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upsert column parameters #2810

Open
SoumayaMauthoorMOJ opened this issue May 7, 2024 · 1 comment
Open

Upsert column parameters #2810

SoumayaMauthoorMOJ opened this issue May 7, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@SoumayaMauthoorMOJ
Copy link

SoumayaMauthoorMOJ commented May 7, 2024

Is your idea related to a problem? Please describe.
The api for glue columns includes Parameters. However none of the aws-sdk-pandas functions allows you to set or update the column Parameters so we have to resort to boto3 update_table.

Describe the solution you'd like
Create new function similar to upsert_table_parameters but for columns:

pars = wr.catalog.upsert_column_parameters(
    parameters={"source": "mysql", "destination":  "datalake"},
    database="...",
    table="...",
    column="..."
)

This could be used to flag primary keys, foreign keys, and various other attributes which Glue data catalog does not currently support

@SoumayaMauthoorMOJ SoumayaMauthoorMOJ added the enhancement New feature or request label May 7, 2024
@SoumayaMauthoorMOJ SoumayaMauthoorMOJ changed the title Support for column parameters Upsert column parameters May 7, 2024
@jaidisido
Copy link
Contributor

My worry is that we would be polluting the library with additional APIs that have little value. By extension, we can also add upsert_columns_comments, upsert_columns_types. Aim of the library is to facilitate pandas operations and none are involved here. Keen to hear from other contributors though

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants