Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SageMakerTransformOperator does not deduplicate model name #39502

Open
1 of 2 tasks
mxuan0 opened this issue May 8, 2024 · 2 comments
Open
1 of 2 tasks

SageMakerTransformOperator does not deduplicate model name #39502

mxuan0 opened this issue May 8, 2024 · 2 comments
Assignees
Labels
area:providers good first issue kind:bug This is a clearly a bug provider:amazon-aws AWS/Amazon - related issues

Comments

@mxuan0
Copy link

mxuan0 commented May 8, 2024

Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

8.7.1

Apache Airflow version

2.7.2

Operating System

amazon linux AMI

Deployment

Amazon (AWS) MWAA

Deployment details

No response

What happened

The SageMakerTransformOperator deduplicates job name before creating the job, but does not deduplicate the model name before creating the model. Upon retry, the operator would fail due to model already exist error.

What you think should happen instead

The operator should apply the same behavior (fail or add timestamp) to both the job name and the model name.

How to reproduce

create a DAG containing a SageMakerTransformOperator step. In the operator config, fix both ModelName and TransformJobName. In airflow UI, run the task until completion in airflow once, and the run the same task for the second time by "clear task".

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@mxuan0 mxuan0 added area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels May 8, 2024
Copy link

boring-cyborg bot commented May 8, 2024

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

@mateuslatrova
Copy link
Contributor

Hi! I would like to work on this issue after triage!

@rawwar rawwar added the provider:amazon-aws AWS/Amazon - related issues label May 9, 2024
@eladkal eladkal added good first issue and removed needs-triage label for new issues that we didn't triage yet labels May 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:providers good first issue kind:bug This is a clearly a bug provider:amazon-aws AWS/Amazon - related issues
Projects
None yet
Development

No branches or pull requests

4 participants