Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] table and column comments not being added with vault_insert_by_rank #208

Closed
erikwilkins-doosan opened this issue Aug 31, 2023 · 4 comments
Assignees
Labels
bug Something isn't working dbt-approach Issues related to implementation (or lack) of dbt functionality

Comments

@erikwilkins-doosan
Copy link

erikwilkins-doosan commented Aug 31, 2023

Describe the bug
After setting "persist_docs" in dbt_project.yml, yml descriptions are not being added to Snowflake when model is using the vault_insert_by_rank materialization

Environment
dbt version: dbt_cloud v1.5
automate_dv version: 0.9.6
Database/Platform: Snowflake

To Reproduce
Steps to reproduce the behavior:

  1. Add the following to dbt_project.yml under models: to add yml comments to table and column
    +persist_docs:
    relation: true
    columns: true
  2. Setup model to use vault_insert_by_rank
    {{ config(materialized='vault_insert_by_rank', rank_column='automate_dv_rank', rank_source_models='STAGE MODEL') }}
  3. Build Model
  4. Check COMMENT column on INFORMATION_SCHEMA.COLUMNS in Snowflake where TABLE_NAME = 'Model'

Expected behavior
COMMENTs to be sent to Snowflake similar to dbt standard customizations when database object is created

AB#5357

@erikwilkins-doosan erikwilkins-doosan added the bug Something isn't working label Aug 31, 2023
@DVAlexHiggs DVAlexHiggs added the dbt-approach Issues related to implementation (or lack) of dbt functionality label Feb 21, 2024
@DVAlexHiggs
Copy link
Member

DVAlexHiggs commented Feb 21, 2024

Hello!

Apologies for the (massively) delayed response on this. This issue is due to a feature implemented in dbt which we haven't kept up-to-date with. persist_docs is handled by calling helper macros provided by dbt in custom materialisations, and we just haven't added that functionality in. Another example of this is the on_schema_change functionality.

For the custom materialisations we offer, this needs to happen. Though we plan to deprecate these materialisations eventually, we want to get them to a better state before we do so.

I've added the 'dbt-approach' label to highlight what I've described above.

@erikwilkins-doosan
Copy link
Author

@DVAlexHiggs , thanks for responding to this. after this issue was created and after upgrading AutomateDV version to latest version for improved Satellite insert methodology, we pivoted from using vault_insert_by_rank custom materialization back to incremental . At this time, I see no need to leave this issue open.

@DVAlexHiggs
Copy link
Member

DVAlexHiggs commented Feb 21, 2024

@DVAlexHiggs , thanks for responding to this. after this issue was created and after upgrading AutomateDV version to latest version for improved Satellite insert methodology, we pivoted from using vault_insert_by_rank custom materialization back to incremental . At this time, I see no need to leave this issue open.

Thanks for the quick reply! That makes sense and we're glad you were able to transition away from this to use incremental, as that was one of the aims of the improvements to the Satellite macro. I'm going to leave this open until I've discussed with the team internally as to whether we still want to make these changes prior to deprecation.

@DVAlexHiggs DVAlexHiggs added dbt-approach Issues related to implementation (or lack) of dbt functionality and removed dbt-approach Issues related to implementation (or lack) of dbt functionality labels May 15, 2024
@DVAlexHiggs
Copy link
Member

Hello,
Apologies for the long time coming on this one. With the release of v0.11.0 today, we've decided to deprecate our vault_insert_by_x materialisations and focus on improving other areas of AutomateDV, so we won't be making that change.

If anyone comes across this thread and would like any guidance, please reach out to me on Slack, LinkedIn or via email. Thanks!

@DVAlexHiggs DVAlexHiggs closed this as not planned Won't fix, can't repro, duplicate, stale Jun 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working dbt-approach Issues related to implementation (or lack) of dbt functionality
Projects
None yet
Development

No branches or pull requests

2 participants