Skip to content

Schema-Grants need to run before pipeline deployment #4573

@MeinAccount

Description

@MeinAccount

Describe the issue

My DAB contains:

  • Schema
  • Grants on the schema for the SP used by pipeline
  • Lakeflow Declarative Pipeline

Occasionally the DAB deploy will fail with the following message:

Error: cannot create pipeline: User does not have CREATE TABLE on Schema 'CATALOG_NAME.*'.

  with databricks_pipeline.load,
  on bundle.tf.json line 5326, in resource.databricks_pipeline.load:

Rerunning the deploy typically succeeds, as the schema permissions will have been granted.

Configuration

resources:
  schemas:
    src:
      catalog_name: ${var.CATALOG}
      name: src
      grants:
      - principal: ${var.SP}
        privileges:
        - CREATE_TABLE
        - CREATE_MATERIALIZED_VIEW
  pipelines:
    load:
      name: load
      catalog: ${resources.schemas.src.catalog_name}
      schema: ${resources.schemas.src.name}
      serverless: true
      event_log:
        catalog: ${resources.schemas.src.catalog_name}
        schema: ${resources.schemas.src.name}
        name: pipeline_log
      root_path: [...]
      libraries:
      - glob:
          include: [...]
      run_as:
        service_principal_name: ${var.SP}

OS and CLI version

Dab deploy from the Azure Databricks Workspace.

Metadata

Metadata

Assignees

No one assigned

    Labels

    BugSomething isn't workingDABsDABs related issues

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions