Skip to content

How to set multiple source in config file at same time #4325

Answered by lingtaolf
lingtaolf asked this question in Q&A
Discussion options

You must be logged in to vote

Solved:


env {
  spark.app.name = "seatunnel_2023-02-01"
  spark.driver.cores = 2
  spark.rpc.message.maxSize = 2047
  spark.executor.instances = 200 
  spark.driver.maxResultSize="5g"
  spark.driver.memory = "20g"
  spark.executor.cores = 5 
  spark.executor.memory = "10g"
  spark.sql.catalogImplementation = "hive"
}

source {
    hive {
      pre_sql = "select .... from A'" 
      parallelism = 6 
      table_name = "A"
      metastore_uri = "xxxxx"
      result_table_name = "result_table_A"
    }
    hive {
      pre_sql = "select .... from B" 
      parallelism = 6 
      table_name = "B"
      metastore_uri = "xxxxx"
      result_table_name = "result_table_B"
    }
transform {}

sink…

Replies: 1 comment

Comment options

You must be logged in to vote
0 replies
Answer selected by lingtaolf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
1 participant