Builder

org.apache.spark.sql.SparkSession.builder => com.snowflake.snowpark.Session.builder

SparkSession builder transformation consists of the following:

1. Use the com.snowflake.snowpark.Session class

2. Read the configuration from the connection.properties file

3. Replace the method using create instead of getOrCreate

Builder methods that are not supported by snowflake will be registered as an issue:

• master

• appName

• enableHiveSupport

• withExtensions

• option

• config

For a better description of how to create a Session for Snowpark, please visit https://docs.snowflake.com/en/developer-guide/snowpark/scala/creating-session.html.

Input Code:

val spark = SparkSession.builder()
           .master("local")
           .appName("testApp")
           .config("spark.sql.broadcastTimeout", "3600")
           .getOrCreate()

Output Code:

val spark = Session.builder.configFile("connection.properties")
   /*/*EWI: SPRKSCL1112 => Function org.apache.spark.sql.Builder.master is not supported*/
   .master("local")
   /*EWI: SPRKSCL1112 => Function org.apache.spark.sql.Builder.appName is not supported*/
   .appName("testApp")
   /*EWI: SPRKSCL1112 => Function org.apache.spark.sql.Builder.config is not supported*/
   .config("spark.sql.broadcastTimeout", "3600")*/
   .create