SparkConf

The transformation consists of replacing a SparkConf call with a Session builder call. This change makes the “my_conf” variable understandable using Snowflake.

Input Code:

from pyspark.conf import SparkConf
my_conf = SparkConf(loadDefaults=True)

Output Code:

#EWI: SPRKPY1005 => pyspark.conf.SparkConf is not required
#from pyspark.conf import SparkConf
pass
my_conf = Session.builder.configs({"user" : "my_user", "password" : "my_password", "account" : "my_account", "role" : "my_role", "warehouse" : "my_warehouse", "database" : "my_database", "schema" : "my_schema"}).create()#EWI: SPRKPY1005 => pyspark.conf.SparkConf is not required