SPRKPY1006

pyspark.context.SparkContext is not required

Category

Warning.

Description

This issue appears when the tool detects the usage of pyspark.context.SparkContext, which is not required in Snowflake.

Input code:

from pyspark import SparkContext

sql_context1 = SQLContext(my_sc1)
sql_context2 = SQLContext(sparkContext=my_sc2)

Output code:

#EWI: SPRKPY1006 => pyspark.context.SparkContext is not required
#from pyspark import SparkContext
pass

#EWI: SPRKPY1006 => pyspark.context.SparkContext is not required
my_sc1 = my_conf1
#EWI: SPRKPY1006 => pyspark.context.SparkContext is not required
my_sc2 = my_conf2
#EWI: SPRKPY1006 => pyspark.context.SparkContext is not required
my_sc3 = my_conf3

Recommendation

• This is an unnecessary parameter being removed with a warning comment being inserted. There should be no action from the user.

• For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your your account representative.