SPRKPY1007

pyspark.sql.context.SQLContext is not required

Category

Warning.

Description

This issue appears when the tool detects the usage of pyspark.sql.context.SQLContext, which is not required.

Input code:

from pyspark import SQLContext

my_sc1 = SparkContext(var1, var2, var3, var4, var5, var6, var7, my_conf1)
my_sc2 = SparkContext(conf=my_conf2)
my_sc3 = SparkContext(val, conf=my_conf3)

Output code:

#EWI: SPRKPY1007 => pyspark.sql.context.SQLContext is not required
#from pyspark import SQLContext
pass

#EWI: SPRKPY1007 => pyspark.sql.context.SQLContext is not required
sql_context1 = my_sc1
#EWI: SPRKPY1007 => pyspark.sql.context.SQLContext is not required
sql_context2 = my_sc2

Recommendation

• This is an unnecessary parameter being removed with a warning comment being inserted into the source code. There should be no action from the user.

• For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your your account representative.