SPRKPY1008

pyspark.sql.context.HiveContext is not required

Category

Warning.

Description

This issue appears when the tool detects the usage of pyspark.sql.context.HiveContext, which is not required.

Input code:

from pyspark import HiveContext

hive_context1 = HiveContext(my_sc1)
hive_context2 = HiveContext(sparkContext=my_sc2)

Output code:

#EWI: SPRKPY1008 => pyspark.sql.context.HiveContext is not required
#from pyspark import HiveContext
pass

#EWI: SPRKPY1008 => pyspark.sql.context.HiveContext is not required
hive_context1 = my_sc1
#EWI: SPRKPY1008 => pyspark.sql.context.HiveContext is not required
hive_context2 = my_sc2

Recommendation

• The pyspark.sql.context.HiveContext usage should be removed.

• For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your your account representative.