Links

SPRKPY1008

pyspark.sql.context.HiveContext is not required

Category

Warning.

Description

This issue appears when the tool detects the usage of pyspark.sql.context.HiveContext which is not required.
Input code:
from pyspark import HiveContext
hive_context1 = HiveContext(my_sc1)
hive_context2 = HiveContext(sparkContext=my_sc2)
Output code:
#EWI: SPRKPY1008 => pyspark.sql.context.HiveContext is not required
#from pyspark import HiveContext
pass
#EWI: SPRKPY1008 => pyspark.sql.context.HiveContext is not required
hive_context1 = my_sc1
#EWI: SPRKPY1008 => pyspark.sql.context.HiveContext is not required
hive_context2 = my_sc2

Recommendation

  • The pyspark.sql.context.HiveContext usage should be removed.
  • For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your sales engineer and they can direct your support needs.