Links

SPRKPY1049

pyspark.sql.session.SparkSession.sparkContext has workaround

Description

This issue appears when the tool detects the usage of pyspark.sql.session.SparkSession.sparkContext which has a workaround.
Input code:
print("APP Name :"+spark.sparkContext.appName())
Output code:
#EWI: SPRKPY1049 => pyspark.sql.session.SparkSession.sparkContext has a workaround, see documentation for more info
print("APP Name :"+spark.sparkContext.appName())

Scenario

property SparkSession.sparkContext
Action: SparkContext is not supported in SnowPark but you can access the methods and properties from SparkContext directly from the Session instance.
Example:
# Pyspark
print("APP Name :"+spark.sparkContext.appName())
can be used in SnowPark removing the sparkContext as:
#Manual adjustment in SnowPark
print("APP Name :"+spark.appName());

Recommendation

  • For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your sales engineer and they can direct your support needs.