SPRKPY1048

pyspark.sql.session.SparkSession.conf has workaround

Description

This issue appears when the tool detects the usage of pyspark.sql.session.SparkSession.conf which has a workaround. Note: This workaround is deprecated because a later tool version supports the conf snowflake function as direct.

Input code:

spark.conf.set("spark.sql.crossJoin.enabled", "true")

Output code:

#EWI: SPRKPY1048 => pyspark.sql.session.SparkSession.conf has a workaround, see documentation for more info
spark.conf.set("spark.sql.crossJoin.enabled", "true")

Scenario

property SparkSession.conf

Action: SparkSession.conf is used to pass some specific settings only used by Pyspark and doesn't apply to Snowpark. You can remove or comment the code

#spark.conf.set("spark.sql.crossJoin.enabled", "true")

Recommendation

• For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your sales engineer and they can direct your support needs.