Links

SPRKPY1051

pyspark.sql.session.SparkSession.Builder.master has workaround

Description

This issue appears when the tool detects pyspark.sql.session.SparkSession.Builder.master usage which has a workaround.
Input code:
spark = SparkSession.builder.master("local[1]")
Output code:
#EWI: SPRKPY1051 => pyspark.sql.session.SparkSession.Builder.master has a workaround, see documentation for more info
spark = Session.builder.master("local[1]")

Scenario

builder.master(master: str) → pyspark.sql.session.SparkSession.Builder
Action: pyspark.sql.session.SparkSession.Builder.master is used to set up a Spark Cluster. Snowpark doesn't use Spark Clusters so you can remove or comment the code. Example:
# spark = Session.builder.master("local[1]")

Recommendation

  • For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your sales engineer and they can direct your support needs.