SPRKPY1051

pyspark.sql.session.SparkSession.Builder.master has workaround

Description

This issue appears when the tool detects pyspark.sql.session.SparkSession.Builder.master usage which has a workaround.

Input code:

spark = SparkSession.builder.master("local[1]") 

Output code:

#EWI: SPRKPY1051 => pyspark.sql.session.SparkSession.Builder.master has a workaround, see documentation for more info
spark = Session.builder.master("local[1]") 

Scenario

builder.master(master: str) → pyspark.sql.session.SparkSession.Builder

Action: pyspark.sql.session.SparkSession.Builder.master is used to set up a Spark Cluster. Snowpark doesn't use Spark Clusters so you can remove or comment the code. Example:

# spark = Session.builder.master("local[1]") 

Recommendation

• For more support, you can email us at [email protected]. If you have a contract for support with Snowflake, reach out to your sales engineer and they can direct your support needs.