SPRKPY1002

Element is not supported

Category

Conversion error.

Description

This issue appears when the tool detects the usage of an element that is not supported in Snowpark, and does not have it's own error code associated with it. This is the generic error code used by the SMA for an unsupported element.

Additional Considerations

Some elements do not have a direct equivalent in Snowpark but that doesn’t mean that similar actions cannot be achieved. For example, in issues with this code and description:

#EWI: SPRKPY1002 => pyspark.sql.readwriter.DataFrameReader.load is not supported
cases = session.read.load("/FileStore/Case.csv", format = "csv", sep = ",", inferSchema = "true", header = "true")

This happens because there are some differences between the Spark and Snowpark API options for parameters in the load function:

SPARK	SNOWPARK
sep	FIELD_DELIMITER
inferSchema	INFER_SCHEMA
header	When reading you can use PARSE_HEADER to indicate if you need to consider the presence of a header for inference, or SKIP_HEADER

A workaround could still exist. In this case, using the option function could allow you to pull from a csv file in a stage:

cases = session.read.option("FIELD_DELIMITER",",").option("INFER_SCHEMA",True).option("PARSE_HEADER",True).csv("@mystage/Case.csv")

Even though though the option above is not supported, this does not mean that a solution cannot be found. It only means that the tool itself cannot find the solution.

Pyspark.ml Considerations

If you have encountered an unsupported element from a pyspark.ml library, consider some alternative approached. There are additional guides available to walkthrough issues related to ml such as this one from Snowflake.

Recommendation

• Check if the source code has the correct syntax. (You can use the issues.csv file to determine where the conversion error(s) are occurring.) If the syntax is correct, report that you encountered a conversion error on a particular element using the Report an Issue option in the SMA. Include the line of code that was causing the error in the description when you file this issue.
• For more support, you can email us at [e-mail]. If you have a contract for support with Snowflake, reach out to your your account representative.