SPRKSCL1164

The parameter of org.apache.spark.sql.DataFrameReader.option function is not defined.

This issue code has been deprecated

Message: The parameter is not defined for org.apache.spark.sql.DataFrameReader.option

Category: Warning

Description

This issue appears when the SMA detects that giving parameter of org.apache.spark.sql.DataFrameReader.option is not defined.

Scenario

Input

Below is an example of undefined parameter for org.apache.spark.sql.DataFrameReader.option function.

spark.read.option("header", True).json(path)

Output

The SMA adds the EWI SPRKSCL1164 to the output code to let you know that giving parameter to the org.apache.spark.sql.DataFrameReader.option function is not defined.

/*EWI: SPRKSCL1164 => The parameter header=True is not supported for org.apache.spark.sql.DataFrameReader.option*/
spark.read.option("header", True).json(path)

Recommended fix

Check the Snowpark documentation for reader format option here, in order to identify the defined options.

Additional recommendations

• For more support, you can email us at [email protected] or post an issue in the SMA.