SPRKSCL1172

org.apache.spark.sql.types.StructField.apply

Message: Snowpark does not support StructFiled with metadata parameter.

Category: Warning

Description

This issue appears when the SMA detects that org.apache.spark.sql.types.StructField.apply with org.apache.spark.sql.types.Metadata as parameter. This is because Snowpark does not supported the metadata parameter.

Scenario

Input

Below is an example of the org.apache.spark.sql.types.StructField.apply function that generates this EWI. In this example, the apply function is used to generate and instance of StructField.

val result = StructField("f1", StringType(), True, metadata)

Output

The SMA adds the EWI SPRKSCL1172 to the output code to let you know that metadata parameter is not supported by Snowflake.

/*EWI: SPRKSCL1172 => Snowpark does not support StructFiled with metadata parameter.*/
val result = StructField("f1", StringType(), True, metadata)

Recommended fix

Snowpark has an equivalent com.snowflake.snowpark.types.StructField.apply function that receives three parameters. Then, as workaround, you can try to remove the metadata argument.

val result = StructField("f1", StringType(), True, metadata)

Additional recommendations

• For more support, you can email us at [email protected] or post an issue in the SMA.