SPRKPY1087
pyspark.sql.dataframe.DataFrame.writeTo
Message: The pyspark.sql.dataframe.DataFrame.writeTo function is not supported, but it has a workaround.
Category: Warning.
Description
The pyspark.sql.dataframe.DataFrame.writeTo function is not supported. The workaround is to use Snowpark DataFrameWriter SaveAsTable method instead.
Scenario
Input
Below is an example of a use of the pyspark.sql.dataframe.DataFrame.writeTo
function, the dataframe df
is written into a table name Personal_info
.
df = spark.createDataFrame([["John", "Berry"], ["Rick", "Berry"], ["Anthony", "Davis"]],
schema=["FIRST_NAME", "LAST_NAME"])
df.writeTo("Personal_info")
Output
The SMA adds the EWI SPRKPY1087
to the output code to let you know that this function is not supported, but has a workaround.
df = spark.createDataFrame([["John", "Berry"], ["Rick", "Berry"], ["Anthony", "Davis"]],
schema=["FIRST_NAME", "LAST_NAME"])
#EWI: SPRKPY1087 => pyspark.sql.dataframe.DataFrame.writeTo is not supported, but it has a workaround.
df.writeTo("Personal_info")
Recommended fix
The workaround is to use Snowpark DataFrameWriter SaveAsTable method instead.
df = spark.createDataFrame([["John", "Berry"], ["Rick", "Berry"], ["Anthony", "Davis"]],
schema=["FIRST_NAME", "LAST_NAME"])
df.write.saveAsTable("Personal_info")
Additional recommendations
For more support, you can email us at sma-support@snowflake.com or post an issue in the SMA.
Last updated