Write

org.apache.spark.sql.Dataframe.write => com.snowflake.snowpark.Dataframe.write

Write transformation consists on replace equivalent methods. Also, transformations add EWIs to not supported methods and also for unsupported or not required options.

Methods supported by Snowpark are: async, mode, option, saveAsTable.

Options supported by Snowpark are: dbtable, which value is used as a parameter to the saveAsTable method.

Check here for more details about methods supported by Snowpark

Input Code:

 df.write.
      format("net.snowflake.spark.snowflake").
      option("sfUrl", s"https://$ACCOUNT.snowflakecomputing.com:443").
      option("sfAccount", ACCOUNT).
      option("sfUser", USER).
      option("sfPassword", PASSWORD).
      option("sfSchema", SCHEMA).
      option("sfDatabase", DB).
      option("sfWarehouse", WAREHOUSE).
      option("sfRole", ROLE).
      option("truncate_table", "ON").
      option("usestagingtable", "OFF").
      option("dbtable", tablename).mode("overwrite").save()

Output Code:

df.write.mode("overwrite").saveAsTable(tablename)
      /*EWI: SPRKSCL1105 => Writer format value is not supported.format("net.snowflake.spark.snowflake")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfUrl", s"https://$ACCOUNT.snowflakecomputing.com:443")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfAccount", "ACCOUNT")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfUser", "USER")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfPassword", "PASSWORD")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfSchema", "SCHEMA")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfDatabase", "DB")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfWarehouse", "WAREHOUSE")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("sfRole", "ROLE")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("truncate_table", "ON")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("usestagingtable", "OFF")
      EWI: SPRKSCL1106 => Writer option is not supported.
      option("dbtable", tablename)
      /*EWI: SPRKSCL1112 => Function org.apache.spark.sql.DataFrameWriter.save is not supported*/
      .save()*/

Last updated