SPRKPY1017

pyspark.sql.functions.date_add has a workaround

This issue code has been deprecated since Spark Conversion Core Version 4.8.0

pyspark.sql.functions.date_add has a workaround

Category: Warning.

Description

This issue appears when the tool detects the usage of pyspark.sql.functions.date_add which has a workaround.

Scenario

Input

In this example we use date_add to calculate the date 5 days after the current date for the dataframe df.

col = df.select(date_add(df.colName, 5))

Output

SMA returns the EWI SPRKPY1017 over the line where date_add is used, so you can use to identify where to fix.

#EWI: SPRKPY1017 => pyspark.sql.functions.date_add has a workaround, see documentation for more info
col = df.select(date_add(df.colName, 5))

Recommended fix

Import snowflake.snowpark.functions, which contains an implementation for date_add (and alias dateAdd) function.

from snowflake.snowpark.functions import date_add

col = df.select(date_add(df.dt, 1))

Additional recommendation

• For more support, you can email us at [email protected] or post an issue in the SMA.