SPRKPY1086
pyspark.ml.linalg.VectorUDT
Message: pyspark.ml.linalg.VectorUDT is not supported.
Category: Warning
Description
The pyspark.ml.linalg.VectorUDT is not supported.
Scenario
Input code
VectorUDT is a data type to represent vector columns in a DataFrame.
data = [
(1, Vectors.dense([10.0, 20.0])),
(2, Vectors.dense([25.0, 30.0])),
(3, Vectors.dense([50.0, 60.0]))
]
schema = StructType([
StructField("Id", IntegerType(), True),
StructField("VectorCol", VectorUDT(), True),
])
df = SparkSession.createDataFrame(data, schema=schema)
Output code
The tool adds this EWI SPRKPY1086
on the output code to let you know that this function is not supported by Snowpark.
data = [
(1, Vectors.dense([10.0, 20.0])),
(2, Vectors.dense([25.0, 30.0])),
(3, Vectors.dense([50.0, 60.0]))
]
#EWI: SPRKPY1086 => The pyspark.ml.linalg.VectorUDT function is not supported.
schema = StructType([
StructField("Id", IntegerType(), True),
StructField("VectorCol", VectorUDT(), True),
])
df = spark.createDataFrame(data, schema=schema)
Recommended fix
The pyspark.ml.linalg.VectorUDT does not have a recommended fix.
Additional recommendations
For more support, you can email us at sma-support@snowflake.com or post an issue in the SMA.
Last updated