SPRKPY1086

pyspark.ml.linalg.VectorUDT

Message: pyspark.ml.linalg.VectorUDT is not supported.

Category: Warning

Description

The pyspark.ml.linalg.VectorUDT is not supported.

Scenario

Input code

VectorUDT is a data type to represent vector columns in a DataFrame.

data = [
        (1, Vectors.dense([10.0, 20.0])),
        (2, Vectors.dense([25.0, 30.0])),
        (3, Vectors.dense([50.0, 60.0]))
    ]

schema = StructType([
        StructField("Id", IntegerType(), True),
        StructField("VectorCol", VectorUDT(), True),
    ])

df = SparkSession.createDataFrame(data, schema=schema)

Output code

The tool adds this EWI SPRKPY1086 on the output code to let you know that this function is not supported by Snowpark.

data = [
        (1, Vectors.dense([10.0, 20.0])),
        (2, Vectors.dense([25.0, 30.0])),
        (3, Vectors.dense([50.0, 60.0]))
    ]

#EWI: SPRKPY1086 => The pyspark.ml.linalg.VectorUDT function is not supported.
schema = StructType([
        StructField("Id", IntegerType(), True),
        StructField("VectorCol", VectorUDT(), True),
    ])

df = spark.createDataFrame(data, schema=schema)

Recommended fix

The pyspark.ml.linalg.VectorUDT does not have a recommended fix.

Additional recommendations

• For more support, you can email us at [email protected] or post an issue in the SMA.