Release Notes

Updates on each released version of Snowflake SnowConvert for Spark (Python)

Release 1.0.877 - April 26th, 2023

Python 1.1.25
PythonSnowConvert Core 2.01.090
SparkCommon 1.3.151
  • Added support for Snowpark 1.3.0
  • Added transformations for
    • DataFrameReader chain
    • SparkSession.sparkContext
  • Added Severity column to the Issues Summary table of the detailed report
  • Improved name of the Spark usages inventory file
  • Improved readiness score displayed value when no Spark references were found
  • Fixed button URLs
  • Fixed inconsistencies of the Spark usages inventory locally and in telemetry
  • Fixed RDD metrics in the Spark Usage Summary table of the detailed report
  • Fixed inconsistencies with zero and dash symbols in the reports

Release 1.0.826 - March 29th, 2023

Python 1.1.25
PythonSnowConvert Core 2.01.068
SparkCommon 1.3.131
  • Added support for convert DBC files
  • Added transformation for DataFrameReader.format and DataFrameReader.load
  • Fixed SnowConvert/Snowpark version values transposed

Release 1.0.725 - February 15th, 2023

Python 1.1.11
PythonSnowConvert Core 2.01.022
SparkCommon 1.3.113
  • Added support for Databricks archive files (.dbc extension)
  • Added support for Databricks notebook files (.python extension)
  • Added parallelism to the Spark usages identification process
  • Added support for SnowPark API version 1.1.0
  • Added mapping elements:
  • twelve direct mappings
  • two conversions using helper
  • Improved SPRKPY1038 EWI message
  • Improved registration of EWIs in conversion for columns using attribute access
  • Improved local report names

Release 1.0.691 - February 1st, 2023

Python 1.1.3
PythonSnowConvert Core 2.1.4
SparkCommon 1.3.105
  • Added Net6 compatibility (internal)
  • Added issues.csv report
  • Added sizing table to the detailed report
  • Added support for global variable declaration
  • Added support for inherited symbol identification
  • Added support for accessing columns using attribute access
  • Added in telemetry the version of the mapping that was used
  • Added support for Jupyter Notebooks in GenericScanner
  • Added mapping elements:
    • one direct mapping
    • one conversion using helper
    • six workarounds
    • five not supported identification
  • Improved tool version format in reports, inventories and telemetry
  • Improved syncing of local and remote HTML reports
  • Improved HTML detailed report sync with DOCX detailed report
  • Improved issues table grouping by EWI code
  • Improved import table grouping by package
  • Improved commented output code
  • Improved UI progress phase titles
Bug Fixes
  • Fixed location of EWI messages for complex statements
  • Fixed UI wording when cancelling the execution
  • Fixed typos on reports

Release 1.0.594 - December 28th, 2022

Python 1.0.457
PythonSnowConvert Core 2.0.280
  • Added support for Jupyter Notebooks in Generic Scanner
  • Added conversion percentage in the reports
  • Added 'ElementPackage' column to the import usages inventory
  • Added one direct mapping
  • Added four helpers
  • Added two workarounds
  • Added minor visual improvements to the detailed report
  • Improved one mapping from rename to direct
  • Improved sorting of issues table in the detailed report
  • Fixed columns size of the issue table in the detailed report
  • Fixed an error when adding EWI comment for Column.contains function usage
  • Fixed six mapping statuses that didn't match in the Spark usages inventory

Release 1.0.555 - December 21st, 2022

Python 1.0.457
PythonSnowConvert Core 2.0.259
New Features
  • Added three new workarounds
  • Added margin of error in the Detailed Report description
  • Improved two mapping from rename to direct
  • Improved sorting of issues table in the detailed report
  • Improved displaying of percentages in the detailed report
  • Conversion stage logging messages improved
  • Fixed two mappings
  • Fixed identification of a not supported element

Release 1.0.515 - December 14th, 2022

Python 1.0.457
PythonSnowConvert Core 2.0.241
New Features
  • Support for 'snowpark_extensions'
  • Twelve conversions using the 'snowpark_extensions'
  • Two workarounds added
  • A new spark reference added to the table reference database, including its status.
  • Customer info added to the detailed report
  • EWI SPRKPY1038 wording improvement
  • A spark reference status improved from rename to direct
Bug Fixes
  • A bug in a mapping fixed
  • A broken Spark Core Mapping table fixed

Release 1.0.492 - December 07th, 2022

Python 1.0.455
PythonSnowConvert Core 2.0.233
New Features
  • Addd margin of error in the readiness score
  • Added two new mappings
  • Added EWI for PySpark elements that were not recognized
  • Improved appendix A wording in the detailed report
  • Improved EWI message for PySpark elements that are not defined in the tool's conversion database
Bug Fixes
  • Fixed 'alias' column name in the inventory

Release 1.0.457 - December 01st, 2022

Python 1.0.452
Python SnowConvert Core 2.0.217
New Features
  • Added support to SnowPark API version 1.0.0
  • Added five new workarounds documentation
  • Added execution info to telemetry
  • Added margin of error to the readiness score
  • Improved accuracy in code symbols identification
  • Improvement in the assessment step when logging messages.

Release 1.0.441 - November 23rd, 2022

Python 1.0.449
PythonSnowConvert Core 2.0.210
New Features
  • Added EWI comments to the output code for not defined PySpark elements
  • Added support for inherited symbols
  • Three new mappings added
  • One workaround added
  • Improved readiness score when all the files have errors
  • Improved error message when loading the symbol table
  • Improved handling of generic types
  • One mapping status changed from rename to direct
  • One conversion status changed from workaround to direct mapping
Bug Fixes
  • Fixed markdown conversion issue
  • Fixed syncing issues between PySpark_Mappings_Core table and the tool

Release 1.0.425 - November 17th, 2022

Python 1.0.445
PythonSnowConvert Core 2.0.203
  • Robustness at the loading symbol table
Bug Fixes
  • Fixed detailed report summary table for spark usage values
  • Fixed some parsing errors
  • Fixed EWI code sync issues between the tool and PySpark_Mappings_Core Snowflake DB table and

Release 1.0.415- November 15th, 2022

Python 1.0.441
PythonSnowConvert Core 2.0.199
New Features
  • Added EWI record when an error is detected at loading the symbol table
Bug fixes
  • Fixed new lines issue when converting Jupyter notebook files

Release 1.0.404 - November 11th, 2022

Python 1.0.436

PythonSnowConvert Core 2.0.195

New Features
  • Added basic support to convert Jupyter notebook files
  • Added a value for tracking import usages as an inventory
  • Improve the detailed report (Spark usages grouped by support category and Python Import Call Summary)
  • New mappings added
  • New workarounds added for 'SparkSession.Builder.appName'
  • New EWIs added as comments in the output code
  • Added support to copy non-Python files to the output directory
  • Added PySpark usages identification for id expressions
  • Added an error message when symbol table loading fails
  • Improved imports mapping
  • Improved type hints mapping
  • Improved rename mappings to direct mappings
Bug Fixes
  • Parsing errors
  • The output directory structure for files with parsing errors
  • Fixed 'pyspark.streaming' full names
  • Fixed CLI crashing

Release 1.0.315 - October 21st, 2022

Python 1.0.422
PythonSnowConvert Core 2.0.152
  • Added type inference
  • 5 New mappings supported
  • Detailed report
  • Import Statement conversion
  • Transformation documentation
  • EWIs related to a Project ID logging
  • 4 Pyspark elements conversion status

Release 1.0.280 - October 12th, 2022

Python 1.0.417
PythonSnowConvert Core 2.0.135
  • New transformations
  • Handling unsupported Pyspark elements used in imports
  • Improvements in logging message

Release 1.0.271 - October 05th, 2022

Python 1.0.417
PythonSnowConvert Core 2.0.132
  • Robustness to symbol identification
  • Improving in type resolution
  • Settings button is not refreshing with license change
  • Documentation link in Python version reference

Release 1.0.247 - September 27th, 2022

Python 1.0.410
PythonSnowConvert Core 2.0.126
  • Robustness when parsing Jupypter Notebook files
  • Improvements in resolving symbols with Generics
  • New transformations
  • Total Python files in the report

Release 1.0.220 - September 15th, 2022

Python 1.0.399
PythonSnowConvert Core 2.0.112
  • New support for imports
  • Alias name in inventories for the imports
  • Wrong line number in the inventory for macOS files
  • Identified usages table percentages in the html report
  • Qualification tool showing zero PySpark references
  • Update contact information in the email template

Release 1.0.190 - September 06th, 2022

Python 1.0.392
PythonSnowConvert Core 2.0.100
  • 'SnowConvert Version' and 'Snowpark version' columns to SparkUsagesInventory
  • More functions from pyspark supported
  • Improvements to speed analysis
  • Direct mapping updating

Release 1.0.148 - August 31st, 2022

Python 1.0.381
PythonSnowConvert Core 2.0.71
  • 10 new mappings supported
  • 17 new workaround conversions detected
  • Support for identification of PySpark usages in Jupyter notebook files
  • Automated and Status columns added to SparkReferenceInventory.csv
  • Summary and detailed html report uploading to snowflake
  • Summary and detailed report wordings fixes
  • Email template wording fixes

Release 1.0.107 - August 24th, 2022

Python 1.0.380
PythonSnowConvert Core 2.0.30
  • 30 new mappings supported
  • Identification of pyspark.streaming and pyspark.rdd packages
  • Improvements in identifying imported symbols
  • Email template update
  • Adding "Version information" section to Summary Report
  • Adding "Resources" section to Detailed Report
  • Final screen UI changes
  • Sort SparkReferenceInventory report file
  • Settings button removed
  • Detailed report logos update
  • Percentage values precision on summary and detailed assessment reports

Release 1.0.66 - August 17th, 2022

Python 1.0.377
PythonSnowConvert Core 1.0.61
  • 136 new mappings supported
  • Supported status updated for all functions listed as "Corrected" in the shared spreadsheet
  • Information collected from the requirements.txt file
  • Improvements in identifying chained symbols
  • Line number in SparkReferenceInventory report

Release 1.0.30 - August 9th, 2022

Python 1.0.373
PythonSnowConvert Core 1.0.29
  • Collect all the import usages
  • Improvements identifying PySpark usages (import without module, import with star)
  • Identifying more DataFrame functions as supported
Logging parsing errors

Release 0.1.172 - July 20th, 2022

Python 0.1.172
  • Command line interface.
  • Python code Qualification tool feature.