Release Notes
Updates on each released version of Snowflake SnowConvert for Spark (Python)
Python 1.1.25
PythonSnowConvert Core 2.01.090
SparkCommon 1.3.151
Added
- Added support for Snowpark 1.3.0
- Four new mappings
- Added transformations for
- DataFrameReader chain
- SparkSession.sparkContext
- Added Severity column to the Issues Summary table of the detailed report
Improvements
- Improved name of the Spark usages inventory file
- Improved readiness score displayed value when no Spark references were found
Fixed
- Fixed button URLs
- Fixed inconsistencies of the Spark usages inventory locally and in telemetry
- Fixed RDD metrics in the Spark Usage Summary table of the detailed report
- Fixed inconsistencies with zero and dash symbols in the reports
Python 1.1.25
PythonSnowConvert Core 2.01.068
SparkCommon 1.3.131
Added
- Added support for convert DBC filesImprovements
- Added transformation for DataFrameReader.format and DataFrameReader.load
Fixed
- Fixed SnowConvert/Snowpark version values transposed
Python 1.1.11
PythonSnowConvert Core 2.01.022
SparkCommon 1.3.113
Added
- Added support for Databricks archive files (.dbc extension)
- Added support for Databricks notebook files (.python extension)
- Added parallelism to the Spark usages identification process
- Added support for SnowPark API version 1.1.0
- Added mapping elements:
- twelve direct mappings
- two conversions using helper
Improvements
- Improved SPRKPY1038 EWI message
- Improved registration of EWIs in conversion for columns using attribute access
- Improved local report names
Python 1.1.3
PythonSnowConvert Core 2.1.4
SparkCommon 1.3.105
Added
- Added Net6 compatibility (internal)
- Added issues.csv report
- Added sizing table to the detailed report
- Added support for global variable declaration
- Added support for inherited symbol identification
- Added support for accessing columns using attribute access
- Added in telemetry the version of the mapping that was used
- Added support for Jupyter Notebooks in GenericScanner
- Added mapping elements:
- one direct mapping
- one conversion using helper
- six workarounds
- five not supported identification
Improvements
- Improved tool version format in reports, inventories and telemetry
- Improved syncing of local and remote HTML reports
- Improved HTML detailed report sync with DOCX detailed report
- Improved issues table grouping by EWI code
- Improved import table grouping by package
- Improved commented output code
- Improved UI progress phase titles
Bug Fixes
- Fixed location of EWI messages for complex statements
- Fixed UI wording when cancelling the execution
- Fixed typos on reports
Python 1.0.457
PythonSnowConvert Core 2.0.280
Added
- Added support for Jupyter Notebooks in Generic Scanner
- Added conversion percentage in the reports
- Added 'ElementPackage' column to the import usages inventory
- Added one direct mapping
- Added four helpers
- Added two workarounds
- Added minor visual improvements to the detailed report
Improvements
- Improved one mapping from rename to direct
- Improved sorting of issues table in the detailed report
Bugs
- Fixed columns size of the issue table in the detailed report
- Fixed an error when adding EWI comment for Column.contains function usage
- Fixed six mapping statuses that didn't match in the Spark usages inventory
Python 1.0.457
PythonSnowConvert Core 2.0.259
New Features
- Added three new workarounds
- Added margin of error in the Detailed Report description
Improvements
- Improved two mapping from rename to direct
- Improved sorting of issues table in the detailed report
- Improved displaying of percentages in the detailed report
- Conversion stage logging messages improved
Bugs
- Fixed two mappings
- Fixed identification of a not supported element
Python 1.0.457
PythonSnowConvert Core 2.0.241
New Features
- Support for 'snowpark_extensions'
- Twelve conversions using the 'snowpark_extensions'
- Two workarounds added
- A new spark reference added to the table reference database, including its status.
- Customer info added to the detailed report
Improvements
- EWI SPRKPY1038 wording improvement
- A spark reference status improved from rename to direct
Bug Fixes
- A bug in a mapping fixed
- A broken Spark Core Mapping table fixed
Python 1.0.455
PythonSnowConvert Core 2.0.233
New Features
- Addd margin of error in the readiness score
- Added two new mappings
- Added EWI for PySpark elements that were not recognized
Improvements
- Improved appendix A wording in the detailed report
- Improved EWI message for PySpark elements that are not defined in the tool's conversion database
Bug Fixes
- Fixed 'alias' column name in the inventory
Python 1.0.452
Python SnowConvert Core 2.0.217
New Features
- Added support to SnowPark API version 1.0.0
- Added five new workarounds documentation
- Added execution info to telemetry
- Added margin of error to the readiness score
Improvements
- Improved accuracy in code symbols identification
- Improvement in the assessment step when logging messages.
Python 1.0.449
PythonSnowConvert Core 2.0.210
New Features
- Added EWI comments to the output code for not defined PySpark elements
- Added support for inherited symbols
- Three new mappings added
- One workaround added
Improvements
- Improved readiness score when all the files have errors
- Improved error message when loading the symbol table
- Improved handling of generic types
- One mapping status changed from rename to direct
- One conversion status changed from workaround to direct mapping
Bug Fixes
- Fixed markdown conversion issue
- Fixed syncing issues between PySpark_Mappings_Core table and the tool
Python 1.0.445
PythonSnowConvert Core 2.0.203
Improvements
- Robustness at the loading symbol table
Bug Fixes
- Fixed detailed report summary table for spark usage values
- Fixed some parsing errors
- Fixed EWI code sync issues between the tool and PySpark_Mappings_Core Snowflake DB table and
Python 1.0.441
PythonSnowConvert Core 2.0.199
New Features
- Added EWI record when an error is detected at loading the symbol table
Bug fixes
- Fixed new lines issue when converting Jupyter notebook files
New Features
- Added basic support to convert Jupyter notebook files
- Added a value for tracking import usages as an inventory
- Improve the detailed report (Spark usages grouped by support category and Python Import Call Summary)
- New mappings added
- New workarounds added for 'SparkSession.Builder.appName'
- New EWIs added as comments in the output code
- Added support to copy non-Python files to the output directory
- Added PySpark usages identification for id expressions
- Added an error message when symbol table loading fails
Improvements
- Improved imports mapping
- Improved type hints mapping
- Improved rename mappings to direct mappings
Bug Fixes
- Parsing errors
- The output directory structure for files with parsing errors
- Fixed 'pyspark.streaming' full names
- Fixed CLI crashing
Python 1.0.422
PythonSnowConvert Core 2.0.152
Added
- Added type inference
- 5 New mappings supported
Improvements
- Detailed report
- Import Statement conversion
- Transformation documentation
Fixed
- EWIs related to a Project ID logging
- 4 Pyspark elements conversion status
Python 1.0.417
PythonSnowConvert Core 2.0.135
Added
- New transformations
- Handling unsupported Pyspark elements used in imports
- Improvements in logging message
Python 1.0.417
PythonSnowConvert Core 2.0.132
Added
- Robustness to symbol identification
- Improving in type resolution
Fixed
- Settings button is not refreshing with license change
- Documentation link in Python version reference
Python 1.0.410
PythonSnowConvert Core 2.0.126
Added
- Robustness when parsing Jupypter Notebook files
- Improvements in resolving symbols with Generics
- New transformations
Fixed
- Total Python files in the report
Python 1.0.399
PythonSnowConvert Core 2.0.112
Added
- New support for imports
- Alias name in inventories for the imports
Fixed
- Wrong line number in the inventory for macOS files
- Identified usages table percentages in the html report
- Qualification tool showing zero PySpark references
- Update contact information in the email template
Python 1.0.392
PythonSnowConvert Core 2.0.100
Added
- 'SnowConvert Version' and 'Snowpark version' columns to SparkUsagesInventory
- More functions from pyspark supported
- Improvements to speed analysis
Fixed
- Direct mapping updating
Python 1.0.381
PythonSnowConvert Core 2.0.71
Added
- 10 new mappings supported
- 17 new workaround conversions detected
- Support for identification of PySpark usages in Jupyter notebook files
- Automated and Status columns added to SparkReferenceInventory.csv
- Summary and detailed html report uploading to snowflake
Fixed
- Summary and detailed report wordings fixes
- Email template wording fixes
Python 1.0.380
PythonSnowConvert Core 2.0.30
Added
- 30 new mappings supported
- Identification of pyspark.streaming and pyspark.rdd packages
- Improvements in identifying imported symbols
- Email template update
- Adding "Version information" section to Summary Report
- Adding "Resources" section to Detailed Report
- Final screen UI changes
- Sort SparkReferenceInventory report file
Fixed
- Settings button removed
- Detailed report logos update
- Percentage values precision on summary and detailed assessment reports
Python 1.0.377
PythonSnowConvert Core 1.0.61
Added
- 136 new mappings supported
- Supported status updated for all functions listed as "Corrected" in the shared spreadsheet
- Information collected from the requirements.txt file
- Improvements in identifying chained symbols
Fixed
- Line number in SparkReferenceInventory report
Python 1.0.373
PythonSnowConvert Core 1.0.29
Added
- Collect all the import usages
- Improvements identifying PySpark usages (import without module, import with star)
- Identifying more DataFrame functions as supported
Fixed
Logging parsing errors
Python 0.1.172
Added
- Command line interface.
- Python code Qualification tool feature.
Last modified 25d ago