Dbutils

The dbutils provide commands that enable you to work with your DBX environment from notebooks.

During the notebook migration process from DBX to Snowflake, a persistent use of the dbutils module has been identified. As a result it was proposed to simulate the functionalities of dbutils methods, which led to the development of sfutils (Snowflake utility).

The SMA tool is responsible for creating the sfutils file and renaming the usage of dbutils utilities, for example:

Input code:

csv_fiel = "data.csv"
with open(csv_fiel, "r") as file:
    data = file.read()

new_csv_file_path = "/Volumes/main/default/my-volume/data_random.csv"
dbutils.fs.put(new_csv_file_path, data)

Output code:

csv_fiel = "data.csv"
with open(csv_fiel, "r") as file:
    data = file.read()

new_csv_file_path = "/Volumes/main/default/my-volume/data_random.csv"
sfutils.fs.put(new_csv_file_path, data)

Dbutils.fs module

Snowflake does not have a traditional file system, all file-related operations (such as copying, moving, removing, etc.) must be performed through stages. In order to unify storage logic and simplify the replication of this module, a specific stage called SFFS (Snowflake FileSystem) has been defined. This centralized stage operates as a single point of entry and exit for the functions implemented in sfutils.py.

The sfutils file is responsible for creating this stage. However, if the automatic creation is unsuccessful, you can try to create it manually by executing the following SQL command:

CREATE STAGE IF NOT EXISTS "SFFS"