sempy_labs.data_pipeline package

Module contents

sempy_labs.data_pipeline.create_data_pipeline(name: str, description: str | None = None, workspace: str | UUID | None = None)

Creates a Fabric data pipeline.

This is a wrapper function for the following API: Items - Create Data Pipeline.

Service Principal Authentication is supported (see here for examples).

Parameters:

  • name (str) – Name of the data pipeline.

  • description (str, default=None) – A description of the environment.

  • workspace (str | uuid.UUID, default=None) – The Fabric workspace name or ID. Defaults to None which resolves to the workspace of the attached lakehouse or if no lakehouse attached, resolves to the workspace of the notebook.

sempy_labs.data_pipeline.delete_data_pipeline(name: str | UUID, workspace: str | UUID | None = None)

Deletes a Fabric data pipeline.

This is a wrapper function for the following API: Items - Delete Data Pipeline.

Service Principal Authentication is supported (see here for examples).

Parameters:

  • name (str | uuid.UUID) – Name or ID of the data pipeline.

  • workspace (str | uuid.UUID, default=None) – The Fabric workspace name. Defaults to None which resolves to the workspace of the attached lakehouse or if no lakehouse attached, resolves to the workspace of the notebook.

sempy_labs.data_pipeline.get_data_pipeline_definition(name: str | UUID, workspace: str | UUID | None = None, decode: bool = True) → dict | DataFrame

Obtains the definition of a data pipeline.

Parameters:

  • name (str or uuid.UUID) – The name or ID of the data pipeline.

  • workspace (str | uuid.UUID, default=None) – The Fabric workspace name or ID. Defaults to None which resolves to the workspace of the attached lakehouse or if no lakehouse attached, resolves to the workspace of the notebook.

  • decode (bool, default=True) – decode : bool, default=True If True, decodes the data pipeline definition file into .json format. If False, obtains the data pipeline definition file a pandas DataFrame format.

Returns:

A pandas dataframe showing the data pipelines within a workspace.

Return type:

dict | pandas.DataFrame

sempy_labs.data_pipeline.list_data_pipelines(workspace: str | UUID | None = None) → DataFrame

Shows the data pipelines within a workspace.

This is a wrapper function for the following API: Items - List Data Pipelines.

Service Principal Authentication is supported (see here for examples).

Parameters:

workspace (str | uuid.UUID, default=None) – The Fabric workspace name or ID. Defaults to None which resolves to the workspace of the attached lakehouse or if no lakehouse attached, resolves to the workspace of the notebook.

Returns:

A pandas dataframe showing the data pipelines within a workspace.

Return type:

pandas.DataFrame