Export Classes

Export pipeline lineage data to various formats.

from clpipe import JSONExporter, CSVExporter, GraphVizExporter

JSONExporter

Export pipeline to JSON format.

Methods

export

Export to a dictionary.

JSONExporter.export(
    graph: Pipeline,
    include_metadata: bool = True
) -> Dict[str, Any]

Returns: Dictionary with: - columns: List of column data - edges: List of lineage edges - tables: List of table data

Example:

data = JSONExporter.export(pipeline)
print(f"Exported {len(data['columns'])} columns")

export_to_file

Export directly to a JSON file.

JSONExporter.export_to_file(
    graph: Pipeline,
    file_path: str,
    include_metadata: bool = True,
    indent: int = 2
)

Example:

JSONExporter.export_to_file(pipeline, "lineage.json")

Output Format

{
  "columns": [
    {
      "full_name": "output.total",
      "table_name": "output",
      "column_name": "total",
      "expression": "SUM(amount)",
      "operation": "aggregate",
      "node_type": "output",
      "description": "Total revenue",
      "owner": "analytics-team",
      "pii": false,
      "tags": ["financial"]
    }
  ],
  "edges": [
    {
      "from": "raw.amount",
      "to": "output.total",
      "edge_type": "aggregate",
      "transformation": "SUM",
      "query_id": "query_0"
    }
  ],
  "tables": [
    {
      "table_name": "output",
      "is_source": false,
      "created_by": "query_0"
    }
  ]
}

CSVExporter

Export column and table metadata to CSV files.

Methods

export_columns_to_file

Export column metadata to CSV.

CSVExporter.export_columns_to_file(
    graph: Pipeline,
    file_path: str
)

Example:

CSVExporter.export_columns_to_file(pipeline, "columns.csv")

Output columns: - full_name - table_name - column_name - expression - operation - node_type - description - owner - pii - tags

export_tables_to_file

Export table metadata to CSV.

CSVExporter.export_tables_to_file(
    graph: Pipeline,
    file_path: str
)

Example:

CSVExporter.export_tables_to_file(pipeline, "tables.csv")

GraphVizExporter

Export to DOT format for visualization with GraphViz.

Methods

export

Export to DOT string.

GraphVizExporter.export(
    graph: Pipeline,
    layout: str = "TB",
    show_source_only: bool = False,
    max_columns: Optional[int] = None
) -> str

Parameters: - layout: Graph direction ("TB" top-bottom, "LR" left-right) - show_source_only: Only show source columns - max_columns: Maximum columns per table (for large pipelines)

Example:

dot = GraphVizExporter.export(pipeline, layout="LR")
print(dot)

export_to_file

Export directly to a DOT file.

GraphVizExporter.export_to_file(
    graph: Pipeline,
    file_path: str,
    **kwargs
)

Example:

GraphVizExporter.export_to_file(pipeline, "lineage.dot")

Rendering

Convert DOT to image using GraphViz:

# PNG
dot -Tpng lineage.dot -o lineage.png

# SVG
dot -Tsvg lineage.dot -o lineage.svg

# PDF
dot -Tpdf lineage.dot -o lineage.pdf

Or use Python graphviz library:

import graphviz

dot_string = GraphVizExporter.export(pipeline)
graph = graphviz.Source(dot_string)
graph.render("lineage", format="png", cleanup=True)

Convenience Method

Pipeline has a built-in JSON export:

# Equivalent to JSONExporter.export(pipeline)
data = pipeline.to_json()