elementary-lineage
Data lineage made simple, reliable, and automated.
Effortlessly track the flow of data, understand dependencies and analyze impact.
Features
- Visualization: In browser visual representation of the data lineage graph.
- DWH lineage: Effortlessly map data flow in the data warehouse.
- Accuracy: Reflects the actual state in the DWH.
- Plug-and-play: No need for code changes.
Coming soon:
- Monitoring: Present data about freshness, volume and schema on the lineage graph.
- Lineage history: Store data about lineage versions and changes.
- Column level lineage: Add column-level granularity.
- Full lineage: Integrate with downstream and upstream tools to create a full graph.
Quick start
pip install elementary-lineage
# The tool is named edl (Elementary Data Lineage),
# run it to validate the installation:
edl --help
Now we need a connection file in a simple YAML called profiles.yml. Here is a template to create a Snowflake connection profile.
For further instructions go to our quickstart page.
If you usedbt, you can start right away by running this command with the path to your profiles.yml and the relevant profile name:
edl -d ~/profiles_dir -p <profile_name>
If you like what we are building, support us with
Documentation
Our full documentation is available here.
Community & Support
For additional information and help, you can use one of these channels:
Slack (live chat with the team, feature requests, community support, discussions, etc.)
- Contact us directly at [email protected]
Integrations
- Snowflake
- BigQuery
- Redshift
Ask us for integrations on Slack or as a Github issue.
License
Elementary lineage is licensed under Apache License 2.0. See the LICENSE file for licensing information.
elementary-data
