Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
-
Updated
May 30, 2024 - Java
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
I'm documenting my learning of Data Engineering here.
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.
An orchestration platform for the development, production, and observation of data assets.
Apache Superset is a Data Visualization and Data Exploration Platform
Fancy stream processing made operationally mundane
Multiwoven Documentation
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
lakeFS - Data version control for your data lake | Git for data
Developer-first embedded analytics
Supercharge BigQuery with BigFunctions
100% Python stream processing with Streaming DataFrames
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.
Open Source Feature Flagging and A/B Testing Platform
The open source high performance ELT framework powered by Apache Arrow
SQL stream processing, analytics, and management. We decouple storage and compute to offer instant failover, dynamic scaling, speedy bootstrapping, and efficient joins.
Service for bulk-loading data to databases with automatic schema management (Redshift, Snowflake, BigQuery, ClickHouse, Postgres, MySQL)
Add a description, image, and links to the data-engineering topic page so that developers can more easily learn about it.
To associate your repository with the data-engineering topic, visit your repo's landing page and select "manage topics."