Multi-source operations datamart
Stage Oracle tables, a public CSV URL, and Microsoft SQL Server data into common PostgreSQL tables using incremental load mode, so scheduled runs add new records without reloading every source from scratch.
Apply business-logic SQL transformations across those staged sources, then publish the curated output to two targets: a PostgreSQL datamart for BI tools and full-copy snapshots in an AWS S3 bucket.
- Incremental staging for adding new records
- Scheduled transformation jobs with two downstream targets
- Error monitoring, run history, alerts, and data quality checks