Key highlights
50 percent
boost in change analysis and control
60 percent
savings in data dictionary creation
Challenges
The organization lacked a data change management process, leading to inconsistencies and errors.
The team did not define recovery time objectives (RTO) or recovery point objectives (RPO), making data restoration more difficult.
The team did not create an entity relationship (ER) diagram or implement a process for tracking database changes, resulting in a lack of accountability.
Solution
AWS-native zero-ETL pipelines, a three-tier Redshift architecture with data sharing, Airflow-based orchestration, automated MLOps runbooks, and cost-optimized DevOps
Zensar partnered with the customer team to operationalize an AWS-first architecture with Zero-ETL ingestion and curated Redshift layers. Data sharing bridged BI, EDL, and food demand clusters, giving analysts secure, near-real-time access without physical data movement. Airflow orchestrated multi-account pipelines with parameterization, alerts, and metadata-driven execution. MLOps standardized model deployment via a Kedro runbook, integrating CI/CD and operations. Return-leg pipelines captured forecasts from partners. EMR, Glue, and SQL jobs were optimized to cut execution times by 50%+. Cluster RPUs were right-sized (128 → 64), and lower environments were shut down when idle, achieving ~50% cost savings.
1
Developed metadata and documentation for 496 tables, including column definitions, ensuring clarity.
2
Created a data dictionary and ER diagram for the first time to enhance understanding of the data structure.
3
Implemented a standardized change management process with tracking and approval mechanisms for improved efficiency and transparency.
Impact
Improved data management
Created metadata, documentation, and a data dictionary to enhance data accessibility.
Enhanced data integrity and recovery
Defined backup procedures to strengthen data integrity and recovery.
Streamlined change management
Standarized the change management process to improve handling, reduce errors, and boost system reliability.