ETL Modernization Architect - 1646

aKube Inc; United States; Remoto; USD 88.000 - 118.000; Tempo pieno

Descrizione del lavoro

aKube Inc. is seeking a hands-on senior architect to lead modernization of legacy ETL/ELT pipelines. You will assess IBM DataStage jobs, design scalable Databricks-based data solutions on AWS, and establish governance and CI/CD standards.

You will guide migration planning, testing strategies, and production readiness while driving cutover activities and knowledge transfer to engineering teams.

Competenze

• Extensive experience designing enterprise ETL/ELT architectures.
• Hands-on experience with IBM DataStage and Databricks modernization projects.
• Strong experience with Databricks, Delta Lake, Unity Catalog, Photon, and Spark.
• Strong knowledge of AWS data services including Glue, Redshift, and Lambda.
• Proficiency in Python and Unix/Linux scripting.
• Experience with Terraform and automated deployment pipelines.
• Knowledge of data governance, observability, lineage, and operational readiness.
• Experience defining testing and production-readiness standards.
• Experience in Agile/Scrum environments and PI planning.

Mansioni

• Define the architecture and migration strategy for modernizing legacy ETL and ELT pipelines.
• Assess IBM DataStage jobs, databases, and data warehouses for migration readiness.
• Design scalable data solutions using Databricks Spark on AWS.
• Establish architecture standards, reusable frameworks, and governance controls.
• Design CI/CD pipelines for automated builds, testing, and deployment.
• Lead technical design reviews, migration planning, and artifact validation.
• Define parity, functional, UAT, regression, and performance testing strategies.
• Ensure schema validation, data quality, lineage, security, and production readiness.
• Guide cutover, go-live, hypercare, and operational stabilization activities.
• Oversee the decommissioning of legacy DataStage jobs and related components.
• Create operational documentation and conduct knowledge-transfer sessions.
• Provide technical direction to developers and engineering teams.

Descrizione del lavoro

Onsite/Hybrid/Remote: Remote Duration: 12 months Rate Range: $75 in W2 Work Authorization: GC and US Citizens Only

Must Have

ETL/ELT architecture and modernization IBM DataStage Databricks and Apache Spark Delta Lake, Unity Catalog, and Photon AWS Glue, Redshift, and Lambda Python and Unix/Linux scripting Terraform and CI/CD Large-scale database and data warehouse migrations Data governance, quality, and observability

Responsibilities

• Define the architecture and migration strategy for modernizing legacy ETL and ELT pipelines.
• Assess IBM DataStage jobs, databases, and data warehouses for migration readiness.
• Design scalable data solutions using Databricks Spark on AWS.
• Establish architecture standards, reusable frameworks, and governance controls.
• Design CI/CD pipelines for automated builds, testing, and deployment.
• Lead technical design reviews, migration planning, and artifact validation.
• Define parity, functional, UAT, regression, and performance testing strategies.
• Ensure schema validation, data quality, lineage, security, and production readiness.
• Guide cutover, go-live, hypercare, and operational stabilization activities.
• Oversee the decommissioning of legacy DataStage jobs and related components.
• Create operational documentation and conduct knowledge-transfer sessions.
• Provide technical direction to developers and engineering teams.

Qualifications

• Extensive experience designing enterprise ETL/ELT architectures.
• Hands-on experience with IBM DataStage and Databricks modernization projects.
• Strong experience with Databricks, Delta Lake, Unity Catalog, Photon, and Spark.
• Strong knowledge of AWS data services, including Glue, Redshift, and Lambda.
• Experience designing large-scale database and data warehouse migration programs.
• Proficiency in Python and Unix/Linux scripting.
• Experience with Terraform and automated deployment pipelines.
• Knowledge of data governance, observability, lineage, and operational readiness.
• Experience defining testing and production-readiness standards.
• Experience working in Agile/Scrum environments and PI planning.

Nice to Have

• GitLab or Azure DevOps experience
• JIRA experience
• Automated data-testing framework experience
• Experience leading enterprise cutovers and hypercare activities
• Experience mentoring data engineering teams

Originally posted on Himalayas