Remote ETL Modernization Engineer (Databricks/AWS)

ZENITH INFOTEK LLC; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Zenith Infotek LLC is seeking an experienced ETL Modernization Developer to join the Data Management team, modernizing legacy ETL/ELT pipelines and migrating large-scale data workloads in a cloud-native environment.

The role focuses on redesigning DataStage jobs into modern frameworks (Databricks Spark on AWS), building pipelines with Python and Unix scripting, and establishing CI/CD for automated testing and deployment.

Qualifications

• Experience in migrating and modernizing ETL/ELT pipelines.
• Background in IBM DataStage and Databricks-based environments.
• Knowledge of cloud services (AWS) and CI/CD tooling.
• Experience with PySpark, SQL, Python, and shell scripting.

Responsibilities

• Assess existing ETL/ELT jobs and data warehouses for migration readiness.
• Redesign DataStage jobs into modern frameworks (Databricks Spark on AWS).
• Develop, test, and deploy ETL pipelines using Python and Unix scripting.
• Implement CI/CD pipelines for automated build, test, and deployment.
• Perform parity, unit, functional, and performance testing; validate data quality.
• Support migration cutover, go-live, and post-migration stabilization.
• Plan decommissioning of legacy DataStage jobs and artifacts.

Skills

IBM DataStage development

PySpark

SQL

Shell Scripting

Python

Databricks Spark

Tools

Databricks

Delta Lake

Unity Catalog

Photon

AWS

Glue

Lambda

Redshift

GitLab

ADO

JIRA

PyTest

Job description

Zenith Infotek LLC is seeking an experienced ETL Modernization Developer to join the Data Management team, modernizing legacy ETL/ELT pipelines and migrating large-scale data workloads in a cloud-native environment.

The role focuses on redesigning DataStage jobs into modern frameworks (Databricks Spark on AWS), building pipelines with Python and Unix scripting, and establishing CI/CD for automated testing and deployment.