Remote ETL Modernization Engineer (Databricks/AWS)

Zenith Infotek LLC.

United States

Remote

USD 120,000 - 160,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Zenith Infotek LLC is seeking an experienced ETL Modernization Developer to join the Data Management team, modernizing legacy ETL/ELT pipelines and migrating large-scale data workloads in a cloud-native environment.

The role focuses on redesigning DataStage jobs into modern frameworks (Databricks Spark on AWS), building pipelines with Python and Unix scripting, and establishing CI/CD for automated testing and deployment.

Qualifications

  • Experience in migrating and modernizing ETL/ELT pipelines.
  • Background in IBM DataStage and Databricks-based environments.
  • Knowledge of cloud services (AWS) and CI/CD tooling.
  • Experience with PySpark, SQL, Python, and shell scripting.

Responsibilities

  • Assess existing ETL/ELT jobs and data warehouses for migration readiness.
  • Redesign DataStage jobs into modern frameworks (Databricks Spark on AWS).
  • Develop, test, and deploy ETL pipelines using Python and Unix scripting.
  • Implement CI/CD pipelines for automated build, test, and deployment.
  • Perform parity, unit, functional, and performance testing; validate data quality.
  • Support migration cutover, go-live, and post-migration stabilization.
  • Plan decommissioning of legacy DataStage jobs and artifacts.

Skills

IBM DataStage development
PySpark
SQL
Shell Scripting
Python
Databricks Spark

Tools

Databricks
Delta Lake
Unity Catalog
Photon
AWS
Glue
Lambda
Redshift
GitLab
ADO
JIRA
PyTest

Job description

Job Description – ETL Modernization Developer
Location: Across USA any Location – Remote
Only Independent Visa Holders
Role Overview
We are seeking an experienced ETL Modernization Developer to join our Data Management team supporting customer’s Enterprise AI & Data initiatives. The role involves modernizing legacy ETL/ELT pipelines, migrating large-scale data workloads, and ensuring operational readiness in a cloud-native environment. The developer will work closely with
business, application, and technical teams to deliver high-quality, governed, and production-ready data solutions.

Key Responsibilities
Assessment & Planning
Analyze existing ETL/ELT jobs, databases, and data warehouses for migration readiness.
Document requirements and migration artifacts for approval.
Design & Development
Redesign and re-engineer legacy IBM DataStage jobs into modern frameworks (Databricks Spark on AWS preferred).
Develop, test, and deploy ETL pipelines using Python, Unix scripting, and cloud-native tools.
Implement CI/CD pipelines for automated build, test, and deployment.
Testing & Validation
Conduct parity checks, unit testing, functional testing, UAT, and performance testing.
Ensure schema validation, data quality, regression, and performance test coverage.
Execution & Cutover
Support migration cutover, go-live, and post-migration stabilization (hypercare).
Coordinate with application teams for redeployment and complete testing.
Decommissioning
Plan and execute decommissioning of legacy DataStage jobs and artifacts.
Knowledge Transfer & Training
Deliver operational readiness documentation and conduct KT sessions with Customer engineers.
Provide training support for application and business teams.
Required Skills & Experience

Technical Skills
Strong experience in IBM DataStage development.
Experience in Databricks, Delta Lake, Unity Catalog, Photon
Proficiency in PySpark, SQL, Shell Scripting and Python.
Hands-on experience with Databricks Spark (DBX) on AWS for job verification and pipeline development.
Familiarity with AWS services such as Glue, Lambda, Redshift.
Knowledge of CI/CD tools (GitLab, ADO, JIRA) and automated testing frameworks.
Automated ETL Testing tools, PyTest and XML/JSON parsing.

Domain Knowledge
Expertise in ETL/ELT migration and modernization.
Understanding of large-scale DB/DWH workloads and complex pipelines.
Experience in hypercare support and KT with application teams.

Soft Skills
Strong communication and collaboration skills to work with cross-functional teams.
Ability to document requirements, test plans, and operational readiness artifacts.
Problem-solving mindset with focus on quality and continuous improvement.

This is a remote position.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Remote DataStage Migration & ETL Validation Engineer
Remote DataStage Migration & ETL Validation Engineer

Zenith Infotek LLC. • United States

Remote
USD 120,000 - 180,000
Azure Data Engineer
Azure Data Engineer

MSR Technology Group • Springfield (IL)

Remote
USD 120,000 - 170,000
Senior Data Engineer - Databricks & Cloud Modernization
Senior Data Engineer - Databricks & Cloud Modernization

Compunnel Inc. • Philadelphia

Hybrid
USD 140,000 - 170,000
Data Engineer
Data Engineer

Compunnel Inc. • Philadelphia

Remote
USD 140,000 - 170,000
Senior Databricks Engineer
Senior Databricks Engineer

Robotic Process Automation LLC • Austin (TX)

On-site
USD 83,000 - 118,000
Databricks Engineer / Remote / Contract
Databricks Engineer / Remote / Contract

Motion Recruitment • Boston (MA)

Remote
USD 83,000 - 152,000
Data Engineer
Data Engineer

Root It Services • Snowflake (AZ)

Hybrid
USD 104,000 - 146,000
Senior Databricks Engineer
Senior Databricks Engineer

Ethereum Technologies LLC • Austin (TX)

On-site
USD 85,000 - 116,000
Senior Databricks Engineer
Senior Databricks Engineer

Valiant Technologies LLC • Austin (TX)

On-site
USD 83,000 - 118,000
Databricks Engineer / Remote / Contract
Databricks Engineer / Remote / Contract

Kelly Services • Boston (MA)

Remote
USD 96,000 - 165,000