AWS ETL Engineer: Data Pipelines & Cloud Migration

Valiant Technologies LLC; Chicago (IL); Presencial; USD 120 000 - 170 000; Tempo integral

Resumo da oferta

Valiant Technologies LLC is seeking a highly experienced Software Development Engineer to lead and support migrating ETL jobs and databases from on-premises to the AWS Cloud. The role demands hands-on development to build new ETL solutions, deep knowledge of existing on‑prem ETL jobs, and guidance for the team.

You will own job execution, data validation, data quality, and compliance within cloud environments.

Qualificações

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 8+ years of experience in ETL development, data warehousing, AWS cloud services, and API development.
• Comprehensive proficiency in all AWS services with a strong understanding of data integration, transformation, and loading processes.
• Expertise in scripting languages, including DataStage or Informatica, Shell scripting, Perl, and Python.
• Experience with API development and frameworks such as REST and SOAP.
• Advanced skills utilizing the AWS cloud platform and services, including AWS Glue, Step Functions, Airflow DAGS, Lambda, S3, EC2, and others.
• Proficiency in SQL with both on‑premises relational databases and AWS databases.
• Experience with version control tools like Git or Bitbucket.
• Familiarity with implementing Jenkins pipelines and CI/CD processes.
• Demonstrated experience in data analysis/ validation, utilizing tools such as Excel, Python, and Tableau.
• AWS certifications and expertise in AI/ML are advantageous.
• Familiarity with automotive industry knowledge, data warehousing concepts, and adherence to developer best practices.
• Experience working within Agile development methodologies.

Responsabilidades

• Leverage development experience with DataStage or Informatica, Korn Shell, Python, and Perl scripting on Unix/Linux platforms.
• Design, develop, and implement ETL Glue PySpark processes to extract, transform, and load data into AWS data warehouses and data lakes (Redshift, Aurora PostgreSQL).
• Develop scalable and efficient full-stack applications, including front-end and back-end components or APIs, as required.
• Perform data analysis/validation to minimize errors and support business decisions across the software lifecycle.
• Collaborate with data analysts, data architects, and team members to analyze source data, improve accuracy and integrity, and deliver solutions.
• Automate and orchestrate ETL processes, applications, and APIs across on‑premises and cloud environments.
• Provide production support including monitoring and troubleshooting ETL processes and data integrity.
• Develop documentation covering processes, data mappings, data flows, APIs, and support procedures.
• Perform impact analysis and communicate technical concepts to larger audiences.
• Ensure data and application security and compliance with relevant regulations.
• Participate in code reviews and follow best practices with peers.
• Stay updated with ETL, full‑stack, data analytics, and cloud technologies.

Conhecimentos

ETL development

AWS cloud

Python

Shell scripting

DataStage

Informatica

SQL

Data analysis

Airflow

Formação académica

Bachelor's degree in CS/IT

Ferramentas

DataStage

Informatica

Python scripting

Unix/Linux

Git

Jenkins

AWS Glue

Redshift

Aurora PostgreSQL

Airflow (DAGs)

Descrição da oferta de emprego

Valiant Technologies LLC is seeking a highly experienced Software Development Engineer to lead and support migrating ETL jobs and databases from on-premises to the AWS Cloud. The role demands hands-on development to build new ETL solutions, deep knowledge of existing on‑prem ETL jobs, and guidance for the team.

You will own job execution, data validation, data quality, and compliance within cloud environments.