Senior ETL & AWS Cloud Engineer

MACHINE LEARNING TECHNOLOGIES LLC; Chicago (IL); On-site; USD 96,000 - 165,000; Full time

Job summary

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced Software Engineer to lead the migration of ETL jobs from on-prem to AWS. You will build ETL solutions, understand existing jobs, and guide the team on data quality, compliance, and production support.

Strong knowledge of DataStage/Informatica, Python, and AWS is required across a broad data analytics stack. The role involves orchestration of ETL processes, development of scalable cloud-based data pipelines, and collaboration with data

Qualifications

• More than 8 years of experience in ETL development, data warehousing, AWS cloud services, and API development.
• Comprehensive proficiency in AWS services with data integration, transformation, and loading.
• Scripting expertise including DataStage/Informatica, Shell, Perl, and Python.
• API development with REST and SOAP.
• AWS ecosystem skills (Glue, Step Functions, Airflow, Lambda, S3, EC2, Redshift, Aurora).
• Experience with Git/Bitbucket, Jenkins pipelines, and CI/CD.
• Proven data analysis/validation using Excel, Python, Tableau is advantageous.
• Familiarity with automotive industry data-warehousing concepts and agile methods.

Responsibilities

• Lead migration of ETL jobs and on-premises databases to AWS Cloud.
• Build new ETL solutions as needed and document data flows and mappings.
• Design and implement ETL Glue PySpark processes for Redshift and Aurora PostgreSQL.
• Develop full-stack capabilities and APIs as required by projects.
• Analyze data quality, validate data, and ensure regulatory compliance.
• Automate ETL processes and orchestrate workflows across hybrid environments.
• Provide production support, monitoring, troubleshooting, and issue resolution.
• Create comprehensive documentation for processes, data mappings, and APIs.

Skills

ETL development

Data warehousing

AWS cloud services

API development

CI/CD

Agile

Data analysis

Education

Bachelor's degree in Computer Science, Information Technology, or a related field

Tools

DataStage

Informatica

Korn Shell

Python

Perl

REST

SOAP

SQL

Git/Bitbucket

Jenkins

Airflow

AWS Glue

Lambda

S3

Aurora

Redshift

EC2

Step Functions

DB2 on AIX

Job description

MACHINE LEARNING TECHNOLOGIES LLC is seeking an experienced Software Engineer to lead the migration of ETL jobs from on-prem to AWS. You will build ETL solutions, understand existing jobs, and guide the team on data quality, compliance, and production support.

Strong knowledge of DataStage/Informatica, Python, and AWS is required across a broad data analytics stack. The role involves orchestration of ETL processes, development of scalable cloud-based data pipelines, and collaboration with data