ETL / AWS Software Engineer

Cloud Analytics Technologies, LLC; Chicago (IL); On-site; USD 120,000 - 180,000; Full time

Job summary

Cloud Analytics Technologies, LLC is seeking an experienced Software Development Engineer to lead the migration of ETL jobs and databases from on‑premises to AWS. You will build ETL solutions, understand existing ETL jobs, and guide the team through data validation, quality and compliance.

Responsibilities include designing and implementing PySpark ETL processes, deploying to Redshift/Aurora, and ensuring robust production support.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 8+ years of ETL development, data warehousing, AWS cloud services, and API development.
• Proficiency in AWS services and data integration, transformation and loading.
• Scripting: DataStage or Informatica, Shell, Perl, Python.
• Experience with API development (REST/SOAP).
• AWS services: Glue, Step Functions, Airflow DAGs, Lambda, S3, EC2, and more.
• SQL proficiency across on-prem DB2 and AWS databases (RDS, Aurora, Redshift).
• Version control with Git or Bitbucket.
• CI/CD with Jenkins.
• Experience in data analysis/validation with Excel, Python, Tableau.
• AWS certifications and AI/ML knowledge are advantageous.

Responsibilities

• Lead migration of ETL jobs from on‑premises to AWS cloud.
• Design, develop and implement ETL using DataStage or Informatica and PySpark.
• Develop full‑stack components or APIs as required.
• Validate data quality, ensure data compliance and business rules.
• Collaborate with data analysts and architects to improve data accuracy.
• Automate ETL processes and orchestrate workflows across on‑premises and cloud.
• Provide production support, monitoring and issue resolution for ETL pipelines.
• Document processes, data mappings, data flows and APIs.

Skills

ETL development

Data warehousing

AWS cloud services

API development

Python

Perl scripting

Unix/Linux

Shell scripting

SQL

Data analysis

Tableau

Education

Bachelor's degree in Computer Science or related field

Tools

DataStage

Informatica

Git/Bitbucket

Jenkins

AWS Glue/Step Functions

Airflow

Job description

TITLE - ETL / AWS Software Engineer (F2F Interview)

Job Description

Client is seeking a highly experienced Software Development Engineer to lead and support the migration of ETL jobs and databases from on-premises environments to the AWS Cloud. This position requires a hands‑on developer who can build new ETL solutions if necessary, thoroughly understand all existing on‑premises ETL jobs, and serve as an expert to guide the team. Key responsibilities include job execution, data validation, ensuring data quality, and adherence to data compliance and business rules. The role also encompasses understanding AWS infrastructure-as‑code, developing ETL solutions to replicate on‑premises processes in the cloud, orchestrating and validating jobs, and providing comprehensive production support. Candidates must possess technical development expertise and a strong background in data analytics.

Responsibilities

• Leverage development experience with DataStage or Informatica, Korn Shell, Python, and Perl scripting on Unix/Linux platforms.
• Be able to design, develop, and implement ETL Glue PySpark processes to extract, transform, and load data from various sources into AWS cloud-based data warehouses and data lakes, specifically Redshift and Aurora PostgreSQL.
• Be able to develop scalable and efficient full-stack applications, including both front‑end and back‑end components or APIs, as required.
• Conduct business analysis and perform data analysis/ validation to minimize errors and inconsistencies, extract meaningful insights, and support business decision‑making throughout the software development lifecycle.
• Collaborate closely with data analysts, data architects, and other team members to analyze source data, enhance data accuracy and integrity, implement data reporting, and understand data and application requirements to deliver effective solutions.
• Automate and orchestrate ETL processes, applications, and APIs to optimize performance and scalability across both on‑premises and cloud environments.
• Proactively provide production support, including monitoring, troubleshooting, and resolving issues for ETL processes, configurations, and data integrity.
• Develop comprehensive documentation covering overall processes, data mappings, data flows, APIs, application code, and support procedures.
• Perform impact analysis and effectively communicate technical processes and concepts to larger audiences.
• Ensure data and application security and compliance with all relevant data protection regulations.
• Participate in code reviews, adhere to best practices, and provide constructive feedback to peers.
• Stay up to date with the latest industry trends and best practices in the areas of ETL development, full‑stack development, data analysis, and cloud technologies.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• More than 8 years of experience in ETL development, data warehousing, AWS cloud services, and API development.
• Comprehensive proficiency in all AWS services with a strong understanding of data integration, transformation, and loading processes.
• Expertise in scripting languages, including DataStage or Informatica, Shell scripting, Perl, and Python.
• Experience with API development and frameworks such as REST and SOAP.
• Advanced skills utilizing the AWS cloud platform and services, including AWS Glue, Step Functions, Airflow DAGS, Lambda, S3, EC2, and others.
• Proficiency in SQL with both on‑premises relational databases (such as DB2 on AIX) and AWS databases (such as Amazon RDS, Amazon Aurora PostgreSQL, and Redshift).
• Experience with version control tools like Git or Bitbucket.
• Familiarity with implementing Jenkins pipelines and CI/CD processes.
• Demonstrated experience in data analysis/ validation, utilizing tools such as Excel, Python, and Tableau.
• AWS certifications and expertise in AI/ML are advantageous.
• Familiarity with automotive industry knowledge, data warehousing concepts, and adherence to developer best practices.
• Experience working within Agile development methodologies.

CLOUD ANALYTICS TECHNOLOGIES is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Equal Opportunity Employer

CLOUD ANALYTICS TECHNOLOGIES LLC is an equal opportunity employer inclusive of female, minority, disability and veterans, (M/F/D/V). Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. CLOUD ANALYTICS TECHNOLOGIES LLC will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will CLOUD ANALYTICS TECHNOLOGIES LLC require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract

Some of our Business Ventures - Knowledge Analytics

Do restrictive necessities, your capability for advanced analytics or your business model need custom‑made applications and infrastructure configurations? opt for our hosted managed services to urge comprehensive analytics solutions supported industry‑leading Cloud Analytics Technologies, LLC code - delivered by the Cloud Analytics Technologies, LLC consultants UN agency understand these solutions best. Through a mix of with‑it technology and dedicated service and support, Cloud Analytics Technologies, LLC hosted managed services will facilitate you: