Data Engineer — PySpark, Spark, Airflow & AWS

Infosys Limited; Atlanta (GA); Hybrid; USD 120,000 - 180,000; Full time

Benefits offered by this job

Long-term disability

Health and dependent care accounts

Insurance (various)

Job summary

Infosys Limited is seeking a Technology Consultant 2 in Atlanta, GA to design, develop, and optimize data pipelines and analytics solutions. You will work with PySpark, Python, SQL, and Spark, contributing to end-to-end data processing and integration projects.

The role emphasizes collaboration with data scientists and engineering teams, implementing ETL/ELT workflows, and ensuring robust production systems while using AWS, Airflow, Snowflake, and Databricks.

Qualifications

• Design, develop, and maintain scalable data pipelines using PySpark, Python, SQL, and Apache Spark.
• Experience in data engineering, big data processing, or a closely related role.
• Hands-on proficiency in Python, SQL, PySpark, and Apache Spark.

Responsibilities

• Contribute to requirements elicitation by documenting assigned parts of business requirements.
• Facilitate software design discussions and document decisions to guide development teams.
• Participate in coding and integrate new features into existing applications while maintaining stability.
• Conduct code reviews and maintain code repositories.
• Implement test strategies, analyze results, and coordinate bug fixes to uphold quality.
• Develop user training programs, documentation, and support frameworks for new software.

Skills

PySpark

Python

SQL

Apache Spark

Data pipelines

ETL/ELT pipelines

Airflow

AWS

Snowflake

Databricks

MLflow

Problem solving

Education

Bachelor’s degree in a related field

Tools

Apache Airflow

AWS (S3, Glue, Redshift, Lambda)

Snowflake

Databricks

MLflow

Job description

Infosys Limited is seeking a Technology Consultant 2 in Atlanta, GA to design, develop, and optimize data pipelines and analytics solutions. You will work with PySpark, Python, SQL, and Spark, contributing to end-to-end data processing and integration projects.

The role emphasizes collaboration with data scientists and engineering teams, implementing ETL/ELT workflows, and ensuring robust production systems while using AWS, Airflow, Snowflake, and Databricks.