Data Engineer - ETL Pipelines, PySpark & AWS

Compunnel, Inc.; Charlotte (NC); On-site; USD 90,000 - 120,000; Full time

Job summary

A prominent tech company seeks a skilled Data Engineer to design and implement data integration solutions. The role demands proficiency in Python and PySpark, along with experience in AWS services. Responsibilities include developing ETL pipelines, collaborating with teams, and ensuring scalable data solutions. Ideal candidates should have strong SQL skills and experience with Agile methodologies.

Qualifications

• Hands-on expertise in Python and PySpark is a must.
• Experience with various AWS services required.
• Strong SQL programming skills, including performance tuning.

Responsibilities

• Design and implement data integration solutions.
• Develop and maintain ETL pipelines for data processing.
• Collaborate with cross-functional teams to deliver data solutions.

Skills

Python

PySpark

AWS Glue

AWS Lambda

SQL programming

UNIX scripting

REST APIs

Agile

Tools

Postgres

SQL Server

Oracle

DataStage

Informatica

Pentaho

Bitbucket

Jenkins

Job description

A prominent tech company seeks a skilled Data Engineer to design and implement data integration solutions. The role demands proficiency in Python and PySpark, along with experience in AWS services. Responsibilities include developing ETL pipelines, collaborating with teams, and ensuring scalable data solutions. Ideal candidates should have strong SQL skills and experience with Agile methodologies.