Senior Data Platform Engineer - Redshift, PySpark, AWS Glue

Compunnel, Inc.; Pennsylvania; On-site; USD 95,000 - 155,000; Full time

Job summary

Compunnel, Inc. is seeking a data engineer to support and maintain the data pipeline platform powering performance reporting data staging capabilities for internal teams.

You will design, develop, and optimize ETL/ELT pipelines using PySpark and AWS Glue, manage Redshift clusters, monitor reliability, troubleshoot issues, and collaborate with product, engineering, and business stakeholders to deliver timely, quality data across the platform.

Qualifications

• Experience with AWS Glue and PySpark
• Experience with Amazon Redshift
• Strong SQL and data modeling skills
• Data pipeline development experience
• Cloud data engineering experience
• Data analysis and validation
• Troubleshooting and issue resolution
• Operational support and monitoring

Responsibilities

• Manage and maintain Amazon Redshift consumer cluster infrastructure, ensuring platform stability, performance, scalability, and operational reliability.
• Design, develop, enhance, and support data pipelines built using PySpark and AWS Glue to ingest, transform, and deliver data across the platform.
• Monitor, troubleshoot, and optimize ETL/ELT workflows, data processing jobs, and cloud infrastructure to meet service level expectations.
• Perform data analysis, root cause investigation, and data validation activities to identify and resolve data quality, performance, and operational issues.
• Collaborate with product, engineering, and business teams to support data integration, reporting, and modernization initiatives.
• Implement engineering best practices related to automation, monitoring, performance tuning, and cloud-native data processing.
• Support production releases, incident resolution, and continuous improvement of the AWS-based data platform.

Skills

AWS Glue

PySpark

Amazon Redshift

SQL

Data pipeline development

Cloud data engineering

Data analysis

Troubleshooting

Operational support

Job description

plain
JOB SUMMARY
The contractor will be responsible for supporting and maintaining the data pipeline platform that powers performance reporting data staging capabilities.

Key Responsibilities
• Manage and maintain Amazon Redshift consumer cluster infrastructure, ensuring platform stability, performance, scalability, and operational reliability.
• Design, develop, enhance, and support data pipelines built using PySpark and AWS Glue to ingest, transform, and deliver data across the platform.
• Monitor, troubleshoot, and optimize ETL/ELT workflows, data processing jobs, and cloud infrastructure to meet service level expectations.
• Perform data analysis, root cause investigation, and data validation activities to identify and resolve data quality, performance, and operational issues.
• Collaborate with product, engineering, and business teams to support data integration, reporting, and modernization initiatives.
• Implement engineering best practices related to automation, monitoring, performance tuning, and cloud-native data processing.
• Support production releases, incident resolution, and continuous improvement of the AWS-based data platform.

Required Qualifications
• AWS Glue
• PySpark
• Amazon Redshift
• SQL
• Data pipeline development
• Cloud data engineering
• Data analysis
• Troubleshooting
• Operational support