Data Engineer: Scalable ETL & Analytics Pipelines

Compunnel, Inc.; Pennsylvania; Hybrid; USD 85,000 - 125,000; Full time

Job summary

Compunnel, Inc. is seeking a Data Engineer to design, develop, and maintain scalable data pipelines that support enterprise analytics. You will leverage AWS, Snowflake, SQL, and Python to deliver robust ETL solutions and governance.

The role requires collaborating with cross-functional teams, translating business requirements into technical designs, and ensuring data quality through comprehensive testing and deployment in a hybrid environment.

Qualifications

• Bachelor's degree or equivalent experience in a data-related field.
• 5+ years in data analytics/engineering or related roles.
• Strong SQL and Python skills with ETL development experience.
• Experience with AWS services and Snowflake data platform.
• Knowledge of data governance and CI/CD practices.
• Excellent communication and cross-functional collaboration.

Responsibilities

• Design, develop, and maintain scalable ETL processes for data integration.
• Build and optimize databases for real-time and batch analytics.
• Integrate data from multiple sources to support BI initiatives.
• Troubleshoot pipelines to ensure data integrity and reliability.
• Translate business needs into technical data solutions.
• Develop SQL queries, data processing scripts, and reusable code.
• Collaborate with stakeholders and lead development from design to deployment.
• Perform testing/validation and support data governance initiatives.

Skills

SQL

Python

ETL development

AWS

Snowflake

Data governance

Analytical thinking

Communication

Education

Bachelor's degree or equivalent

Tools

AWS Glue

Redshift

Job description

We are seeking a Data Engineer to design, develop, and maintain scalable data pipelines that enable efficient data integration, processing, and analytics across enterprise platforms. This role is responsible for building ETL solutions, optimizing databases, supporting business intelligence initiatives, and leveraging modern cloud-based data technologies. The ideal candidate will have experience with AWS, Snowflake, SQL, Python, ETL development, and data governance while collaborating with cross-functional teams to deliver high-quality data solutions.

Key Responsibilities

• Design, develop, and maintain scalable ETL (Extract, Transform, Load) processes to support enterprise data integration.

• Build and optimize database systems for real-time and batch analytical processing.

• Integrate large-scale data from multiple sources to support business intelligence and analytics initiatives.

• Troubleshoot data pipelines, software, and processes to ensure data consistency, integrity, and reliability.

• Translate business requirements into technical designs and scalable data solutions.

• Develop complex SQL queries, data processing programs, reports, and reusable code.

• Ensure code is well-structured, documented, maintainable, and reusable.

• Collaborate with business stakeholders to understand data requirements and deliver effective solutions.

• Lead all phases of solution development from design through implementation and deployment.

• Communicate technical concepts and solution designs to business partners and technical teams.

• Perform comprehensive testing and validation of data solutions to ensure quality and accuracy.

• Provide data analysis guidance and technical support to internal teams.

• Develop and deliver training materials, job aids, and knowledge-sharing sessions on data tools and technologies.

• Perform regression testing and implement new software releases into production environments.

• Collaborate with vendors to resolve software issues and support production deployments.

• Participate in special projects and perform additional responsibilities as needed.

Required Qualifications

• Bachelor's degree or equivalent combination of education and experience.

• 5+ years of experience in data analytics, data engineering, programming, database administration, or data management.

• Experience with data integration and ETL development.

• Experience with machine learning concepts and supporting data platforms.

• Experience with data governance principles and best practices.

• Strong SQL and Python programming skills.

• Experience with AWS services including Glue ETL, Lambda, Redshift, S3, and related cloud technologies.

• Experience building and supporting AWS infrastructure and CI/CD pipelines.

• Experience with Snowflake data platform.

• Strong analytical, troubleshooting, and problem-solving skills.

• Excellent communication and collaboration skills.

• Ability to work in a hybrid environment with onsite presence three days per week.

Preferred Qualifications

• Experience designing enterprise-scale data pipelines and analytics platforms.

• Experience supporting business intelligence and reporting initiatives.

• Experience with cloud-native data architectures and modern data engineering best practices.