Data Engineer -Intermediate

Ampcus, Inc; New York (NY); On-site; USD 110,000 - 180,000; Full time

Job summary

Ampcus Inc. is seeking a skilled Data Engineer to design, build, and manage scalable ETL pipelines for a centralized data lake and Snowflake data warehouse.

The role focuses on automating data ingestion, transformation, and aggregation workflows to enable reliable analytics and data-driven decision-making. In this intermediate-level position, you will design robust data pipelines, implement orchestration with ControlM and Airflow, and work with PySpark and Spark(Java) on AWS.

Qualifications

• Experience with data lake architectures and large-scale data processing.
• Hands-on experience with AWS services (S3, EC2, EMR, Glue).
• Proven expertise in building ETL pipelines for analytics and reporting.
• Snowflake data loading, transformations, and performance optimization.
• Experience with workflow automation and scheduling tools such as ControlM and Apache Airflow.
• Proficiency in PySpark for distributed data processing.
• Strong programming experience with Apache Spark using Java.
• Good understanding of data modeling, partitioning, and performance tuning concepts.
• Exposure to CI/CD practices for data pipelines.
• Experience in Agile or DevOps environments.

Responsibilities

• Design, develop, and maintain ETL pipelines for ingesting data into the data lake and Snowflake.
• Automate data processing, aggregation, and analytical workflows for availability and performance.
• Manage orchestration and scheduling of pipelines with ControlM and Airflow.
• Develop scalable data transformations using PySpark and Spark (Java).
• Work with large, structured and semi-structured datasets on AWS.
• Ensure data quality, integrity, and reliability across pipelines.
• Optimize pipelines for performance, cost, and scalability.
• Collaborate with analytics, data science, and business teams to define requirements.
• Monitor, troubleshoot, and resolve pipeline failures and bottlenecks.
• Follow best practices for data engineering, security, and documentation.

Skills

Data lake

AWS Services

ETL pipelines

Snowflake

Workflow automation

PySpark

Spark (Java)

Data modeling

Tools

ControlM

Apache Airflow

Job description

Ampcus Inc. is a certified global provider of a broad range of Technology and Business consulting services. We are in search of a highly motivated candidate to join our talented Team.

Job Title: Data Engineer - Intermediate Location: Manhattan West, NY - Onsite

Job Description:

We are seeking a skilled Data Engineer to design, build, and manage scalable ETL pipelines supporting a centralized data lake and Snowflake data warehouse. The role focuses on automating data ingestion, transformation, and aggregation workflows to enable reliable analytics and data-driven decision-making.

Key Responsibilities

• Design, develop, and maintain robust ETL pipelines for ingesting data into the enterprise data lake and Snowflake environment.
• Automate data processing, aggregation, and analytical workflows to improve data availability and performance.
• Implement and manage orchestration and scheduling of data pipelines using ControlM and Apache Airflow.
• Develop scalable data transformation logic using PySpark and Apache Spark (Java).
• Work with large, structured and semi-structured datasets on AWS infrastructure.
• Ensure data quality, integrity, and reliability across data pipelines.
• Optimize data pipelines for performance, cost, and scalability.
• Collaborate with analytics, data science, and business teams to understand data requirements.
• Monitor, troubleshoot, and resolve pipeline failures and performance bottlenecks.
• Follow best practices for data engineering, security, and documentation.

Required Skills & Qualifications

• Strong experience with data lake architectures and large-scale data processing.
• Hands-on experience with AWS services (e.g., S3, EC2, EMR, Glue, or related).
• Proven expertise in building ETL pipelines for analytics and reporting use cases.
• Solid working knowledge of Snowflake, including data loading, transformations, and performance optimization.
• Experience with workflow automation and scheduling tools such as ControlM and Apache Airflow.
• Proficiency in PySpark for distributed data processing.
• Strong programming experience with Apache Spark using Java.
• Good understanding of data modeling, partitioning, and performance tuning concepts.

Preferred Qualifications (Nice to Have)

• Exposure to CI/CD practices for data pipelines.
• Experience working in Agile or DevOps environments.

Ampcus is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veterans or individuals with disabilities.