Spark Data Engineer | PySpark, AWS ECS, Snowflake

Quadrant IQ Solutions LLC; New York (NY); On-site; USD 100,000 - 130,000; Full time

Job summary

A leading data solutions company in New York is seeking a Mid-Senior level Data Engineer. The role involves designing and optimizing high-performance Spark-based data pipelines using Python/PySpark. Candidates should have strong experience with AWS services and data warehousing tools such as Snowflake or Teradata. This is a full-time position that promises great opportunities for growth in a dynamic environment.

Qualifications

• Expert-level proficiency with Python & PySpark, with strong experience in Apache Spark.
• Hands-on experience with data warehousing tools like Snowflake or Teradata.
• Experience deploying data workloads on AWS ECS and using AWS Glue for data governance.

Responsibilities

• Design and optimize high-performance Spark-based data pipelines using Python/PySpark.
• Develop ETL/ELT workflows for large-scale structured and semi-structured data.
• Collaborate with analytics and engineering teams for data integration.

Skills

Python & PySpark

AWS Cloud

ECS (Elastic Container Service)

AWS Glue

Data Warehousing

Snowflake

Teradata

Job description

A leading data solutions company in New York is seeking a Mid-Senior level Data Engineer. The role involves designing and optimizing high-performance Spark-based data pipelines using Python/PySpark. Candidates should have strong experience with AWS services and data warehousing tools such as Snowflake or Teradata. This is a full-time position that promises great opportunities for growth in a dynamic environment.