Snowflake Data Engineer

Programmers.io; Town of Florida (NY); On-site; USD 110,000 - 150,000; Full time

Job summary

Programmers.io in New York is seeking an Experienced Data Engineer to design, develop, and optimize large-scale data pipelines. You will work with Java, Python, PySpark and focus on performance, data quality, and reliability.

Responsibilities include end-to-end production support, DevOps practices, and deploying with Snowflake for analytics, along with Spark, Hadoop, Hive, and cloud environments (AWS/Azure).

Qualifications

• Experience designing and optimizing large-scale data pipelines.
• Proficiency with Java, Python, and PySpark in production environments.
• Strong knowledge of Snowflake, Spark, Hadoop, Hive, and cloud data architectures.

Responsibilities

• Design, develop, and optimize large-scale data pipelines.
• Provide end-to-end production support, monitoring and reliability.
• Implement performance tuning, data quality, and CI/CD practices.

Skills

Java

Python

PySpark

SQL

NoSQL

Snowflake

Tools

Spark

Hadoop

Hive

Git

Jenkins

Docker

Kubernetes

Terraform

Airflow

AWS

Azure

Job description

Job Summary:

Experienced Data Engineer with a strong foundation in Java-based ecosystems and a proven track record in designing, developing, and optimizing large-scale data pipelines.

Adept in handling both structured and unstructured data using a combination of SQL, NoSQL, and PySpark, with a deep focus on performance tuning and data quality.

Expertise in Snowflake for modern data warehousing and analytics, coupled with robust DevOps and CI/CD practices to ensure smooth development and deployment workflows.

Demonstrated ability to provide end-to-end production support, ensuring data pipeline reliability, uptime, and scalability.

Designing, developing, and optimizing large-scale data pipelines

Languages & Frameworks: Java, Python, PySpark

Databases: SQL (Oracle, PostgreSQL), NoSQL (MongoDB, Cassandra), Snowflake

Big Data Tools: Spark, Hadoop, Hive

DevOps & CI/CD: Git, Jenkins, Docker, Kubernetes, Terraform, Airflow

Cloud: AWS / Azure (as applicable)

Performance Tuning: SQL query optimization, job parallelism, resource allocation

Production Support: Monitoring, alerting, incident response, root cause analysis