Senior Big Data Engineer: Hadoop & Spark to Snowflake

Veriipro; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

Veriipro in New York City is seeking a Senior Big Data Engineer with 8+ years of IT experience and hands-on expertise in Hadoop, Spark, Python, and Snowflake. You will design scalable data pipelines, migrate data from Hadoop to Snowflake using PySpark, and develop enterprise ETL solutions.

The role requires building and optimizing ETL workflows, writing complex SQL, ensuring data quality, and collaborating with cross-functional teams to deliver robust data platforms for analytics.

Qualifications

• 8+ years of IT experience in data engineering or big data development.
• Hands-on experience with Hadoop, Spark, Python, and Snowflake.
• Experience migrating enterprise data from Hadoop to Snowflake using PySpark.
• Strong ETL, data integration, and data transformation knowledge.
• Strong SQL skills and ability to optimize complex queries.
• Excellent analytical and communication skills.
• Unix/Linux experience preferred.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Hadoop, Spark, Python, and Snowflake.
• Migrate data from Hadoop to Snowflake using PySpark frameworks and libraries.
• Build and optimize ETL workflows for large-scale data ingestion, transformation, and validation.
• Develop and optimize complex SQL queries for data processing and analytics.
• Ensure data quality, performance, and reliability across enterprise data platforms.
• Collaborate with cross-functional teams to gather requirements, troubleshoot issues, and deliver data solutions.

Skills

Hadoop

Apache Spark

Python

Snowflake

SQL

ETL

Unix/Linux

Data Migration

Tools

PySpark

Job description

Veriipro in New York City is seeking a Senior Big Data Engineer with 8+ years of IT experience and hands-on expertise in Hadoop, Spark, Python, and Snowflake. You will design scalable data pipelines, migrate data from Hadoop to Snowflake using PySpark, and develop enterprise ETL solutions.

The role requires building and optimizing ETL workflows, writing complex SQL, ensuring data quality, and collaborating with cross-functional teams to deliver robust data platforms for analytics.