Data Engineer

VSG Business Solutions LLC; New York (NY); On-site; USD 120,000 - 170,000; Full time

Job summary

VSG Business Solutions LLC in New York seeks a Data Engineer to build and manage ETL pipelines for a Data Lake and Snowflake environments. You will design scalable data processing solutions and optimize Spark/PySpark applications in a distributed data platform.

Strong Python and live-coding skills are essential, along with hands-on experience in ETL, Control-M, Airflow, and AWS Big Data services. Java is not required, scripting knowledge is welcome.

Qualifications

• Strong Python programming and core Python expertise.
• Expert-level Spark / PySpark experience.
• Hands-on ETL development with large-scale data pipelines.
• Strong data structures and algorithms knowledge.
• Production experience with distributed data processing.
• Experience with AWS Big Data services (EMR, Glue).
• Snowflake and Data Lake familiarity.
• Experience with Control-M and Airflow for orchestration.

Responsibilities

• Build and manage ETL pipelines for Data Lake and Snowflake.
• Automate data analysis and aggregation processes.
• Design scalable data processing solutions.
• Develop and optimize large-scale Spark/PySpark applications.
• Contribute to system architecture and production-quality code.

Skills

Python

Spark PySpark

ETL development

Data Structures & Algorithms

Distributed data processing

Airflow

Control-M

Problem solving

Live coding

Project explanations

Tools

Snowflake

AWS EMR

AWS Glue

Data Lake

Data orchestration

Job description

Data Engineer (601/602/603)

Must-Have Skills:

• Strong Python programming (core Python, not just PySpark)
• Expert-level Apache Spark / PySpark
• Hands-on ETL development with large-scale data pipelines
• Strong Data Structures & Algorithms
• Production experience with distributed data processing
• AWS Big Data services (EMR, Glue)
• Snowflake
• Data Lake
• Control-M
• Airflow
• Strong software engineering fundamentals
• Excellent problem-solving and live coding skills
• Ability to explain projects and technologies confidently

Job Responsibilities:

• Build and manage ETL pipelines for Data Lake and Snowflake
• Automate data analysis and aggregation processes
• Design scalable data processing solutions
• Develop and optimize large-scale Spark/PySpark applications
• Contribute to system architecture and production-quality code

Additional Notes:

• Strong Data Engineering background is mandatory.
• Heavy hands-on experience with PySpark, ETL, and Control-M is required.
• Java is NOT required. Any scripting language experience is acceptable.
• Open to 601, 602, and 603 level candidates.