Senior Data Engineer

BCforward; New York (NY); On-site; USD 140,000 - 190,000; Full time

Job summary

BCforward in New York, NY is seeking a Senior Big Data Engineer to architect and deliver scalable data pipelines for a data lake and Snowflake environment.

Ideal candidate has strong PySpark and AWS Glue/EMR experience, plus Python and Java, and can drive data processing for financial services projects.

Qualifications

• 5–10 years of experience in big data engineering.
• Strong PySpark, AWS Glue/EMR, Snowflake, Python and Java skills.
• Experience with data structures and algorithms.

Responsibilities

• Create and manage ETL pipelines for data lake and Snowflake.
• Design scalable data processing solutions for large-scale data processing.
• Automate data analysis and aggregation to support business insights.
• Collaborate with cross-functional teams to optimize data workflows and ensure data quality.

Skills

PySpark

AWS Glue ETL

AWS EMR

Snowflake

Python

Java

Data Structures & Algorithms

Job description

• Create/manage ETL pipelines for the data lake and Snowflake; automate data analysis/aggregation work.
• Design, build, and manage scalable ETL pipelines for a data lake and Snowflake environment, ensuring efficient data ingestion, transformation, and loading.
• Automate data analysis and aggregation processes to support business insights and reporting.
• Collaborate with cross-functional teams to optimize data workflows and maintain data quality.

Key Skills:

Experience: 5–10 years

Project: Large-scale Financial Data Exchange & Data Security platform handling transaction processing and data movement.

Must-Have Skills:

• PySpark (Strong hands-on experience)
• AWS Glue ETL
• AWS EMR
• Snowflake
• Python
• Java
• Data Structures & Algorithms

Preferred Domain: Banking, Financial Services, Payments, Transaction Processing.

Interview Process: Zoom interview with Live Coding; onsite may be required.

Ideal Candidate: Senior Big Data Engineer with strong PySpark, AWS Glue/EMR, Snowflake, and Python skills, capable of building scalable financial data processing solutions.