Regulatory Data Engineer - Cloud Spark/PySpark & Delta Lake

Selby Jennings; New York (NY); Hybrid; USD 100,000 - 130,000; Full time

Job summary

A leading financial services firm is seeking a Data Engineer to join their Regulatory Technology and Controls team. This hybrid role focuses on working with high-volume trading data to enhance regulatory reporting and trade recordkeeping. The ideal candidate should have over 4 years of experience in cloud data technologies like Synapse and Spark, along with strong coding skills in Python or similar languages. This position includes collaborating with various teams to implement data solutions and improve decision-making processes.

Qualifications

• 4+ years of experience with Synapse, Spark, or similar cloud data technologies.
• 4+ years of development experience in a modern programming language (Python, C++, Java, etc.).
• Experience with GitHub or similar tools for version control and CI/CD.
• Strong communication and collaboration skills.

Responsibilities

• Maintain and enhance cloud‑based Synapse/Spark data pipelines for regulatory reporting.
• Migrate legacy SQL Server/T‑SQL processes to cloud‑based Synapse, PySpark, and Delta Lake.
• Understand Equity trading workflows and how they map to data models.
• Analyze high‑volume datasets using Python, Synapse, and Power BI.
• Implement data reconciliations and support critical production reporting.
• Partner with business teams to develop dashboards and reporting tools.

Skills

Python

SQL

Azure

Spark

Banking

Regulatory Reporting

Trading

Education

Bachelor's or Master's degree in a STEM field

Tools

GitHub

CI/CD

Job description

A leading financial services firm is seeking a Data Engineer to join their Regulatory Technology and Controls team. This hybrid role focuses on working with high-volume trading data to enhance regulatory reporting and trade recordkeeping. The ideal candidate should have over 4 years of experience in cloud data technologies like Synapse and Spark, along with strong coding skills in Python or similar languages. This position includes collaborating with various teams to implement data solutions and improve decision-making processes.