Senior Data Engineer: Scalable Data Pipelines & Analytics

UJA Federation of New York; New York (NY); On-site; USD 110,000 - 130,000; Full time

Job summary

UJA-Federation of New York is seeking an experienced Data Engineer to join the Data Engineering team in New York. The role emphasizes Python development for data-intensive applications, with hands-on SQL and PySpark to build scalable pipelines and improve data quality for analytics, reporting, and machine learning initiatives.

This position involves designing and maintaining ETL/ELT workflows, ensuring data governance and security, and collaborating with data scientists, analysts, and business

Qualifications

• 5+ years of professional experience in data engineering or related roles.
• Strong proficiency in Python, with experience in data-focused libraries (e.g., PySpark, Pandas).
• Hands-on experience designing and consuming RESTful APIs or GraphQL endpoints.
• Proficiency in SQL and relational databases (e.g., SQL Server, PostgreSQL, MySQL).
• Experience with data lakes and data warehouses.
• Familiarity with cloud environments, with Azure experience preferred.
• Familiarity with Microsoft Fabric (Data Lakehouse, Fabric pipelines, or Fabric Data Warehouses) is a strong plus.
• Hands-on experience with Azure DevOps tools: pipelines, repositories, boards, and artifacts.
• Strong knowledge of CI/CD pipelines and version control (Git).
• Excellent problem-solving, communication, and collaboration skills.

Responsibilities

• Design, build, and maintain robust ETL/ELT pipelines and data workflows using Python.
• Implement best practices for data architecture, modeling, and processing at scale.
• Collaborate with team members to leverage Azure DevOps for CI/CD, release management, and automated testing.
• Ensure compliance with data governance, quality, and security standards.
• Mentor junior engineers and contribute to the growth of engineering best practices.
• Partner with stakeholders (data scientists, analysts, business teams) to deliver reliable data solutions.

Skills

Python

PySpark

SQL

REST/GraphQL

CI/CD

Git

Data Warehousing

Tools

Azure DevOps

Azure Pipelines

Git

Azure

Job description

UJA-Federation of New York is seeking an experienced Data Engineer to join the Data Engineering team in New York. The role emphasizes Python development for data-intensive applications, with hands-on SQL and PySpark to build scalable pipelines and improve data quality for analytics, reporting, and machine learning initiatives.

This position involves designing and maintaining ETL/ELT workflows, ensuring data governance and security, and collaborating with data scientists, analysts, and business