Data Engineer: Pipelines & Insights (Python/Azure)

UJA-Federation of New York; New York (NY); On-site; USD 110,000 - 130,000; Full time

Job summary

UJA-Federation of New York's Data Engineering team seeks an experienced Data Engineer to design and implement scalable data pipelines using Python, PySpark, and SQL. The role supports analytics, reporting, and machine learning initiatives across cross-functional partners.

The ideal candidate will build robust ETL/ELT workflows, improve data quality, and collaborate on CI/CD practices with Azure DevOps while ensuring governance and security standards.

Qualifications

• 5+ years of professional experience in data engineering or related roles.
• Strong proficiency in Python, with proven experience in data-focused libraries (PySpark, Pandas).
• Hands-on experience designing and consuming RESTful APIs or GraphQL endpoints.
• Proficiency in SQL and relational databases (SQL Server, PostgreSQL, MySQL).
• Solid experience with data lakes and data warehouses.
• Familiarity with cloud environments is a plus, with Azure experience preferred.
• Familiarity with Microsoft Fabric (Data Lakehouse, Fabric pipelines, or Fabric Data Warehouses) is a strong plus.
• Hands-on experience with Azure DevOps tools: pipelines, repositories, boards, and artifacts.
• Strong knowledge of CI/CD pipelines and version control (Git).
• Excellent problem-solving, communication, and collaboration skills.

Responsibilities

• Design, build, and maintain robust ETL/ELT pipelines and data workflows using Python.
• Implement best practices for data architecture, modeling, and processing at scale.
• Collaborate with team members to leverage Azure DevOps for CI/CD, release management, and automated testing.
• Ensure compliance with data governance, quality, and security standards.
• Mentor junior engineers and contribute to the growth of engineering best practices.
• Partner with stakeholders to deliver reliable data solutions.

Skills

Python

PySpark

Pandas

RESTful APIs

GraphQL

SQL

CI/CD

Git

Data Lakes

Data Warehouses

Azure Cloud

Fabric Data Lakehouse

Azure DevOps

REST API Design

Tools

Azure DevOps

SQL Server

PostgreSQL

MySQL

Azure

Microsoft Fabric

Docker

Kubernetes

Terraform

Bicep

ARM Templates

Job description

UJA-Federation of New York's Data Engineering team seeks an experienced Data Engineer to design and implement scalable data pipelines using Python, PySpark, and SQL. The role supports analytics, reporting, and machine learning initiatives across cross-functional partners.

The ideal candidate will build robust ETL/ELT workflows, improve data quality, and collaborate on CI/CD practices with Azure DevOps while ensuring governance and security standards.