Data Engineer: Big Data Pipelines & Cloud Infra

New York City Department of Transportation; New York (NY); On-site; USD 110,000 - 150,000; Full time

Job summary

New York City Department of Transportation is seeking an IT Infrastructure Engineer specializing as a Data Engineer to support the Enterprise Data Science and Engineering Unit (EDSE). The role focuses on building scalable data pipelines with Apache NiFi, managing Snowflake data warehouses, and administering Linux cloud environments (Azure, AWS, or GCP).

You will collaborate with analysts to deliver production-ready datasets and analytics solutions.

Qualifications

• Bachelor's degree in computer science, engineering or related field and 4+ years of relevant experience in datacenter/cloud engineering or IT infrastructure.
• Alternative 8+ years of relevant experience in datacenter/cloud engineering or IT infrastructure can substitute the degree.
• Experience designing and operating data pipelines and data warehouses is preferred.

Responsibilities

• Architect, implement, and maintain scalable data pipelines using Apache NiFi for ETL and real-time data integration.
• Design, optimize, and manage Snowflake data warehouses including schema design and security controls.
• Administer Linux-based cloud environments (Azure, AWS, or GCP) and automate deployments and monitoring.

Skills

Python

SQL

Data engineering

CI/CD

Education

Bachelor's degree in computer science, engineering or related field

Tools

Apache NiFi

Snowflake

Linux

Terraform

Ansible

Git

Job description

New York City Department of Transportation is seeking an IT Infrastructure Engineer specializing as a Data Engineer to support the Enterprise Data Science and Engineering Unit (EDSE). The role focuses on building scalable data pipelines with Apache NiFi, managing Snowflake data warehouses, and administering Linux cloud environments (Azure, AWS, or GCP).

You will collaborate with analysts to deliver production-ready datasets and analytics solutions.