Data Engineer

NYC Department of Sanitation (DSNY); New York (NY); On-site; USD 90,000 - 120,000; Full time

Job summary

The New York City Department of Sanitation (DSNY) is seeking a Data Engineer to design, develop, and maintain data management systems and integration solutions for the Operations Management Division (OMD). You will support critical reporting, analytics, and operational initiatives across the DSNY project portfolio.

Responsibilities include building data models, optimizing SQL queries, and developing ETL pipelines using tools like Databricks, Informatica PowerCenter, and Pentaho.

Qualifications

• Bachelor’s degree in computer science, engineering or a related field and four years of satisfactory full-time experience related to datacenter engineering and operations, cloud engineering and operations, complex IT infrastructure engineering.
• Bachelor’s degree and eight years or equivalent experience in datacenter engineering and operations, cloud engineering and operations, complex IT infrastructure engineering.

Responsibilities

• Design, development, implementation, enhancement, documentation, and maintenance of database management systems, data integration solutions, and related software functions supporting DSNY Operations Management Division.
• Perform data analysis, data profiling, data modeling, source-to-target mapping, and reporting for enterprise data and analytics projects.
• Develop and maintain SQL-based data solutions, including complex queries, stored procedures, and data transformation processes.
• Collaborate with stakeholders to design, develop, and support data integration pipelines using ETL tools such as Databricks, Informatica PowerCenter, Pentaho Data Integration, and similar technologies.
• Develop and maintain technical documentation including data models, data dictionaries, mapping documents, ETL specifications, and reporting artifacts.
• Support BI and reporting initiatives using platforms like OBIEE, Power BI, Tableau, and other enterprise reporting tools.
• Participate in data architecture and integration strategy design to improve data sharing across DSNY and City agencies.

Skills

SQL

Python

pySpark

Data modeling

ETL/ELT

Data integration

OBIEE

Power BI

Tableau

Databricks

Cloud platforms

Unix/Linux

Education

Bachelor’s degree in CS/Engineering

Tools

Databricks

Informatica PowerCenter

Pentaho Data Integration (Kettle)

Oracle BI/OBIEE

Job description

The New York City Department of Sanitation (DSNY) keeps New York City healthy, safe, and clean by collecting, recycling, and disposing of waste, cleaning City streets and vacant lots, and clearing snow and ice. DSNY is the nation's largest municipal sanitation agency, with nearly 10,000 employees, 59 district garages, and a fleet of more than 5,000 trucks, cars, and other types of equipment. The Department clears litter, snow, and ice from approximately 6,500 miles of City streets and removes debris from vacant lots as well as abandoned vehicles from City streets.

Under general supervision, with considerable latitude for the exercise of independent judgment and initiative, the incumbent will be responsible for the design, development, implementation, enhancement, documentation, and maintenance of database management systems, data integration solutions, custom applications, and related software functions supporting the Department of Sanitation's Operations Management Division (OMD).

The selected candidate will serve as a Data Engineer, performing complex data engineering, data integration, and analytical functions within a multi-platform enterprise environment. The role will support critical reporting, analytics, and operational initiatives across the DSNY Operations Management project portfolio.

Key Responsibilities:

• Perform data analysis, data profiling, data modeling, source-to-target mapping, and reporting activities for enterprise data and analytics projects.
• Develop and maintain conceptual, logical, and physical data models to support current and future operational, analytical, and reporting requirements.
• Analyze legacy and strategic source systems to assess data quality, identify inconsistencies, and recommend corrective actions to improve data integrity and governance.
• Collaborate with business stakeholders, technical teams, and external partners to design, develop, and support complex data integration pipelines using ETL tools such as Databricks, Informatica PowerCenter, Pentaho Data Integration (Kettle), and similar technologies.
• Develop, maintain, and enhance technical documentation, including data models, data dictionaries, source-to-target mapping documents, ETL specifications, process flows, and reporting artifacts.
• Develop and maintain SQL-based data solutions, including complex queries, stored procedures, database objects, and data transformation processes.
• Support business intelligence and reporting initiatives through the development of reports, dashboards, and analytical solutions utilizing platforms such as Oracle Business Intelligence Enterprise Edition (OBIEE), Power BI, and other enterprise reporting tools.
• Participate in the design and implementation of innovative data architecture and integration strategies to improve data sharing and interoperability across multiple business domains within DSNY and across City agencies.
• Support and maintain internal IT systems, applications, and databases, ensuring reliability, performance, and adherence to agency standards.

Required Qualifications:

• Strong proficiency in SQL and Python programming. Additional experience with pySpark preferred.
• Experience with data modeling, data analysis, ETL development, and database design.
• Experience with enterprise reporting and analytics platforms such as Power BI, OBIEE, Tableau, or similar tools.
• Knowledge of modern data platforms and cloud-based analytics environments; experience with Databricks is highly desirable.

Preferred Qualifications:

• Experience with Microsoft .NET technologies, including ASP.NET, C#, TypeScript, React or other modern web application framework.
• Experience with Java development and related technologies.
• Familiarity with data warehousing, data governance, cloud platforms, and enterprise data integration best practices.
• The ideal candidate will possess strong analytical and problem-solving skills, excellent communication abilities, and the capacity to work effectively with

IT INFRASTRUCTURE ENGINEER - 95714

Minimum Qualifications

• 1. A baccalaureate degree from an accredited college in computer science, engineering or a related field and four years of satisfactory full-time experience related to datacenter engineering and operations, cloud engineering and operations, complex IT infrastructure engineering; or,
• 2. A baccalaureate degree from an accredited college and eight years of satisfactory full-time experience related to datacenter engineering and operations, cloud engineering and operations, complex IT infrastructure engineering; or,
• 3. Education and/or experience which is equivalent to "1" or "2" above.

Preferred Skills

The successful candidate should possess extensive hands-on experience in the following areas: Mandatory Skills - Database design, development, and administration - Advanced SQL programming and query optimization - Data integration, ETL/ELT development, and data transformation processes - Enterprise reporting, analytics, and business intelligence solutions - Data modeling, data profiling, and data quality analysis Preferred Skills - Python and pySpark programming - Unix/Linux environments and shell scripting - Databricks and modern cloud-based data platforms - Oracle Business Intelligence Enterprise Edition (OBIEE) - Power BI, Tableau, or similar analytics platforms - C++, Java, C#, and ASP.NET development - Data warehousing, data governance, and enterprise architecture concepts - Designing and implementing enterprise-scale data integration solutions - Developing and supporting cloud and hybrid data platforms - Building automated reporting and analytics solutions - Leading cross-functional technical projects and data modernization initiatives Qualifying experience should demonstrate substantial involvement in the analysis, design, development, implementation, administration, or management of enterprise technology solutions. The following types of experience do not qualify: - Routine use of commercial or pre-packaged software without responsibility for system design, programming, implementation, enhancement, or administration. - Basic word processing, spreadsheet, or office productivity software usage. - Data entry, record maintenance, or transactional processing activities. - Operation of computer hardware, consoles, or peripheral devices without systems development responsibilities. - Use of calculators or other non-programmatic tools as a primary job function. Candidates should demonstrate experience involving complex technical problem-solving, systems development, data engineering, database management, application development, or enterprise analytics solutions.

55a Program

This position is also open to qualified persons with a disability who are eligible for the 55-a Program. Please indicate at the top of your resume and cover letter that you would like to be considered for the position through the 55-a Program.

Public Service Loan Forgiveness

As a prospective employee of the City of New York, you may be eligible for federal loan forgiveness programs and state repayment assistance programs. For more information, please visit the U.S. Department of Education’s website at https://studentaid.gov/pslf/.

Residency Requirement

New York City Residency is not required for this position

Additional Information

The City of New York is an inclusive equal opportunity employer committed to recruiting and retaining a diverse workforce and providing a work environment that is free from discrimination and harassment based upon any legally protected status or protected characteristic, including but not limited to an individual's sex, race, color, ethnicity, national origin, age, religion, disability, sexual orientation, veteran status, gender identity, or pregnancy.