Data Engineer-Integration Specialist

City of Philadelphia, PA; Philadelphia (Philadelphia County); On-site; USD 95,000 - 140,000; Full time

Job summary

City of Philadelphia, PA is seeking a Data Engineer within the Office of Innovation & Technology. You will develop and optimize data pipelines, integrating diverse datasets to support city operations and open data initiatives.

Work with Python, SQL, Airflow, and modern data tools to improve data governance, accessibility, and quality for multiple departments and public data portals.

Qualifications

• Bachelor's degree in CS, data science, information systems, GIS, or related field.
• Experience in data engineering, data management and integration.
• Proven track record with Python, SQL, and database management.
• Familiarity with modern data engineering tech stack and CI/CD basics.

Responsibilities

• Design, develop, and optimize data pipelines and integrations.
• Collaborate with partners to improve data accessibility and quality.
• Support enterprise data warehouse and open data initiatives.
• Document technical specs and coordinate with stakeholders.

Skills

Data engineering

Python

SQL

ETL/ELT

CI/CD

Apache Airflow

Education

Bachelor's degree in Computer Science or related field

Tools

Docker

Terraform

Postgres

PostGIS

ArcGIS Online

CARTO

MuleSoft iPaaS

Job description

With a workforce of over 25,000 people, and opportunities in more than 1,000 different job categories, the City of Philadelphia is the fifth largest city in the United States and one of the largest employers in southeastern Pennsylvania. As an employer, the City of Philadelphia operates through the guiding principles of service, integrity, respect, accountability, collaboration, diversity and inclusion. Through these principles, we strive to effectively deliver services, to resolve the challenges facing our city, and to make Philadelphia a place where all our residents have the opportunity to reach their potential.

The Office of Innovation & Technology (OIT) is the central IT agency for the City of Philadelphia headed by the Chief Information Officer (CIO). OIT oversees all major information and communications technology initiatives for the City of Philadelphia - increasing the effectiveness of the information technology infrastructure, where the services provided are advanced, optimized, and responsive to the needs of the City of Philadelphia’s businesses, residents, and visitors. OIT responsibilities include: identifying the most effective approach for implementing new information technology directions throughout city government; improving the value of the city’s technology assets and the return on the city’s technology investments; ensuring data security continuity; planning for continuing operations in the event of disruption of information technology or communications services; and supporting accountable, efficient and effective government across every city department, board, commission and agency.

Job Description

Data Engineering at the City is a unique opportunity for meaningful, exciting work and professional development using state-of-the art technologies and software development best practices. This is a position on a diverse team of engineers, analysts, and GIS specialists, contributing to developing, maintaining, and optimizing our data pipelines and integration strategies, assisting departments and agencies in innovating, and enhancing the City’s service to its residents.

The data engineer/integration specialist will develop an intimate understanding of the City’s diverse data and contribute to improving the City’s data engineering infrastructure, pipelines, models, and integrations. The team uses a blend of open source, custom developed and off-the-shelf tools, including Python, Bash, SQL, DBT, GIS, Docker, Terraform, Apache Airflow, Jenkins, Postgres, PostGIS, AWS, GitHub, Github actions, MuleSoft as an iPaaS and SAS API providers like ArcGIS Online and CARTO.

Some prominent services the team is responsible for include DataBridge, OIT’s enterprise data warehouse, AIS, the City’s custom address information systems which supports public address search and internal geocoding, and Databridge-Airflow, a custom workflow generator and orchestrator for departmental integrations ( https://metadata.phila.gov/ ) and open data publishing (https://opendataphilly.org/). Some examples of OIT’s commitment to serving the public through the effective and innovative integration of City open data include:

Essential Functions

• Working with business partners of varying technical ability to understand how their data is produced, stored, and updated, designing enhancements to optimize enterprise integrations and public data accessibility.
• Design and develop new and expand and refine existing data pipelines using Apache Airflow, custom Python-based software, and other tools to connect diverse systems of record, centralized databases, and SaaS environments, improving data platform components, data governance, master data management, data standardization, enrichment and quality assurance.
• Play a key role in enterprise integration and open data initiatives, developing and supporting critical systems like the City’s DataBridge, Databridge-Airflow and AIS.
• Write clear, descriptive technical documentation for systems/applications, partner with data stewards of various technical levels and communicate technical details to non‑technical stakeholders.

Qualifications

• Bachelor’s degree in Computer Science, Data Science, Information Systems, Geography/GIS, or a related field. Equivalent work experience may be considered.
• Demonstrated experience in data engineering or a similar role, with a focus on data management/integration/ETL/ELT.
• Proven track record with Python, SQL, and database management.
• Experience with working with modern data engineering tech stack.
• Familiarity with development of custom ETL/CI/CD tools and the ability to