Senior Data Engineer – Enterprise Data Platforms & Spark

Tap Growth ai; Pittsburgh (Allegheny County); On-site; USD 140,000 - 180,000; Full time

Job summary

Tap Growth ai is seeking a Senior Data Engineer to design, develop, and maintain secure, scalable enterprise data platforms. You will build reliable data pipelines, optimize processing workflows, and implement governance and security best practices to support analytics and reporting initiatives.

Responsibilities include ETL/ELT development with Databricks, Delta Lake, and Azure Data Factory, as well as data modeling, API integration, and streaming solutions with Kafka.

Qualifications

• Bachelor's degree or equivalent in a related technical field.
• 6+ years in data engineering or enterprise data platforms.
• Strong SQL development and performance tuning abilities.
• Experience with Databricks ecosystem and cloud data services.

Responsibilities

• Design, develop, test, deploy, and maintain enterprise data engineering solutions.
• Build and optimize ETL/ELT pipelines using Databricks, Spark, Delta Lake, and ADF.
• Develop data ingestion, transformation, and integration frameworks for analytics.
• Write and optimize complex SQL queries and data transformation logic.
• Design scalable data models for BI, analytics, and AI initiatives.
• Develop and integrate REST APIs and event-driven architectures.
• Maintain streaming solutions with Apache Kafka and monitor pipelines.

Skills

Databricks

Spark PySpark

Python

SQL

Scala

Education

Bachelor's degree or equivalent

Tools

Databricks

Delta Lake

Azure Data Factory

Apache Kafka

Job description

Tap Growth ai is seeking a Senior Data Engineer to design, develop, and maintain secure, scalable enterprise data platforms. You will build reliable data pipelines, optimize processing workflows, and implement governance and security best practices to support analytics and reporting initiatives.

Responsibilities include ETL/ELT development with Databricks, Delta Lake, and Azure Data Factory, as well as data modeling, API integration, and streaming solutions with Kafka.