Senior Data Engineer – Databricks Lakehouse & Pipelines

Bitful Consulting; Elm (PA); On-site; USD 120,000 - 180,000; Full time

Job summary

Bitful Consulting is seeking an experienced Data Engineer to design and maintain scalable data pipelines across finance, sales, and operations. You will work in a Databricks lakehouse environment, ingesting data with PySpark and Python, and building datasets for Tableau and Spotfire.

Strong SQL, data modeling and warehouse design are essential, along with cloud platform experience and CI/CD practices. You will mentor teammates and collaborate with business stakeholders to deliver robust data

Qualifications

• 7+ years of experience in data engineering and production pipelines.
• Strong SQL, data modeling and data warehousing fundamentals.
• Experience with Databricks, PySpark, Python and Spark in cloud environments.

Responsibilities

• Build and maintain scalable data pipelines and datasets for reporting and analytics.
• Ingest, validate, clean, and transform data from multiple sources using Python and PySpark.
• Develop data models and datasets for Tableau and Spotfire.
• Integrate data warehouses with modern cloud analytics platforms.
• Improve pipeline reliability, data quality and performance.
• Implement automated testing, monitoring, and CI/CD practices.
• Collaborate with analysts and stakeholders to deliver practical data solutions.
• Provide technical guidance and review code.

Skills

SQL

Python

PySpark

Databricks

Spark

Data modeling

ETL/ELT

CI/CD

Tableau

Spotfire

Data warehousing

Cloud platforms

Communication

Tools

Tableau

Spotfire

Oracle

Informatica

R

Data virtualization

Source control

CI/CD tools

Job description

Bitful Consulting is seeking an experienced Data Engineer to design and maintain scalable data pipelines across finance, sales, and operations. You will work in a Databricks lakehouse environment, ingesting data with PySpark and Python, and building datasets for Tableau and Spotfire.

Strong SQL, data modeling and warehouse design are essential, along with cloud platform experience and CI/CD practices. You will mentor teammates and collaborate with business stakeholders to deliver robust data