Senior Data Engineer: Databricks Lakehouse & BI Pipelines

Bitful Consulting; Florida; On-site; USD 120,000 - 180,000; Full time

Job summary

Bitful Consulting in Florida is seeking an experienced Senior Data Engineer to design, build and maintain data pipelines in a Databricks lakehouse environment. You will work with Python, PySpark and SQL to ingest and transform data, create analytics-ready datasets for Tableau and Spotfire, and collaborate with analysts and stakeholders to deliver scalable data solutions.

The role emphasizes data quality, CI/CD practices, and mentoring teammates, with opportunities to influence reporting,

Qualifications

• 7+ years building and supporting production data pipelines.
• Strong SQL, data modeling, and data warehousing fundamentals.
• Hands-on with Databricks, Apache Spark, and cloud data platforms.
• Proficient in Python and PySpark.
• Experience ETL/ELT and integrating data from multiple sources.
• Experience preparing reliable datasets for Tableau or similar BI tools.
• Familiarity with source control, automated testing and CI/CD.
• Ability to troubleshoot complex data issues and optimize pipelines.
• Strong communication skills and collaboration with technical teams.
• Experience with Spotfire, Oracle, Informatica, or data virtualization preferred.
• Databricks certification preferred.

Responsibilities

• Developing and maintaining scalable data pipelines and reusable datasets within a Databricks lakehouse environment
• Ingesting, validating, cleaning and transforming structured and unstructured data from multiple sources using Python, PySpark and SQL
• Building data models and reporting datasets that support Tableau and established analytics tools such as Spotfire
• Integrating established data warehouse and reporting systems with modern cloud analytics platforms
• Improving pipeline reliability, data quality and processing performance
• Implementing automated testing, monitoring, documentation and CI/CD practices
• Partnering with analysts, architects, developers and business stakeholders to deliver practical data solutions
• Providing technical guidance, reviewing code and mentoring other data engineer

Skills

SQL

Python

PySpark

Databricks

Apache Spark

Data modeling

Data warehousing

ETL/ELT

CI/CD practices

Tableau/Spotfire

Tools

Tableau

Spotfire

Oracle

Informatica

R

CI/CD tooling

Airflow

Job description

Bitful Consulting in Florida is seeking an experienced Senior Data Engineer to design, build and maintain data pipelines in a Databricks lakehouse environment. You will work with Python, PySpark and SQL to ingest and transform data, create analytics-ready datasets for Tableau and Spotfire, and collaborate with analysts and stakeholders to deliver scalable data solutions.

The role emphasizes data quality, CI/CD practices, and mentoring teammates, with opportunities to influence reporting,