Senior Data Engineer: Pipelines & Lakehouse Expert

Bitful Consulting; Lorida (FL); On-site; USD 130,000 - 180,000; Full time

Job summary

Bitful Consulting is seeking a senior data engineer to design and maintain scalable data pipelines in a Databricks lakehouse environment across finance, sales, and marketing. You will ingest and transform data using Python, PySpark and SQL, and build data models for BI tools like Tableau and Spotfire.

You will collaborate with analysts and stakeholders, ensure data quality, implement CI/CD practices, and mentor junior engineers as part of a fast-moving data platform team based in Florida.

Qualifications

• 7+ years of experience building production data pipelines.
• Strong SQL skills and data modeling/warehousing knowledge.
• Hands-on with Databricks, Spark, and cloud data platforms.
• Proficiency in Python and PySpark for ETL/ELT.
• Experience preparing datasets for Tableau or BI tools.

Responsibilities

• Develop scalable data pipelines and reusable datasets in a Databricks lakehouse.
• Ingest, validate, clean, and transform data from multiple sources using Python, PySpark and SQL.
• Build data models and reporting datasets for Tableau and Spotfire.
• Integrate data warehouse/reporting with cloud analytics platforms.
• Improve pipeline reliability, data quality, and performance.
• Implement automated testing, monitoring, documentation, and CI/CD practices.
• Collaborate with analysts, architects, developers, and stakeholders to deliver data solutions.
• Provide technical guidance, review code, and mentor other data engineers.

Skills

SQL

Python

PySpark

ETL/ELT pipelines

Data modeling

Data warehousing

CI/CD practices

Troubleshooting data issues

Communication with stakeholders

Tools

Databricks

Apache Spark

Tableau

Spotfire

Informatica

R

Data virtualization

Oracle

Job description

Bitful Consulting is seeking a senior data engineer to design and maintain scalable data pipelines in a Databricks lakehouse environment across finance, sales, and marketing. You will ingest and transform data using Python, PySpark and SQL, and build data models for BI tools like Tableau and Spotfire.

You will collaborate with analysts and stakeholders, ensure data quality, implement CI/CD practices, and mentor junior engineers as part of a fast-moving data platform team based in Florida.