Senior Data Engineer: Build Scalable Pipelines & Dashboards

Bitful Consulting; Tallahassee (FL); Presencial; USD 120 000 - 180 000; Tempo integral

Resumo da oferta

Bitful Consulting seeks an experienced Data Engineer to design and maintain scalable data pipelines in a Databricks lakehouse. You will ingest, validate and transform data from multiple sources using Python, PySpark and SQL, building datasets for Tableau and other BI tools.

The role emphasizes data modeling, warehousing, and collaboration with analysts, architects and stakeholders. Strong background in CI/CD, testing and data quality is essential.

Qualificações

• 7+ years of experience in data engineering, including building and supporting production data pipelines.
• Strong SQL skills and a solid understanding of data modeling, database design and data warehousing
• Hands-on experience with Databricks, Apache Spark and cloud data platforms
• Strong proficiency in Python and PySpark
• Experience developing ETL/ELT pipelines and integrating data from multiple source systems
• Experience preparing reliable datasets for Tableau or comparable business intelligence tools
• Familiarity with source control, automated testing and CI/CD practices
• Ability to troubleshoot complex data issues and optimize pipeline performance
• Strong communication skills and experience collaborating with technical teams and business stakeholders
• Experience with Spotfire, Oracle, Informatica, R or data virtualization preferred

Responsabilidades

• Develop and maintain scalable data pipelines and reusable datasets within a Databricks lakehouse environment
• Ingest, validate, clean and transform structured and unstructured data from multiple sources using Python, PySpark and SQL
• Build data models and reporting datasets that support Tableau and established analytics tools
• Integrate established data warehouse and reporting systems with modern cloud analytics platforms
• Improve pipeline reliability, data quality and processing performance
• Implement automated testing, monitoring, documentation and CI/CD practices
• Partner with analysts, architects, developers and business stakeholders to deliver practical data solutions
• Provide technical guidance, reviewing code and mentoring other data engineer

Conhecimentos

SQL

Python

PySpark

Databricks

ETL/ELT pipelines

Data modeling

Data warehousing

CI/CD practices

Testing automation

Communication

Spotfire

Oracle

Informatica

R

Ferramentas

Spotfire

Oracle

Informatica

R

Descrição da oferta de emprego

Bitful Consulting seeks an experienced Data Engineer to design and maintain scalable data pipelines in a Databricks lakehouse. You will ingest, validate and transform data from multiple sources using Python, PySpark and SQL, building datasets for Tableau and other BI tools.

The role emphasizes data modeling, warehousing, and collaboration with analysts, architects and stakeholders. Strong background in CI/CD, testing and data quality is essential.