Senior Data Engineer

Bitful Consulting; Lorida (FL); Presencial; USD 130.000 - 180.000; Jornada completa

Descripción de la vacante

Bitful Consulting is seeking a senior data engineer to design and maintain scalable data pipelines in a Databricks lakehouse environment across finance, sales, and marketing. You will ingest and transform data using Python, PySpark and SQL, and build data models for BI tools like Tableau and Spotfire.

You will collaborate with analysts and stakeholders, ensure data quality, implement CI/CD practices, and mentor junior engineers as part of a fast-moving data platform team based in Florida.

Formación

• 7+ years of experience building production data pipelines.
• Strong SQL skills and data modeling/warehousing knowledge.
• Hands-on with Databricks, Spark, and cloud data platforms.
• Proficiency in Python and PySpark for ETL/ELT.
• Experience preparing datasets for Tableau or BI tools.

Responsabilidades

• Develop scalable data pipelines and reusable datasets in a Databricks lakehouse.
• Ingest, validate, clean, and transform data from multiple sources using Python, PySpark and SQL.
• Build data models and reporting datasets for Tableau and Spotfire.
• Integrate data warehouse/reporting with cloud analytics platforms.
• Improve pipeline reliability, data quality, and performance.
• Implement automated testing, monitoring, documentation, and CI/CD practices.
• Collaborate with analysts, architects, developers, and stakeholders to deliver data solutions.
• Provide technical guidance, review code, and mentor other data engineers.

Conocimientos

SQL

Python

PySpark

ETL/ELT pipelines

Data modeling

Data warehousing

CI/CD practices

Troubleshooting data issues

Communication with stakeholders

Herramientas

Databricks

Apache Spark

Tableau

Spotfire

Informatica

R

Data virtualization

Oracle

Descripción del empleo

• You’ll build and maintain data pipelines and datasets that support reporting, visualization, machine learning and data-driven decision making across finance, sales, supply chain, marketing and other business functions.
• Your responsibilities will include:
  • Developing and maintaining scalable data pipelines and reusable datasets within a Databricks lakehouse environment
  • Ingesting, validating, cleaning and transforming structured and unstructured data from multiple sources using Python, PySpark and SQL
  • Building data models and reporting datasets that support Tableau and established analytics tools such as Spotfire
  • Integrating established data warehouse and reporting systems with modern cloud analytics platforms
  • Improving pipeline reliability, data quality and processing performance
  • Implementing automated testing, monitoring, documentation and CI/CD practices
  • Partnering with analysts, architects, developers and business stakeholders to deliver practical data solutions
  • Providing technical guidance, reviewing code and mentoring other data engineer

Requirements

• 7+ years of experience in data engineering, including building and supporting production data pipelines
• Strong SQL skills and a solid understanding of data modeling, database design and data warehousing
• Hands-on experience with Databricks, Apache Spark and cloud data platforms
• Strong proficiency in Python and PySpark
• Experience developing ETL/ELT pipelines and integrating data from multiple source systems
• Experience preparing reliable datasets for Tableau or comparable business intelligence tools
• Familiarity with source control, automated testing and CI/CD practices
• Ability to troubleshoot complex data issues and optimize pipeline performance
• Strong communication skills and experience collaborating with technical teams and business stakeholders
• Experience with Spotfire, Oracle, Informatica, R or data virtualization preferred
• Databricks certification preferred