Senior Big Data Engineer: FastAPI & Databricks Lead

IQVIA; Warszawa; On-site; PLN 189,000 - 350,000; Full time

Benefits offered by this job

Technical training

Conferences access

Global opportunities

Competitive compensation

Job summary

IQVIA is seeking a Senior Big Data Engineer to design, develop, and support data engineering solutions for processing millions of electronic medical records. You will work with a dynamic Systems Engineering team within GIO, applying Agile, DevOps, and TDD practices in a fast-paced environment.

You will lead data pipeline development on Databricks, with SQL, Python, and Scala, and collaborate across architecture, QA, and cross-functional teams to ensure reliable production systems.

Qualifications

• Bachelor’s Degree in Information Technology, Software Engineering, Computer Science, Mathematics, or another related field.
• 6 years of experience in data engineering.
• Strong understanding of Databricks platform.
• Experience with Big Data Batch processing and Azure Cloud Services.
• Practical knowledge of Apache Spark ecosystem, including Delta and Parquet data formats.
• Experience with complex ETL design and implementation.
• Experience with data analysis, data profiling, data integration, data quality, data warehousing, data reporting.
• Experience with CI/CD automation processes and tools such as Jenkins/GitLab.
• Familiarity with AI-powered productivity tools such as Copilot or Cursor.
• Practical experience with FastAPI for building production-grade APIs is a plus.
• Experience working in an Agile environment.
• Knowledge of algorithms & design patterns and how to apply them effectively.
• Experience with large data volume (TBs’) processing.

Responsibilities

• Oversee and mentor junior/mid-level engineers and provide technical direction.
• Delegate tasks with clear expectations aligned to delivery goals.
• Own delivery of assigned workstreams with quality and timeliness.
• Build pipelines for processing large volumes of medical data.
• Participate in analysis, system design, development, deployment and maintenance.
• Conduct peer code reviews and enforce standards.
• Contribute to defining and adopting technical standards.
• Create comprehensive automated unit and integration tests.

Skills

SQL

Python

Scala

Databricks

Apache Spark

Azure Cloud

ETL design

Jenkins/GitLab

FastAPI

Agile/Scrum

Education

Bachelor's Degree in IT / CS / Math or related field

Tools

Jenkins

GitLab

Delta Lake

Parquet

Job description

IQVIA is seeking a Senior Big Data Engineer to design, develop, and support data engineering solutions for processing millions of electronic medical records. You will work with a dynamic Systems Engineering team within GIO, applying Agile, DevOps, and TDD practices in a fast-paced environment.

You will lead data pipeline development on Databricks, with SQL, Python, and Scala, and collaborate across architecture, QA, and cross-functional teams to ensure reliable production systems.