Senior Big Data Engineer — FastAPI & Databricks (Healthcare)

IQVIA; Warszawa; On-site; PLN 189,000 - 350,000; Full time

Job summary

IQVIA is seeking an experienced data engineering expert to design, develop and support scalable data solutions for healthcare datasets. You will profile data, own solution design, and collaborate with a global engineering team to deploy production-grade platforms handling millions of records.

The role emphasizes building robust ETL pipelines, working with Databricks, Spark, and Azure cloud services, and mentoring others in an Agile environment.

Qualifications

• Bachelor's degree in Information Technology, Software Engineering, Computer Science, Mathematics, or a related field.
• 6 years of experience with SQL, Python and Scala.
• 6 years of experience in data engineering.
• Strong understanding of Databricks platform.
• Experience with Big Data batch processing and Azure Cloud Services.
• Proficiency with Apache Spark ecosystem, including Delta and Parquet data formats.
• Experience with complex ETL design and implementation.
• Experience with data analysis, profiling, integration, quality, warehousing, and reporting.
• Experience with CI/CD automation using Jenkins or GitLab.
• Familiarity with AI-powered productivity tools and FastAPI is a plus.

Responsibilities

• Oversee and mentor junior and mid-level engineers, providing technical direction.
• Delegate tasks with clear delivery goals and expectations.
• Own delivery of assigned workstreams with quality and timeliness.
• Build pipelines for processing large amounts of medical data.
• Participate in analysis, system design, development, deployment and maintenance.
• Conduct peer code reviews and ensure high standards.
• Contribute to technical standards and automated tests (unit and integration).
• Collaborate with Architects and QA to ensure quality and alignment.

Skills

SQL

Python

Scala

Data engineering

Databricks

Big data

Azure

Apache Spark

Delta Lake

Parquet

Jenkins/GitLab

Education

Bachelor's Degree in Information Technology / Software Engineering / Computer Science

Tools

Databricks

Delta Lake

Parquet

FastAPI

Job description

IQVIA is seeking an experienced data engineering expert to design, develop and support scalable data solutions for healthcare datasets. You will profile data, own solution design, and collaborate with a global engineering team to deploy production-grade platforms handling millions of records.

The role emphasizes building robust ETL pipelines, working with Databricks, Spark, and Azure cloud services, and mentoring others in an Agile environment.