Data Engineer, Databricks

Indegene; Raleigh (NC); On-site; USD 120,000 - 180,000; Full time

Job summary

Indegene in Raleigh, NC is seeking a Data Engineer to design, develop, and optimize large-scale data pipelines and analytics solutions in Life Sciences. This role requires deep Databricks, AWS, and PySpark expertise to support commercial operations.

You will partner with Commercial Analytics, Data Science, IT and business stakeholders to deliver scalable data products for sales insights, targeting, incentives, forecasting, and field reporting. HIPAA and SOC2 familiarity preferred.

Qualifications

• 5–10+ years of experience in Data Engineering or Big Data Analytics.
• Hands-on experience with Databricks, Genie, Delta Lake and MLflow.
• Proficiency in PySpark, Python, SQL and distributed computing.
• Deep experience with AWS services (S3, Glue, Lambda, EMR, Redshift, IAM).
• Experience with Life Sciences Commercial datasets.

Responsibilities

• Design, develop, and optimize large-scale data pipelines.
• Partner with Commercial Analytics, Data Science, IT, and Business teams to deliver scalable data solutions.
• Ingest, harmonize, and model Life Sciences datasets (Claims, Specialty Pharmacy).
• Build commercial data marts for sales reporting, targeting, incentives, forecasting, and field reporting.
• Ensure data quality, lineage, governance, and compliance with HIPAA, SOC2.

Skills

PySpark

Python

SQL

Distributed computing

CI/CD

Git

DevOps

Data modeling

ETL/ELT

Tools

Databricks

Genie

Delta Lake

MLflow

AWS

S3

Glue

Lambda

EMR

Redshift

Job description

We are a technology-led healthcare solutions provider. We are driven by our purpose to enable healthcare organizations to be future-ready. We offer accelerated, global growth opportunities for talent that is bold, industrious and nimble. With Indegene, you gain a unique career experience that celebrates entrepreneurship and is guided by passion, innovation, collaboration and empathy. To explore exciting opportunities at the convergence of healthcare and technology, check out www.careers.indegene.com

What if we told you that you can move to an exciting role in an entrepreneurial organization without the usual risks associated with it?

We understand that you are looking for growth and variety in your career at this point and we would love for you to join us in our journey and grow with us. At Indegene, our roles come with the excitement you require at this stage of your career with the reliability you seek. We hire the best and trust them from day 1 to deliver global impact, handle teams and be responsible for the outcomes while our leaders support and mentor you.

We are a profitable rapidly growing global organization and are scouting for the best talent for this phase of growth. With us, you are at the intersection of two of the most exciting industries of healthcare and technology. We offer global opportunities with fast-track careers while you work with a team that is fueled by purpose. The combination of these will lead to a truly differentiated experience for you.

Position Overview

The Data Engineer will be responsible for designing, developing, and optimizing large‑scale data pipelines and analytical solutions supporting Life Sciences Commercial Operations. This role requires deep expertise in Databricks (including Genie), AWS cloud services, and PySpark, with strong domain knowledge of pharmaceutical/biotech commercial datasets such as IQVIA, Symphony, DDD, NPA, Xponent, Claims, Specialty Pharmacy, and CRM/Sales data.

The ideal candidate will partner closely with Commercial Analytics, Data Science, and Business Stakeholders to deliver high‑quality, scalable data products that enable sales insights, targeting, forecasting, incentive compensation, and field performance reporting.

Key Responsibilities

• Databricks Development
• Build, optimize, and maintain data pipelines using Databricks, Delta Lake, and Genie‑powered workflows.
• Develop notebooks, jobs, workflows, and ML‑ready datasets within the Databricks environment.
• Design and manage data ingestion, storage, and processing using AWS services such as S3, Glue, Lambda, EMR, Redshift, and Athena.
• Implement secure, scalable architectures following AWS best practices (IAM, VPC, encryption, monitoring).
• Build distributed data transformation pipelines using PySpark for high‑volume commercial datasets.
• Optimize PySpark jobs for performance, cost efficiency, and reliability.
• Implement unit testing, CI/CD, and code versioning using Git‑based workflows.
• Life Sciences Commercial Data Expertise
• Ingest, harmonize, and model datasets including:
• Claims (medical, pharmacy)
• Specialty Pharmacy data feeds
• Build commercial data marts supporting:
• Sales reporting and dashboards
• Targeting and segmentation
• Incentive compensation
• Forecasting and demand analytics
• Field performance insights
• Cross‑Functional Collaboration
• Work with Commercial Analytics, Data Science, IT, and Business teams to translate requirements into scalable data solutions.
• Ensure data quality, lineage, governance, and compliance with industry standards (HIPAA, GxP, SOC2).

Required Qualifications

• 5–10+ years of experience in Data Engineering or Big Data Analytics.
• Strong hands‑on experience with Databricks, Genie, Delta Lake, and MLflow.
• Advanced proficiency in PySpark, Python, SQL, and distributed computing.
• Deep experience with AWS (S3, Glue, Lambda, EMR, Redshift, IAM).
• Proven experience working with Life Sciences Commercial datasets.
• Strong understanding of data modeling, ETL/ELT, and cloud architecture.
• Experience with CI/CD, Git, DevOps, and automated workflow orchestration.

Preferred Qualifications

• Experience with Databricks Unity Catalog and Lakehouse architecture.
• Familiarity with LLM‑based automation or AI‑assisted analytics.
• Experience supporting Commercial Operations, Sales Leadership, and Field Teams.
• Knowledge of incentive compensation methodologies and targeting algorithms.
• Experience with BI tools (Tableau, Power BI, Qlik).
• Strong analytical and problem‑solving skills.
• Excellent communication and documentation abilities.
• Ability to work in fast‑paced, cross‑functional environments.
• High attention to detail and commitment to data accuracy.

EQUAL OPPORTUNITY

Indegene is proud to be an Equal Employment Employer and is committed to the culture of Inclusion and Diversity. We do not discriminate on the basis of race, religion, sex, colour, age, national origin, pregnancy, sexual orientation, physical ability, or any other characteristics. All employment decisions, from hiring to separation, will be based on business requirements, candidate’s merit and qualification.

We are an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, colour, religion, sex, national origin, gender identity, sexual orientation, disability status, protected veteran status, or any other characteristics.