Principal Data Engineer

InCommon; Bengaluru; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

InCommon is hiring a Principal Data Engineer for a US-based healthtech company. You will drive end-to-end data architecture, build scalable pipelines, and collaborate with Product, Sales, and Clinical Operations to rewire healthcare data ecosystems.

You will lead data infrastructure design, ensure data quality, and empower teams with data tools while balancing speed and technical debt in a fast-growing setup.

Qualifications

• 7+ years experience as a Data Engineer with quantitative degree.
• Entrepreneurial, mission-driven and able to present ideas clearly.
• Advanced SQL and relational DB expertise.
• Experience building and optimizing large data pipelines and data sets, healthcare experience a plus.
• Root cause analysis on data/processes to drive improvements.
• Strong analytics with unstructured data, and data transformation.
• Proficient in data structures, metadata, dependencies, and workloads.
• Experience with messaging/streaming and scalable data stores.
• Excellent cross-functional collaboration in dynamic environments.
• Nice-to-have: Snowflake, NoSQL, Python/Java/Golang, Spark, Kafka, Airflow.

Responsibilities

• Be a data champion empowering others to leverage data.
• Understand the complex data ecosystem and translate requirements into a platform roadmap.
• Lead design and implementation of scalable data infrastructure and data-driven products.
• Provide technical leadership and establish best practices.
• Define and maintain data pipeline architectures with observability and reliability.
• Assemble large, complex datasets meeting functional and non-functional requirements.
• Build ETL/ELT pipelines for diverse data sources using SQL.
• Collaborate with executives, product, clinical, data, and design teams to support data needs.
• Create data tools for analytics and data scientist teams to drive product innovation.

Skills

7+ years Data Engineer

Advanced SQL

Big data pipelines

Root cause analysis

Unstructured data handling

Data transformation

Metadata management

Workload management

Data architecture

Data integration

Cross-functional collaboration

Cloud data warehouse (Snowflake)

SQL/NoSQL databases

Golang

Python

Java

C++

Scala

Spark

Kafka

Airflow

Education

Graduate degree in Computer Science/Statistics/Informatics/Information Systems

Tools

Snowflake

Relational SQL databases

NoSQL databases

Airflow

Spark

Kafka

Job description

InCommon is hiring for a US based healthtech company on a mission to provide dignified, high quality care to patients with complex health needs. The company delivers primary, behavioral, and social care through a combination of at-home and virtual care, supported by 24/7 access to dedicated care teams. Powered by technology, evidence-based care models, and strong partnerships across the healthcare ecosystem, the company is focused on improving health outcomes and addressing long standing gaps in care.

About the role:

As a Principal Data Engineer, you will be:

• A mission-critical, early part of our growing team (and company)
• Collaborate across the organization with various teams, including Product, Sales, and Clinical Operations, to rewire health care: Your choices will matter, and you will work across various teams to help execute the choices.
• Drive the change in healthcare: You will build the products to integrate highly fragmented and dispersed healthcare services as an end-to-end experience.
• Be part of building a great engineering culture, maintaining the balance and right tradeoff for building products for speed and tech debt.

Desired skills and experience:

Required:

• 7+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
• You are entrepreneurial and mission-driven and can present your ideas with clarity and confidence clarity and confidence
• Advanced working SQL knowledge and experience working with relational databases
• Experience building and optimizing 'big data' data pipelines, architectures, and data sets. A definite plus with healthcare experience
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
• Strong analytic skills related to working with unstructured datasets
• Build processes supporting data transformation, data structures, metadata, dependency, and workload management
• A successful history of manipulating, processing, and extracting value from large disconnected datasets
• Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores
• Strong project management and organizational skills
• Experience supporting and working with cross-functional teams in a dynamic environment
• The following is a list of software/tools that would be nice to have, but not required:
• Experience with cloud-based data warehouse: Snowflake
• Experience with relational SQL and NoSQL databases
• Experience with object-oriented/object function scripting languages: Golang, Python, Java, C++, Scala, etc.
• Experience with big data tools: Spark, Kafka, etc.
• Experience with data pipeline and workflow management tools like Airflow

Responsibilities will include:

• Be a data champion and seek to empower others to leverage the data to its full potential
• Understand our complex data ecosystem
• Work with the product team and stakeholders to translate business requirements for data across the company into a technical roadmap and architecture for the platform
• Act as the leading data domain expert and owns platform data architecture
• Lead the technical design and implementation of reliable, scalable, and efficient data infrastructure, data-driven products, and software solutions for external and internal customers
• Provide technical leadership to define overall data engineering best practices, standards, and architectural approaches and drive technical excellence. Identify design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Create and maintain optimal data pipeline architecture with high observability and robust operational characteristics
• Assemble large, complex data sets that meet functional / non-functional business requirements
• Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL
• Work with stakeholders, including the Executive, Product, Clinical, Data, and Design teams, to assist with data-related technical issues and support their data infrastructure needs.
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader