Senior/Lead Data Engineer

TechDigital Group; California (MO); Hybrid; USD 140,000 - 190,000; Full time

Job summary

TechDigital Group is seeking a hands-on Senior/Lead Data Engineer with 8+ years of experience to design, build, and support scalable cloud data solutions for healthcare reporting, analytics, and operational use cases.

The role requires strong expertise in Snowflake, Informatica IDMC, ETL/ELT, dimensional data warehousing, Python, CI/CD, and AWS or Databricks, with technical ownership expected at the Lead level.

Qualifications

• 8+ years in data engineering, ETL development, and enterprise data warehousing.
• Strong Snowflake expertise in SQL, data loading, security, performance tuning, scalability, and cost optimization.
• Strong IDMC development experience with CDI mappings, tasks, taskflows, parameters, deployment, and monitoring.
• Proficiency in Python and SQL, plus Git-based CI/CD using Jenkins, Azure DevOps, GitHub Actions, or similar tools.
• Hands-on experience with AWS data services and/or Databricks/Spark, including data quality, governance, and production support.
• Strong analytical, troubleshooting, communication, documentation, and stakeholder-management skills.

Responsibilities

• Design, develop, test, deploy, and support scalable batch and near-real-time data pipelines using Snowflake and IDMC.
• Implement ETL/ELT solutions, dimensional models, and SCD Types 1 and 2 for enterprise DW.
• Develop SQL and Python components, reusable frameworks, automation, and validation utilities.
• Build CI/CD pipelines for data code, IDMC assets, Snowflake objects, testing, and environment promotion.
• Optimize Snowflake performance, scalability, reliability, security, and cost across AWS and/or Databricks workloads.
• Implement data quality, lineage, governance, monitoring, and audit controls.
• Troubleshoot production issues and collaborate with architecture, QA, DevOps, analytics, and business teams.
• Lead design and code reviews, mentor engineers, and drive engineering standards and ownership.

Skills

Snowflake SQL

Python

ETL/ELT design

CI/CD pipelines

AWS / Databricks

Data warehousing

Analytical troubleshooting

Stakeholder management

Education

Bachelor's degree

Tools

Informatica IDMC

Jenkins

Azure DevOps

GitHub Actions

AWS

Databricks

Job description

Position Summary

Seeking a hands-on Senior/Lead Data Engineer with 8+ years of experience to design, build, and support scalable cloud data solutions for healthcare reporting, analytics, and operational use cases. The role requires strong expertise in Snowflake, Informatica IDMC, ETL/ELT, dimensional data warehousing, Python, CI/CD, and AWS or Databricks, with technical ownership expected at the Lead level.

Key Responsibilities

• Design, develop, test, deploy, and support scalable batch and near-real-time data pipelines using Snowflake and IDMC.


• Implement ETL/ELT solutions, dimensional models, and SCD Types 1 and 2 for enterprise DW.


• Develop efficient SQL and Python components, reusable frameworks, automation, and validation utilities.


• Build CI/CD pipelines for data code, IDMC assets, Snowflake objects, testing, and environment promotion.


• Optimize Snowflake performance, scalability, reliability, security, and cost; support AWS and/or Databricks workloads.


• Implement data quality, reconciliation, lineage, monitoring, governance, and audit controls.


• Troubleshoot production issues and partner with architecture, QA, DevOps, analytics, and business teams.


• Lead design and code reviews, mentor engineers, and drive engineering standards and solution ownership.

Required Qualifications

• 8+ years in data engineering, ETL development, and enterprise data warehousing.


• Strong Snowflake expertise in SQL, data loading, security, performance tuning, scalability, and cost optimization.


• Strong IDMC development experience with CDI mappings, tasks, taskflows, parameters, deployment, and monitoring.


• Proficiency in Python and SQL, plus Git-based CI/CD using Jenkins, Azure DevOps, GitHub Actions, or similar tools.


• Hands-on experience with AWS data services and/or Databricks/Spark, including data quality, governance, and production support.


• Strong analytical, troubleshooting, communication, documentation, and stakeholder-management skills.

Preferred Qualifications

• Bachelor's degree in a relevant field or equivalent experience.


• Healthcare data experience across claims, eligibility, provider, clinical, or pharmacy domains.


• Exposure to Snowpark, PySpark, AWS Glue/S3/Lambda, streaming, or automated data testing.


• Snowflake, Informatica, AWS, or Databricks certification is a plus.