Senior Data Engineer

Compunnel Inc.; Eden Prairie (MN); On-site; USD 130,000 - 180,000; Full time

Job summary

Compunnel Inc. seeks a Senior Data Engineer to design and implement scalable data architectures using SQL, Snowflake, Databricks, and Azure Data Factory. The role includes QA and business process analysis to ensure accurate data delivery and optimized workflows.

The candidate will collaborate with analytics, product, and business teams to support data integration and advanced reporting, while upholding governance and compliance standards.

Qualifications

• Advanced SQL proficiency with complex queries and performance tuning.
• Hands-on experience with Snowflake, Databricks, and Azure Data Factory.
• Strong governance and data quality QA processes.

Responsibilities

• Design scalable, secure data architectures and models.
• Build and optimize ETL/ELT pipelines with ADF and Databricks.
• Enhance storage and retrieval in Snowflake and relational databases.
• Collaborate with analytics, product, and business teams on data integration and reporting.
• Perform data validation, reconciliation, and regression testing for pipelines.
• Develop automated QA frameworks for data processing.

Skills

SQL

Snowflake

Databricks

Azure Data Factory

ETL/ELT

Data governance

QA frameworks

Business process analysis

Data modeling

Analytical thinking

Communication

Documentation

Healthcare domain knowledge

Python

Power BI

Tableau

Tools

Power BI

Tableau

Job description

The Senior Data Engineer will design, build, and optimize scalable data solutions leveraging SQL, Snowflake, Databricks, and Azure Data Factory (ADF).

In addition to data engineering, this role includes responsibilities in Quality Engineering and Business Process Analysis to ensure accurate data delivery, optimized workflows, and adherence to enterprise governance.

The ideal candidate possesses a blend of technical expertise, analytical thinking, and cross-functional collaboration skills.

Key Responsibilities

• Design and implement scalable and secure data architectures and data models.
• Build, maintain, and optimize ETL/ELT pipelines using Azure Data Factory (ADF) and Databricks.
• Enhance data storage and retrieval efficiency across Snowflake and relational databases.
• Ensure adherence to data governance, security, and compliance standards.
• Collaborate with analytics, product, and business teams to support data integration and advanced reporting.

Quality Engineering

• Perform data validation, reconciliation, and regression testing for ETL/ELT pipelines.
• Develop automated QA frameworks for continuous testing of data processes.
• Enforce data quality standards and ensure compliance with enterprise and regulatory requirements.

Business Process Analysis

• Analyze and document existing business processes to identify optimization opportunities.
• Develop standard operating procedures (SOPs) for data workflows and operations.
• Work with stakeholders to implement automation initiatives and process efficiency improvements.

Required Qualifications

Technical Expertise

• Advanced proficiency in SQL, including complex queries and performance tuning.
• Hands-on experience with Snowflake, Databricks, and Azure Data Factory (ADF).

Additional Skills

• Strong understanding of data governance, data quality frameworks, and QA processes.
• Experience conducting business process analysis and documentation.
• Excellent analytical thinking and communication skills.

Preferred Qualifications

• Experience in the healthcare domain.
• Familiarity with cloud-native architectures and Agile methodologies.
• Knowledge of Python and data visualization tools (Power BI or Tableau).

Certifications

• Relevant certifications such as Microsoft DP-203 or Azure Databricks (preferred).