Senior Data Engineer

InfoStride; Raleigh (NC); Hybrid; USD 120,000 - 180,000; Full time

Job summary

NC DHHS is seeking a Senior Data Analyst/Engineer to design, manage, and optimize enterprise data pipelines, data warehouses, and analytics solutions in Microsoft Azure. You will transform complex data into business insights and drive advanced analytics using Azure Synapse Analytics, Microsoft Fabric, and Azure AI services.

The role involves designing ETL/ELT pipelines, optimizing Synapse/Spark pools, writing complex SQL/PLSQL, and building dashboards and predictive models while enforcing data

Qualifications

• Experience in enterprise data analytics, engineering, and designing complex ETL workflows.
• Expertise in Azure Synapse Analytics, Azure Data Factory, and cloud data warehousing.
• Strong SQL and PL/SQL skills including optimization and tuning.
• Hands-on PySpark, Python or Scala experience.
• Knowledge of RBAC and cloud security.

Responsibilities

• Design, build, and maintain robust enterprise ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines, or PL/SQL procedures.
• Manage and optimize Azure Synapse Analytics (SQL and Spark pools) and Microsoft Fabric Lakehouse.
• Develop, test, and tune complex SQL and PL/SQL scripts, stored procedures, packages, and functions for data integration.
• Collaborate with business users to define analytical goals and build Power BI-ready datasets.
• Integrate AI/ML insights using Azure ML and Azure OpenAI Service.
• Enforce data governance, quality, and security via Microsoft Purview and Entra ID.
• Build cross-functional reporting, dashboards, and predictive analytics models

Skills

Azure Synapse Analytics

Azure Data Factory

SQL

PL/SQL

PySpark/Python/Scala

RBAC/Security

Power BI data models

Agile methodologies

Data analytics

ETL workflows

Tools

Azure Synapse Analytics

Synapse Pipelines

Job description

Work Arrangements: Hybrid

Interview Type: In Person Only

Overview:

NC DHHS requires the services of a Senior Data Analyst / Engineer that will help us design, manage, and optimize enterprise data pipelines, data warehouses, and analytics solutions within Microsoft Azure. This role transforms complex data into business insights and drives advanced analytics using Azure Synapse Analytics, Microsoft Fabric, and Azure AI services.

The key responsibilities for this position will include the following:

• Design, build, and maintain robust enterprise ETL/ELT pipelines using Azure Data Factory, Synapse Pipelines, or PL/SQL procedures.
• Manage and optimize Azure Synapse Analytics (SQL and Spark pools) and Microsoft Fabric Lakehouse.
• Develop, test, and tune complex SQL and PL/SQL scripts, stored procedures, packages, and functions to support data integration.
• Collaborate with business users to define analytical goals and build Power BI-ready datasets.
• Integrate AI and machine learning insights using Azure Machine Learning and Azure OpenAI Service.
• Enforce data governance, quality, and security via Microsoft Purview and Microsoft Entra ID.
• Build cross-functional reporting, dashboards, and predictive analytics models

Skill

• Experience in enterprise data analytics, engineering, and designing complex ETL workflows.
• Expertise in Azure Synapse Analytics, Azure Data Factory, and cloud data warehousing
• Advanced SQL and PL/SQL skills, including query optimization, performance tuning, and indexing strategies
• Hands-on experience with PySpark, Python, or Scala
• Experience with data modeling, performance tuning, and cost optimization in Azure
• Strong knowledge of Microsoft Entra ID, role-based access control (RBAC), and cloud security
• Familiarity with Power BI semantic models and dataflows
• Knowledge of Azure OpenAI or generative AI data integration
• Ability to organize, prioritize, and follow complex and/or detailed technical procedures
• Experience with Agile (Scrum, Kanban, or SAFe) development