Azure Data Engineer

Novacruitix; Minnesota; Vor Ort; USD 120.000 - 160.000; Vollzeit

Zusammenfassung

Novacruitix is seeking a data engineering expert in Minnesota to design and optimize scalable data pipelines using Databricks, Azure Data Factory, and PySpark. Build AI-driven insights with code and low-code tools, while modeling data with star/snowflake schemas and open formats like Delta Lake and Iceberg.

Collaborate with stakeholders to deliver performant data products, leveraging Azure cloud services. Leadership and agile delivery are highly valued in this role.

Qualifikationen

• Proficiency in Snowflake data modeling, querying, and performance optimization.
• Strong experience in Azure Data Factory for data integration and orchestration.
• Hands-on knowledge of Databricks for data engineering, processing, and ML.
• Ability to design, develop, and optimize complex ETL data pipelines.
• Strong problem-solving skills and ability to troubleshoot data pipelines.
• Excellent communication to interact with technical and non-technical stakeholders.
• Strong leadership to guide technical teams towards project delivery.
• Experience in Agile methodologies and project management practices.

Aufgaben

• Lead design and optimization of end-to-end data pipelines.
• Collaborate with data scientists and analysts to deliver reliable data products.
• Implement data models using Star/Snowflake schemas and modern open formats.
• Develop notebooks and jobs in Databricks to support ETL/ELT processes.
• Mentor team members and manage agile project milestones.

Kenntnisse

Leadership

Communication

Agile methodology

Project management

Problem solving

Tools

Snowflake

Azure Data Factory

Databricks

PySpark

Python

Databricks SQL

GitHub Copilot

Delta Lake

Iceberg

Databricks Genie

Jobbeschreibung

Job Summary

Databricks, Azure Data Factory, Databricks workflows, PySpark, Python, Databricks SQLConfig driven data pipelinesDatabricks Genie conversational tool for AI driven insights using no code/low-code and code completion options using GitHub CopilotData Modeling: Star Schema and Snowflake SchemaOpen table formats: Delta Lake and IcebergCloud technologies: Azure is preferredData virtualization schemes

Key Responsibilities

Databricks, Azure Data Factory, Databricks workflows, PySpark, Python, Databricks SQLConfig driven data pipelinesDatabricks Genie conversational tool for AI driven insights using no code/low-code and code completion options using GitHub CopilotData Modeling: Star Schema and Snowflake SchemaOpen table formats: Delta Lake and IcebergCloud technologies: Azure is preferredData virtualization schemes

Skill Requirements

1. 1. Proficiency in snowflake, including data modeling, querying, and performance optimization.
2. 2. Strong experience in azure data factory (adf) for data integration and orchestration.
3. 3. Handson knowledge of data bricks for data engineering, data processing, and machine learning.
4. 4. Ability to design, develop, and optimize complex data pipelines for etl processes.
5. 5. Strong problem-solving skills and the ability to troubleshoot data pipeline issues.
6. 6. Excellent communication skills to interact with technical and nontechnical stakeholders effectively.
7. 7. Strong leadership skills to guide and motivate technical teams towards project delivery and success.
8. 8. Experience in agile methodologies and project management practices for efficient project execution.

Other Requirements

• 1.Relevant certifications in Snowflake, Azure Data Factory (ADF), DataBricks are a plus.