Data Architect (Microsoft Fabric & Azure Databricks)

Padmore Global Connections LLC; Atlanta (GA); Hybrid; USD 130,000 - 190,000; Part time

Job summary

Padmore Global Connections LLC is seeking an experienced Data Architect to design and implement enterprise data solutions with Microsoft Fabric and Azure Databricks for integration with state systems.

The role focuses on scalable data architecture, Delta Lake data flows, data governance with Purview, and real-time ETL/ELT across the analytics platform.

Qualifications

• 6+ years of experience in data architecture and engineering.
• 2+ years hands-on experience with Azure Databricks and Spark.
• Recent experience with Microsoft Fabric platform.

Responsibilities

• Design end-to-end data architecture leveraging Microsoft Fabric's capabilities.
• Design data flows within the Microsoft Fabric environment.
• Implement OneLake storage strategies.
• Configure Synapse Analytics workspaces.

Skills

Azure Databricks

Microsoft Fabric

Data Architecture

Data Integration

Delta Lake

Purview

Synapse Analytics

Security & Compliance

Education

Bachelor's degree in Computer Science or related field

Tools

Microsoft Fabric

Azure Synapse

Delta Lake

Job description

About the job Data Architect (Microsoft Fabric & Azure Databricks)

Interview Type: Either Web Cam or In Person

Work Arrangement: Hybrid

Engagement Type: Contract

Short Description:

Supervises the coordination of design and security for computer databases to store, track, and maintain a large volume of critical business information.

Complete Description

The Department of Early Care & Learning (DECAL) is seeking an experienced Data Architect to design and implement enterprise data solutions using Microsoft Fabric and Azure Databricks for integration with state-level systems. This role will focus on creating scalable data architecture that enables seamless data flow between IES Gateway and our analytics platform. The ideal candidate will have deep expertise in modern data architecture, with specific experience in Microsoft's data platform and Delta Lake architecture.

Must be physically located in Georgia

On-site: Tuesday to Thursday, per manager's discretion

Mandatory in-person meetings:

• All Hands
• Enterprise Applications
• DECAL All Staff

Work arrangements subject to management's decision

While the intent may be a long-term tenure, this position is subject to annual budget restrictions. The initial contract is through the end of this fiscal year and is anticipated to be renewed July 1st.

Key Responsibilities

• Design end-to-end data architecture leveraging Microsoft Fabric's capabilities.
• Design data flows within the Microsoft Fabric environment.
• Implement OneLake storage strategies.
• Configure Synapse Analytics workspaces.

Integration Design

• Architect data integration patterns between IES Gateway and the analytics platform using Azure Databricks and Microsoft Fabric.
• Design Delta Lake architecture for IES Gateway data.
• Implement medallion architecture (Bronze/Silver/Gold layers).
• Establish data quality frameworks.

Lakehouse Architecture

• Implement modern data lakehouse architecture using Delta Lake, ensuring data reliability and performance.

Data Governance

• Establish data governance frameworks incorporating Microsoft Purview for data quality, lineage, and compliance.
• Configure Microsoft Purview policies.
• Establish data masking for sensitive information.
• Design scalable data pipelines using Azure Databricks for ETL/ELT processes and real-time data integration.

Performance Optimization

• Implement performance tuning strategies for large-scale data processing and analytics workloads.
• Implement partitioning strategies.
• Design caching mechanisms.

Security Framework

Design and implement security patterns aligned with federal and state requirements for sensitive data handling.

Required Qualifications

Education

Bachelors degree in computer science or related field.

Experience

• 6+ years of experience in data architecture and engineering.
• 2+ years hands-on experience with Azure Databricks and Spark.
• Recent experience with Microsoft Fabric platform.

Technical Skills

Microsoft Fabric Expertise

• Data Integration: Combining and cleansing data from various sources.
• Data Pipeline Management: Creating, orchestrating, and troubleshooting data pipelines.
• Analytics Reporting: Building and delivering detailed reports and dashboards to derive meaningful insights from large datasets.
• Data Visualization Techniques: Representing data graphically in impactful and informative ways.