Azure Data Engineer: Scalable Pipelines, BI & AI Enablement

Jobtome; Washington (District of Columbia); On-site; USD 110,000 - 170,000; Full time

Job summary

Jobtome is seeking an experienced Azure Data Engineer to design and implement scalable data pipelines using Azure Databricks, Fabric, and related services. You will transform data with Python and PySpark, ensure data quality, and support analytics and reporting across multiple data sources.

You will collaborate with data analysts and business stakeholders to standardize data models and enable BI solutions with Power BI, leveraging Cosmos DB and AI-enabled features where appropriate.

Qualifications

• Experience in designing data ingestion pipelines using Azure data services.
• Strong Python, PySpark skills for large-scale data processing.
• Proficiency in SQL/PLSQL for data transformation and validation.
• Experience with data governance, quality checks, and data lineage.
• Proven ability to implement ETL/ELT pipelines and data models.
• Experience with BI tools, especially Power BI, for analytics.

Responsibilities

• Design, develop, and maintain scalable data ingestion pipelines using Microsoft Azure and Azure Databricks.
• Perform data transformation, cleansing, validation, and schema normalization using Python, PySpark, and SQL.
• Ingest and standardize internal and external partner data, ensuring accurate SKU, partner, and channel mapping.
• Implement data quality checks, reconciliation processes, and validation rules across datasets.
• Monitor data pipelines, troubleshoot failures, and perform exception handling and root cause analysis.
• Manage data refresh schedules, pipeline orchestration, and operational monitoring workflows.
• Support onboarding of new data feeds and integrations across enterprise systems.
• Develop standardized data models to support analytics, reporting, and business intelligence requirements.
• Work with Microsoft Fabric and Power BI to support data analytics and reporting solutions.
• Utilize Azure Cosmos DB and related data services where applicable.
• Implement data governance, security, compliance, data lineage, auditability, and traceability standards.
• Support inventory data pipelines and deliver reliable, high-quality datasets for business reporting.
• Collaborate with data analysts, business stakeholders, and cross-functional teams to understand data requirements.
• Explore Microsoft Foundry capabilities to support AI-enabled data solutions where required.

Skills

Azure data services

Python

PySpark

SQL/PLSQL

Power BI

Data governance

Data modeling

ETL/ELT

Troubleshooting

Tools

Azure Databricks

Microsoft Fabric

Power BI

Cosmos DB

Microsoft Foundry

Power Platform

Job description

Jobtome is seeking an experienced Azure Data Engineer to design and implement scalable data pipelines using Azure Databricks, Fabric, and related services. You will transform data with Python and PySpark, ensure data quality, and support analytics and reporting across multiple data sources.

You will collaborate with data analysts and business stakeholders to standardize data models and enable BI solutions with Power BI, leveraging Cosmos DB and AI-enabled features where appropriate.