Azure Data Engineer: Spark, Pipelines & Power BI Pro

TestingXperts; Jackson (MI); On-site; USD 85,000 - 125,000; Full time

Job summary

TestingXperts is seeking a hands-on Azure Data Engineer to design and implement end-to-end data solutions. You will work with Azure Synapse Analytics, Data Factory and Databricks, building pipelines, datasets, and linked services while ensuring robust deployment through CI/CD controls.

You will leverage Python (PySpark), SQL stored procedures, and APIs to process data, troubleshoot issues, and deliver data insights via Power BI and notebooks.

Qualifications

• Hands-on experience with Azure data services including Synapse, Data Factory, Databricks, Storage and Key Vault.
• Strong skills in building data pipelines, datasets, linked services and pipelines.
• Experience deploying code across environments (Dev -> QA -> Prod) with CI/CD pipelines and Git.
• Proficiency in writing SQL stored procedures, functions, and using dynamic SQL with PySpark as needed.
• Hands-on with Python libraries (PySpark, Pandas, NumPy) and connecting to MySQL/Oracle via Python.
• Ability to access data via APIs and troubleshoot integration issues.
• Familiarity with Spark Pools, Notebooks, and Power BI for data visualization.

Responsibilities

• Develop end-to-end data pipelines and data flows in Azure environments.
• Manage deployment and CI/CD processes across development, QA, and production.
• Implement and optimize SQL stored procedures and Python routines for data processing.
• Collaborate with teams to integrate APIs and ensure data quality and reliability.
• Enable data access for analytics and BI tools like Power BI using notebooks and Spark.

Skills

Azure Synapse

Azure Data Factory

Azure Databricks

Azure Storage

Azure Key Vault

CI/CD pipelines

SQL Stored Procedures

Python (PySpark)

Power BI

Notebooks

APIs

Job description

TestingXperts is seeking a hands-on Azure Data Engineer to design and implement end-to-end data solutions. You will work with Azure Synapse Analytics, Data Factory and Databricks, building pipelines, datasets, and linked services while ensuring robust deployment through CI/CD controls.

You will leverage Python (PySpark), SQL stored procedures, and APIs to process data, troubleshoot issues, and deliver data insights via Power BI and notebooks.