Azure Data Engineer : Jackson, MI (Hybrid)_W2 Only

TestingXperts Inc. DBA Damcosoft; Jackson (MI); On-site; USD 90,000 - 130,000; Full time

Job summary

TestingXperts Inc. DBA Damcosoft seeks a data engineering specialist with hands-on experience in Azure Synapse, Data Factory and Databricks. You will design end-to-end data pipelines, implement SQL stored procedures, and leverage Python PySpark for data processing.

Strong CI/CD and API data access skills are essential, plus Power BI for visualization. Collaborate across teams to move solutions from development to production, optimize data flows, and troubleshoot data issues to ensure robust data

Qualifications

• Hands-on experience with Azure Synapse Analytics, Azure Data Factory and Databricks
• Strong experience in control flow, data transformations, and data pipelines
• Proficient in deploying code across Dev-QA-Prod with GitHub and CI/CD pipelines
• Extensive SQL development with stored procedures and data integration
• Proficient in Python libraries (PySpark, Pandas, NumPy) and data access via APIs
• Experience with Power BI for data visualization and reporting

Responsibilities

• Design and implement end-to-end data pipelines on Azure
• Develop and optimize SQL procedures, functions and data flows
• Maintain CI/CD pipelines and version control for data projects
• Collaborate across teams to deploy data solutions from development to production
• Create notebooks and scripts to process and analyze data
• Troubleshoot data issues and mentor users on libraries and tools

Skills

Azure Synapse

Azure Data Factory

Azure Databricks

Azure Storage

Azure Key Vault

SQL

CI/CD

Python

PySpark

Power BI

APIs

Tools

GitHub

Notebooks

Spark Pools

Job description

• Hands on Experience in Azure Synapse Analytics, Azure Data Factory and Data Bricks, Azure Storage, Azure Key Vault, SQL Pools CI/CD Pipeline Designing and other Azure services like functions, logic apps - Linked services, Various Runtimes, Datasets, Pipelines, Activities
• Strong Hands-on Experience in Various Activities like Control flow logic and conditions (For Each, if, switch, until), Lookup, Stored procedure, scripts, validations, Copy Data, Data flow, Azure functions, Notebooks, SQL Pool Stored procedures etc.
• Strong hands on exp in deployment of code throughout landscape (Dev -> QA -> Prod), Git Hub, CI/CD pipelines etc.
• Strong hands on creating the SQL stored procedures - Functions, Stored Procedures, how to call one SP into another, how to process record-by-record - Dynamic SQL - Data Bricks - Must have
• Strong background about the Python libraries like PySpark, Pandas, NumPy, MySQL, Oracle, PySpark libraries
• Must have strong hands on to get data through APIs
• Must be able to install libraries and help users to troubleshoot issues
• Must have knowledge to get the data through stored procedures via Python
• Should be able to debug the Python code Hands on experience in Spark Pools, Spark
• Should be able to merge data/delta loads through Notebooks
• Must have strong background about the Python libraries and PySpark - Hands on Exp on Power BI