Azure Data Engineer

Shrive Technologies; Jackson (MS); On-site; USD 95,000 - 125,000; Full time

Job summary

Shrive Technologies is seeking a data engineering professional with hands-on expertise in Azure Synapse Analytics, Data Factory, and Databricks. You will design and deploy data pipelines, manage CI/CD for data solutions, and integrate Python-based analytics with Power BI reporting.

Strong experience with PySpark, APIs, and cloud-native data storage is required to drive scalable data flows from ingestion to consumption.

Qualifications

• Hands-on experience with Azure data services and SQL data objects.
• Deep knowledge of data integration, pipelines and CI/CD concepts.
• Proficiency in Python data tooling (PySpark, Pandas) and API data access.

Skills

Azure Synapse

Azure Data Factory

Azure Databricks

Azure Storage

Azure Key Vault

SQL Pools

CI/CD Pipelines

PySpark

Power BI

APIs

Tools

Databricks

Spark

Notebooks

Azure DevOps

SQL

Job description

• Hands on Experience in Azure Synapse Analytics, Azure Data Factory and Data Bricks, Azure Storage, Azure Key Vault, SQL Pools CI/CD Pipeline Designing and other Azure services like functions, logic apps - Linked services, Various Runtimes, Datasets, Pipelines, Activities
• Strong Hands-on Experience in Various Activities like Control flow logic and conditions (For Each, if, switch, until), Lookup, Stored procedure, scripts, validations, Copy Data, Data flow, Azure functions, Notebooks, SQL Pool Stored procedures etc.
• Strong hands on exp in deployment of code throughout landscape (Dev -> QA -> Prod), Git Hub, CI/CD pipelines etc.
• Strong hands on creating the SQL stored procedures - Functions, Stored Procedures, how to call one SP into another, how to process record-by-record - Dynamic SQL - Data Bricks - Must have
• Strong background about the Python libraries like PySpark, Pandas, NumPy, MySQL, Oracle, PySpark libraries
• Must have strong hands on to get data through APIs
• Must be able to install libraries and help users to troubleshoot issues
• Must have knowledge to get the data through stored procedures via Python
• Should be able to debug the Python code Hands on experience in Spark Pools, Spark
• Should be able to merge data/delta loads through Notebooks
• Must have strong background about the Python libraries and PySpark - Hands on Exp on Power BI