Data Engineer

Talentify; Cleveland (OH); On-site; USD 100,000 - 150,000; Full time

Job summary

Diverse Lynx LLC in Cleveland, OH seeks an experienced data engineer to design and implement end-to-end Azure data solutions using Synapse Analytics, Data Factory and Databricks, building scalable pipelines and secure data storage.

The role requires hands-on expertise with SQL, Python (PySpark), APIs, and CI/CD deployments across Dev/QA/Prod, plus the ability to optimize data flows and collaborate with cross-functional teams.

Qualifications

• Hands-on experience with Azure data services and large-scale data pipelines.

Responsibilities

• Design and deploy end-to-end Azure data solutions including pipelines and storage.
• Implement data integration with Synapse, Data Factory, Databricks and related services.
• Develop and optimize SQL procedures, Python-based data processing and Spark notebooks.
• Coordinate Dev/QA/Prod deployments via GitHub CI/CD pipelines and troubleshoot issues.
• Create robust data access, security patterns and data quality checks.

Skills

Azure Synapse Analytics

Azure Data Factory

Databricks

SQL

Python / PySpark

APIs

CI/CD pipelines

Power BI

Tools

GitHub

Job description

Job Description

• Hands on Experience in Azure Synapse Analytics, Azure Data Factory and Data Bricks, Azure Storage, Azure Key Vault, SQL Pools CI/CD Pipeline Designing and other Azure services like functions, logic apps - Linked services, Various Runtimes, Datasets, Pipelines, Activities
• Strong Hands-on Experience in Various Activities like Control flow logic and conditions (For Each, if, switch, until), Lookup, Stored procedure, scripts, validations, Copy Data, Data flow, Azure functions, Notebooks, SQL Pool Stored procedures etc.
• Strong hands on exp in deployment of code throughout landscape (Dev -> QA -> Prod), Git Hub, CI/CD pipelines etc.
• Strong hands on creating the SQL stored procedures - Functions, Stored Procedures, how to call one SP into another, how to process record-by-record - Dynamic SQL - Data Bricks - Must have
• Strong background about the Python libraries like PySpark, Pandas, NumPy, MySQL, Oracle, PySpark libraries
• Must have strong hands on to get data through APIs
• Must be able to install libraries and help users to troubleshoot issues
• Must have knowledge to get the data through stored procedures via Python
• Should be able to debug the Python code Hands on experience in Spark Pools, Spark
• Should be able to merge data/delta loads through Notebooks
• Must have strong background about the Python libraries and PySpark - Hands on Exp on Power BI

Disclaimer: Diverse Lynx LLC is an Equal Opportunity Employer. All applicants and employees are evaluated without discrimination, based solely on their qualifications, ability, competence and performance. This email and its attachments may contain confidential or proprietary information and is intended only for the recipient(s). If you received this message in error, please disregard it and notify the sender. If you no longer wish to receive our communications, you may unsubscribe at any time.
Security Notice: Our official website is www.diverselynx.com We do not operate or authorize any other websites representing Diverse Lynx LLC.