Data Engineer

Programmers.io; Weehawken (NJ); On-site; USD 90,000 - 120,000; Full time

Job summary

A leading IT services firm is seeking an Azure Data Engineer to build data solutions and optimize SQL queries. The candidate should have experience with Big Data technologies like Spark and Kafka, and proficiency in Azure data services. This full-time position offers opportunities to work on complex data pipelines in a mid-senior level role.

Qualifications

• Strong proficiency with SQL and data modeling.
• Experience with Azure data services and cloud infrastructure.
• Experience with Spark, Kafka, Hive or equivalent Big Data technologies.

Responsibilities

• Write and optimize complex SQL queries.
• Build data pipelines and applications to process datasets.
• Design data solutions using API and Streaming ingestion methods.

Skills

SQL optimization

Data modelling

ETL design

Big Data technologies (Spark, Kafka, Hive)

API ingestion

Dev-Ops processes

Azure Cosmos DB

Azure and AWS Cloud Infrastructure

Python

Job description

Overview

Programmers.io is currently looking for an Azure Data Engineer.

FULL TIME ROLE+ BENEFITS (NO C2C) - NO SPONSORSHIP IS PROVIDED

If you are interested, please apply or share your updated resume at [e-mail].

Responsibilities

• Expert level skills writing and optimizing complex SQL
• Experience with complex data modelling, ETL design, and using large databases in a business environment
• Experience with building data pipelines and applications to stream and process datasets at low latencies
• Fluent with Big Data technologies like Spark, Kafka and Hive
• Designing and building data pipelines using API ingestion and Streaming ingestion methods
• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential
• Experience in developing NO SQL solutions using Azure Cosmos DB is essential
• Thorough understanding of Azure and AWS Cloud Infrastructure offerings
• Working knowledge of Python is desirable
• Designing and implementing scalable and secure data solutions

Qualifications

• Strong proficiency with SQL and data modeling
• Experience with Azure data services and cloud infrastructure
• Experience with Spark, Kafka, Hive or equivalent Big Data technologies
• Experience building data pipelines and streaming/ingestion architectures
• Familiarity with CI/CD, IaC (Infrastructure as Code), and DevOps practices
• Proficiency in Python or similar scripting languages is desirable

Job Details

• Seniority level: Mid-Senior level
• Employment type: Full-time
• Job function: Information Technology
• Industries: IT Services and IT Consulting

Referrals increase your chances of interviewing at Programmers.io by 2x