Member of Technical Staff – Data Engineer

Microsoft Innovation Center; New York (NY); Hybrid; USD 161,000 - 287,000; Full time

Job summary

An innovative firm is seeking a talented Data Engineer to join their dynamic team in New York. This role involves architecting and implementing a scalable data platform that processes vast amounts of data daily. The ideal candidate will have extensive experience in data engineering, utilizing technologies such as SPARK, Hadoop, and cloud platforms like Azure. You will collaborate with cross-functional teams to ensure high-quality code and maintainability, all while embracing a culture of growth and inclusivity. If you're passionate about building robust data solutions and eager to stay ahead of industry trends, this opportunity is perfect for you.

Qualifications

• 8+ years of experience in data engineering or equivalent.
• 5+ years of experience with large-scale data processing platforms.

Responsibilities

• Design and own the architecture of the data platform.
• Write high-quality code in SPARK using Scala, Python, and Java.

Skills

Data Engineering

SPARK

Scala

Python

Java

Hadoop

Kafka

Cloud Platforms

Distributed Systems

Education

Bachelor's Degree

Master's Degree

Tools

Azure

AWS

GCP

Job description

Member of Technical Staff – Data Engineer

Join to apply for the Member of Technical Staff – Data Engineer role at Microsoft Innovation Center.

Overview

Microsoft AI (MS AI) is seeking an experienced Data Engineer to architect, design, and implement a scalable data platform processing 50TB/day of data from AI systems. The role requires technical expertise, problem-solving skills, and proficiency in data platform technologies. The candidate should enjoy building maintainable, scalable data platforms using current tech stacks and design patterns.

As part of Microsoft’s mission to empower organizations worldwide, we value a growth mindset, innovation, collaboration, and inclusivity. This position is based in New York, NY, with an in-office requirement of 3 days a week.

Responsibilities

• Design and own the architecture of the data platform.
• Write high-quality code in SPARK using Scala, Python, and Java.
• Collaborate with platform, infrastructure, application engineers, and AI researchers.
• Ensure code security, testing, and maintainability.
• Overcome roadblocks to deliver solutions iteratively.
• Work in a fast-paced, design-driven environment.
• Uphold our culture and values.

Qualifications

Required

• Bachelor's Degree in a related field with 8+ years of experience, or Master's Degree with 6+ years, or equivalent experience.
• 5+ years building large-scale data processing platforms with technologies like Hadoop, Kafka, columnar databases, etc.
• 5+ years deploying scalable services on cloud platforms such as Azure, AWS, or GCP.
• 2+ years working with distributed systems at scale.

Preferred

• 10+ years in building data platforms handling large data volumes with languages like Python, C#, Java, Spark, SQL.
• Strong problem-solving, coding, and documentation skills.
• Excellent communication and teamwork abilities.
• Passion for learning new technologies and industry trends.

The typical base pay range is USD $161,600 - $286,200 annually, with location-based variations. Benefits and additional pay details are available here.