Senior Data Engineer: AI-Driven Full-Stack Cloud

Newmark; Chicago (IL); Hybrid; USD 190,000 - 230,000; Full time

Job summary

Newmark in Chicago, IL is seeking a Senior Data Engineer with Full Stack capabilities to own cloud-native platforms end to end, spanning front-end experiences, backend services and data layers. You will set technical direction, design scalable systems and drive AI-enabled automation to accelerate delivery.

You will design and maintain data pipelines using Azure Databricks, collaborate with cross-functional teams, mentor engineers, and ensure security and governance across the stack.

Qualifications

• 12+ years in data engineering with hands-on Azure Databricks experience.
• Strong expertise with Azure Databricks, Data Factory, and Data Lake Storage.
• Proficient in Python and SQL; solid distributed systems design and API concepts.

Responsibilities

• Design and maintain scalable data pipelines on Azure Databricks.
• Partner with business and research teams to translate data requirements into designs.
• Lead ingestion, transformation, modeling, and orchestration for large datasets.
• Optimize Spark performance, reliability, and cost; tune pipelines.
• Build batch and near real-time data processing workflows.
• Establish data quality, monitoring, and operational support.
• Mentor engineers and raise architectural standards across teams.
• Own frontend, backend, and data layer integration in a cloud-native stack.
• Architect multi-agent AI workflows and ensure governance across systems.

Skills

Azure Databricks

Python

SQL

Distributed systems

API design

Cloud platforms

Data modeling

DevOps

Mentoring

Education

Bachelor's degree in CS/Engineering

Tools

Docker

Kubernetes

LangChain

AutoGen

Semantic Kernel

MCP

Job description

Newmark in Chicago, IL is seeking a Senior Data Engineer with Full Stack capabilities to own cloud-native platforms end to end, spanning front-end experiences, backend services and data layers. You will set technical direction, design scalable systems and drive AI-enabled automation to accelerate delivery.

You will design and maintain data pipelines using Azure Databricks, collaborate with cross-functional teams, mentor engineers, and ensure security and governance across the stack.