Senior Data Engineer — AI-Driven Full-Stack Architect

Newmark; Chicago (IL); Hybrid; USD 190,000 - 230,000; Full time

Job summary

Newmark seeks a Senior Data Engineer/Full Stack to own cloud-native architecture spanning front-end, back-end, and data layers. You will drive agentic AI-powered automation and mentor engineers while partnering with product and security teams to deliver scalable, secure data platforms.

You will lead data pipelines, governance, and collaboration across cross-functional teams to accelerate engineering velocity and product capabilities within a hybrid Chicago environment.

Qualifications

• 12+ years of data engineering experience with Azure Databricks.
• Strong expertise in Azure Databricks, Azure Data Factory, and Azure Data Lake Storage.
• Proficiency in Python and SQL.
• Experience with distributed systems, microservices, and API design.
• Experience with cloud platforms (Azure, AWS, GCP) and cloud-native architecture.
• Hands-on with agentic AI, LLM-powered apps, or AI orchestration frameworks.
• Proven track record leading large technical initiatives.
• Experience with CI/CD, IaC, and DevOps.

Responsibilities

• Design and maintain scalable data pipelines using Azure Databricks.
• Translate complex data requirements into robust designs aligned with enterprise standards.
• Lead data ingestion, transformation, modeling, and orchestration for large datasets.
• Tune Spark performance and optimize pipelines for cost and reliability.
• Build batch and near real-time data processing workflows.
• Establish data quality, monitoring, and operational support.
• Support analytics and data science with well-modeled data assets.
• Ensure data security, governance, and compliance.
• Collaborate with platforms, infrastructure, and security teams.

Skills

Azure Databricks

Python

SQL

Distributed systems

Microservices

API design (REST/GraphQL)

Cloud platforms (Azure/AWS/GCP)

Agentic AI / LLM

CI/CD /DevOps

Data modeling

Education

Bachelor’s degree

Tools

LangChain

AutoGen

Semantic Kernel

MCP

Docker

Kubernetes

Kafka

Job description

Newmark seeks a Senior Data Engineer/Full Stack to own cloud-native architecture spanning front-end, back-end, and data layers. You will drive agentic AI-powered automation and mentor engineers while partnering with product and security teams to deliver scalable, secure data platforms.

You will lead data pipelines, governance, and collaboration across cross-functional teams to accelerate engineering velocity and product capabilities within a hybrid Chicago environment.