Senior Data Engineer

83zero; Baltimore (MD); On-site; USD 120,000 - 150,000; Full time

Job summary

83zero is seeking a Senior Data Engineer in Baltimore, Maryland, focused on designing and building scalable cloud-native data platforms. This role involves collaboration with Data Scientists and ML Engineers, implementing advanced analytics, AI, and producing robust data capabilities.

The ideal candidate will have over 5 years of experience, strong expertise in technologies like Spark and Kafka, and a passion for scalable engineering. Join a dynamic company committed to transforming industries through data technology.

Qualifications

• 5+ years’ experience building and operating large-scale data platforms in cloud or hybrid environments.
• Strong experience designing modern data architectures including lakehouse, data mesh, or equivalent frameworks.
• Hands-on expertise with distributed processing technologies.

Responsibilities

• Design and develop scalable cloud-native data platforms and architectures.
• Build robust ingestion and transformation pipelines across various data sources.
• Collaborate with Data Scientists and ML Engineers to operationalise AI solutions.

Skills

Python

SQL

PySpark / Spark

Kafka

Databricks

dbt

Docker

Job description

Type: Permanent

No Sponsorship Available

About the Company

Our client is a global AI and data technology business focused on transforming critical industries through advanced data platforms, AI, and intelligent infrastructure solutions. They partner with governments, utilities, telecoms, healthcare providers, and enterprise organisations to deliver scalable, real-world impact across highly complex environments.

The organisation operates across the full technology lifecycle, from architecture and engineering through to deployment and operational capability building, leveraging cloud-native technologies and modern data platforms.

The Opportunity

We’re looking for a Senior Data Engineer to design, build, and optimise scalable data platforms supporting advanced analytics, machine learning, AI, and operational intelligence solutions. This role will focus on building modern cloud-native data architectures capable of handling large-scale, multi-source data workloads across batch, streaming, and near real-time environments. You’ll work closely with Data Scientists, ML Engineers, Product teams, and senior stakeholders to deliver robust, production-grade data capabilities.

The environment is highly engineering-led, with a strong focus on:

• Data as a Product
• Scalability & reliability
• Observability & governance
• Modern lakehouse architectures
• Distributed processing frameworks
• AI & advanced analytics enablement

Key Responsibilities

• Design and develop scalable cloud-native data platforms and architectures
• Build robust ingestion and transformation pipelines across APIs, IoT/telemetry, legacy systems, and structured data sources
• Develop distributed data processing capabilities using technologies such as Spark and Kafka
• Implement modern lakehouse/data platform architectures supporting analytics, GIS, AI, and machine learning workloads
• Establish best practices around observability, monitoring, data quality, lineage, and platform reliability
• Design secure, governed data environments with strong access controls, encryption, and auditability
• Collaborate with Data Scientists and ML Engineers to operationalise AI and advanced analytics solutions
• Optimise platform performance, scalability, and cost efficiency
• Contribute to architectural strategy and mentor junior engineers within the team
• Support CI/CD, infrastructure automation, and cloud deployment best practices

Required Experience

• 5+ years’ experience building and operating large-scale data platforms in cloud or hybrid environments
• Strong experience designing modern data architectures including lakehouse, data mesh, or equivalent frameworks
• Hands-on expertise with distributed processing technologies such as Spark/PySpark
• Experience with streaming/event-driven architectures using Kafka or similar technologies
• Strong Python and SQL development skills
• Experience working with modern data tooling such as dbt, Docker, and cloud-native platforms
• Exposure to AWS, Azure, or GCP environments
• Strong understanding of data governance, security access controls, encryption, and auditability
• Experience integrating complex and heterogeneous data sources
• Ability to design highly available, production-grade data systems
• Experience enabling machine learning and AI workloads through scalable data infrastructure
• Experience within regulated, public sector, utilities, infrastructure, or government environments
• Knowledge of Infrastructure-as-Code and CI/CD practices
• Exposure to AI/ML operationalisation and feature engineering pipelines
• Experience working within highly distributed, multi-stakeholder organisations
• Python
• SQL
• PySpark / Spark
• Kafka
• Databricks
• dbt
• Docker

Personal Attributes

• Strong systems-thinking and problem-solving mindset
• Comfortable operating within fast-paced, complex environments
• Collaborative and highly communicative
• Passionate about scalable engineering and platform reliability
• Proactive, delivery-focused, and accountable
• Strong stakeholder engagement skills across technical and non-technical teams