Senior Data Engineer / Lead Data Engineer

Archer; United States; On-site; USD 120,000 - 180,000; Full time

Job summary

Archer, in collaboration with Version 1, seeks a Senior Data Engineer to advance modern cloud-native data platforms for analytics and AI at scale. You will own end-to-end data pipelines, ensure production-grade quality, and help enterprise clients adopt governed data services.

You will design and operate pipelines on AWS, with Delta Lake/Medallion patterns, mentor engineers, and collaborate with data scientists and platform teams to deliver reliable, cost-efficient data solutions.

Qualifications

• 6–8 years of hands-on data engineering experience.
• Strong Databricks experience including Delta Lake and Medallion Architecture.
• Design, build and support scalable data pipelines from enterprise source systems.
• Develop data models and curated data products within the ISP data platform.
• Implement MDM, data quality controls, monitoring and remediation workflows.
• Nice to have: experience enabling ML/GenAI workloads and data tooling exposure.

Responsibilities

• Designing, building, and operating reliable data pipelines using modern ETL/ELT approaches.
• Engineering cloud-native data platforms on AWS with Azure exposure where required.
• Working with Databricks, PostgreSQL, Aurora, and distributed processing frameworks.
• Ensuring solutions are production-ready: secure, tested, observable, and cost-efficient.
• Optimising data performance, reliability, and data quality in live environments.
• Collaborating with analysts, data scientists, AI engineers, and platform teams.
• Mentoring junior engineers and contributing to shared standards and accelerators.

Skills

Databricks

AWS

Delta Lake

Medallion Architecture

Data pipelines

MDM

Data quality

AI integration

Tools

PostgreSQL

Aurora

Spark

Job description

Archer is collaborating with Version 1 to connect them with exceptional professionals for this role.

We are looking for a Senior Data Engineer to join Version 1’s Data & AI Practice, working on modern data platforms for enterprise and regulated clients. You will be a hands‑on engineer with strong production experience, contributing to the design, build, and operation of scalable, cloud‑native data solutions that enable analytics and AI at scale.

This is a delivery‑focused role for an experienced engineer who enjoys solving complex data challenges, takes ownership of outcomes, and applies strong engineering discipline in real‑world environments.

What You’ll Be Doing

• Designing, building, and operating reliable data pipelines using modern ETL/ELT approaches
• Engineering cloud‑native data platforms, primarily on AWS (with Azure exposure where required)
• Working with technologies such as Databricks, PostgreSQL, Aurora, and distributed processing frameworks
• Ensuring solutions are production‑ready: secure, tested, observable, and cost‑efficient
• Optimising data performance, reliability, and data quality in live environments
• Collaborating with analysts, data scientists, AI engineers, and platform teams
• Mentoring junior engineers and contributing to shared standards and accelerators

What You’ll Bring

• 6-8 years’ experience in hands‑on data engineering roles
• Strong Databricks experience (Data Engineering, Delta Lake, Medallion Architecture)
• Design, build and support scalable data pipelines from enterprise source systems
• Build and maintain data models, mapping tables and curated data products within the ISP data platform
• Implement Master Data Management (MDM), data quality controls, monitoring and remediation workflows
• Develop automated data quality and completeness reporting, including AI-assisted reporting capabilities
• Support the industrialisation of AI-enabled applications, transitioning from file-based solutions to governed Databricks-backed data services
• Experience integrating AI/analytics tools with enterprise data platforms and supporting operationalisation of proof-of-concepts into production solutions

Nice to Have

• Experience enabling ML, AI, or GenAI workloads through strong data foundations
• Exposure to modern BI and analytics tooling
• Experience working in regulated or security‑conscious environments
• Consulting or client‑facing delivery experience