Senior Lead Software Engineer - Data & AI Platform Engineer

ARCHER

Jersey City (NJ)

On-site

USD 180,000 - 260,000

Full time

12 days ago

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
Backup Childcare
Tuition Reimbursement
Mental Health Support
Financial Coaching

Job summary

ARCHER is seeking a senior data platform engineer to lead the design and delivery of scalable data pipelines and governance capabilities. The role focuses on building an Agentic AI lakehouse to automate provisioning and operations, partnering with product, analytics, and business teams to deliver secure, reliable production solutions.

You will work with Python/Java/SQL, Databricks, Spark, Airflow, and cloud data services on distributed platforms, driving data modeling, optimization, and CI/CD

Qualifications

  • Requires 5+ years of applied software engineering experience.
  • Strong programming skills in Python, Java, or SQL.
  • Hands-on experience with distributed data processing, Databricks, orchestration, cloud data services, and production data platforms.
  • Understanding of data modeling, query optimization, CI/CD, resiliency, and security.
  • Experience with Agentic AI, LLMs, RAG, MCP, vector databases, and embedding-based retrieval.

Responsibilities

  • Lead the design and delivery of scalable data pipelines and governed data platforms.
  • Drive an Agentic AI lakehouse initiative to automate provisioning and operations.
  • Collaborate with product, analytics, and business teams to deliver secure, reliable production solutions.

Skills

Distributed data processing
Data modeling
CI/CD
Security
Agentic AI
LLMs
RAG
Vector databases
Embedding-based retrieval
Python
Java
SQL

Tools

Databricks
Apache Spark
Apache Flink
Apache Airflow
AWS Data Services
Kubernetes
Terraform

Job description

Lead the design and delivery of scalable data pipelines and governed data platform capabilities, including cataloging, lineage, data quality, and analytics-ready data models. Drive an Agentic AI lakehouse initiative to automate platform provisioning and operations, while partnering with product, analytics, and business teams to deliver secure, reliable production solutions.

Requirements
  • Requires at least five years of applied software engineering experience, strong programming skills in Python, Java, or SQL, and hands-on experience with distributed data processing, Databricks, orchestration, cloud data services, and production data platforms.
  • Candidates should also understand data modeling, query optimization, CI/CD, resiliency, and security; experience with Agentic AI, LLMs, RAG, MCP, vector databases, and embedding-based retrieval is specified.
Key Skills
  • Python
  • Java
  • SQL
  • Apache Spark
  • Apache Flink
  • Databricks
  • Apache Airflow
  • AWS Data Services
  • Kubernetes
  • Data Modeling
  • Data Pipelines
  • Agentic AI
  • Large Language Models
  • Retrieval-Augmented Generation
  • CI/CD
  • Terraform
Benefits
  • Health Care Coverage
  • On-Site Health And Wellness Centers
  • Retirement Savings Plan
  • Backup Childcare
  • Tuition Reimbursement
  • Mental Health Support
  • Financial Coaching
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Sr Lead Software Engineer - Data Engineering
Sr Lead Software Engineer - Data Engineering

JP Morgan Chase • Wilmington (DE)

On-site
USD 140,000 - 190,000
Health Care Coverage
On-Site Health and Wellness Centers
Retirement Savings Plan
+4
Data Engineer
Data Engineer

FutureTech Consultants, LLC • Alpharetta (GA)

On-site
USD 120,000 - 190,000
Software Engineer III - Agentic Data Quality Platform
Software Engineer III - Agentic Data Quality Platform

JP Morgan Chase • Plano (TX)

On-site
USD 120,000 - 160,000
Health Care Coverage
On-Site Health and Wellness Centers
Retirement Savings Plan
+4
Lead Software Engineer - AI Application (Corporate Technology)
Lead Software Engineer - AI Application (Corporate Technology)

ARCHER • Jersey City (NJ)

On-site
USD 180,000 - 260,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+6
Lead AI Software Engineer
Lead AI Software Engineer

Humana • Louisville (KY)

On-site
USD 140,000 - 200,000
Medical Benefits
Dental Benefits
Vision Benefits
+9
Lead AI Software Engineer
Lead AI Software Engineer

Humana • Nashville (TN)

On-site
USD 180,000 - 240,000
Medical Insurance
Dental Insurance
Vision Insurance
+10
Principal Software Engineer - Databricks (Chief Data Analytics Office - AIML Data Platforms)
Principal Software Engineer - Databricks (Chief Data Analytics Office - AIML Data Platforms)

ARCHER • Jersey City (NJ)

On-site
USD 180,000 - 260,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4
Sr. Data Engineer, Data Platform
Sr. Data Engineer, Data Platform

Mirion Technologies • United States

On-site
USD 120,000 - 160,000
Lead Software Engineer - Agentic AI, Java/Python (Commercial & Investment Bank)
Lead Software Engineer - Agentic AI, Java/Python (Commercial & Investment Bank)

ARCHER • Jersey City (NJ)

On-site
USD 180,000 - 260,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4
Lead Software Engineer - Databricks (Corporate Technology)
Lead Software Engineer - Databricks (Corporate Technology)

ARCHER • Jersey City (NJ)

On-site
USD 140,000 - 210,000
Health Care Coverage
On-Site Health And Wellness Centers
Retirement Savings Plan
+4