Lead Data Platform Engineer: AI-Driven Pipelines

Next Frontier Capital; Plano (TX); On-site; USD 180,000 - 240,000; Full time

Job summary

JPMorgan Chase in Texas is seeking a Lead Software Engineer to design, develop, and maintain scalable data processing solutions using Databricks, Python, and AWS. You will collaborate with cross-functional teams to deliver data solutions that support business objectives.

The role emphasizes building a control plane for enterprise pipelines, API/SDK development, data quality, security, observability, and mentoring engineers while adopting enterprise AI-assisted development practices.

Qualifications

• Formal training or certification on software engineering concepts and 5+ years applied experience

Responsibilities

• Design, develop, and maintain scalable data pipelines on Databricks/AWS.
• Build a control plane for enterprise data pipelines.
• Develop self-service APIs/SDKs and onboarding templates.
• Lead SDLC activities and provide production support.
• Collaborate with stakeholders on data management strategy.
• Mentor engineers and promote AI-assisted development with governance.
• Ensure data quality, security, and observability.
• Drive adoption of enterprise AI-assisted engineering practices.

Skills

Data management

ETL/ELT

Python

PySpark

SQL

Databricks

AWS

APIs/SDKs

CI/CD

Observability

SRE

AI-assisted

Education

Formal software training

Tools

Databricks

Jenkins

Spinnaker

Sonar

Job description

JPMorgan Chase in Texas is seeking a Lead Software Engineer to design, develop, and maintain scalable data processing solutions using Databricks, Python, and AWS. You will collaborate with cross-functional teams to deliver data solutions that support business objectives.

The role emphasizes building a control plane for enterprise pipelines, API/SDK development, data quality, security, observability, and mentoring engineers while adopting enterprise AI-assisted development practices.