Senior Data Platform Lead: Cloud, Spark & Delta Lake

Remotedxb; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

Remotedxb is seeking an experienced Data Platform Architect to design and evolve enterprise-grade data platforms and pipelines for credit risk products and analytics. You will implement scalable ETL/ELT frameworks using Databricks, Spark, and Delta Lake on cloud platforms, ensuring data quality, governance, and observability.

You will lead cross-functional engineering initiatives, promote standards, and mentor teams while shaping the technical roadmap and delivery priorities.

Qualifications

• Design large-scale data engineering solutions and distributed processing systems.
• Proficient with Databricks, Spark, Delta Lake, SQL, Hadoop, and Python.
• Experience building cloud-based data platforms on Azure/AWS/GCP.
• Expertise ETL/ELT pipelines and streaming architectures.
• Strong data modeling for analytical and operational workloads.
• Knowledge of CI/CD, IaC, and DevOps practices.
• Proven ability to lead technical initiatives across teams.
• Advanced English proficiency.

Responsibilities

• Lead design, development, and evolution of enterprise data platforms and pipelines.
• Architect scalable ETL/ELT frameworks with Databricks, Spark, Delta Lake, and cloud tech.
• Establish data quality, lineage, governance, observability, and monitoring.
• Migrate legacy data assets to cloud-based architectures and Lakehouse platforms.
• Partner with Risk, Product, Architecture, and Engineering to translate requirements.
• Define engineering standards, testing frameworks, and deployment automation.
• Lead design reviews and influence Airflow and workflow orchestration directions.
• Optimize large-scale data workloads for performance, reliability, and cost.
• Mentor engineers and raise technical capability across the org.
• Shape roadmap planning, technology evaluation, and delivery priorities.
• Support regulatory, compliance, security, and audit controls.
• Conduct technical interviews and assess engineering talent.

Skills

Data architecture

Data engineering

Databricks

Apache Spark

Delta Lake

SQL

Hadoop

Python

Cloud platforms

ETL/ELT pipelines

Streaming architectures

Data modeling

CI/CD

Infrastructure as code

DevOps practices

Leadership

English proficiency

Education

Bachelor's degree in CS/Engineering

Tools

Airflow

GitHub Actions

CI/CD tooling

Job description

Remotedxb is seeking an experienced Data Platform Architect to design and evolve enterprise-grade data platforms and pipelines for credit risk products and analytics. You will implement scalable ETL/ELT frameworks using Databricks, Spark, and Delta Lake on cloud platforms, ensuring data quality, governance, and observability.

You will lead cross-functional engineering initiatives, promote standards, and mentor teams while shaping the technical roadmap and delivery priorities.