Senior Software Engineer (Data Platform)

globalizationpartners; United States; Remote; USD 120,000 - 170,000; Full time

Benefits offered by this job

Health insurance

Parental leave

Sabbatical after 5 years

Job summary

globalizationpartners seeks a Senior Engineer to help shape a large-scale, AI-native Data Platform powering internal services and workflows. You will bridge platform engineering and data engineering, building reusable frameworks and self-service tooling in a remote-first, distributed team.

You will own design for streaming and batch processing, optimize Databricks, and guide data quality through schema contracts, while mentoring engineers to sustain scalable solutions across teams.

Qualifications

• 5+ years of experience building and operating production-grade data systems at massive scale.
• Deep, hands-on mastery of the Databricks and Spark ecosystem, including Delta Lake, DLT, Spark UI debugging, and performance tuning.
• Proven experience designing real-time or streaming architectures (such as Spark Structured Streaming, Kafka, or Kinesis) in production.
• Demonstrated ability to manage and optimize cloud costs in a high-growth environment.
• Prior experience building APIs, tools, or frameworks consumed by other internal engineering teams.

Responsibilities

• Lead the design and rollout of internal SDKs and self-service frameworks that let distributed engineering teams ingest and transform data independently.
• Build reusable patterns for both batch and real-time event processing, shifting the team's mindset from pipeline construction to platform engineering.
• Own the cost-effectiveness of the Databricks ecosystem, tuning Spark execution plans, optimizing shuffle partitions, and implementing auto-scaling strategies to manage DBU consumption and cloud spend.
• Implement Schema-on-Write validation and Data Contracts so that incoming data from hundreds of services meets strict quality standards before entering the Bronze layer.
• Partner with data architects and stewards to enforce PII handling, security standards, and metadata lineage across the platform.
• Champion AI-assisted development tools to accelerate the SDLC and mentor engineers through code reviews emphasizing maintainability and scalability.

Skills

Databricks & Spark

Streaming architectures

Cloud cost optimization

API development

Distributed systems

Tools

Delta Lake

Delta Live Tables

Spark UI

Kafka

Kinesis

Job description

Role overview

Join as a Senior Engineer focused on a large-scale, AI-native Data Platform that powers hundreds of internal services and downstream product workflows. This role sits at the intersection of platform engineering and data engineering, designing reusable frameworks, self-service tooling, and high-performance processing engines rather than one-off pipelines. The position is suited to someone who enjoys bridging architectural strategy and hands‑on implementation in a remote-first, distributed engineering environment.

Responsibilities

• Lead the design and rollout of internal SDKs and self-service frameworks that let distributed engineering teams ingest and transform data independently.
• Build reusable patterns for both batch and real-time event processing, shifting the team's mindset from pipeline construction to platform engineering.
• Own the cost-effectiveness of the Databricks ecosystem, tuning Spark execution plans, optimizing shuffle partitions, and implementing auto-scaling strategies to manage DBU consumption and cloud spend.
• Implement Schema-on-Write validation and Data Contracts so that incoming data from hundreds of services meets strict quality standards before entering the Bronze layer.
• Partner with data architects and stewards to enforce PII handling, security standards, and metadata lineage across the platform.
• Champion AI-assisted development tools to accelerate the SDLC and mentor engineers through code reviews emphasizing maintainability and scalability.

Requirements

• 5+ years of experience building and operating production-grade data systems at massive scale.
• Deep, hands‑on mastery of the Databricks and Spark ecosystem, including Delta Lake, DLT, Spark UI debugging, and performance tuning.
• Proven experience designing real-time or streaming architectures (such as Spark Structured Streaming, Kafka, or Kinesis) in production.
• Demonstrated ability to manage and optimize cloud costs in a high‑growth environment.
• Prior experience building APIs, tools, or frameworks consumed by other internal engineering teams.

Nice to have

• Background in Lakehouse architecture patterns and medallion data modeling.
• Familiarity with AI-assisted coding tools and integrating them into team workflows.
• Experience mentoring engineers on distributed computing best practices.

Benefits and work setup

• Remote-first working model with flexibility and autonomy over scheduling.
• Generous paid parental leave and flexible time off policies.
• Medical, dental, and vision insurance, plus spending accounts.
• Sabbatical eligibility after five years of service.
• Annual bonus opportunity for non-sales roles, with compensation benchmarked against market data using gender-neutral criteria.