Senior Data Engineer

Sigma Software

United States

Remote

USD 140,000 - 190,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Sigma Software is seeking a Senior Data Engineer to design and scale cloud-native data platforms for a pioneering fusion energy company.

You will work on greenfield data initiatives, build near-real-time pipelines, and modernize large-scale data infrastructure using Databricks and AWS.

Join Sigma Software to work on innovative global projects where technology advances science and sustainability, driving engineering excellence and AI adoption within the data platform ecosystem.

Qualifications

  • 5+ years of experience in data engineering.
  • 3+ years production experience with Databricks and Delta Lake.
  • Strong Python, PySpark, and SQL skills.
  • Deep hands-on AWS expertise (S3, IAM, KMS, Lambda, etc.).
  • Experience with CI/CD and Terraform.
  • Experience with data quality tooling such as Great Expectations, Soda, pyDeequ, or Databricks Labs DQX.

Responsibilities

  • Design and build scalable cloud-native data platforms from greenfield to production.
  • Define and enforce data platform standards including Delta Lake and medallion architecture.
  • Refactor Spark and PySpark scripts for performance and maintainability.
  • Drive adoption of AI tooling and agentic workflows within the data engineering team.
  • Collaborate with the Customer’s scientific and engineering teams.
  • Monitor platform performance and cloud costs.

Skills

Python
PySpark
SQL
Databricks
AWS
Event-driven design
CI/CD
Data quality

Tools

Unity Catalog
Lakeflow Jobs/Workflows
Terraform
Databricks Asset Bundles
Git workflows
Great Expectations
Soda
pyDeequ

Job description

  • Design and build scalable cloud-native data platforms from greenfield to production
  • Design and implement near-real-time ingestion pipelines using event-driven patterns
  • Define and enforce data platform standards including Data Lake/Lakehouse principles, medallion architecture, and data contracts
  • Refactor, optimize, and modernize Spark and PySpark scripts for performance and maintainability
  • Introduce best practices for code quality, testing, and CI/CD across data pipelines
  • Drive adoption of AI tooling and agentic workflows within the data engineering team
  • Ensure data quality, observability, and reliability across pipelines and platforms
  • Design and deliver self-service tooling and microservices that abstract platform complexity
  • Collaborate directly with the Customer’s scientific and engineering teams
  • Monitor platform performance, reliability, and cloud infrastructure costs
  • 5+ years of experience in data engineering
  • 3+ years of production experience with Databricks and Delta Lake
  • Strong expertise in medallion architecture and event-driven asynchronous pipeline design
  • Advanced proficiency in Python, PySpark, and SQL
  • Deep hands-on experience with Databricks, including Unity Catalog, Lakeflow Jobs/Workflows, and job optimization
  • Strong AWS expertise including S3, IAM, KMS, and services such as AWS Batch, ECR, EventBridge, Lambda, or API Gateway
  • Experience with infrastructure as code and CI/CD using Terraform and Databricks Asset Bundles
  • Practical experience with Git workflows, pull request reviews, and automated testing
  • Experience configuring secure CI pipelines using OIDC or service principals
  • Practical MLflow experience including experiment tracking, model registration, and reproducibility practices
  • Strong understanding of data quality, lineage, provenance, schema contracts, and versioning
  • Experience with data quality frameworks such as Great Expectations, Soda, pyDeequ, or Databricks Labs DQX
  • Experience with monitoring, observability, and cloud cost optimization
  • Upper-Intermediate English level or higher

WILL BE A PLUS

  • Experience packaging workloads in Docker containers and running batch jobs on AWS
  • Familiarity with experiment optimization tools such as scikit-optimize, BoTorch, Ax, and scipy
  • Experience building feature sets and train/test splits that avoid data leakage
  • Experience with regulated or high-security environments including data residency requirements
  • Experience disabling or restricting AI features in secured environments
PERSONAL PROFILE
  • Strong analytical mindset
  • Ownership and accountability
  • Ability to communicate complex technical concepts clearly
  • Proactive approach to problem-solving
  • Attention to scalability and maintainability
  • Collaborative and team-oriented attitude

Join an ambitious deep-tech initiative shaping the future of clean energy. We are looking for a Senior Data Engineer to design and scale cloud-native data platforms for a pioneering fusion energy company working on the world’s first commercial laser fusion power plant.

In this role, you will work on greenfield data engineering initiatives, build near-real-time pipelines, and modernize large-scale data infrastructure using Databricks and AWS technologies.

We at Sigma Software create opportunities to work on innovative global projects where technology directly impacts the future of science and sustainability. This role offers the chance to contribute to breakthrough research while driving engineering excellence, automation, and AI adoption within the data platform ecosystem.

CUSTOMER

Our Customer is a German-American deep-tech company building the world’s first commercial laser fusion power plant. The company develops direct-drive laser fusion technology and is commercializing the 2021 breakthrough at Lawrence Livermore, where scientists achieved fusion ignition for the first time.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Cloud Data Platform Engineer - Databricks & AI
Senior Cloud Data Platform Engineer - Databricks & AI

Sigma Software • United States

Remote
USD 140,000 - 190,000
Middle/Senior Data Platform Engineer – DataOps
Middle/Senior Data Platform Engineer – DataOps

Sigma Software Group • United States

Remote
USD 120,000 - 170,000
Senior Data Engineer (Cybersecurity)
Senior Data Engineer (Cybersecurity)

Sigma Software • United States

Hybrid
USD 120,000 - 160,000
Data Engineer
Data Engineer

Sigma • New York (NY)

On-site
USD 140,000 - 180,000
Equity
Generous health benefits
Flexible time off policy
+5
Senior Data Engineer - Databricks
Senior Data Engineer - Databricks

UNAVAILABLE • McLean (VA)

On-site
USD 120,000 - 160,000
Data Engineer
Data Engineer

Sigma • San Francisco (CA)

On-site
USD 140,000 - 180,000
Equity
Generous health benefits
Flexible time off policy
+5
Head of Engineering - Inbound Data
Head of Engineering - Inbound Data

spgi • New York (NY)

On-site
USD 185,000 - 265,000
Data Engineer
Data Engineer

Sigma Computing • New York (NY)

On-site
USD 140,000 - 180,000
Equity
Flexible time off
Paid bonding time for new parents
+4
Databricks Architect/ Senior Data Engineer
Databricks Architect/ Senior Data Engineer

OZ • Boca Raton (FL)

On-site
USD 120,000 - 160,000
Competitive compensation
Comprehensive benefits package
Remote work options
Senior Data Engineer
Senior Data Engineer

83zero Ltd • Baltimore (MD)

On-site
USD 120,000 - 150,000