Senior Data Engineer: Databricks Lakehouse & Data Products

AmeriLife; Washington (District of Columbia); On-site; USD 143,000 - 160,000; Full time

Benefits offered by this job

PTO

Health insurance

Dental insurance

Vision insurance

Retirement savings

Disability insurance

Life insurance

Job summary

AmeriLife seeks an experienced Senior Data Engineer to design, build, and optimize enterprise data products and modern data pipelines on Databricks. You will develop scalable, metadata-driven data solutions powering enterprise reporting, analytics, AI, and BI.

The ideal candidate brings deep Databricks, Spark, Delta Lake, and Lakehouse expertise, with hands-on experience shaping enterprise-grade data engineering solutions and collaboration across teams.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.
• 7+ years of Data Engineering experience.
• 3+ years of hands-on Databricks experience.
• Strong expertise in Apache Spark (PySpark and SQL).
• Deep knowledge of Delta Lake, Unity Catalog, Workflows, and Lakehouse architecture.
• Advanced SQL development skills.
• Experience with Azure cloud services.
• Git, CI/CD, and DevOps practices.
• Experience with performance tuning and optimization.

Responsibilities

• Design and develop scalable Spark-based ETL and ELT pipelines.
• Build enterprise data products using Medallion Architecture.
• Develop Bronze, Silver, Gold, and Semantic data layers.
• Engineer reusable enterprise data assets for business consumption.
• Optimize large-scale distributed data processing workloads.
• Build native Databricks solutions using Spark, Delta Lake, Unity Catalog, Workflows, SQL Warehouses, and Delta Live Tables.
• Optimize cluster performance and workload efficiency.
• Implement automated deployment pipelines and CI/CD practices.
• Develop scalable Lakehouse solutions.
• Design dimensional models for business consumption.
• Build Data Vault 2.0 models for enterprise integration.
• Develop semantic models and curated data products.
• Support Master Data Management initiatives.
• Implement enterprise data quality validations.
• Develop metadata-driven engineering solutions.
• Enable lineage, governance, and traceability.
• Ensure production-ready engineering standards.
• Partner with Architects, Product Owners, Business Analysts, and Data Scientists.
• Participate in architecture reviews and code reviews.
• Mentor junior engineers.
• Drive engineering best practices across the organization.

Skills

Databricks

Apache Spark

SQL

Azure cloud

CI/CD

Git

DevOps practices

Performance tuning

Education

Bachelor's degree in Computer Science, Engineering, Information Systems, or related field

Tools

dbt

Terraform

Azure DevOps

Delta Lake

Unity Catalog

Workflows

Delta Live Tables

Spark

Job description

AmeriLife seeks an experienced Senior Data Engineer to design, build, and optimize enterprise data products and modern data pipelines on Databricks. You will develop scalable, metadata-driven data solutions powering enterprise reporting, analytics, AI, and BI.

The ideal candidate brings deep Databricks, Spark, Delta Lake, and Lakehouse expertise, with hands-on experience shaping enterprise-grade data engineering solutions and collaboration across teams.