Senior Data Engineer - Databricks & Lakehouse Lead

AmeriLife; Town of Florida (NY); On-site; USD 143,000 - 160,000; Full time

Benefits offered by this job

PTO and comprehensive benefits package

Job summary

AmeriLife is seeking a Senior Data Engineer to design, build, and optimize enterprise data products and modern data pipelines on Databricks. You will develop scalable, metadata-driven data solutions powering enterprise reporting, analytics, AI, and BI.

The role requires deep Databricks, Spark, Delta Lake, and Lakehouse architecture expertise, with a focus on enterprise-grade data engineering. Collaboration with architects, product owners, and data scientists is essential.

Qualifications

• Bachelor's degree in Computer Science, Engineering, Information Systems, or related field.

Responsibilities

• Design scalable Spark-based ETL/ELT pipelines.
• Build data products using Medallion Architecture.
• Develop Bronze, Silver, Gold, and Semantic data layers.
• Engineer reusable enterprise data assets for business consumption.
• Optimize large-scale distributed data processing workloads.
• Build native Databricks solutions using Spark, Delta Lake, Unity Catalog, Workflows, SQL Warehouses, and Delta Live Tables.
• Optimize cluster performance and workload efficiency.
• Implement automated deployment pipelines and CI/CD practices.
• Develop scalable Lakehouse solutions.
• Design dimensional models for business consumption.
• Build Data Vault 2.0 models for enterprise integration.
• Develop semantic models and curated data products.
• Support Master Data Management initiatives.
• Implement enterprise data quality validations.
• Develop metadata-driven engineering solutions.
• Enable lineage, governance, and traceability.
• Ensure production-ready engineering standards.
• Partner with Architects, Product Owners, Business Analysts, and Data Scientists.
• Participate in architecture reviews and code reviews.
• Mentor junior engineers.
• Drive engineering best practices across the organization.

Skills

7+ years Data Engineering

Databricks

Apache Spark (PySpark & SQL)

Delta Lake, Unity Catalog, Workflows,

SQL development

ETL/ELT pipelines

Azure cloud services

Git, CI/CD, DevOps

Performance tuning

Education

Bachelor's degree in CS/Engineering/IS

Tools

Azure DevOps

Terraform

dbt

Job description

AmeriLife is seeking a Senior Data Engineer to design, build, and optimize enterprise data products and modern data pipelines on Databricks. You will develop scalable, metadata-driven data solutions powering enterprise reporting, analytics, AI, and BI.

The role requires deep Databricks, Spark, Delta Lake, and Lakehouse architecture expertise, with a focus on enterprise-grade data engineering. Collaboration with architects, product owners, and data scientists is essential.