Senior Data Engineer: Lakehouse on Databricks (Flexible Hours)

Namirial; Town of Italy (NY); Hybrid; USD 57,000 - 62,000; Full time

Benefits offered by this job

Meal vouchers

Flexible working hours

Home office

International team

Job summary

Namirial is seeking a Data Engineer to join the IT Platform & AI Team, building scalable data architectures on a Databricks Lakehouse Platform on AWS. You will own end-to-end data pipelines, governance, and reliability, shaping how data supports analytics and ML across the business.

The role requires 5+ years in data engineering, strong PySpark/SQL, and experience with streaming pipelines. Collaborative, security-minded, and growth-oriented mindset is essential.

Qualifications

• 5+ years of experience as a Data Engineer in complex enterprise environments.
• Strong experience with Databricks Lakehouse Platform, including Delta Lake and Unity Catalog.
• Solid working knowledge of AWS cloud services and cloud-based data architectures.
• Strong proficiency in Python, SQL, and Apache Spark (PySpark / Spark SQL).
• Experience developing and operating both batch and streaming pipelines, including Spark Structured Streaming.
• Experience with orchestration solutions such as Databricks Lakeflow Jobs and/or external orchestrators (Airflow, Prefect, Dagster, AWS Step Functions).
• Solid understanding of data modeling techniques, including dimensional modeling and Lakehouse modeling approaches.
• Understanding of data governance, data quality, security-by-design principles, and compliance requirements.
• Experience working with Parquet, Delta, Avro, and JSON.
• Knowledge of Kafka or Amazon Kinesis.
• Familiarity with Terraform, CloudFormation, and Databricks Asset Bundles.
• Experience with CI/CD pipelines using GitLab CI, Jenkins, or similar solutions.
• Good command of English, both written and spoken.

Responsibilities

• Design and implement end-to-end data architectures on a Databricks Lakehouse Platform following Medallion architecture principles (Bronze, Silver, Gold layers).
• Build and operate scalable ETL/ELT pipelines for batch and streaming workloads using PySpark, Spark SQL, and Spark Structured Streaming.
• Develop and manage Databricks infrastructure, including Delta Lake tables, Databricks SQL, and Lakeflow Jobs.
• Implement data governance, lineage, and access control using Unity Catalog, ensuring security-by-design and audit readiness.
• Define and implement data quality checks, monitoring, logging, and observability practices across the data ecosystem.
• Establish engineering best practices around coding standards, testing, CI/CD, and Infrastructure as Code.
• Collaborate with analytics, machine learning, security, and engineering teams to enable reliable and efficient data-driven solutions.
• Contribute to team growth through collaboration, knowledge sharing, and technical guidance.
• Support the continuous improvement of platform reliability and scalability.

Skills

Databricks Lakehouse Platform

Python

SQL

Apache Spark (PySpark / Spark SQL)

Delta Lake

Unity Catalog

Lakeflow Jobs

Airflow / Prefect / Dagster

Terraform

Kafka / Kinesis

Tools

Databricks Lakehouse Platform

Delta Lake

Unity Catalog

Lakeflow

dbt

Airflow

Dagster

Prefect

Terraform (Databricks provider)

GitLab CI / Jenkins

Job description

Namirial is seeking a Data Engineer to join the IT Platform & AI Team, building scalable data architectures on a Databricks Lakehouse Platform on AWS. You will own end-to-end data pipelines, governance, and reliability, shaping how data supports analytics and ML across the business.

The role requires 5+ years in data engineering, strong PySpark/SQL, and experience with streaming pipelines. Collaborative, security-minded, and growth-oriented mindset is essential.