Databricks & Spark Data Engineer | Azure Lakehouse

Ethereum Technologies; Quincy (MA); On-site; USD 77,000 - 110,000; Full time

Job summary

Ethereum Technologies in Quincy, MA seeks a Data Engineer to develop and optimize Spark jobs and data pipelines using Azure Databricks. You will design lakehouse solutions and implement streaming data transformations.

Candidates should have strong experience with Databricks, Spark SQL, and DevOps integration; Docker/Kubernetes experience is a plus. This is a hands-on, fast-paced role supporting live data workloads.

Qualifications

• Apache Spark and Spark SQL expertise for large-scale data processing.
• ETL/ELT development and data pipeline orchestration.
• Scala and PySpark development with SQL performance tuning.

Responsibilities

• Develop and optimize Spark jobs, data transformations, and processing workflows.
• Implement streaming data solutions using Azure Databricks.
• Design and implement data warehouse/lakehouse solutions.
• Monitor, debug, and resolve issues in live Databricks jobs.
• Troubleshoot DevOps pipelines, Azure, and AKS services.
• Build data ingestion, streaming, and transformation solutions.

Skills

Spark & SQL

ETL pipelines

Scala / PySpark

Azure Databricks

Databricks expertise

Unity Catalog

Auto Loader

Lakehouse architecture

Databricks Serverless

Dataset & DataFrame

Cluster sizing

Job orchestration

Structured Streaming

Tools

Docker

Kubernetes

Job description

Ethereum Technologies in Quincy, MA seeks a Data Engineer to develop and optimize Spark jobs and data pipelines using Azure Databricks. You will design lakehouse solutions and implement streaming data transformations.

Candidates should have strong experience with Databricks, Spark SQL, and DevOps integration; Docker/Kubernetes experience is a plus. This is a hands-on, fast-paced role supporting live data workloads.