Databricks Integration Engineer

Ethereum Technologies; Boston (MA); On-site; USD 180,000 - 240,000; Full time

Job summary

Ethereum Technologies is seeking a seasoned Databricks architect to design scalable, secure data and AI platforms on the Databricks Lakehouse. You will define the target architecture, standards, and reusable patterns, and guide ingestion, transformation, governance and production operations.

You will lead architecture reviews, optimize performance and cost, establish CI/CD and monitoring, and partner with stakeholders across data, security, and vendor teams while mentoring engineering staff.

Qualifications

• 12-15 years of data engineering, integration, or cloud/data architecture experience.
• Hands-on Databricks and cloud architecture expertise.
• Strong knowledge of Apache Spark, Delta Lake, Unity Catalog, SQL, and Python.
• Experience designing enterprise-scale data and AI platforms.
• Strong understanding of governance, security, performance tuning, and cost optimization.
• Experience with CI/CD and production monitoring.
• Excellent communication and stakeholder-management skills.

Responsibilities

• Design scalable and secure Databricks Lakehouse architectures.
• Define target architecture, engineering standards, and reusable frameworks.
• Lead data ingestion, transformation, governance, and production operations.
• Conduct architecture reviews and technology evaluations.
• Drive performance and cost optimization across the platform.
• Establish CI/CD, monitoring, and operational best practices.
• Partner with client stakeholders, data engineers, analysts, security teams, and vendors to translate business requirements into technical solutions.
• Provide technical leadership and mentoring to engineering teams.

Skills

Databricks expert

Cloud architecture

Data engineering

Apache Spark

Delta Lake

Unity Catalog

SQL

Python

Enterprise data platforms

Governance & security

CI/CD

Monitoring

Stakeholder management

GenAI exposure

Education

Bachelor's degree in CS or related field

Tools

Databricks

Cloud platforms (AWS/Azure/GCP)

CI/CD tooling

Delta Lake

Apache Spark

Unity Catalog

Python

SQL

Azure

Job description

US Citizen, Green Card, H-1B, OPT-EAD, CPT, Other

Job Description

We are seeking an experienced Databricks expert to design and deliver scalable, secure data and AI platforms on the Databricks Lakehouse. The role will define the target architecture, establish engineering standards, and guide the implementation of data ingestion, transformation, governance, performance optimization, and production operations.

Key Responsibilities

• Design scalable and secure Databricks Lakehouse architectures.
• Define target architecture, engineering standards, and reusable frameworks.
• Lead data ingestion, transformation, governance, and production operations.
• Conduct architecture reviews and technology evaluations.
• Drive performance and cost optimization across the platform.
• Establish CI/CD, monitoring, and operational best practices.
• Partner with client stakeholders, data engineers, analysts, security teams, and vendors to translate business requirements into technical solutions.
• Provide technical leadership and mentoring to engineering teams.

Required Skills

• 12-15 years of overall experience in data engineering, integration, or cloud/data architecture.
• Strong hands‑on expertise with Databricks and cloud architecture.
• Strong knowledge of Apache Spark, Delta Lake, Unity Catalog, SQL, and Python.
• Experience designing enterprise‑scale data and AI platforms.
• Strong understanding of data governance, security, performance tuning, and cost optimization.
• Experience with CI/CD and production monitoring.
• Excellent communication and stakeholder-management skills.

Preferred Skills

• Exposure to Generative AI (GenAI) solution patterns.
• Understanding of enterprise architecture principles and standards.

We are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.