Databricks

Ameri-partners; New Jersey; Presencial; USD 180.000 - 230.000; Jornada completa

Descripción de la vacante

Ameri Partners seeks a Databricks SME to lead design and delivery of enterprise-scale data platforms and Lakehouse solutions. The role requires deep hands-on expertise with Databricks, Apache Spark, Delta Lake, Unity Catalog, and modern data engineering practices.

The SME will provide technical leadership across architecture, data pipelines, governance, performance, and production deployment, coordinating with cross-functional teams and clients to translate requirements into scalable solutions.

Formación

• 10+ years of experience in data engineering, analytics engineering, or data/platform architecture.
• 5+ years of hands-on Databricks experience.
• Strong expertise with Apache Spark, Delta Lake, Unity Catalog, PySpark/Scala, and SQL.
• Experience designing and supporting enterprise-scale cloud data platforms.
• Strong understanding of data architecture, governance, performance optimization, and distributed data processing.
• Experience working in client-facing or cross-functional technical environments.
• Ability to lead architecture reviews and solve complex technical problems independently.

Responsabilidades

• Design scalable Databricks Lakehouse architectures using Delta Lake, Unity Catalog, and Bronze/Silver/Gold data patterns.
• Develop and optimize ETL/ELT pipelines using Apache Spark, PySpark, Scala, SQL, and Databricks Workflows.
• Establish data governance, lineage, access controls, and data-quality standards.
• Improve Spark workloads, query performance, storage design, and overall platform efficiency.
• Support enterprise Databricks environments across Azure, AWS, or GCP.
• Contribute to MLflow, MLOps, model deployment, and production machine-learning workflows.
• Review solution architectures and provide technical direction for complex data-platform initiatives.
• Troubleshoot difficult performance, integration, and scalability issues.
• Partner with engineering, analytics, architecture, and business teams to translate requirements into scalable technical solutions.
• Lead technical discussions and provide guidance to project teams and stakeholders.

Conocimientos

Data engineering

Architectural leadership

Client-facing communication

Performance optimization

Herramientas

Databricks

Apache Spark

Delta Lake

Unity Catalog

PySpark

Scala

SQL

MLflow

Descripción del empleo

Ameri Partners is seeking a Databricks Subject Matter Expert (SME) to lead the design and delivery of enterprise-scale data platforms and Lakehouse solutions.

This role requires deep hands-on expertise with Databricks, Apache Spark, Delta Lake, Unity Catalog, cloud data platforms, and modern data engineering practices. The SME will provide technical leadership across architecture, data pipelines, governance, performance, and production deployment.

Key Responsibilities

• Design scalable Databricks Lakehouse architectures using Delta Lake, Unity Catalog, and Bronze/Silver/Gold data patterns.
• Develop and optimize ETL and ELT pipelines using Apache Spark, PySpark, Scala, SQL, and Databricks Workflows.
• Establish data governance, lineage, access controls, and data-quality standards.
• Improve Spark workloads, query performance, storage design, and overall platform efficiency.
• Support enterprise Databricks environments across Azure, AWS, or GCP.
• Contribute to MLflow, MLOps, model deployment, and production machine-learning workflows.
• Review solution architectures and provide technical direction for complex data-platform initiatives.
• Troubleshoot difficult performance, integration, and scalability issues.
• Partner with engineering, analytics, architecture, and business teams to translate requirements into scalable technical solutions.
• Lead technical discussions and provide guidance to project teams and stakeholders.

Required Qualifications

• 10+ years of experience in data engineering, analytics engineering, or data/platform architecture.
• 5+ years of hands-on Databricks experience.
• Strong expertise with Apache Spark, Delta Lake, Unity Catalog, PySpark/Scala, and SQL.
• Experience designing and supporting enterprise-scale cloud data platforms.
• Strong understanding of data architecture, governance, performance optimization, and distributed data processing.
• Experience working in client-facing or cross-functional technical environments.
• Ability to lead architecture reviews and solve complex technical problems independently.

Qualified Databricks professionals with strong architecture and hands-on engineering experience are encouraged to apply through this posting.