Databricks Data Engineer - Lead Cloud Solutions

Inizio Partners Corp; New York (NY); On-site; USD 120,000 - 150,000; Full time

Job summary

A consulting firm in New York is seeking a Databricks Data Engineer to design and implement data solutions tailored to client needs. The ideal candidate will have over 5 years of experience with Databricks and a strong understanding of data architecture. Responsibilities include engaging with clients, defining coding standards for ETL processes, and developing comprehensive architecture solution roadmaps. The role requires excellent communication skills and the ability to work independently.

Qualifications

• 5+ years experience in architecture, design, and implementation using Databricks.
• Experience in designing and implementing scalable, fault-tolerant systems.
• Deep understanding of big data computing technologies such as Databricks and Snowflake.
• In-depth knowledge of spark internals and Databricks runtime environment.

Responsibilities

• Design and develop data systems tailored to client-specific requirements.
• Engage with clients to evaluate needs and provide strategic recommendations.
• Define coding standards for ETL processes ensuring best practices.
• Architect and implement CI/CD pipelines for Databricks environments.

Skills

Databricks

SQL

Python

Agile Methodologies

Communication Skills

Education

B.S. Degree in a data-centric field

Tools

Airflow

Git

Azure DevOps

Power BI

Kubernetes

Splunk

Job description

A consulting firm in New York is seeking a Databricks Data Engineer to design and implement data solutions tailored to client needs. The ideal candidate will have over 5 years of experience with Databricks and a strong understanding of data architecture. Responsibilities include engaging with clients, defining coding standards for ETL processes, and developing comprehensive architecture solution roadmaps. The role requires excellent communication skills and the ability to work independently.