DataBricks Engineer

Russell Tobin; New York (NY); Hybrid; USD 156,000 - 162,000; Full time

Benefits offered by this job

Healthcare coverage

Dental & Vision

Supplemental coverage

401(k) retirement

Life & disability insurance

Employee assistance program

Legal support

Home insurance

Pet insurance

Employee discounts

Job summary

Russell Tobin is seeking a Databricks Data Engineer in New York with 7+ years in data engineering and 3+ years on Databricks. You will design and implement data pipelines, transform data for analytics and incentive processing, and collaborate with cross-functional teams. Strong Python/Scala, SQL, and data modeling skills are required, with SIC domain experience a plus.

The role is hybrid in New York, offering a contract of 6+ months with possible extension and pay rate $75-78/hour on W2, DOE.

Qualifications

• 7+ years in data engineering with 3+ years on Databricks.
• Experience with Delta Lake, Delta Live Tables, Unity Catalog, Databricks Workflows.
• Proficient in Python/Scala and SQL.
• ETL/ELT, data modeling and pipeline orchestration expertise.
• SIC design, quota Allocation, attainment tracking and payouts familiarity.
• Experience integrating SIC platforms with data pipelines is a plus.
• Ability to translate complex business rules into scalable data architectures.
• Databricks certification is a plus.

Responsibilities

• Design, develop and implement Databricks data solutions.
• Ingest, transform and optimize data for analytics and SIC processing.
• Collaborate with cross-functional teams and stakeholders.
• Document architectures and ensure data quality and performance.

Skills

Databricks

Python/Scala

SQL

ETL/ELT

Data modeling

SIC understanding

Stakeholder management

Data pipelines

Tools

Delta Lake

Delta Live Tables

Unity Catalog

Databricks Workflows

Job description

DataBricks Engineer
Contract (6 Months + Possible Extension)
New York, NY (Hybrid)

Pay Rate: $75-78 an hour on W2 (DOE)

DataBricks Engineer
Contract (6 Months + Possible Extension)
New York, NY (Hybrid)

Pay Rate: $75-78 an hour on W2 (DOE)
Job Description:
We are seeking a skilled and experienced Databricks Data Engineer to join our team. As a Databricks Data Engineer, you will be responsible for designing, developing, and implementing data engineering solutions using the Databricks platform to support data ingestion, transformation, performance reporting, analytics, sales incentive compensation processing, and enterprise data pipeline processes.

Requirements:

• At least 7 years of experience in data engineering, with a minimum of 3 years hands-on experience building and managing Databricks solutions.
• Extensive experience with Databricks platform components: Delta Lake, Delta Live Tables, Unity Catalog and Databricks Workflows.
• Strong proficiency in Python and/or Scala for data engineering; SQL expertise required.
• Strong experience with core data engineering practices including data ingestion, transformation (ETL/ELT), data modeling, and pipeline orchestration.
• Functional and technical experience with Sales Incentive Compensation (SIC) — including incentive plan design, quota allocation, attainment tracking, commission calculations, and payout processing.
• Familiarity with SIC platforms (e.g., Anaplan, Oracle ICM, Varicent) and experience integrating them with data engineering platforms is a strong plus.
• Ability to understand complex incentive compensation rules and model them accurately in data pipelines and transformation logic.
• Experience integrating Databricks with cloud platforms (Azure, AWS, or GCP) and data sources such as Azure Data Factory, Event Hubs, Kafka, or equivalent.
• Passion and ability to create holistic, end-to-end, integrated data solutions.
• Stakeholder management skills; ability to challenge constructively and translate complex business requirements into scalable data architectures — not just gather requirements.
• Strong analytical and problem-solving skills, with the ability to understand complex data requirements and implement reliable, efficient solutions.
• Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams and stakeholders.
• Must work well in a team environment while being a resourceful, independent self-starter able to work effectively with minimal direction.
• Skilled at documenting complex data architectures and pipelines in a professional manner.
• Ability to work independently and manage multiple tasks and priorities in a fast-paced environment.
• Passion for data quality, performance optimization, and delivering excellent end-user data experiences.
• Databricks Certified Data Engineer Associate or Professional certification is a plus.
  

#RTA

Russell Tobin offers eligible employee’s comprehensive healthcare coverage (medical, dental, and vision plans), supplemental coverage (accident insurance, critical illness insurance and hospital indemnity), 401(k)-retirement savings, life & disability insurance, an employee assistance program, legal support, auto, home insurance, pet insurance and employee discounts with preferred vendors.

Equal Employment Opportunity Russell Tobin is an equal opportunity employer. We do not discriminate on the basis of the race, religious creed, color, national origin, ancestry, physical disability, mental disability, reproductive health decision making, medical condition, genetic information, marital status, sex, gender, gender identity, gender expression, age, sexual orientation, veteran or military status, or any other characteristic protected by applicable federal, state, or local law.

Fair Chance Employment Russell Tobinis a Fair Chance employer. We consider all qualified applicants, including those with criminal histories, in a manner consistent with applicable state and local Fair Chance laws and ordinances, including, the California Fair Chance Act and all applicable local Fair Chance ordinances.

Accommodations We are committed to providing reasonable accommodations to applicants and employees with disabilities. If you require a reasonable accommodation to participate in the application or interview process, or to perform the essential functions of this role, please contact us.
Under the San Francisco Lactation in the Workplace Ordinance, we will provide written notice of lactation accommodation rights, and this notice will automatically be given upon hiring, any inquiry of parental leave or lactation accommodation.