Senior Databricks Engineer – Data Platform & Analytics

Ventures Unlimited Inc; O’Fallon (MO); On-site; USD 90,000 - 120,000; Full time

Job summary

A renowned IT consulting firm is seeking a Recruitment Manager to lead IT recruitment in O’Fallon, Missouri. This mid-senior level role focuses on building and optimizing data pipelines using Databricks, along with ensuring data observability and effective recruitment strategies. The ideal candidate will have at least 8+ years of experience in IT recruitment and software development, with a strong emphasis on Python, real-time data processing, and CI/CD practices. Join a dynamic team and impact the future of data-driven solutions.

Qualifications

• 8+ years of experience in software development and delivery.
• Strongly prefer experience with real-time data processing and data streaming.
• Experience with CI/CD and agile methodologies.

Responsibilities

• Design, develop, and optimize data pipelines on Databricks.
• Build data transformation workflows using dbt and TDD.
• Implement data observability practices across platforms.

Skills

Data observability tools

Databricks

Python

Spark SQL

Power BI

CI/CD

Real-time data processing

SQL development

Cloud infrastructure (AWS, Azure, GCP)

Agile methodologies

Education

Bachelor's degree in Computer Science or related field

Tools

Git

Jenkins

Kafka

Job description

A renowned IT consulting firm is seeking a Recruitment Manager to lead IT recruitment in O’Fallon, Missouri. This mid-senior level role focuses on building and optimizing data pipelines using Databricks, along with ensuring data observability and effective recruitment strategies. The ideal candidate will have at least 8+ years of experience in IT recruitment and software development, with a strong emphasis on Python, real-time data processing, and CI/CD practices. Join a dynamic team and impact the future of data-driven solutions.