Lead Databricks Engineer

Falcon Smart IT (FalconSmartIT); Columbus (OH); On-site; USD 120,000 - 180,000; Full time

Job summary

Falcon Smart IT is seeking a DiTA/Predictive Attrition developer to implement, modify and maintain data and ML pipelines in our cloud environment. You will collaborate with data engineers, architects and business stakeholders to ensure reliable data integration and modeling.

The role emphasizes Databricks Unity Catalog, Terraform, Python and MLOps practices, plus observability via dashboards to monitor performance and impact on attrition predictions.

Qualifications

• Data integration, transformation and loading in cloud environments.
• Collaborate with data engineers, architects and business stakeholders.
• Experience with Databricks, Unity Catalog, SQL warehouses, CI/CD, and MLOps.

Responsibilities

• Implement, modify and maintain DiTA/Predictive Attrition development.
• Ensure observability with monitoring dashboards.
• Collaborate with cross-functional teams to deliver scalable data solutions.

Skills

Databricks

Terraform

Python

Generative AI

SQL

MLOps

CI/CD

Tools

Databricks Unity Catalog

AWS SageMaker

AWS ECS

AWS Lambda

SSM parameters

EventBridge

Terraform

OpenAI/Claude

GCP LLM models

Job description

Primary Skill : Data Bricks, Terraform, Databricks Unity Catalog

Job Requirements

We are seeking a DiTA/Predictive Attrition developer responsible for implementation, modification and maintenance.

The role involves working closely with data engineers, architects, and business stakeholders to ensure data integration, transformation, and loading processes within the cloud environment. Databricks, Genie one, Unity catalog, SQL warehouse, compute setup DABS pipeline, I/CD, MLOPs principles, AWS MLOPs(Sage maker, ECS, Lambda SSM parameter, event bridge)Terraform, python, Generative AI (open AI, Claud, GCP LLM models) Observability (Monitoring dashboards)Work Experience 7-10Years