Senior Databricks Engineer

Codvo.ai; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Codvo.ai in Pune, Maharashtra, is looking for a Full-Stack Data Engineer specializing in Data & ML Automation using Databricks. The candidate will develop and automate workflows, validate data, and build internal tools and dashboards using frameworks like React and Streamlit. Key responsibilities include creating CLI tools and implementing testing frameworks for ETL/ELT pipelines. Familiarity with Databricks API and CI/CD workflows is essential for success in this role.

Qualifications

• Experience with fullstack development, particularly in data engineering.
• Familiarity with automation tooling and testing frameworks.
• Understanding of ML workflows and data validation processes.

Responsibilities

• Develop CLI tools and scripts to automate workflows.
• Automate Databricks workflows and MLOps operations.
• Design and implement integration tests for ML inference workflows.
• Build internal applications using frameworks like React.

Skills

Databricks APIs

Terraform

React

Streamlit

Integration tests

CI/CD workflows

Tools

Power BI

Tableau

Looker

Job description

Data & ML Automation (Databricks) (Pune)

Pune, Maharashtra

Work Type: Full Time

Job Title: Full-Stack Data Engineer – Data & ML Automation (Databricks)

Overview

We are seeking a Fullstack Data Engineer who can operate at the intersection of data engineering, fullstack development, and testing/automation. This role will focus on building internal automation tooling, validating data and ML workflows, and developing lightweight web interfaces or dashboards that help teams interact with data and model outputs more effectively. The ideal candidate is a versatile engineer who can seamlessly switch between CLI tooling, pipeline testing, backend glue code, and rapid visualization apps.

Key Responsibilities

Automation & Tooling

• Develop CLI tools, scripts, and utilities to automate repetitive workflows across the data platform, including:
  • Pipeline execution
  • Governance and metadata synchronization
  • Environment setup
  • Test harness creation
• Automate Databricks workflows, job deployments, environment provisioning, and MLOps operations using:
  • Databricks APIs
  • Terraform
  • Databricks SDK

Data & ML Workflow Testing

• Design and implement:
  • Integration tests
  • End‑to‑end pipeline tests
  • Validation frameworks for Databricks ETL/ELT pipelines and ML inference workflows
• Improve reliability, observability, and overall engineering productivity across the data & ML team.

Internal Tools & Dashboards

• Build quick internal applications using React, Streamlit, or similar frameworks to:
  • Visualize data flows
  • Provide model inference demos
  • Enable operational or configuration controls
• Develop internal productivity and monitoring dashboards

Engineering Practices

• Apply best practices around:
  • Spec‑driven development
  • Coding agents and automation patterns
  • CI/CD workflows for data/ML systems

Nice‑to‑Have Skills

• Experience creating dashboards and visualizations using Databricks AI/BI or any modern BI tool (Power BI, Tableau, Looker, etc.)