Databricks Data Engineer

Veracy Consulting; Delhi; On-site; INR 1,500,000 - 3,000,000; Full time

Job summary

Veracy Consulting in Delhi, India is seeking skilled Databricks Engineers to design, build, and maintain modern, cloud-based data solutions. Join a collaborative team turning messy data into reliable pipelines and scalable data platforms.

The role focuses on Databricks, Apache Spark, Python and SQL, with emphasis on data quality, governance, and delivering robust ETL/ELT processes for energy trading and risk management data.

Qualifications

• 2–6 years of experience in data engineering or similar role.
• Strong Databricks experience.
• Proficient Python and SQL skills.
• Familiarity with cloud data platforms, ideally Azure.
• Experience with large datasets and data governance.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
• Build and optimize ETL/ELT processes for structured and semi-structured data.
• Write clean, efficient Python and SQL code.
• Collaborate with stakeholders to translate data requirements into robust solutions.
• Integrate data from multiple source systems into cloud platforms.
• Monitor, troubleshoot, and improve pipeline performance and data quality.
• Champion testing, documentation, and version control best practices.

Skills

Databricks

Python

SQL

Data pipelines

ETL/ELT

Azure

Git

Team communication

Tools

Delta Lake

Databricks Workflows

Unity Catalog

CI/CD

Job description

Veracy Consulting provides Consulting and Development services to Energy Trading & Risk Management clients. We’re growing our data team and looking for skilled Databricks Engineers to help design, build, and maintain modern, cloud-based data solutions. If you love turning messy data into reliable, scalable pipelines - and enjoy solving tough technical problems alongside a collaborative team — we’d love to hear from you.

What you’ll do

• Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark
• Build and optimize ETL/ELT processes for structured and semi-structured data
• Write clean, efficient, maintainable code in Python and SQL
• Partner with business and technical stakeholders to translate data requirements into robust solutions
• Integrate data from multiple source systems into cloud-based platforms
• Monitor, troubleshoot, and improve pipeline performance, reliability, and data quality
• Champion best practices in testing, documentation, version control, security, and data governance
• Contribute to code reviews, technical discussions, and continuous improvement

What we’re looking for

• 2-6 years in data engineering, data development, or a similar role
• Strong experience with Databricks
• Strong Python and SQL skills
• Solid grasp of data pipelines, ETL/ELT, and modern data engineering practices
• Experience with large datasets and optimizing data-processing workloads
• Familiarity with cloud data platforms — ideally Azure
• Comfortable with Git and collaborative development practices
• A structured, solution-oriented mindset and a sharp eye for data quality
• Strong communicator and team player

Nice to have

• Delta Lake, Databricks Workflows, and Unity Catalog
• CI/CD pipelines and infrastructure-as-code
• Supporting reporting, analytics, or BI use cases
• Work on impactful, modern cloud-data projects
• Grow your Databricks and data-engineering expertise in a collaborative environment
• Build scalable platforms that support real business decisions