Data Engineer - IT

Guardian; Gurgaon; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Guardian is seeking a data engineer with strong Databricks and Spark expertise for work in India. The role focuses on building scalable data pipelines, optimizing Spark SQL performance, and implementing ETL processes across data warehouses and lakes.

Collaboration with teams on governance and deployment best practices is expected. Candidates should have hands-on experience with Delta Lake, Unity Catalog, or similar governance tools, and familiarity with CI/CD concepts and Git-based workflows.

Qualifications

• Strong hands-on experience with Databricks and Spark stack.
• Proficient in PySpark, SQL, ETL design and data modeling.
• Experience building scalable data pipelines for structured/semi-structured data.

Skills

Databricks

PySpark

Advanced SQL

ETL concepts

Data warehousing

Data modeling

Data pipelines

Delta Lake

Spark optimization

Git

Tools

Git

Jenkins

Delta Lake

Job description

Job Description:

Job Description:

Required Skills & Experience
Strong hands-on experience in:

• Databricks (Lakehouse platform, notebooks, jobs, clusters)
• PySpark / Apache Spark
• Advanced SQL (joins, window functions, performance tuning)
• Solid understanding of ETL concepts, data warehousing, and data modeling
• Experience building scalable data pipelines for structured and semi-structured data
• Familiarity with data lake / delta lake architecture
• Good understanding of performance optimization techniques in Spark and SQL
• Experience with version control (Git) and deployment practices
• Strong problem-solving and analytical skills

Good to Have

• Experience with cloud platforms (AWS)
• Exposure to CI/CD pipelines (Jenkins)
• Knowledge of Delta Lake, Unity Catalog, or similar governance tools
• Understanding of data quality frameworks / monitoring tools
• Experience working in Agile / Scrum teams

Location:

This position can be based in any of the following locations:

Chennai, Gurgaon