Data Engineer / Data Scientist

Lever, Inc.; India; Remote; INR 1,800,000 - 3,000,000; Full time

Benefits offered by this job

Remote work

Cloud and ML tech exposure

Collaborative environment

Job summary

Lever, Inc. in India is seeking a Data Engineer / Data Scientist to design and optimize scalable data pipelines and ML solutions in a cloud-first environment. You will work with Python, PySpark, Azure Databricks, Delta Lake, and real-time streaming technologies to support analytics and AI initiatives.

As an individual contributor, you will own deliverables while collaborating with tech and business teams. Remote-friendly role offering exposure to modern cloud and ML technologies such as Kafka

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, IT, or related field.
• 5–8 years of professional experience in data engineering, data science, or cloud analytics.
• Strong hands-on experience with Python and PySpark in Azure environments.
• Knowledge of Azure services, Databricks, and MLOps practices.

Responsibilities

• Develop scalable data processing and ML solutions across cloud environments.
• Build, maintain, and optimize batch and real-time streaming pipelines.
• Develop Spark DataFrame transformations and data workflows.
• Debug and optimize Spark applications and Databricks jobs.
• Implement Delta Lake for reliability, versioning, and performance.
• Create APIs for data and ML applications using Python/Scala.
• Support ML initiatives, MLOps, and model deployment activities.
• Collaborate with engineering, data science, and business teams.
• Configure Databricks clusters and notebook workflows.
• Ensure data quality through validation checks and monitoring.

Skills

Python

PySpark

Azure

MLops

Data pipelines

Education

Bachelor's degree in CS/Engineering

Tools

Azure Databricks

Kafka

Event Hubs

Delta Lake

APIs

Databricks notebooks

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer / Data Scientist based in India.

This role offers the opportunity to build scalable data and machine learning solutions within a technology-driven, collaborative environment. You will design and optimize batch and streaming data pipelines using Python, PySpark, and Azure Databricks. The position combines data engineering with machine learning, MLOps, and model deployment to support impactful analytics and AI initiatives. You will work extensively with Azure services, Spark, Delta Lake, APIs, and cloud-based data architectures. As an individual contributor, you will take ownership of assigned deliverables while partnering closely with technical and business teams. The role is well suited to an experienced data professional who enjoys solving complex data challenges and working across modern cloud and ML technologies.

Accountabilities

You will develop, optimize, and support modern data processing and machine learning solutions across cloud-based environments. The role requires strong technical ownership, practical problem-solving, and collaboration across engineering, data science, and business teams.

• Develop scalable data processing solutions using Python, PySpark, and Azure Databricks.
• Build, maintain, and optimize batch and real-time streaming data pipelines.
• Develop Spark DataFrame-based transformations and data processing workflows.
• Debug, troubleshoot, and optimize Spark applications and Databricks jobs.
• Implement Delta Lake solutions to improve data reliability, versioning, and query performance.
• Develop APIs using Python or Scala for data and machine learning applications.
• Support machine learning initiatives, MLOps workflows, and model deployment activities.
• Work with Azure services for data ingestion, storage, security, integration, and processing.
• Configure and manage Databricks job clusters, compute environments, and notebook workflows.
• Build and execute DataFrame-based data validation and quality checks.
• Develop pipelines using Event Hubs, Kafka, IoT sources, or other real-time data technologies.
• Support data quality monitoring and production troubleshooting.
• Implement secure integrations between Azure services using managed identities and secrets.
• Contribute to CI/CD practices for data engineering and machine learning workloads.
• Collaborate with technical and business stakeholders while independently managing assigned deliverables.
• Apply performance tuning techniques to Spark applications and Databricks workloads.

Requirements

The ideal candidate brings 5–8 years of relevant experience across data engineering, data science, machine learning, or cloud analytics, with strong hands‑on capabilities in Python, PySpark, Azure, and Databricks. You should be comfortable developing production‑ready data solutions, troubleshooting distributed processing workloads, and contributing to machine learning and MLOps initiatives.

• Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, Information Technology, or a related discipline.
• 5–8 years of relevant professional experience in data engineering, data science, machine learning, or cloud analytics.
• Strong hands‑on expertise in Python and PySpark.
• Good knowledge of Microsoft Azure and Azure Databricks.
• Hands‑on experience with MLOps practices and tools.
• Practical experience supporting machine learning projects.
• Basic understanding of machine learning model deployment.
• Strong experience developing and debugging Spark-based applications.
• Hands‑on experience with Databricks notebook development.
• Strong knowledge of Spark DataFrames using PySpark or Scala.
• Experience optimizing Spark jobs and Databricks workloads.
• Experience developing APIs using Python or Scala.
• Working knowledge of Azure Event Hubs, Storage Accounts, Key Vault, Service Bus, Azure Functions, and Azure Data Lake Storage.
• Understanding of Databricks job clusters and compute configurations.
• Experience implementing cloud-based data solutions on Azure.
• Knowledge of real‑time streaming technologies such as Kafka.
• Experience developing batch and streaming pipelines using Event Hubs, Kafka, or IoT data sources.
• Hands‑on experience implementing Delta Lake solutions.
• Working knowledge of GitHub or similar version‑control platforms.
• Exposure to MLflow or comparable tools for experiment tracking and model lifecycle management is beneficial.
• Experience with CI/CD for data and machine learning workloads is a plus.
• Knowledge of data quality validation, monitoring, and production support is advantageous.
• Strong analytical and problem‑solving abilities.
• Ability to work independently while collaborating effectively with cross‑functional project teams.

Benefits

• Full‑time position.
• Remote work arrangement.
• Immediate requirement with an opportunity to join a technology‑focused data and AI environment.
• Opportunity to work with modern cloud technologies including Microsoft Azure and Azure Databricks.
• Hands‑on exposure to Python, PySpark, Spark, Delta Lake, streaming, and MLOps.
• Opportunity to contribute to machine learning projects and model deployment initiatives.
• Exposure to real‑time data technologies such as Kafka, Event Hubs, and IoT data sources.
• Opportunities to work across data engineering, machine learning, and cloud analytics.
• Collaboration with technical and business teams on impactful data initiatives.
• Scope for continued development in cloud, data engineering, and machine learning technologies.