Associate III - Data Engineering

UST; Dadri; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

UST is seeking a software engineer with 3–5 years of experience in infrastructure or data platforms. You will work with Python/PySpark, SQL, and Terraform to manage data pipelines and cloud infrastructure.

You will engage with Databricks Unity Catalog and DABs, build RESTful APIs with FastAPI/Flask, and contribute to MLOps and data engineering initiatives across Spark/Delta Lake ecosystems.

Qualifications

• 3 to 5 years of software engineering experience, preferably in infrastructure or data platforms.
• Proficiency in Python/PySpark and strong SQL knowledge.
• Experience with Terraform or similar Infrastructure-as-Code tools and at least one cloud platform (GCP/AWS).
• Strong background in Databricks Unity Catalog, medallion architecture, and Databricks Asset Bundles (DABs).
• Proficiency in Python and API frameworks such as FastAPI or Flask, and RESTful APIs.

Responsibilities

• Design, develop, and maintain data platform components and pipelines.
• Collaborate on API development using FastAPI or Flask.
• Contribute to MLOps and data engineering tasks, including Spark/Delta Lake/MLflow workflows.

Skills

Python

PySpark

SQL

CI/CD

MLOps

Data Modeling

Workflow Automation

Databricks

FastAPI

Flask

Tools

Terraform

Databricks

MLflow

Delta Lake

Fivetran

Lakeflow Connect

Job description

Role Description

3 to 5 years of Software engineering experience, ideally in infrastructure or data platform roles.

• Proficiency in Python/PySpark and strong knowledge of SQL.
• Experience with Terraform or similar Infrastructure-as-Code (IaC) tools.
• Strong knowledge of at least one cloud platform (GCP/AWS)
• Solid CI/CD experience
• Data & MLOps Expertise:
• Strong Background in Databricks, specifically Unity Catalog, medallion architecture, and Databricks Asset Bundles (DABs).
• Proficiency in Python and API frameworks such as FastAPI or Flask.
• Strong experience designing, developing, and maintaining RESTful APIs.
• Good understanding of data engineering concepts such as data modeling and workflow orchestration.
• Experience with MLOps and Data Engineering like having worked on Spark/Delta Lake/MLflow/ETL frameworks (e.g., Fivetran, Lakeflow Connect).
• Operational Excellence:
• Experience with Github workflows, including container versioning.
• Proficiency with incident management and observability tools (e.g., PagerDuty, system monitoring).
• An AI as a partner mindset.
• A collaborative approach, including asking questions, navigating unclear requirements, and actively contributing to knowledge sharing/community enablement.

Role Description

3 to 5 years of Software engineering experience, ideally in infrastructure or data platform roles.

• Proficiency in Python/PySpark and strong knowledge of SQL.
• Experience with Terraform or similar Infrastructure-as-Code (IaC) tools.
• Strong knowledge of at least one cloud platform (GCP/AWS)
• Solid CI/CD experience
• Data & MLOps Expertise:
• Strong Background in Databricks, specifically Unity Catalog, medallion architecture, and Databricks Asset Bundles (DABs).
• Proficiency in Python and API frameworks such as FastAPI or Flask.
• Strong experience designing, developing, and maintaining RESTful APIs.
• Good understanding of data engineering concepts such as data modeling and workflow orchestration.
• Experience with MLOps and Data Engineering like having worked on Spark/Delta Lake/MLflow/ETL frameworks (e.g., Fivetran, Lakeflow Connect).
• Operational Excellence:
• Experience with Github workflows, including container versioning.
• Proficiency with incident management and observability tools (e.g., PagerDuty, system monitoring).
• An AI as a partner mindset.
• A collaborative approach, including asking questions, navigating unclear requirements, and actively contributing to knowledge sharing/community enablement.

Skills

• Databricks
• FastAPI
• Terraform
• Python
• ETL
• Infrastructure as Code
• MLOps
• Data Modeling
• Workflow Automation
• PySpark
• Flask