AWS Data Engineer

Mindfire Solutions; Delhi; On-site; INR 1,200,000 - 2,400,000; Full time

Benefits offered by this job

Generative AI exposure

LangChain experience

LLM/NLP experience

AWS Bedrock familiarity

ML deployment basics

Job summary

Mindfire Solutions in Delhi is seeking an experienced Data Engineer to design, develop, and maintain scalable ETL/ELT pipelines, data lakes, and warehouse solutions, leveraging Python, SQL, and AWS services.

You will build real-time data ingestion, containerised apps with Docker and ECS, and implement IaC with Terraform and CloudFormation, while upholding security and governance standards.

Qualifications

• Bachelor's or Master's degree in CS/IT or related field.
• 3–5 years of hands-on Data Engineering and AWS cloud experience.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines, data lakes, and data warehouse solutions.
• Build and optimize data ingestion frameworks for batch and real-time processing.
• Develop and deploy containerised applications using Docker, Amazon ECR, and Amazon ECS.
• Design and implement RESTful APIs for system integrations using modern frameworks (e.g., FastAPI).
• Implement Infrastructure as Code (IaC) using Terraform and AWS CloudFormation.
• Establish and maintain CI/CD pipelines for automated build, test, and deployment workflows.
• Ensure adherence to data security, governance, and compliance standards (e.g., encryption, access control).
• Monitor, troubleshoot, and optimise data workflows for performance and reliability.

Skills

Python

SQL

FastAPI

PySpark

AWS Lambda

API Gateway

CI/CD

Data security & governance

Problem-solving

Communication

Collaboration

Education

Bachelor's or Master's in CS/IT

Tools

Docker

AWS ECS

Terraform

AWS CloudFormation

LangChain

Job description

Core Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines, data lakes, and data warehouse solutions.


• Build and optimize data ingestion frameworks for batch and real-time processing.


• Develop and deploy containerised applications using Docker, Amazon ECR, and Amazon ECS.


• Design and implement RESTful APIs for system integrations using modern frameworks (e.g., FastAPI).


• Implement Infrastructure as Code (IaC) using Terraform and AWS CloudFormation.


• Establish and maintain CI/CD pipelines for automated build, test, and deployment workflows.


• Ensure adherence to data security, governance, and compliance standards (e.g., encryption, access control).


• Monitor, troubleshoot, and optimise data workflows for performance and reliability.

Required Skills

• Strong proficiency in Python and SQL for data processing and transformation.


• Hands-on experience with FastAPI for API development.


• Experience with distributed data processing frameworks such as PySpark.


• Solid experience with AWS services, including: AWS Lambda, AWS DMS, API Gateway.


• Experience with containerization and orchestration (Docker, ECS).


• Strong understanding of cloud-native architecture and best practices.


• Excellent problem-solving, communication, and collaboration skills.

Nice to have

• Exposure to Generative AI and Agentic AI concepts.


• Hands-on experience with LLM frameworks such as LangChain, LlamaIndex, or CrewAI.


• Experience working with LLMs and NLP models (e.g., GPT, BERT, LLaMA, Mistral, Gemini).


• Familiarity with LLM-as-a-Service platforms such as AWS Bedrock or Hugging Face.


• Basic understanding of ML model deployment and lifecycle management.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.


• 3-5 years of hands-on experience in Data Engineering and AWS cloud environments.