Senior AI ML/ Gen AI Engineer

Quadratyx; Chennai District; On-site; INR 1,500,000 - 2,400,000; Full time

Job summary

Quadratyx is seeking a hands-on Data Engineer to design and operate scalable batch and streaming data pipelines using PySpark and modern backend frameworks. You will build Python-based microservices, APIs, and collaborate with AI/GenAI teams to enable LLM and RAG-powered applications.

The role requires strong coding skills, distributed systems experience, and comfort working in a fast-paced data-driven environment. Hyderabad/Chennai based, on-site work with modern data platforms.

Qualifications

• Must have strong hands-on PySpark and distributed data processing.
• Practical Python programming expertise.
• Experience building APIs/backend services (Flask/FastAPI).
• Good understanding of Big Data and streaming architectures.

Responsibilities

• Design, develop, and optimize scalable batch and streaming pipelines.
• Build real-time streaming solutions with Kafka/Flink where applicable.
• Create and maintain Python-based microservices and APIs.
• Collaborate with AI/GenAI teams to support LLM/RAG apps.
• Ensure reliability, monitoring, and maintainability of data systems.

Skills

PySpark

Python

Linux

API design

Distributed systems

Tools

Flask

FastAPI

Apache Flink

Docker

Job description

We are a product-centric insight & automation services company globally. We help the world’s organizationsmake better & faster decisions using the power of insight & intelligent automation.We build and operationalize their next-gen strategy, through Big Data, Artificial Intelligence, MachineLearning, Unstructured Data Processing and Advanced Analytics. Quadratyx can boast of more extensiveexperience in data sciences & analytics than most other companies in India. We firmly believe in
Excellence Everywhere.

Job / Role Information

Designation:

Function:

• Technical

Location:

• Hyderabad, Chennai

Job Description

Purpose of the Job/ Role:

• We are looking for a highly skilled and hands-on Data Engineer with strong expertise in Big Data technologies, distributed data processing, and backend engineering. The ideal candidate should have practical experience in building scalable batch and real-time data pipelines using frameworks such as PySpark, Kafka, and streaming platforms.
• The candidate should also possess strong Python programming skills and experience building APIs/microservices using modern backend frameworks.
• This role is best suited for engineers who enjoy solving large-scale data engineering problems and working closely with AI/GenAI-driven applications.
• We are specifically looking for candidates with strong hands-on coding and distributed systems experience, and not candidates whose experience is predominantly limited to traditional ETL tools, low-code cloud pipelines, or Snowflake-centric workflows.

Key Requisites:

• Design, develop, and optimize scalable batch and streaming data pipelines
• Develop real-time streaming solutions using Kafka and/or Apache Flink
• Build and maintain APIs and backend services using Python-based microservice frameworks
• Work with structured and semi-structured datasets across multiple storage systems
• Optimize pipeline performance, scalability, and fault tolerance
• Collaborate with AI/ML and GenAI teams to support LLM and RAG-based applications
• Participate in architecture discussions, debugging, code reviews, and production deployments
• Ensure reliability, monitoring, and maintainability of data engineering systems

Mandatory Skills

• Strong hands-on experience with PySpark and distributed data processing (must-have)
• Strong programming expertise in Python
• Hands-on experience building APIs and backend services using:
• Flask
• FastAPI
• Good understanding of Big Data ecosystem and streaming architectures

Experience with:

• Apache Flink (preferred)
• Experience building end-to-end batch and real-time data engineering pipelines
• Good understanding of scalable system design, partitioning, optimization, and distributed processing concepts
• Understanding of microservices architecture and API integrations
• Experience working in Linux-based environments
• Strong analytical and problem-solving skills

GenAI / AI Engineering Expectations

Candidates should also have practical exposure to modern GenAI concepts and tools, including:

• Strong analytical and problem-solving skills
• Working knowledge of LLMs and prompt engineering
• Understanding of RAG (Retrieval-Augmented Generation) concepts

Experience using GenAI frameworks/tools such as:

• LangChain
• LlamaIndex (good to have)
• Exposure to vector databases, embeddings, or AI-assisted workflows is a plus

Good to Have / Bonus Skills

• Experience with QuestDB
• Exposure to real-time analytics and streaming systems
• Understanding of distributed systems internals
• Experience with Docker/containerization and deployment workflows
• Familiarity with monitoring and logging frameworks
• Knowledge of cloud platforms is an added advantage
• This role is focused on core data engineering, distributed systems, and backend engineering.

Candidates with predominantly:

• Snowflake-only experience
• may not be the right fit for this requirement unless they also possess strong hands-on Big Data engineering and programming experience.

Working Relationships

Reporting to

• Vice President

External Stakeholders

• Clients

Preferred Candidate Profile

• Strong ownership mindset and problem-solving ability
• Minimum 5-8 years of work experience as a Data Manager in an IT organization (preferably Analytics / Big Data/ Data Science / AI background).
• Comfortable working independently on complex engineering problems
• Ability to work in fast-paced product/research-oriented environments
• Passion for learning modern AI and data technologies
• Good communication and collaboration skills

Quadratyx is an equal opportunity employer - we will never differentiate candidates on the basis of religion, caste, gender, language, disabilities or ethnic group.

Quadratyx reserves the right to place/move any candidate to any companylocation, partner location or customer location globally, in the bestinterest of Quadratyx business.