Senior AIML/ Gen AI Engineer

Quadratyx; Chennai District; On-site; INR 900,000 - 1,500,000; Full time

Benefits offered by this job

Equal opportunity employer

Global locations

Job summary

Quadratyx is seeking a Senior Data Engineer focused on Big Data and GenAI to design and operate scalable data pipelines using PySpark, Kafka, and streaming platforms. You will develop Python-based microservices, APIs, and backend services, collaborating with AI/GenAI teams to support LLM/RAG applications.

Ideal candidates have hands-on experience with distributed systems, real-time analytics, Linux environments, and a strong foundation in data engineering and backend engineering.

Qualifications

• Strong hands-on experience with PySpark and distributed data processing.
• Strong programming expertise in Python.
• Hands-on experience building APIs and backend services using Flask or FastAPI.
• Good understanding of Big Data ecosystem and streaming architectures.

Responsibilities

• Design, develop, and optimize scalable batch and streaming data pipelines.
• Build large-scale distributed data processing systems using PySpark.
• Develop real-time streaming solutions using Kafka and/or Apache Flink.
• Build and maintain APIs and backend services using Python-based microservice frameworks.
• Work with structured and semi-structured datasets across multiple storage systems.
• Optimize pipeline performance, scalability, and fault tolerance.
• Collaborate with AI/ML and GenAI teams to support LLM and RAG-based applications.
• Participate in architecture discussions, debugging, code reviews, and production deployments.
• Ensure reliability, monitoring, and maintainability of data engineering systems.

Skills

PySpark

Python

Flask

FastAPI

Microservices

Kafka

Flink

APIs

Linux

LangChain

LlamaIndex

Vector databases

Tools

Kafka

Flink

PostgreSQL

Linux

Docker

Job description

About Quadratyx

We are a product-centric insight & automation services company globally. We help the world’s organizations make better & faster decisions using the power of insight & intelligent automation. We build and operationalize their next-gen strategy, through Big Data, Artificial Intelligence, Machine Learning, Unstructured Data Processing and Advanced Analytics. Quadratyx can boast of more extensive experience in data sciences & analytics than most other companies in India. We firmly believe in

About Quadratyx

We are a product-centric insight & automation services company globally. We help the world’s organizations make better & faster decisions using the power of insight & intelligent automation. We build and operationalize their next-gen strategy, through Big Data, Artificial Intelligence, Machine Learning, Unstructured Data Processing and Advanced Analytics. Quadratyx can boast of more extensive experience in data sciences & analytics than most other companies in India. We firmly believe in

Excellence Everywhere.

Senior AI ML/ Gen AI Engineer

Job / Role Information

Designation

• Senior Data Engineer – Big Data & GenAI

Function

• Technical

Location

• Hyderabad, Chennai

Job Description

Purpose of the Job/ Role

• We are looking for a highly skilled and hands-on Data Engineer with strong expertise in Big Data technologies, distributed data processing, and backend engineering. The ideal candidate should have practical experience in building scalable batch and real-time data pipelines using frameworks such as PySpark, Kafka, and streaming platforms.
• The candidate should also possess strong Python programming skills and experience building APIs/microservices using modern backend frameworks.
• This role is best suited for engineers who enjoy solving large-scale data engineering problems and working closely with AI/GenAI-driven applications.
• We are specifically looking for candidates with strong hands-on coding and distributed systems experience, and not candidates whose experience is predominantly limited to traditional ETL tools, low-code cloud pipelines, or Snowflake-centric workflows.

Key Requisites

• Design, develop, and optimize scalable batch and streaming data pipelines
• Build large-scale distributed data processing systems using PySpark
• Develop real-time streaming solutions using Kafka and/or Apache Flink
• Build and maintain APIs and backend services using Python-based microservice frameworks
• Work with structured and semi-structured datasets across multiple storage systems
• Optimize pipeline performance, scalability, and fault tolerance
• Collaborate with AI/ML and GenAI teams to support LLM and RAG-based applications
• Participate in architecture discussions, debugging, code reviews, and production deployments
• Ensure reliability, monitoring, and maintainability of data engineering systems

Mandatory Skills

• Strong hands-on experience with PySpark and distributed data processing (must-have)
• Strong programming expertise in Python
• Hands‑on experience building APIs and backend services using:
• Flask
• FastAPI
• or similar microservice frameworks
• Good understanding of Big Data ecosystem and streaming architectures

Experience With

• Apache Kafka
• Apache Flink (preferred)
• PostgreSQL
• Experience building end-to-end batch and real‑time data engineering pipelines
• Good understanding of scalable system design, partitioning, optimization, and distributed processing concepts
• Understanding of microservices architecture and API integrations
• Experience working in Linux‑based environments
• Strong analytical and problem‑solving skills

GenAI / AI Engineering Expectations

Candidates should also have practical exposure to modern GenAI concepts and tools, including:

• Strong analytical and problem‑solving skills
• Working knowledge of LLMs and prompt engineering
• Understanding of RAG (Retrieval‑Augmented Generation) concepts

Experience Using GenAI Frameworks/tools Such As

• LangChain
• LlamaIndex (good to have)
• Exposure to vector databases, embeddings, or AI‑assisted workflows is a plus

Good to Have / Bonus Skills

• Experience with QuestDB
• Exposure to real‑time analytics and streaming systems
• Understanding of distributed systems internals
• Experience with Docker/containerization and deployment workflows
• Familiarity with monitoring and logging frameworks
• Knowledge of cloud platforms is an added advantage

Important Note

• This role is focused on core data engineering, distributed systems, and backend engineering.

Candidates With Predominantly

• Snowflake‑only experience
• Low‑code ETL pipeline development
• Pure cloud ETL orchestration backgrounds
• may not be the right fit for this requirement unless they also possess strong hands‑on Big Data engineering and programming experience.

Working Relationships

• Vice President

External Stakeholders

• Clients

Preferred Candidate Profile

• Strong ownership mindset and problem‑solving ability
• Minimum 5-8 years of work experience as a Data Manager in an IT organization (preferably Analytics / Big Data/ Data Science / AI background).
• Comfortable working independently on complex engineering problems
• Ability to work in fast‑paced product/research‑oriented environments
• Passion for learning modern AI and data technologies
• Good communication and collaboration skills

Quadratyx is an equal opportunity employer - we will never differentiate candidates on the basis of religion, caste, gender, language, disabilities or ethnic group.

Quadratyx reserves the right to place/move any candidate to any company location, partner location or customer location globally, in the best interest of Quadratyx business.