Senior Data Engineer (with AI/ML experience) India

IDT Corporation; Bengaluru; On-site; INR 6,433,823 - 9,191,176; Full time

Benefits offered by this job

Competitive salary

Stability and growth opportunities

Referral program

Professional training support

Company-supported English classes

Job summary

A leading communications services firm is seeking a skilled Data/ML Engineer to join their team in Bengaluru. You will design and maintain data pipelines and integrate advanced machine learning techniques into workflows. The ideal candidate should have at least 5 years of experience in data engineering, strong Python and SQL skills, and familiarity with big data technologies. The role offers significant flexibility in terms of remote working and promises continuous opportunities for professional growth.

Qualifications

• 5+ years of experience in building scalable data infrastructure.
• Excellent communication skills in English.
• Strong experience with Python for data tasks.
• Deep understanding of vector databases and RAG architectures.
• Experience with cloud platforms for managed LLM deployments.

Responsibilities

• Design and maintain scalable data pipelines.
• Optimize workflows for semantic representation of unstructured data.
• Implement analytics and dashboard solutions.
• Manage processes for prompt engineering and model fine-tuning.
• Document all data processes and workflows.

Skills

Data engineering

Machine learning techniques

Python programming

Cloud platforms (AWS/Azure)

SQL

Big data technologies

Version control systems

Business Intelligence

Tools

Apache Spark

Hadoop

Kafka

Job description

Company Overview

Idt is a communications and financial services company founded in 1990 and headquartered in New Jersey, US. Today it is an industry leader in prepaid communication and payment services and one of the world’s largest international voice carriers. It is listed on the NYSE, employs over 1500 people across 20+ countries, and has revenues in excess of $1.5 billion.

About the role

We are looking for a skilled Data/ML Engineer to join our BI team and take an active role in designing, building, and maintaining the end-to-end data pipeline, architecture and design that powers our warehouse, LLM-driven applications, and AI-based BI. If you’re looking for a company that will give you the maximum flexibility in choosing a location to work, this opportunity is for you! The interview process will be conducted in English.

Responsibilities

• Design, develop, and maintain scalable data pipelines to support ingestion, transformation, and delivery into centralized feature stores, model-training workflows, and real-time inference services.
• Build and optimize workflows for extracting, storing, and retrieving semantic representations of unstructured data to enable advanced search and retrieval patterns.
• Architect and implement lightweight analytics and dashboarding solutions that deliver natural language query experience and AI-backed insights.
• Define and execute processes for managing prompt engineering techniques, orchestration flows, and model fine‑tuning routines to power conversational interfaces.
• Oversee vector data stores and develop efficient indexing methodologies to support retrieval‑augmented generation (RAG) workflows.
• Partner with data stakeholders to gather requirements for language‑model initiatives and translate them into scalable solutions.
• Create and maintain comprehensive documentation for all data processes, workflows and model deployment routines.
• Remain informed and learn emerging methodologies in data engineering, MLOps and LLM operations.

Requirements

• Data & ML Engineer with a proven 5+ year history of building scalable infrastructure.
• Excellent English communication skills.
• Effective oral and written communication skills with BI team and user community.
• Demonstrated experience in utilizing Python for data engineering tasks, including transformation, advanced data manipulation, and large‑scale data processing.
• Deep understanding of vector databases and RAG architectures, and how they drive semantic retrieval workflows.
• Skilled at integrating open‑source LLM frameworks into data engineering workflows for end‑to‑end model training, customization, and scalable inference.
• Experience with cloud platforms like AWS or Azure Machine Learning for managed LLM deployments.
• Hands‑on experience with big data technologies including Apache Spark, Hadoop, and Kafka for distributed processing and real‑time data ingestion.
• Experience designing complex data pipelines extracting data from RDBMS, JSON, API and flat‑file sources.
• Demonstrated skills in SQL and PL/SQL programming, advanced mastery in Business Intelligence and data warehouse methodologies, and hands‑on experience in one or more relational database systems and cloud‑based database services such as Snowflake/Redshift.
• Understanding of software engineering principles and skills working on Unix/Linux/Windows operating systems, and experience with Agile methodologies.
• Proficiency in version control systems, with experience managing code repositories, branching, merging, and collaborating within a distributed development environment.
• Interest in business operations and a comprehensive understanding of how robust BI systems drive corporate profitability by enabling data‑driven decision‑making and strategic insights.

Pluses

• Experience with vector databases such as DataStax AstraDB, and developing LLM‑powered applications using popular open‑source frameworks like LangChain and LlamaIndex— including prompt engineering, retrieval‑augmented generation (RAG), and orchestration of intelligent workflows.
• Familiarity with evaluating and integrating open‑source LLM frameworks—such as Hugging Face Transformers/LLaMA‑4 across end‑to‑end workflows, including fine‑tuning and inference optimization.
• Knowledge of MLOps tooling and CI/CD pipelines to manage model versioning and automated deployments.

What we offer

• Remote work opportunity!
• B2B Employment ($, gross).
• Stable job with long‑term growth perspective.
• Competitive salary with annual performance review.
• Really good hardware.
• An exciting and challenging job with talented people around.
• Continuous learning and career growth opportunities.
• Compensation for professional training, seminars, and conferences.
• Referral program – get rewarded for helping us grow the team with talented people.
• Company‑supported English classes to enhance your professional growth.

Please attach CV in English.

Only accepting applicants from INDIA