Business Data Insights Senior Specialist

APM Terminals; Chennai District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Maersk is seeking a Data & AI Engineer to own end-to-end data-to-AI lifecycle using Azure components and modern ML tools. You will design robust data pipelines, transform data, and deploy ML and generative AI solutions, focusing on reliability and scale.

Collaborating with stakeholders across analytics, product, and engineering, you will implement RAG pipelines, prompt engineering, and MLOps practices to deliver production-ready AI features that drive business value.

Qualifications

• Graduate or postgraduate degree in Computer Science, Information Technology, Data Engineering, AI or related field.
• Experience with Azure Data Lake, Data Factory and Databricks is highly desirable.
• Experience building data/AI pipelines, MLOps, and deploying ML models in cloud environments.

Responsibilities

• Design and develop scalable data pipelines using Azure Data Factory, Data Lake, and Databricks.
• Build ETL/ELT processes to feed analytics and AI model consumption.
• Develop and deploy ML and generative AI models, including RAG pipelines and prompt engineering.
• Collaborate with business stakeholders, data scientists, BI teams, and product teams.
• Monitor and optimize performance, cost, and latency in production.

Skills

Azure Data Factory

Azure Data Lake Storage

Azure Databricks

SQL

Python

ML/AI model deployment

RAG pipelines

MLOps

LLM prompt engineering

Vector databases

Data modeling

Education

Bachelor's or Master's in CS/IT or related

Tools

Power BI

Informatica

Teradata

Docker

Kubernetes

Git

Job description

Job Description Role: Data & AI Engineer

We are looking for a skilled and versatile Data & AI Engineer who can own the full data-to-AI lifecycle — designing and building robust data pipelines using Azure Data Lake, Azure Data Factory, and Azure Databricks, and then using that data to build, fine-tune, and deploy machine learning and generative AI solutions with Azure Machine Learning and Azure OpenAI. The role requires strong hands‑on experience across both disciplines: developing ETL/ELT pipelines, transforming structured and semi-structured data, and ensuring reliable data availability, as well as building RAG pipelines, integrating LLM-based solutions, and operationalizing models with MLOps best practices. The candidate should be comfortable working with business stakeholders, data scientists, BI teams, product teams, and technical teams to understand requirements, design scalable data and AI solutions, and deliver measurable business value end to end — from raw data to production‑ready AI features.

Role: Data & AI Engineer

Work Experience 5+ years of combined experience in data engineering and AI/ML engineering, including building scalable data pipelines and cloud data platforms as well as developing and deploying machine learning or generative AI solutions, in an Agile or DevOps environment.

Role Summary

We are looking for a skilled and versatile Data & AI Engineer who can own the full data-to-AI lifecycle — designing and building robust data pipelines using Azure Data Lake, Azure Data Factory, and Azure Databricks, and then using that data to build, fine‑tune, and deploy machine learning and generative AI solutions with Azure Machine Learning and Azure OpenAI. The role requires strong hands‑on experience across both disciplines: developing ETL/ELT pipelines, transforming structured and semi-structured data, and ensuring reliable data availability, as well as building RAG pipelines, integrating LLM-based solutions, and operationalizing models with MLOps best practices. The candidate should be comfortable working with business stakeholders, data scientists, BI teams, product teams, and technical teams to understand requirements, design scalable data and AI solutions, and deliver measurable business value end to end — from raw data to production‑ready AI features.

Key Skills

• 4+ years of experience spanning data engineering and AI/ML engineering, with strong exposure to data integration, ETL/ELT development, and cloud-based data and AI platforms.
• Hands‑on experience with Azure Data Factory for building, scheduling, monitoring, and managing data pipelines.
• Working knowledge of Azure Data Lake Storage for storing, organizing, and managing large volumes of data.
• Experience with Azure Databricks, including notebooks, Spark SQL, PySpark, data transformation, and performance optimization.
• Strong SQL and Python skills for querying, transformation, data validation, troubleshooting, and performance tuning.
• Hands‑on experience with Azure Machine Learning or Azure AI Studio for training, deploying, and managing ML models.
• Practical experience with Azure OpenAI Service or similar LLM platforms, including prompt engineering, fine‑tuning, and model integration.
• Experience building RAG pipelines using vector databases (e.g., Azure AI Search, FAISS, Pinecone) for grounding LLM responses in enterprise data.
• Experience with ML/AI frameworks such as PyTorch, TensorFlow, scikit‑learn, LangChain, or Semantic Kernel.
• Good understanding of data lake architecture (bronze, silver, gold / medallion architecture) and MLOps practices (model versioning, CI/CD for ML, monitoring, retraining).
• Experience with data modelling, data quality checks, data profiling, reconciliation, feature engineering, and model evaluation.
• Exposure to DevOps practices such as Git, CI/CD, containerization (Docker, Kubernetes), version control, and deployment processes.
• Working knowledge of Power BI or similar reporting tools, and knowledge of legacy ETL tools (Informatica, SSIS, Teradata, Oracle) will be an added advantage.

Key Responsibilities

• Design, develop, test, deploy, and maintain scalable data pipelines using Azure Data Factory, Azure Data Lake, Azure Databricks, SQL, and related technologies.
• Build automated ETL/ELT pipelines to ingest, transform, validate, and publish data for analytics, reporting, and AI model consumption.
• Design, build, and fine‑tune machine learning and generative AI models to solve business problems, using the curated data pipelines as the foundation.
• Develop RAG pipelines and integrate LLM-based solutions with enterprise data sources and applications.
• Work with structured, semi‑structured, and unstructured datasets, preparing them for both analytics and model training/evaluation.
• Develop data and model transformation logic using SQL, PySpark, Spark SQL, Databricks notebooks, and Python ML frameworks.
• Create and maintain reliable data flows and AI services across raw, curated, and consumption‑ready layers, including production model inference.
• Perform data exploration, validation, reconciliation, model evaluation, and performance testing to ensure accuracy and reliability end to end.
• Collaborate with business stakeholders, data scientists, BI developers, product owners, and architects to convert requirements into technical data and AI solutions.
• Support data migration, system integration, and AI feature rollout, from legacy platforms to cloud-based data and AI platforms.
• Monitor pipeline and model performance in production, troubleshoot failures, optimize processing/inference cost and latency, and ensure timely availability.
• Implement data quality rules, responsible AI practices, exception handling, logging, audit checks, and operational controls.
• Contribute to documentation of data flows, model architecture, source‑to‑target mappings, technical designs, and operational support procedures.
• Participate in E2E product lifecycle activities including design, development, testing, deployment, run support, retraining, refactoring, and decommissioning of outdated solutions.
• Continuously improve existing data and AI engineering processes through automation, standardization, and reusable components.

Business and Technical Skills

Strong understanding of both data engineering and AI/ML concepts — data pipelines, data integration, data modelling, model lifecycle, and cloud-based platforms. Ability to understand business requirements and translate them into scalable technical designs spanning data and AI. Experience in working with data structures, storage systems, model architectures, data quality frameworks, and system integrations. Good understanding of enterprise data flows, upstream and downstream dependencies, and AI/reporting consumption patterns. Ability to quickly analyze existing data and AI solutions and recommend improvements, automation opportunities, or alternative approaches. Strong problem‑solving skills with the ability to troubleshoot issues across pipelines, databases, models, and application layers. Experience working in Agile teams with E2E ownership of deliverables, from data ingestion through to deployed AI features. Ability to work with both technical and non‑technical stakeholders. Good documentation skills, including technical design documents, model cards, process flows, data mapping, and support guides.

General Skills

Strong team player with the ability to work independently when required. High ownership mindset with a focus on delivering reliable and scalable data and AI solutions. Open to learning new technologies and applying them to improve existing processes. Strong analytical thinking and attention to detail. Ability to understand end‑to‑end business processes and data/AI dependencies. Good communication skills with the ability to explain technical data and AI concepts in a simple and clear manner. Proactive approach toward automation, optimization, and continuous improvement.

Qualifications

Graduate or postgraduate degree in Computer Science, Information Technology, Data Engineering, Artificial Intelligence, Data Science, or a related field. Relevant certifications in Azure Data Engineering, Azure Databricks, Azure Data Factory, Azure Machine Learning, or generative AI technologies will be an added advantage.

Preferred Technology Stack

• Cloud & Data Platform: Azure Data Lake, Azure Data Factory, Azure Databricks
• Cloud & AI Platform: Azure Machine Learning, Azure AI Studio, Azure OpenAI Service
• Programming & Querying: SQL, Python, PySpark, Spark SQL
• AI/ML Frameworks: PyTorch, TensorFlow, scikit‑learn, LangChain, Semantic Kernel
• Data & AI Engineering: ETL, ELT, data pipelines, data modelling, data quality, RAG pipelines, prompt engineering, MLOps
• Vector Search & Data: Azure AI Search, FAISS, Pinecone, vector databases
• BI & Reporting: Power BI, SSRS or equivalent visualization tools
• DevOps & Delivery: Git, Azure DevOps, CI/CD, Docker, Kubernetes, Agile delivery practices
• Legacy or Enterprise Systems: Informatica, SSIS, Oracle, Teradata, SQL Server

Maersk is committed to a diverse and inclusive workplace, and we embrace different styles of thinking. Maersk is an equal opportunities employer and welcomes applicants without regard to race, colour, gender, sex, age, religion, creed, national origin, ancestry, citizenship, marital status, sexual orientation, physical or mental disability, medical condition, pregnancy or parental leave, veteran status, gender identity, genetic information, or any other characteristic protected by applicable law. We will consider qualified applicants with criminal histories in a manner consistent with all legal requirements. We are happy to support your need for any adjustments during the application and hiring process. If you need special assistance or an accommodation to use our website, apply for a position, or to perform a job, please contact us by emailing [e-mail].

A.P. Moller - Maersk is an integrated container logistics company working to connect and simplify its customer's supply chains. As the global leader in shipping services, the company operates in 130 countries and employs roughly 100,000 people. With simple end‑to‑end offering of products and digital services, seamless customer engagement and a superior end‑to‑end delivery network, Maersk enables its customers to trade and grow by transporting goods anywhere - all over the world.