Senior AI Engineer

Latent View Analytics Limited; Chennai District; On-site; INR 3,000,000 - 4,200,000; Full time

Job summary

LatentView Analytics Limited is seeking a Data Engineer to design and build scalable ETL/ELT pipelines and data infrastructure for analytics and AI initiatives. You will work with Data Architects, Scientists, and Analysts to deliver robust data solutions across cloud platforms.

The role emphasizes Python/PySpark development, SQL optimization, and end-to-end pipeline ownership with opportunities to mentor juniors and implement best practices.

Qualifications

• 4–10 years of experience in Data Engineering or related field.
• Proficient in Advanced SQL with complex queries and optimization.
• Hands-on Python and PySpark for data transformation and processing.
• Experience building end-to-end ETL/ELT pipelines.
• Hands-on in at least one cloud data platform and warehouse service.
• Experience with workflow orchestration tools such as Airflow or similar.
• Strong problem-solving and cross-functional collaboration.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines ingesting data from diverse sources.
• Write and optimize advanced SQL queries for extraction, transformation and reporting.
• Build data models, warehouses, and semantic layers to support analytics.
• Leverage cloud data platforms to enhance processing, storage, and scalability.
• Automate and orchestrate data workflows using Airflow, dbt, Cloud Composer, or Data Factory.
• Collaborate with Data Architects, Scientists, Analysts, and business teams to translate requirements.
• Validate data quality, perform reconciliations, and ensure governance across pipelines.
• Document pipelines and processes; mentor junior engineers and contribute to best practices.

Skills

Advanced SQL

Python

PySpark

ETL/ELT pipelines

Cloud platforms

Airflow

Education

Bachelor's or Master's degree in a quantitative field

Tools

Snowflake

BigQuery

Databricks

Microsoft Fabric

Airflow

Job description

Founded in 2006, LatentView Analytics began with a shared passion for the world of data. Today, over 20 years on, we've grown into a close-knit community of people united by that same drive — solving real business challenges with data and AI. We work with industry leaders worldwide, specializing in end-to-end analytics that goes beyond the buzzwords to deliver genuine business impact. Our focus has stayed consistent since day one: helping clients derive meaningful insights and drive growth through a thoughtful, sustainable approach to data analytics and AI.

Role Overview

We are hiring Data Engineers to work across pipeline development, cloud data platforms, data modeling, and orchestration functions supporting our clients. You will design and build the data infrastructure that powers analytics, reporting, and AI/ML initiatives, working closely with Data Architects, Data Scientists, and business stakeholders. The specific focus of your role — cloud platform, specialization, and scope of ownership — will be matched to your experience and skills.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT data pipelines to ingest, transform, and load data from diverse sources

Write and optimize advanced SQL queries for data extraction, transformation, validation, and reporting

Build and maintain data models, warehouses, and semantic layers to support analytics, reporting, and downstream consumption

Work with cloud data platforms and services to enhance data processing, storage, and scalability

Automate and orchestrate end-to-end data workflows using tools such as Airflow, dbt, Cloud Composer, or Data Factory

Collaborate with cross-functional stakeholders — Data Architects, Data Scientists, Analysts, and business teams — to translate requirements into technical solutions

Perform data validation, reconciliation, and quality checks to ensure accuracy, consistency, and governance across pipelines

Document data pipelines, schemas, and technical processes to support knowledge sharing and maintainability

Troubleshoot pipeline failures, performance bottlenecks, and data quality anomalies

Mentor junior engineers and contribute to best practices, code reviews, and continuous improvement (scope matched to seniority)

Skills & Experience

4–10 years of relevant experience in Data Engineering, Data Pipeline Development, or a related field

Advanced SQL — complex joins, CTEs, window functions, and query performance optimization

Hands-on proficiency in Python and/or PySpark for data transformation, automation, and processing

Proven experience building and maintaining ETL/ELT pipelines end-to-end

Hands-on expertise in at least one cloud data platform (AWS, GCP, or Azure) and its associated warehouse/lake service (Snowflake, BigQuery, Databricks, or Microsoft Fabric)

Experience with workflow orchestration tools (Airflow, dbt, Cloud Composer, Data Factory, or similar)

Strong problem-solving skills and the ability to collaborate effectively with technical and non-technical stakeholders

Good to Have(any one)

Marketing/Media Data Engineering & MLOps: Data mart and harmonization design across disparate sources, MLOps and model deployment, model monitoring, cost optimization

Unstructured Data & GenAI-Adjacent Engineering (GCP): Document AI, Vertex AI, embeddings and vector search, semantic data modeling, Agentic AI/LangChain exposure

Cloud-Native Pipeline Engineering (AWS): S3, Lambda, SNS, Step Functions, NumPy/Pandas, independent end-to-end ownership as an individual contributor

Databricks DataOps & BI Enablement: Databricks and PySpark at scale, data validation and reconciliation, BI tool exposure (Power BI, Tableau, or Looker), Git/CI-CD discipline

Microsoft Fabric Platform Engineering: Data Factory/Dataflows Gen2, Medallion architecture (Bronze/Silver/Gold, Delta Lake, OneLake), Power BI DAX/Direct Lake, real-time streams (Eventstreams/KQL), Purview governance

Team & Delivery Leadership: Mentoring, code reviews, sprint/delivery ownership, cross-functional and business stakeholder management

Qualifications

Bachelor's or Master's degree in a quantitative field (Computer Science, Engineering, or related) or equivalent practical experience

Strong analytical, problem-solving, and communication skills