Data Engineer- LLM

Persistent; Pune District; On-site; INR 2,800,000 - 4,200,000; Full time

Benefits offered by this job

Competitive salary and benefitspackage

Culture focused on talent development

Opportunity to work with cutting‑edge技

Employee engagement initiatives suchas

Annual health check‑ups

Insurance coverage: group term life,

Job summary

Persistent seeks a hands-on Data Engineer- LLM with 6 to 10 years of experience to design, build, and optimize scalable data pipelines and data platforms across Azure, Databricks, and Snowflake. You will collaborate with AI/ML teams, analytics, and business units to enable trusted data products.

You will implement governance, security, and CI/CD practices while shaping an enterprise data architecture that supports analytics, AI initiatives, and cloud migration efforts.

Qualifications

• 6 to 10 years of experience in Data Engineering, Data Warehousing, or Big Data development.
• Strong proficiency in SQL and Python for data engineering, transformation, and analytics workloads.
• Hands‑on experience building and supporting scalable ETL/ELT pipelines.
• Strong expertise with Azure cloud data services and enterprise data platforms.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines and integration frameworks.
• Build and optimize enterprise data warehouses, data lakes, and cloud-native data platforms.
• Develop robust data processing solutions using Azure, Databricks, SQL, Python, and Spark.
• Ensure data quality, integrity, availability, reliability, and performance across enterprise data environments.
• Partner with analytics, AI/ML, reporting, and business teams to deliver trusted data products.
• Support AI/ML initiatives through feature engineering, data preparation, and pipeline development.
• Work with Azure Data Factory, Synapse Analytics, Databricks, Snowflake, and related enterprise data platforms.
• Implement data governance, metadata management, security, and compliance best practices.

Skills

SQL

Python

Azure cloud data services

Databricks

Spark

Data modeling

CI/CD / DevOps

Data governance

Tools

Azure Data Factory

Azure Synapse Analytics

Snowflake

Databricks

Job description

About Position:

We are seeking a hands-on Data Engineer with 6 to 10 years of experience in Data Engineering, Data Warehousing, and Big Data development. The ideal candidate will have strong expertise in Azure cloud technologies, Databricks, SQL, Python, and modern data platform architectures. This role involves building scalable data pipelines, enabling analytics and AI/ML initiatives, and delivering reliable, high-performance data solutions that support enterprise business objectives.

• Role: Data Engineer- LLM
• Location: All Persistent Locations
• Experience: 6 to 10 Years
• Job Type: Full Time Employment

What You'll Do:

• Design, develop, and maintain scalable ETL/ELT data pipelines and integration frameworks.
• Build and optimize enterprise data warehouses, data lakes, and cloud-native data platforms.
• Develop robust data processing solutions using Azure, Databricks, SQL, Python, and Spark.
• Ensure data quality, integrity, availability, reliability, and performance across enterprise data environments.
• Partner with analytics, AI/ML, reporting, and business teams to deliver trusted data products.
• Support AI/ML initiatives through feature engineering, data preparation, and pipeline development.
• Work with Azure Data Factory, Synapse Analytics, Databricks, Snowflake, and related enterprise data platforms.
• Implement data governance, metadata management, security, and compliance best practices.
• Build monitoring, automation, and operational processes to improve platform reliability and efficiency.
• Optimize data models, queries, and processing frameworks for performance and scalability.
• Support structured and unstructured data processing across cloud-based data ecosystems.
• Participate in Agile ceremonies and collaborate with cross‑functional stakeholders.
• Implement CI/CD pipelines, DevOps practices, Infrastructure as Code, and platform automation strategies.
• Troubleshoot and resolve complex data platform, integration, and performance issues.
• Contribute to data architecture decisions and best practices for enterprise data engineering.
• AI/ML Data Enablement and Feature Engineering
• Contact Center Analytics and Reporting Platforms
• Operational Data Mart and KPI Enablement
• Data Pipeline Automation and Optimization Programs
• Enterprise Data Modernization and Cloud Migration Initiatives

Expertise You'll Bring:

• 6 to 10 years of experience in Data Engineering, Data Warehousing, or Big Data development.
• Strong proficiency in SQL and Python for data engineering, transformation, and analytics workloads.
• Hands‑on experience building and supporting scalable ETL/ELT pipelines.
• Strong expertise with Azure cloud data services and enterprise data platforms.
• Experience working with Databricks, Spark, and modern Lakehouse architectures.
• Strong knowledge of data modeling, dimensional modeling, and data warehouse design principles.
• Experience implementing data governance, data quality, lineage, and metadata management practices.
• Hands‑on experience with AI/ML data engineering, feature engineering, and data preparation workflows.
• Ability to manage large-scale structured and unstructured datasets.
• Experience building highly available, scalable, and performance‑optimized data solutions.
• Enterprise Data Engineering and Data Platform development best practices.
• Business Intelligence, Advanced Analytics, AI/ML, and data‑driven solution delivery.
• Azure Data Factory, Azure Synapse Analytics, Databricks, Snowflake, and related cloud technologies.
• Healthcare, payer/provider, claims, operational, or contact center data domains.
• Data Lake, Lakehouse, and Enterprise Data Warehouse architectures.
• CI/CD practices, DevOps methodologies, Infrastructure as Code, and platform automation.
• Data security, privacy, regulatory compliance, and governance frameworks.
• Agile delivery methodologies and cross‑functional team collaboration.
• Data lifecycle management, monitoring, observability, and operational excellence.
• Modern AI and GenAI data platform requirements and architecture.
• Maintain a minimum of 90% weekly usage of GitHub Copilot, Microsoft 365 Copilot, and other approved enterprise AI tools.
• Leverage AI tools to enhance software development, data engineering, documentation, and analytical workflows.
• Utilize GenAI and LLM-based capabilities to improve productivity, code quality, and delivery velocity.
• Stay current with emerging AI technologies, enterprise AI governance, and best practices.
• Apply AI‑assisted engineering approaches while maintaining security, compliance, and quality standards.

Benefits:

• Competitive salary and benefits package
• Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
• Opportunity to work with cutting‑edge technologies
• Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
• Annual health check‑ups
• Insurance coverage: group term life, personal accident, and Mediclaim hospitalisation for self, spouse, two children, and parents

Values-Driven, People-Centric & Inclusive Work Environment:

Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.

• We support hybrid work and flexible hours to fit diverse lifestyles.
• Our office is accessibility‑friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
• If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment.

Let's unleash your full potential at Persistent - persistent.com/careers

Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind.