AI Data Engineer

Corning Inc.; Pune District; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Corning Inc. is seeking a Data Engineer for Advanced Analytics platforms in Pune to develop reliable data ingestion pipelines from multiple data stores to on‑premises and cloud data lakes.

You will implement validation, profiling, and enrichment steps and collaborate with data scientists and domain experts to ensure ready‑to‑use datasets. You will contribute to CI/CD, DevOps support, and cross‑functional collaboration, helping maintain resilience and scalability across Corning’s analytics

Qualifications

• Bachelors degree in CS, engineering, mathematics or related field.
• Experience with ETL/ELT pipelines for data warehousing and cloud/on-prem data lakes.
• Proficient in at least one JVM language (Java/Scala/Kotlin) and Python.
• Experience with AWS services (S3, EC2, DMS, RDS, EMR, RedShift, Lambda, DynamoDB).
• Hands-on with Apache Spark, S3, Parquet and Delta Lake.
• Knowledge of relational and polyglot persistence.
• Familiar with CI/CD and tools: Git (Gitlab), Jira, Terraform, New Relic.
• Familiar with notebooks (JupyterHub).
• Strong communication with users and cross-functional teams.

Responsibilities

• Design, test, deploy and maintain production big-data ingestion pipelines.
• Collaborate with data source teams to define ingestion requirements for structured, unstructured and semi-structured data.
• Define and implement automated validation and profiling capabilities for reliable data delivery.
• Work with data scientists to define data cleansing and enrichment requirements.
• Implement data cleansing and enrichment code using established patterns.
• Validate landed, cleansed and enriched data for immediate use by users.
• Participate in code reviews and share knowledge within the team and broader Corning community.
• Stay up to date with industry standards to improve quality and performance.
• Provide DevOps support to monitor tokens, jobs and system performance.

Skills

Big data engineering

ETL/ELT pipelines

JVM languages (Java/Scala/Kotlin)

Python

Data modeling

CI/CD practices

Communication

Education

Bachelor's degree in CS/Engineering/Math

Tools

AWS

Apache Spark

Delta Lake

Parquet

S3

Terraform

Jira

Gitlab

New Relic

JupyterHub

Job description

Select how often (in days) to receive an alert: Email jobs to me

The company built on breakthroughs.
Join us.

Corning is one of the world’s leading innovators in glass, ceramic, and materials science. From the depths of the ocean to the farthest reaches of space, our technologies push the boundaries of what’s possible.

How do we do this? With our people. They break through limitations and expectations– not once in a career, but every day. They help move our company,and the world, forward.

At Corning, there are endless possibilities for making an impact. You can help connect the unconnected, drive the future of automobiles, transform at-home entertainment, and ensure the delivery of lifesaving medicines. And so much more.

Come break through with us.

Corning’s businesses are ever-evolving to best serve our customers, industries, and consumers. Today, we accelerate and transform life sciences, mobile consumer electronics, optical communications, display, automotive, and solar markets. We are changing the world with:

• Trusted products that accelerate drug discovery, development, and delivery to save lives
• Damage-resistant cover glass to enhance the devices that keep us connected
• Optical fiber, wireless technologies, and connectivity solutions to carry information and ideas at the speed of light
• Precision glass for advanced displays to deliver richer experiences
• Auto glass and ceramics to drive cleaner, safer, and smarter transportation
• Solar polysilicon, wafers, and innovative photovoltaic modules, enabling low-cost solar energy solutions

Overview

The Data Engineer, Advanced Analytics platforms will work with our core platform development team as well as domain experts, application developers, controls engineers and data scientists. Their primary responsibility will be to develop reliable and instrumented data ingestion pipelines that land inbound data from multiple process and operational data stores throughout the company to on-premises and cloud-based data lakes. These pipelines will require data validation and data profiling automation along with version control and CI/CD to ensure ongoing resiliency and maintainability of the inbound data flows supporting our advanced analytics projects.

Responsibilities

As a data engineer for our advanced analytics platforms, your main responsibilities will be:

• Design, test, deploy and maintain production big-data ingestion pipelines using established frameworks, patterns of practice, agile software development and CI/CD practices, working closely with the Principal Software Engineer – Data Ingestion
• Work with cross-organizational data source teams to define data ingestion requirements for structured, unstructured and semi-structured data, pilot their implementation, ensure the data source teams accept the resulting landed data as valid
• Define and implement automated validation and profiling capabilities needed to ensure reliable data delivery, using agile software development and CI/CD practices
• Work with data source teams, domain experts and data scientists to define data cleansing and data enrichment requirements for landed data
• Implement data cleansing and enrichment code using established patterns of practice General - Corning (L4)
• Work with data source teams, domain experts and data scientists to validate landed, cleansed and enriched data, using agile software development and CI/CD practices, while ensuring that the final datasets are directly usable by them without additional processing effort
• Actively participate in code reviews and technical information sharing with your team members and the broader software engineering community at Corning
• Stay up to date with industry standards and technological advancements that will improve the quality, productivity and performance of your work.
• Provide support in a DevOps environment to monitor tokens, jobs and overall system performance.

Qualifications

• Bachelor's degree in computer science, engineering, mathematics, or a related technical discipline
• Understand concepts of big data engineering, developing and maintaining ETL and ELT pipelines for data warehousing, on-premise and cloud data lake environments
• Demonstrated production programming proficiency in at least one modern JVM language such as Java, Scala or Kotlin, as well as an interpreted declarative programming language such as Python
• Entry level experience with AWS platform services, including AWS S3 & EC2, Data Migration Services (DMS), RDS, EMR, RedShift, Lambda, DynamoDB, CloudWatch, CloudTrail
• Hands-on technical familiarity with Apache Spark architecture, S3, parquet and Delta Lake architecture, technologies and tools
• Basic proficiency with both traditional relational and polyglot persistence technologies
• Familiarity with agile software development & continuous integration + continuous deployment methodologies along with supporting tools such as Git (Gitlab), Jira, Terraform, New Relic
• Familiarity with notebook environments including JupyterHub
• Able to Communicate well with users, other technical team to collect requirements, describe data modeling decisions.