Engineer, Data Engineering

News Corp; Bengaluru; On-site; INR 1,500,000 - 2,400,000; Full time

Job summary

News Corp India is seeking a Data Engineer skilled in the GCP ecosystem to design and operate high-performance data environments. You will architect end-to-end ETL/ELT pipelines and ensure data quality while optimizing costs in BigQuery and Dataproc.

You’ll work on a data warehouse-centric platform, collaborating with senior engineers to translate business problems into scalable data solutions, using Python, SQL, Airflow, and Terraform.

Qualifications

• Mid-level Data Engineer with 2–4 years of hands-on experience.
• Strong knowledge of GCP services, especially BigQuery and Dataproc.
• Experience with Dataform for managing SQL workflows.
• Proficiency in Python and SQL; modular, testable code.
• Experience with CI/CD and Infrastructure-as-Code.

Responsibilities

• Develop, deploy, and maintain scalable ETL/ELT pipelines in GCP using Cloud Composer and PySpark on Dataproc.
• Improve query performance with BigQuery features like partitioning and clustering.
• Assist in implementing BigLake components for unified data governance.
• Deploy and manage pipeline code and infrastructure with Terraform and CircleCI.
• Implement logging, alerts, and data quality checks in pipelines.
• Collaborate with senior engineers to translate business requirements into data solutions.
• Proactively learn new GCP services to enhance processing capabilities.

Skills

BigQuery

PySpark

Cloud Composer

Python

SQL

Dataform

Terraform

CircleCI

Airflow

Data Modeling

Tools

Dataproc

BigLake

Terraform

CircleCI

GitHub

Job description

About The Role

We are looking for a Data Engineer with excellent analytical abilities in the GCP Ecosystem. At News Corp India, our Data Warehouse is our primary product. You won't just move data; you will solve critical business problems by architecting high-performance, automated data environments.

We are looking for a Data Engineer with excellent analytical abilities in the GCP Ecosystem. At News Corp India, our Data Warehouse is our primary product. You won't just move data; you will solve critical business problems by architecting high-performance, automated data environments.

This isn't just about moving data from A to B. You’ll be expected to understand the \"why\" behind the data, solving complex business problems through efficient pipeline design and a \"Thinking Data\" mindset - where AI and advanced analytics are simply part of your standard toolkit.

The Technical Core

• Data Warehousing: Expert-level BigQuery (Partitioning, Clustering, Slot Management).


• Lakehouse Architecture: Implementing BigLake for unified governance.


• Distributed Processing: Processing large-scale datasets using PySpark on Dataproc.


• Orchestration & Code: Python, SQL, and Cloud Composer (Airflow).


• DevOps: Terraform (IaC), CircleCI, and GitHub.

Key Responsibilities

The Data Engineer will primarily focus on the development, deployment, and operational maintenance of the data pipelines.

• Pipeline Development & Maintenance: Develop, deploy, and maintain scalable ETL/ELT pipelines in GCP using Cloud Composer and PySpark on Dataproc following established architectural patterns.


• Performance Optimization: Implement and utilize BigQuery features like Partitioning, Clustering, and Materialized Views to improve query performance and contribute to GCP cost management.


• Data Integration: Assist in the implementation of BigLake components to ensure structured and unstructured data is accessible with unified governance.


• Code & Infrastructure Deployment: Utilize Terraform and CircleCI for deploying and managing pipeline code and related infrastructure to ensure a modular and reproducible environment.


• Data Operations & Monitoring: Implement robust logging, alerting, and data quality checks within pipelines to actively monitor data flow and ensure data reliability.


• Collaboration: Work with senior engineers and cross-functional teams to understand business requirements and translate them into efficient data solutions.


• Skill Development: Proactively seek out and learn new GCP services and tools to enhance data processing capabilities.

Skills & Competencies

• Experience: Mid Level - 2 - 4 years of hands-on Data Engineering experience.


• GCP Proficiency: Solid knowledge of GCP services with a focus on BigQuery, DataForm, Cloud Composer, Cloud Storage, Cloud functions and Dataproc.


• Transformation Tools: Proficiency in Dataform for managing version‑controlled SQL workflows and modeling within BigQuery.


• Strong Coding: Excellent Python and SQL skills. You write modular code that is easy to test and maintain.


• Architectural Knowledge: Solid understanding of Data Modeling (Star Schema, Snowflake) and Data Operations.


• Automation Mindset: Experience with CI/CD tools and Infrastructure‑as‑Code is essential.


• Domain Knowledge: Experience in the News or Media industry is a strong plus.

Equal Opportunity Employer

All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, age, national origin, protected veteran status, disability status or any other protected characteristic.

Reasonable Accommodation

We are committed to providing reasonable accommodation for qualified individuals with disabilities in our job application and/or interview process. If you need assistance or accommodation in completing your application or participating in an interview due to a disability, email us at [e-mail]. Please put \"Reasonable Accommodation\" in the subject line and provide a brief description of the type of assistance you need. This inbox will not be monitored for application status updates.

Please refer to the privacy notice at the bottom of this page for submitting any data access, deletion, or other data subject rights requests, where permitted under your local laws and regulations.