Principal Member Technical Staff - Data engineer The Nielsen Company Est. $50,000 - $55,000 Bengaluru, in Engineering Data Data Engineer Bengaluru, in

SportsTechJobs; Bengaluru; On-site; INR 4,766,000 - 5,243,000; Full time

Job summary

The Nielsen Company in Bengaluru is seeking a Principal Member Technical Staff - Data Engineer to lead design and delivery of end-to-end data pipelines, spanning batch to real-time processing across mobile, web, and connected devices.

You’ll apply an AI-first mindset to data engineering, validate data quality, and partner with ML and platform teams to operationalize models. Strong SQL, Python/Scala, and AWS expertise are essential for success.

Qualifications

• Typically 10–15 years of relevant data engineering experience.
• Bachelor’s degree in Computer Science, Engineering, Information Systems or related field; advanced degree or certifications a plus.
• Expert-level SQL and strong programming in Python and/or Scala/Java, with hands-on Spark experience.
• Production data pipelines and data-lake/lakehouse workflows on AWS or similar cloud.
• Experience with S3/Parquet, orchestration, metadata services, and query engines like Athena.

Responsibilities

• Lead design and delivery of batch and near-daily data pipelines from raw inputs to curated outputs.
• Profile telemetry, identify data gaps and translate source formats into reliable processing logic.
• Apply AI-first approach to engineering and operations for data exploration and automation.
• Design and operate real-time streaming alongside batch pipelines with proper latency and backpressure handling.
• Build reusable frameworks for onboarding new apps, data sources and markets.

Skills

SQL

Python

Scala/Java

Apache Spark

Data engineering

Education

Bachelor’s degree in Computer Science/Engineering/Information Systems

Advanced degree or certifications (plus)

Tools

Airflow

AWS

Kinesis

Datadog

Grafana

Job description

# The Nielsen Company is hiring a Principal Member Technical Staff - Data engineer## Get the latest jobs to your inbox!1. Home2. /3. Jobs at The Nielsen Company4. /5. Principal Member Technical Staff - Data engineerBengaluru, inPosted 3 days ago43 viewsUnlock Premium - Save Jobs, Set Alerts & Get Early Access### Job Description**Job Description**Nielsen’s Digital Content AI team builds and operates data-processing pipelines that turn digital audience and interaction signals into reliable, analytics-ready datasets. As a principal-level individual contributor, you will bring a Solution first mindset and strong data understanding to the full lifecycle: understand new data sources, build reusable ingestion and transformation workflows, validate and enrich data, publish it to downstream data platforms, and keep production processing dependable. You will lead multiple cross-functional projects through measurable business deliverables, providing technical direction through expertise and influence rather than formal people management. You will work across mobile, Connected TV (CTV), desktop/laptop and web use cases, partnering with product, data collection, platform, quality and downstream analytics teams. ● Lead the technical design and delivery of batch and near-daily data pipelines, from raw inputs through curated outputs and publication to the Master Data Layer (MDL). ● Profile and interpret incoming telemetry from diverse data sources; identify data gaps and translate source formats into dependable processing logic. ● Apply an AI-first approach to engineering and operations: identify appropriate opportunities for AI-assisted data exploration, code and test generation, classification, anomaly detection and workflow automation; validate outputs rigorously and use AI responsibly with appropriate privacy and security controls. ● Design and operate real-time and streaming event workflows alongside batch pipelines, managing event validation, schema evolution, ordering and duplication, late or replayed events, processing latency, backpressure and failure recovery. ● Build and maintain parsers, configuration-driven workflows, event/session logic, business rules and schema mappings for application usage and in-app activity across mobile, CTV, desktop/laptop and web products. ● Develop data-stitching and transformation logic that correlates relevant sources, derives sessions and engagement measures, and produces consistent intermediate and curated datasets. ● Deliver data processing for streaming/video, e-commerce and advertising use cases, including event classification, purchase and product/category/search activity, app and web usage, and incremental dataset updates. ● Integrate reference and enrichment data—including content metadata and classification inputs—where required; collaborate with machine-learning and platform teams to operationalize model-based classification and enrichment. ● Build reusable frameworks for onboarding new applications, platforms, data sources and markets; support configuration, historical backfills, reprocessing and safe daily refreshes. ● Orchestrate and operate workflows using tools such as Airflow and AWS workflow services; monitor schedules, dependencies, retries, failures and data availability, and troubleshoot production issues through root-cause analysis. ● Implement and evolve automated data-quality controls, including schema and completeness checks, freshness and volume monitoring, null/range/duplicate checks, event consistency, stitching validation and reconciliation between processing layers and MDL. ● Create and use operational dashboards, alerts and job-level telemetry to make pipeline health, data quality, performance and processing costs visible; improve reliability and efficiency based on those signals. ● Optimize distributed processing and cloud data layouts for correctness, scalability, runtime and cost, working with technologies such as Spark, Parquet, Amazon S3, AWS Glue, Athena and EMR. ● Contribute to design reviews, code reviews, testing strategy, release readiness, runbooks and technical documentation; maintain clear data contracts, schemas and operational ownership boundaries. ● Provide technical direction and mentoring to engineers, lead moderately complex initiatives through influence, and communicate risks, dependencies, estimates and delivery status to cross-functional partners. ● Drive multiple projects or workstreams end to end, translating business needs into scope, milestones and technical plans; coordinate dependencies and stakeholders; and remain accountable for quality, adoption, timelines and measurable business outcomes—not people-management responsibilities. ● Handle sensitive data responsibly and follow Nielsen’s security, privacy, access-control and engineering practices.**Qualifications**● Typically 10–15 years of relevant data engineering or data processing experience, or equivalent depth of expertise delivering complex production data systems. ● Bachelor’s degree in Computer Science, Engineering, Information Systems or a related field, or equivalent practical experience; a relevant advanced degree or specialized certification is a plus. ● Expert-level SQL and strong programming skills in Python and/or Scala/Java, with substantial hands-on experience building distributed data-processing workloads using Apache Spark or a comparable framework. ● Proven experience designing, deploying and supporting production data pipelines and data-lake/lakehouse workflows on AWS or a comparable cloud platform. ● Practical experience with object storage and columnar formats such as S3 and Parquet, workflow orchestration, metadata/catalog services, and query engines such as Athena or equivalent. ● Strong understanding of data semantics and modeling, source-to-target lineage, schema evolution, data contracts, event-driven and batch-processing patterns, partitioning, incremental loads, backfills and idempotent/recoverable processing. ● Hands-on knowledge of real-time event processing concepts, including delivery guarantees, event-time handling, ordering, deduplication, replay, latency and backpressure. ● An AI-first engineering mindset and practical judgment about where AI/ML can improve data understanding, processing, quality or operational efficiency, with the ability to verify AI-generated outputs. ● Demonstrated ability to investigate unfamiliar or inconsistent source data, convert business rules into testable processing logic, and diagnose issues across multiple pipeline stages. ● A track record of independently driving multiple complex, cross-functional projects from problem definition through production delivery, with clear ownership of business outcomes and no reliance on formal people-management authority. ● Experience building data-quality checks, reconciliation, monitoring and alerting for production data products. ● Strong analytical and problem-solving skills, sound technical judgment, and a track record of independently owning complex deliverables from discovery through production support. ● Excellent written and verbal communication skills, including the ability to explain technical trade-offs and coordinate delivery across teams.**Additional Information**● Familiarity with stitching diverse telemetry, user activity, and digital interaction signals to derive session- or content-level metrics. ● Experience with content metadata/reference-data enrichment, entity matching, e-commerce event classification, advertising datasets or ML-assisted data processing. ● Experience with AWS services such as Kinesis, Firehose, Lambda, Step Functions, Glue, Athena and EMR, and with Grafana or Datadog for operational observability. ● Experience improving reusable onboarding frameworks, pipeline resilience, data lineage, operational cost visibility or large-scale backfill processes.Please be aware that job-seekers may be at risk of targeting by scammers seeking personal data or money. Nielsen recruiters will only contact you through official job boards, LinkedIn, or email with a nielsen.com domain. Be cautious of any outreach claiming to be from Nielsen via other messaging platforms or personal email addresses. Always verify that email communications come from an @nielsen.com address. If you're unsure about the authenticity of a job offer or communication, please contact Nielsen directly through our official website or verified social media channels.### Skills & TechnologiesApache SparkPythonSQLAWS (S3, Glue, Athena, EMR, Kinesis, Lambda, Step Functions)Apache AirflowParquetStreaming and real-time event processingData pipeline design and data lake/lakehouse architectureData quality, monitoring, and observability (Grafana, Datadog)Scala/JavaSponsored Featured Partner### Explore Biotech CareersDiscover exciting opportunities in biotechnology. Join innovative companies that are advancing healthcare and life sciences through cutting-edge research and development.Remote FriendlyCompetitive SalaryBiotechExplore Biotech JobsLearn More#### Salary InformationSalary: $50,000 - $55,000 This salary estimate is calculated by AI based on the job title, location, company, and market data. Use this as a guide for salary expectations or negotiations. The actual salary may vary based on your experience, qualifications, and company policies.Compare salaries in Bengaluru#### Create a Job AlertInterested in building your career at The Nielsen Company? Get future opportunities sent straight to your email.Create AlertApply for this Job