Senior Member Technical Staff

SmartRecruiters, Inc.; India; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Nielsen is seeking an experienced software engineer to architect and build scalable data pipelines and microservices using Python and Java. You will develop intelligent document processing workflows, integrate AI/ML capabilities, and optimize data strategies with DynamoDB and NoSQL databases.

Real-time streaming with Kafka and cloud-native deployment on AWS are core aspects of the role. Candidate will have 7–10 years of experience in product-focused environments, strong RESTful API skills, and a

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.
• 7 to 10 years of professional software development experience in a product-based company.
• Expert-level proficiency in Python and Java.
• Document parsing experience—extracting structured data from PDFs, HTML, XML, or other unstructured formats.
• Deep hands-on experience with NoSQL databases (DynamoDB) and data modeling for performance.
• Proven experience with Kafka or similar message brokers for building event-driven systems.
• Strong experience with AWS services (Lambda, EC2, S3, RDS, etc.) and building cloud-native applications.
• Infrastructure as Code with K8s and tools like CDK, Terraform, or CloudFormation.
• Data Engineering familiarity with ETL/ELT pipelines and large-scale datasets.
• Architecture understanding of Microservices, RESTful APIs, and distributed systems design.
• Proactive, owner mindset with good communication in fast-paced environments.

Responsibilities

• Architect, design, build, and maintain scalable data pipelines and microservices using Python.
• Develop intelligent document processing pipelines for PDFs, HTML, and unstructured data.
• Integrate AI/ML capabilities using LLMs and related tools to create AI-driven solutions.
• Leverage DynamoDB and NoSQL data stores for efficient data storage and retrieval.
• Implement and manage real-time data streaming with Kafka for event-driven architectures.
• Architect and deploy cloud-native services on AWS with emphasis on security and cost optimization.
• Write clean, maintainable code and lead design reviews with the team.
• Troubleshoot distributed systems issues and optimize performance across the stack.
• Mentor junior engineers and promote rapid experimentation and learning.

Skills

Python

Java

Kafka

RESTful APIs

Distributed systems

Education

Bachelor's or Master's in Computer Science or related field

Tools

DynamoDB

NoSQL databases

AWS services (Lambda, EC2, S3, RDS)

Kubernetes

CDK

Terraform

CloudFormation

Job description

At Nielsen, we are passionate about our work to power a better media future for all people by providing powerful insights that drive client decisions and deliver extraordinary results. Our talented, global workforce is dedicated to capturing audience engagement with content - wherever and whenever it’s consumed. Together, we are proudly rooted in our deep legacy as we stand at the forefront of the media revolution. When you join Nielsen, you will join a dynamic team committed to excellence, perseverance, and the ambition to make an impact together. We champion you, because when you succeed, we do too. We enable your best to power our future.

Job Description

Key Responsibilities:

Architect & Build Scalable Systems: Design, develop, and maintain robust data pipelines and microservices using Python.

Intelligent Document Processing: Develop document parsing and extraction pipelines (for PDFs, HTML, structured/unstructured data) that can reliably handle millions of documents.

AI/ML Integration: Implement and leverage technologies like LLMs (Large Language Models), Langgraph, AgenticAI and Prompt Engineering to create advanced, AI-driven solutions.

Data Strategy: Leverage NoSQL (specifically DynamoDB) and other database technologies to optimize data storage and retrieval for high-scale applications.

Stream Processing: Implement and manage real-time data streaming solutions using Kafka to support event-driven and responsive architectures.

Cloud Infrastructure: Architect and deploy services within AWS, ensuring best practices in security, scalability, and cost-optimization.

Code Excellence & Collaboration: Write clean, maintainable, and highly efficient code, lead rigorous code reviews, and actively collaborate with the team on system design and technical decision-making.

System Optimization: Troubleshoot complex distributed systems issues and optimize performance across the entire tech stack.

Mentorship: Act as a technical pillar for the team, mentoring junior engineers and fostering a culture of rapid experimentation and "failing fast" (and learning faster).

Qualifications

Education: Bachelor’s or Master’s degree in Computer Science, Engineering, or a related technical field.

Experience: 7 to 10 years of professional software development experience, specifically within a product-based company where you have navigated rapid scaling.

Core Tech: Expert-level proficiency in Python and Java.

AI/Document Experience: Document parsing experience—extracting structured data from PDFs, HTML, XML, or other unstructured formats.

Database Expertise: Deep hands-on experience with NoSQL databases (specifically DynamoDB) and a solid understanding of data modeling for performance.

Messaging & Integration: Proven experience with Kafka or similar message brokers for building event-driven systems.

Cloud Native: Strong experience with AWS services (Lambda, EC2, S3, RDS, etc.) and building cloud-native applications.

Infrastructure as Code: Experience with K8s and advanced Infrastructure as Code tools such as CDK, Terraform, or CloudFormation.

Data Engineering: Familiarity with building and maintaining Data Pipelines (ETL/ELT processes) and handling large-scale datasets.

Architecture: Strong understanding of Microservices, RESTful APIs, and distributed systems design.

Mindset & Communication: A proactive, "owner" mindset with the ability to thrive in an ambiguous, fast-paced environment, and excellent communication skills.