Senior Software Engineer - Data Engineering

To The New; India; On-site; INR 4,000,000 - 6,000,000; Full time

Job summary

TO THE NEW is seeking a Senior Software Engineer – Data Engineering to translate business requirements into scalable data solutions, collaborate with cross-functional teams, and deliver high-quality data applications on time.

You will work with Hadoop, Spark, SQL-based stacks, and cloud platforms, focusing on performance, data quality, and timely insights while engaging with customers to showcase new concepts.

Qualifications

• 4+ years in data engineering design and development.
• Strong experience with Python/Scala/Java and Spark SQL.
• Hands-on with Hadoop/HDFS, Hive, Snowflake or BigQuery.
• Strong SQL skills and data modeling abilities.
• Experience with data quality, metadata management, and MDM solutions.

Responsibilities

• Review and understand business requirements ensuring timely completion and quality testing.
• Collaborate with software teams to optimize performance of data applications.
• Engage with customers to define, design and showcase data solutions.
• Research and evaluate new Big Data technologies and trends.
• Work in an agile environment to deliver rapid, high-quality data products.

Skills

Python

Scala

Java

Hadoop

Spark SQL

SQL

Hive

Snowflake

BigQuery

Data Quality

MDM

Tools

Kafka

Flume

Job description

Senior Software Engineer – Data Engineering

• Review and understand business requirements ensuring that development tasks are completed within the timeline provided and that issues are fully tested with minimal defects
• Partner with a software development team to implement best practices and optimize performance of Data applications
• Collaborate across the company and interact with our customers to define, design and showcase new concepts and solutions
• Research on new Big Data technologies, assessing maturity and alignment of technology to business and technology strategy
• Collaborate with other developers to ensure that client needs are met at all times
• Work in a rapid and agile development process to enable increased speed to market against a backdrop of appropriate controls
• Implement good development and testing standards to ensure quality of deliverables
• Rapidly understand and translate clients’ business challenges and concerns into a solution-oriented discussion

Requirements

• At least 4+ years of experience in design and development using Data Engineering technology stack and programming languages
• Mandatory experience on following areas:
• Python/Scala/Java
• Hadoop, HDFS, MR
• Spark SQL, Dataframes, RDDs
• SQL
• Hive / Snowflake/SQL Server/Bigquery
• Good knowledge of the current landscape and ability to visualize technology and industry trends
• Working knowledge of Big Data Integration with Third party / in house built Metadata Management, Data Quality, Master Data Management Solutions.
• Active community involvement through articles / blogs / speaking engagements at conferences

Good to Have

• Spark Streaming, Spark ML
• Kafka/Flume
• Worked with Cloud Based Data Platforms
• Elastic Search

Life is Better Together

A glimpse into the milestones, wins, and everyday moments that shape life at TO THE NEW.

Celebrating milestones. Inspiring what's next

Geek Combat (Hackathon)

Turning bold ideas into real solutions

Marathon

Going the extra mile, together

Diwali celebrations

Bringing festive spirit to work

Empowering every voice, every day

Bootcamp

Learning together. Growing every day