Senior Data Engineer

Aumnitechworks; Pune District; Hybrid; INR 1,500,000 - 2,500,000; Full time

Benefits offered by this job

Medical insurance

Life and Disability insurance

Leaves and holidays

On-site gym

Hybrid work culture

Team events

Job summary

Aumni Techworks in Pune is seeking a Data Engineer to partner with Software Engineering and Product teams to enhance data infrastructure, pipelines, and analytics. Your work will influence organizational insights, customer features, and ML models.

The role requires strong database design, comfort with large data sets, and a passion for building scalable, automated data solutions using Spark, EMR, Python, Airflow, Glue, and S3.

Qualifications

• 5+ years of data engineering experience.
• Experience with data modeling, warehousing and ETL pipelines.
• Proficiency in Python or another modern language.
• Experience delivering end-to-end projects independently.
• Knowledge of distributed systems as it pertains to data storage and computing.
• 3+ years of Postgres/NoSQL experience.
• 3+ years with Spark, Iceberg, EMR, Glue, Data Lake and BI tools like Quicksight.
• Infrastructure as Code using Terraform and GitHub Actions.
• Geospatial and time series data experience.
• Monitoring platforms, especially AWS CloudWatch.

Responsibilities

• Architecture design and implementation of next generation data pipelines, analytics, and BI solutions.
• Manage AWS resources including RDS, EMR, MWAA, and Lambda.
• Build and deliver high quality data architecture and pipelines to support business analysts and data scientists.
• Interface with other technology teams to extract, transform, and load data from a wide variety of data sources.
• Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers.

Skills

5+ years of data engineering

Data modeling

ETL pipelines

Python

End-to-end project delivery

Distributed systems

Geospatial and time series data

Tools

Apache Spark

Apache Iceberg

AWS EMR

AWS Glue

Data Lake

Quicksight

Terraform

GitHub Actions

Postgres

NoSQL

Job description

Role

As a Data Engineer you will work directly with Software Engineering and Product teams to continuously improve our data infrastructure, design, tools and pipelines. Your work will directly influence and drive organizational insights, customer-facing features, and machine learning models.

To be successful in this role, you should have strong database design skills, comfort with large data sets, and an eagerness to invent. You should have a passion for data and analytics with the technical skills needed to build for scale and automation.

Key job responsibilities

• Architecture design and implementation of next generation data pipelines, analytics, and BI solutions
• Manage AWS resources including RDS, EMR, MWAA, and Lambda
• Build and deliver high quality data architecture and pipelines to support business analysts and data scientists
• Interface with other technology teams to extract, transform, and load data from a wide variety of data sources
• Continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers

A day in the life

• Collaborate with Software Engineers, Product Managers, Data Scientists and Business Intelligence Engineers to design, plan and deliver on high priority data initiatives serving internal stakeholders and Pole Star customers.
• Build automated, fault tolerant and scalable data solutions leveraging state of the art technologies including but not limited to Spark, EMR, Python, AirFlow, Glue and S3.
• Look around corners and be creative - Continuously evaluate and improve our strategy, architecture, tooling and codebase to maximize performance, scalability and availability.

BASIC QUALIFICATIONS

• 5+ years of data engineering experience
• Experience with data modeling, warehousing and building ETL pipelines
• Experience programming with at least one modern language, preferably Python
• Experience working on and delivering end to end projects independently
• Knowledge of distributed systems as it pertains to data storage and computing
• 3+ years of analyzing and interpreting data with Postgres, NoSQL etc. experience
• 3+ Years of hands-on experience with big data framework and storage mechanisms like Apache Spark, Iceberg, EMR, Glue, Data Lake, and BI tools like Quicksight
• Infrastructure as Code using Terraform and Github actions
• Geospatial and time series data experience
• Monitoring platforms, especially AWS CloudWatch

About Aumni Techworks:

Aumni Techworks, established in 2016, is a Software Services Company that partners with Product companies to build and manage their dedicated teams in India. So, while you are working for a services company, you are working within a product team and growing with them. We do not take projects, and we have long term (open ended) contracts with our clients. When our clients sign up with us, they are looking at a multi-year relationship. For e.g. Some of the clients we signed up 8 or 6 years, are still with us. We do not move people across client teams and there is no concept of bench. At Aumni, we believe in quality work, and we truly believe that Indian talent is at par with someone in NY, London or Germany. 300+ and growing.

Benefits of working at Aumni Techworks:

• Our award-winning culture reminds us of our engineering days.
• Medical insurance (including Parents), Life and Disability insurance
• 24 leaves + 10 public holidays + leaves for Hospitalization, maternity, paternity and bereavement.
• On site Gym, TT, Carrom, Foosball and Pool table
• Hybrid work culture
• Fitness group / rewards
• Friday Socials, Annual parties, treks.