Data Engineer - Senior 1

Cummins Talent Acquisition; Pune District; On-site; INR 1,400,000 - 2,100,000; Full time

Job summary

Cummins Talent Acquisition in Pune requests an experienced data engineering professional to lead the design, development and maintenance of a data and analytics platform. You will collaborate with business stakeholders and IT teams to deliver scalable analytics solutions and empower analysts and other consumers with timely data.

You will design automated data pipelines, govern metadata and access, optimize database performance, and apply modern cloud and distributed technologies to ensure

Responsibilities

• Designs and automates deployment of distributed data ingestion and transformation pipelines.
• Monitors data quality and data integrity, addressing issues as they arise.
• Implements data governance for metadata, access, and retention for internal and external users.
• Builds reliable, scalable data pipelines using ETL/ELT tools or scripting.
• Optimizes database structures through indexing and table relationships.
• Operates large-scale data storage and processing solutions using distributed/cloud platforms.

Job description

Job Summary:

Leads projects for design, development and maintenance of a data and analytics platform. Effectively and efficiently process, store and make data available to analysts and other consumers. Works with key business stakeholders, IT experts and subject-matter experts to plan, design and deliver optimal analytics and data science solutions. Works on one or many product teams at a time.

Key Responsibilities:

Designs and automates deployment of our distributed system for ingesting and transforming data from various types of sources (relational, event-based, unstructured). Designs and implements framework to continuously monitor and troubleshoot data quality and data integrity issues. Implements data governance processes and methods for managing metadata, access, retention to data for internal and external users. Designs and provide guidance on building reliable, efficient, scalable and quality data pipelines with monitoring and alert mechanisms that combine a variety of sources using ETL/ELT tools or scripting languages. Designs and implements physical data models to define the database structure. Optimizing database performance through efficient indexing and table relationships. Participates in optimizing, testing, and troubleshooting of data pipelines. Designs, develops and operates large scale data storage and processing solutions using different distributed and cloud based platforms for storing data (e.g. Data Lakes, Hadoop, Hbase, Cassandra, MongoDB, Accumulo, DynamoDB, others). Uses innovative and modern tools, techniques and architectures to partially or completely automate the most-common, repeatable and tedious data preparation and integration tasks in order to minimize manual and error-prone processes and improve productivity. Assists with renovating the data management infrastructure to drive automation in data integration and management. Ensures the timeliness and success of critical analytics initiatives by using agile development technologies such as DevOps, Scrum, Kanban Coaches and develops less experienced team members.