Data Engineering Specialist

alliancedata; Bengaluru; On-site; INR 1,800,000 - 3,200,000; Full time

Job summary

alliancedata in Bengaluru seeks a Data Engineering Specialist to advance data pipelines, data warehouses and an enterprise big data platform across multi-datacenter contexts.

You will collaborate with analytics teams to transform requirements into scalable data models, implement ingestion and quality checks, and support dashboard tools with Hive, Hadoop, Spark and SQL.

Qualifications

• Bachelor's Degree in Computer Science or Information Systems.
• 8+ years of data engineering experience.
• Experience with Hadoop, Spark, SQL and data warehousing.

Responsibilities

• Build data pipelines from multiple external data sources.
• Collaborate with stakeholders to translate business requirements into analytical dashboards.
• Maintain Hive/NoSQL data stores and data warehouse interfaces.
• Support data quality and data governance initiatives.

Skills

Data engineering

Project management

SQL

Hadoop/Spark

Data modeling

Communication

Education

Bachelor's degree in Computer Science or Information Systems

Tools

ER Studio

Azure

Hadoop

Spark

Hive

Job description

Every career journey is personal. That's why we empower you with the tools and support to create your own success story.

Be challenged. Be heard. Be valued. Be you ... be here.

Job Summary

The Data Engineering Specialist works on different projects of data engineering to support the use cases, data ingestion pipeline, data warehouse, data lake and enterprise big data platform with multi-datacenter contexts and identify potential process or data warehousing issues. Identify the architecture, infrastructure and interface to data sources, tools supporting automated data loads, security concerns, analytical models and data visualization. The Data Engineer team supports marketing analytic teams with analytical tools that enable our analytics and business communities to do their job easier, faster and smarter. The team brings together data from different internal & external partners and builds a curated Marketing analytics focused data & tools ecosystem. The Data Engineer plays a crucial role in building this ecosystem depending on the Marketing analytics communities need.

Essential Job Functions

Project Management : Builds data pipeline for various data sources, particularly from many external data sources, such as brand data and other data from vendors and ingestion of public data sources. Works with other project team members to understand the use cases, familiar with data sources, provides recommendation on data ingestion processes, complete data ingestion, monitor and identify potential process or data quality issues. Collaborates with internal/external stakeholders to manage data logistics - including data specifications, transfers, structures, and rules.

Collaboration - Collaborates with internal/external stakeholders to manage data logistics - including data specifications, transfers, structures, and rules. Collaborates with business users, business analysts and technical architects in transforming business requirements into analytical workbenches, tools and dashboards reflecting usability best practices and current design trends. Demonstrates analytical, interpersonal and professional communication skills. Learns quickly and works effectively individually and as part of a team.

Process Improvement - Accesses, extracts, and transforms Credit and Retail data from a variety of sources of all sizes (including client marketing databases, 2nd and 3rd party data) using Hadoop, Spark, SQL, Big data technologies etc. Provides automation help to analytical teams around data centric needs using orchestration tools, SQL and possibly other big data/cloud solutions for efficiency improvement.

Project Support - Supports Sr. Specialist and Specialist in new analytical proof of concepts and tool exploration projects. Effectively manages time and computing resources in order to deliver on time/correctly on concurrent projects. Involved in creating POCs to ingest and process streaming data using Spark and HDFS.

Data and Analytics - Answers and trouble shoots questions about data sets and analytical tools; Develops, maintains and enhances new and existing analytics tools/Frameworks to support internal customers/consumers. Ingests data from different sources, process it according to the requirement document in order to store data to Hive or NoSQL database or different warehousing solutions. Manages data coming from different sources, involved in HDFS maintenance, and loading of structured and unstructured data. Applies knowledge in Agile Scrum methodology that leverages the Client BigData platform and used version control tool Git.

Technical Skills - Ensure database designs efficiently. Drive continual improvement and enhancement of existing systems. Collaborate with Data Architects/Engineers for data model management, documentation, and version control. Capture CDE (Critical Data Elements) and work with Business/Data Engineers to translate into Edit check rules, Data Quality check scenarios. Experience on project management framework Agile (Jira toolset). Analysis and query on internal and external data and processes to answer specific business questions and identify opportunities for improvement.

Expert in ER studio, Business process management tool. Expert knowledge on metadata management and related tools. Exposure Azure data environment and familiarity with associated tools would be added advantage. Knowledge and experience in developing software using agile methodologies. Exhaustive experience in Dimensional modeling for Data Marts. Has the ability to operate with a limited level of direct supervision. Experience in modelling a third normal form, Star / Snowflake Data Warehouse system. Good experience in Data Profiling and Analysis

Reports to

Manager or above

Working Conditions / Physical Requirements

Normal office environment

Direct Reports

0

Minimum Requirements

Degree Required: Bachelor's Degree

Area of Study: Computer Science, Information Systems

Years of Work Experience Required: 8+ years

Type / foc