Azure Data Engineer

Hinduja Global Solutions (HGS); Indore District, Hyderabad, Bengaluru; Hybrid; INR 1,200,000 - 2,300,000; Full time

Job summary

Hinduja Global Solutions (HGS) seeks a Data Engineer II to design and maintain scalable data infrastructure, build ETL pipelines, and work with Spark, Hadoop, and cloud data platforms.

You will develop code in Python/Scala/R, create complex SQL queries, and collaborate to translate business needs into data solutions while ensuring data quality and security.

Qualifications

• Bachelor's degree or equivalent experience in a related field.
• 3–6 years of experience in the role or similar fields.
• Knowledge of cloud data platforms and data warehousing concepts.
• Cloud certifications such as GCP Professional Data Engineer or Microsoft Data/AI certifications.

Responsibilities

• Build complex ETL code.
• Create and optimize SQL queries across MongoDB, Oracle, SQL Server, MariaDB, MySQL.
• Develop code using Python, Scala, and R.
• Work with Spark, Hadoop, Kafka and other big data tools.
• Design and maintain ETL/ELT pipelines for corporate reporting and analytics.
• Monitor pipeline health and ensure data quality.

Skills

Agile/Lean Development

AI/Machine Learning

Big Data Software Tools

Cloud Computing

Data Analysis

Data Modeling Skills

Data Mining Software Skills

Data Virtualization Skills

MS Excel Skills

SQL Database Skills

Technical Troubleshooting

Version Control Platform Tools

Education

Bachelor's Degree required, preferably in field related to role

Tools

JIRA

Git

Jenkins

TFS

Shell

PowerShell

Bitbucket

Job description

Data Engineer II

Job Profile Summary

Applies data extraction, transformation and loading techniques in order to connect large data sets from a variety of sources. Creates data collection frameworks for structured and unstructured data. Develops and maintains infrastructure systems (e.g., data warehouses, data lakes) including data access APIs. Prepares and manipulates data using Hadoop or equivalent MapReduce platform. Responsible for adhering to company security policies and procedures and any other relevant policies and standards as directed.

Career Level Summary

• Requires working knowledge and experience in own job discipline and broadens capabilities
• Continues to build knowledge of the company, processes and customers
• Performs a range of assignments related to job discipline
• Uses prescribed guidelines or processes in analyzing situations
• Receives a moderate level of guidance and direction

Critical Competencies

• Excellence: Exceeds expectations by consistently demonstrating accountability, discipline, high performance, and a proven track record of exceptional results.
• Customer-driven: Prioritizes customer needs and satisfaction through collaborative and proactive problem-solving, and an unwavering commitment to customer success.
• Expertise: Possesses deep understanding of customer needs and continually grows and enhances skills to provide customer-focused solutions.
• Agility: Quickly adapts and responds to dynamic customer needs and expectations through innovative solutions.
• Compassion: Cultivates a positive and supportive environment to effectively work together towards a common goal, fostering trust within Rackspace and with external stakeholders.

Key Responsibilities

• Other Incidental tasks related to the job, as necessary.
• Build complex ETL code
• Build complex SQL queries using MongoDB, Oracle, SQL Server, MariaDB, MySQL
• Work on Data and Analytics Tools in the Cloud
• Develop code using Python, Scala, R languages
• Work with technologies such as Spark, Hadoop, Kafka, etc.
• Build complex Data Engineering workflows
• Work with team to solve a variety of problems using machine learning techniques, and implement cloud-based solutions for customers
• Create complex data solutions and build data pipelines
• Establish credibility and build impactful relationships with our customers to enable them to be cloud advocates
• Capture and share industry best practices amongst the community
• Attend and present valuable information at Industry Events
• Design, build, and maintain ETL/ELT pipelines for corporate reporting and analytics platforms
• Model and document data for consumption by BI tools and AI/ML use cases
• Monitor pipeline health, troubleshoot failures, and manage data quality issues

Knowledge

• Basic knowledge on cloud Data platform Services and solutions
• Basic knowledge with neural networks, computer vision, language processing, big data processing, or other machine learning sub-domains
• Basic knowledge of software configuration management environments and tools such as JIRA, Git, Jenkins, TFS, Shell, powershell, Bitbucket, etc

Skills

• Agile/Lean Development Skills
• AI/Machine Learning
• Big Data Software Tools
• Cloud Computing
• Data Analysis
• Data Lake Software Skills
• Data Mining Software Skills
• Data Modeling Skills
• Data Virtualization Skills
• MS Excel Skills
• SQL Database Skills
• Technical Troubleshooting
• Version Control Platform Tools

Education

• Bachelor's Degree required, preferably in field related to role. At the manager's discretion, additional relevant experience may substitute degree requirement

Certifications

• Cloud certifications such as GCP Professional Data Engineer or Microsoft Data / AI certifications.

Experience

• 3 - 6 years of experience in the field of role required