Data Engineer

Innovatics Consulting; Ahmedabad District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Innovatics is seeking a Data Engineer in India to design, implement, and maintain data pipelines and infrastructure for scalable data processing. You will work with cross-functional teams to ensure efficient collection, storage, and processing of data for insightful analytics.

The ideal candidate has 3+ years of data engineering experience, strong Python/SQL skills, and hands-on AWS or Azure expertise, with ETL tooling and data warehousing experience.

Qualifications

• Bachelor's degree in CS, Engineering or related field.
• 3+ years of professional data engineering experience.
• Strong hands-on with AWS or Azure cloud platforms and services.
• Proficiency in Python, Scala or Java.
• Experience with ETL tools and data integration workflows.

Responsibilities

• Design and develop data pipelines and ETL processes for multi-source data.
• Collaborate with data scientists and analysts to meet business needs.
• Build and maintain data warehouses and scalable databases.
• Implement data quality and validation procedures.
• Monitor and troubleshoot data pipelines to ensure data integrity.
• Stay updated on data engineering and cloud trends and propose improvements.
• Coordinate with DevOps for deployment and management of data infra.
• Develop and maintain documentation for data infrastructure and workflows.
• Participate in code reviews and mentor junior data engineers.
• Contribute to a culture of continuous improvement.

Skills

Python

Scala

Java

Education

Bachelor's degree in CS/Engineering

Tools

AWS

Azure

Airflow

AWS Glue

Azure Data Factory

Tableau

Power BI

Hadoop

Spark

SQL/NoSQL

Job description

Job Title: Data Engineer

Company: Innovatics

Job Description:

As a Data Engineer at Innovatics, you will play a crucial role in designing, implementing, and maintaining our data infrastructure and pipelines. You will work closely with cross-functional teams to ensure the efficient collection, storage, and processing of data to drive insightful analysis and enable data-driven decision-making. We are seeking a talented individual with at least 3 years of experience in data engineering, who possesses strong technical skills, particularly in AWS or Azure cloud platforms. Certification in AWS or Azure is a plus.

Responsibilities:

• 1. Design and develop data pipelines and ETL processes to ensure the smooth extraction, transformation, and loading of data from various sources into our data systems.
• 2. Collaborate with stakeholders, including data scientists and analysts, to understand data requirements and develop solutions that meet business objectives.
• 3. Build and maintain scalable and optimized databases and data warehouses to support efficient data storage and retrieval.
• 4. Implement data quality and validation procedures to ensure the accuracy, completeness, and consistency of data.
• 5. Monitor and troubleshoot data pipelines, identifying and resolving issues to ensure data integrity and availability.
• 6. Stay updated with emerging technologies and trends in data engineering, cloud computing, and big data processing to propose innovative solutions and improve existing processes.
• 7. Collaborate with DevOps teams to ensure effective deployment and management of data engineering infrastructure.
• 8. Develop and maintain documentation related to data infrastructure, processes, and workflows.
• 9. Participate in code reviews, provide constructive feedback, and contribute to a culture of continuous improvement.
• 10. Mentor and provide guidance to junior data engineers, fostering their professional growth and development.

Requirements:

• 1. Bachelor's degree in Computer Science, Engineering, or a related field.
• 2. Minimum of 3 years of professional experience in data engineering.
• 3. Strong hands-on experience with AWS/Azure cloud platforms, including proficiency in services such as Amazon S3, EC2, Redshift, Azure Data Lake, Azure SQL Database, etc.
• 4. Solid understanding of data modeling, data warehousing concepts, and database management systems (e.g., SQL, NoSQL).
• 5. Proficiency in programming languages commonly used in data engineering, such as Python, Scala, or Java.
• 6. Experience with data integration tools, ETL frameworks, and workflow schedulers (e.g., Apache Airflow, AWS Glue, Azure Data Factory).
• 7. Knowledge of big data technologies and distributed computing frameworks (e.g., Hadoop, Spark) is desirable.
• 8. Familiarity with data visualization tools (e.g., Tableau, Power BI) is a plus.
• 9. Excellent problem-solving and analytical skills with a keen attention to detail.
• 10. Strong communication and interpersonal skills, with the ability to work effectively in a collaborative team environment.

Good to have:

• 1. Certification in AWS (e.g., AWS Certified Big Data - Specialty) and Azure (e.g., Azure Data Engineer Associate) is a plus.
• 2. Experience working in an Agile/Scrum development environment.
• 3. Familiarity with machine learning concepts and frameworks.

Join our dynamic and innovative team at Innovatics, where you will have the opportunity to contribute to cutting-edge projects and shape the future of data-driven solutions.