Data Engineer

Zohorecruit; Hyderabad; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

AuxoAI is seeking an experienced Data Engineer to join our team in Hyderabad. The role focuses on building and optimizing data pipelines and infrastructure with AWS services like S3, Glue, EMR, Lambda, and Redshift.

You will collaborate with data scientists and analysts to translate data requirements into scalable, reliable solutions. Strong coding skills in Python, Java, or Scala are essential, along with experience in SQL/NoSQL databases and data warehousing concepts.

Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• 4-7 years of prior experience in data engineering, with a focus on designing and building data pipelines.
• Proficiency in AWS services, particularly S3, Glue, EMR, Lambda, and Redshift.
• Strong programming skills in Python, Java, or Scala.
• Experience with SQL and NoSQL databases, data warehousing concepts, and big data technologies.
• Familiarity with containerization technologies (Docker, Kubernetes) and orchestration tools (Apache Airflow) is a plus.

Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes leveraging AWS services such as S3, Glue, EMR, Lambda, and Redshift.
• Collaborate with data scientists and analysts to understand data requirements and implement solutions for analytics and machine learning initiatives.
• Optimize data storage and retrieval mechanisms for performance, reliability, and cost-effectiveness.
• Implement data governance and security best practices to ensure compliance and data integrity.
• Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
• Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhance data engineering capabilities.

Skills

Programming languages

AWS experience

SQL/NoSQL

Big data

Data pipelines

Education

Bachelor's or Master's in CS/Engineering

Tools

Docker

Kubernetes

Apache Airflow

Job description

Role Summary :

AuxoAI is seeking a skilled and experienced Data Engineer to join our dynamic team. The ideal candidate will have6-8years of prior experience in data engineering, with a strong background in AWS (Amazon Web Services)technologies. This role offers an exciting opportunity to work on diverse projects, collaborating with cross functional teams to design, build, and optimize data pipelines and infrastructure.

Responsibilities:

• Design, develop, and maintain scalable data pipelines and ETL processes leveraging AWS services such as S3,
• Glue, EMR, Lambda, and Redshift.
• Collaborate with data scientists and analysts to understand data requirements and implement solutions that support analytics and machine learning initiatives.
• Optimize data storage and retrieval mechanisms to ensure performance, reliability, and cost-effectiveness.
• Implement data governance and security best practices to ensure compliance and data integrity.
• Troubleshoot and debug data pipeline issues, providing timely resolution and proactive monitoring.
• Stay abreast of emerging technologies and industry trends, recommending innovative solutions to enhancedata engineering capabilities.

Requirements

• Bachelor's or Master's degree in Computer Science , Engineering, or a related field.
• 4-7 years of prior experience in data engineering, with a focus on designing and building data pipelines.
• Proficiency in AWS services, particularly S3, Glue, EMR, Lambda, and Redshift.
• Strong programming skills in languages such as Python, Java, or Scala.
• Experience with SQL and NoSQL databases, data warehousing concepts, and big data technologies.
• Familiarity with containerization technologies (e.g., Docker, Kubernetes) and orchestration tools (e.g., Apache Airflow) is a plus.