Big Data Engineer

Iaagiz

Bengaluru

Presencial

INR 3 000 000 - 4 200 000

Tempo integral

Há 6 dias
Torna-te num dos primeiros candidatos

Recebe uma resposta deste empregador

O teu currículo e carta de apresentação, adaptados exatamente ao que estão a contratar.

Ultrapassa os filtros ATS

Resumo da oferta

Iaagiz in Bengaluru seeks a Senior Data Engineer to design and optimize enterprise data pipelines using Spark, Python and Scala on AWS.

You will own data ingestion, transformation, and scalable data replication, collaborating across teams to deploy data tools and automate pipelines. Strong OO design and communication are essential.

Qualificações

  • 4-7 years of experience delivering enterprise software solutions.
  • Proficient with Java, Spark, Scala, Python, AWS Cloud technologies.
  • 2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala.
  • Experience with Agile Development methodologies.
  • Experience with data ingestion and transformation.
  • Secure application development methodologies knowledge.
  • Experience with Airflow and Python preferred.
  • Understanding of automated QA needs related to Big data technology.
  • Strong object-oriented design and analysis skills.
  • Excellent written and verbal communication skills.

Responsabilidades

  • Utilize your software engineering skills including Spark, Python, Scala to analyze disparate, complex systems and collaboratively design new products and services.
  • Integrate new data sources and tools.
  • Implement scalable and reliable distributed data replication strategies.
  • Collaborate with other teams to design, develop and deploy data tools that support both operations and product use cases.
  • Perform analysis of large data sets using components in AWS Cloud.
  • Own product features from the development, testing through to production deployment.
  • Evaluate big data technologies and prototype solutions to improve our data processing architecture.
  • Automate different pipelines.

Conhecimentos

Java
Python
Scala
Spark
AWS
Data modeling
Big data
Agile
Airflow
Security best practices

Ferramentas

Airflow
Hadoop ecosystem
Kafka
Hive
MapReduce

Descrição da oferta de emprego

  • 4-7 years of experience delivering enterprise software solutions
  • Proficient with Java, Spark, Scala, Python, AWS Cloud technologies
  • 2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala
  • Flair for data, schema, data model, how to bring efficiency in big data related life cycle.
  • Experience with Agile Development methodologies.
  • Experience with data ingestion and transformation
  • Have understanding for secure application development methodologies.
  • Experience in with Airflow and Python will be preferred.
  • Understanding of automated QA needs related to Big data technology.
  • Strong object-oriented design and analysis skills
  • Excellent written and verbal communication skills
Responsibilities
  • Utilize your software engineering skills including Spark, Python, Scala to Analyze disparate, complex systems and collaboratively design new products and services
  • Integrate new data sources and tools
  • Implement scalable and reliable distributed data replication strategies
  • Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases
  • Perform analysis of large data sets using components in AWS Cloud
  • Own product features from the development, testing through to production deployment
  • Evaluate big data technologies and prototype solutions to improve our data processing architecture
  • Automate different pipelines

Knowledge and hands-on experience of using AI technologies for development, Testing and automation.

  • Knowledge of Cloud Technologies is a must such as AWS and its services
Obtém a tua avaliação gratuita e confidencial do currículo.

ou arrasta e larga o ficheiro aqui.

Similar jobs

Ofertas semelhantes que vale a pena comparar

Big Data Engineer
Big Data Engineer

Zorba AI • Hyderabad

Presencial
INR 1 200 000 - 1 800 000
Data Engineer
Data Engineer

Amplicomm • Bengaluru

Presencial
INR 1 500 000 - 2 800 000
AWS Big Data Engineer
AWS Big Data Engineer

Zorba AI • Hyderabad

Presencial
INR 1 500 000 - 2 500 000
Data Engineer - ETL/PySpark
Data Engineer - ETL/PySpark

Forward Eye Technologies • Pune District

Presencial
INR 900 000 - 1 300 000
Data Engineer - ETL/PySpark
Data Engineer - ETL/PySpark

Forward Eye Technologies • Dadri

Presencial
INR 1 500 000 - 2 400 000
Big Data Applications Engineer_AWS
Big Data Applications Engineer_AWS

Zorba AI • Hyderabad

Presencial
INR 1 800 000 - 2 600 000
Data Engineer
Data Engineer

Grid Dynamics • Bengaluru

Híbrido
INR 900 000 - 1 400 000
Big Data Engineer
Big Data Engineer

EPAM Systems Inc • Hyderabad, Pune District, Bengaluru

Presencial
INR 1 200 000 - 2 400 000
Big Data Engineer
Big Data Engineer

Infosys • Hyderabad, Pune District, Bengaluru

Presencial
INR 1 200 000 - 2 000 000
Senior Data Engineer
Senior Data Engineer

StartAppss System India Pvt. Ltd. • Indore District

Presencial
INR 1 800 000 - 3 600 000