Big Data Engineer

INTECH Creative Services Pvt. Ltd.; Hyderabad; On-site; INR 1,400,000 - 2,400,000; Full time

Job summary

INTECH Creative Services Pvt. Ltd. in Hyderabad, India seeks adata-focused engineer to design, develop and deploy data tools supporting operations and products. You will own features from development to production, prototype new data architectures, and automate pipelines.

The role requires strong Java/Spark/Scala/Python skills, AWS cloud expertise, and experience with data ingestion, transformation, and agile development. Prior Airflow experience is a plus.

Qualifications

• BS degree in computer science, computer engineering or equivalent.
• 4-7 years of experience delivering enterprise software solutions.
• Proficient with Java, Spark, Scala, Python, AWS Cloud technologies.
• 2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala.
• Flair for data, schema, data model, how to bring efficiency in big data related life cycle.
• Experience with Agile Development methodologies.
• Experience with data ingestion and transformation.
• Understanding for secure application development methodologies.
• Experience with Airflow and Python will be preferred.
• Understanding of automated QA needs related to Big data technology.
• Strong object-oriented design and analysis skills.
• Excellent written and verbal communication skills Responsibilities
• Utilize your software engineering skills including Spark, Python,

Responsibilities

• Collaborate with other teams to design, develop and deploy data tools that support operations and product use cases.
• Own product features from development, testing through production deployment.
• Evaluate big data technologies and prototype solutions to improve data processing architecture.
• Automate different pipelines.
• Design new products and services by analyzing disparate, complex systems.
• Integrate new data sources and tools.
• Implement scalable and reliable distributed data replication strategies.

Skills

Java

Spark

Scala

Python

AWS Cloud

Airflow

Agile

Data ingestion

Transformation

Secure development

OO design

Distributed systems

Communication

Spark Ecosystem

Education

BS degree in computer science or equivalent

Tools

MapReduce

Hive

Kafka

Job description

Collaborate with other teams to design and develop and deploy data tools that support both operations and product use cases

• Perform analysis of large data sets using components in AWS Cloud
• Own product features from the development, testing through to production deployment
• Evaluate big data technologies and prototype solutions to improve our data processing architecture
• Automate different pipelines

Required Qualification:

BS degree in computer science, computer engineering or equivalent

• 4-7 years of experience delivering enterprise software solutions
• Proficient with Java, Spark, Scala, Python, AWS Cloud technologies
• 2+ years of experience across multiple Spark technologies such as MapReduce, Hive, Kafka, Scala
• Flair for data, schema, data model, how to bring efficiency in big data related life cycle.
• Experience with Agile Development methodologies.
• Experience with data ingestion and transformation
• Have understanding for secure application development methodologies.
• Experience in with Airflow and Python will be preferred.
• Understanding of automated QA needs related to Big data technology.
• Strong object-oriented design and analysis skills
• Excellent written and verbal communication skills Responsibilities
• Utilize your software engineering skills including Spark, Python,

Scala to Analyze disparate, complex systems and collaboratively

design new products and services

• Integrate new data sources and tools
• Implement scalable and reliable distributed data replication strategies

Preferred Qualification:

• Knowledge and hands-on experience of using AI technologies for development, Testing and automation.
• Knowledge of Cloud Technologies is a must such as AWS and its services.