Aws Data Engineer

Object Technology Solutions; Hyderabad; Hybrid; INR 1,200,000 - 1,800,000; Full time

Job summary

OTSI Object Technology Solutions in Hyderabad is seeking a Data Engineer with 4+ years of experience to design and develop data pipelines using AWS PySpark, Glue and Snowflake. You will collaborate with business teams to drive data integration, governance and analytics-ready data assets.

This permanent role offers on-site work at Kokapet with regular day shifts (11:00–20:00). You will enhance ETL frameworks, work with S3, Lambda, SNS/SQS, and contribute to cost optimization while ensuring

Qualifications

• Bachelor's or master's degree in CS or equivalent.
• 4+ years of data engineering and analytics experience.
• Experience with AWS and GCP cloud solutions.
• Experience transforming data for data warehouse and lake.
• Proficiency with PySpark, Glue and Snowflake.
• Ability to work with stakeholders across IT and business.

Responsibilities

• Design, develop and maintain data pipelines and ETL processes.
• Collaborate with business teams for data integration and alignment.
• Build and optimize data warehouse and data lake architectures.
• Enhance ETL frameworks and enable cost savings.
• Work with AWS Lambda, SNS, SQS and other services.
• Ensure data governance and analytics readiness.

Skills

AWS

PySpark

Glue

Snowflake

GCP

S3

Lambda

SNS

SQS

Step Functions

Batch

Kafka

PowerBI

Talend Cloud ETL

Education

Bachelor's or Master's in Computer Science

Tools

AWS

PySpark

Glue

Snowflake

GCP

PowerBI

Talend Cloud ETL

Kafka

Job description

Hello ,

Greetings from OTSI..!!

Hope you are doing well! It was nice talking to you!!!

Client : Product based company

Role : Data Engineer

Location: Hyderabad (Kokapet)
Work Mode: WFO(3 Days)- Regular Dayshifts

Shift timings: 11 am to 8 pm

Required Experience: 4+ Years

Interview Mode: (Technical Virtual + Final Face to face mandatory)

Mode of hire Permanent with OTSI Object Technology Solutions, CMMI L5 Organization.

Job Description:

• Sound working experience in AWS cloud tech stack and Snowflake
• Coordinate closely with the Business teams for data integration and alignment
• Design, Develop, Maintain, support, and enhance the business intelligence data backend, including data warehouse and datalakes
• Ability to assess the current environment , requirements and propose the Data Engineering Solutions
• Adapt quickly to change in requirements and be willing to work with different technologies if required
• Ability to build frameworks and pipelines using AWS PySpark and Glue
• Able to enhance various Extraction, Ingestion and Enriched Frameworks that are already built using PySpark and Glue
• Able to build the logic by reading the data from AWS S3 and apply the transformations using Python and GCP
• Able to understand the existing AWS Lambda, SNS and SQS Events
• Able to work with platform team in problem solving and troubleshooting activities.
• Propose Standard and Best Practices by leveraging AWS Services
• Apply creative thinking/approach to arrive at technical solutions that enhance business goals and align with corporate technology strategies.
• Help Application teams in onboarding the feeds in to the built-in frameworks
• Make sure to follow best design practices during the implementation of ETL Frameworks
• Develop prototypes, technical proof of concepts and code with various solutions and convert platform architecture blueprints to real life working artifacts

Qualification:

• Bachelors or Masters Degree in Computer Science or equivalent experience
• At least 3+yrs years of experience in application design, development and analysis
• Experience in AWS Cloud Solutions & GCP. Retail Industry experience preferred.
• Implement data transformations and data structures for data warehouse and lake/repository.
• Manage cloud and/or on-premises solutions for data transfer and storage.
• Process data using spark (PySpark)
• Manage cloud and/or on-premises solutions for data transfer and storage.
• Collaborate and work with data analysts in various functions to ensure that data meets their reporting and analysis needs.
• Experience in creating Various ETL Frameworks in processing or extracting the data from cloud databases by using AWS Services by leveraging Lambda, Glue, PySpark, Step Functions, SNS, SQS and Batch.
• Proven ability to be a strategic thinker to drive the necessary ownership and data governance within the organization
• Ability to evaluate risks and provide business recommendations / solutions in a timely mannerAbility to remain calm in high stress situations while navigating and leading during times of conflict
• Should have sound knowledge in various AWS Services
• Able to understand the existing Frameworks built in AWS PySpark and Glue
• Able to scan ETL Frameworks and propose optimizations and cost savings.
• Able to work independently and able to take key decisions.
• Comfortable interacting with stakeholders at all levels through Business and IT.
• Ability to express complex concepts effectively, both verbally and in writing to IT & Business partners
• Understanding of and experience working in an Agile environment
• Knowledge on Talend Cloud ETL,Kafka, Snowflake Cloud and PowerBI is addon
• Independent. Strong critical thinking, decision making, troubleshooting and problem-solving skills.
• Go-Getter. Possesses strong, planning, execution and multitasking skills and demonstrated ability to reprioritize accordingly. Must be able to manage quickly changing priorities while meeting deadlines.