Data Engineer(AWS EMR, ECS, Step functions, Dynamo DB, Glue, PySpark

Experion; Ernakulam, Thiruvananthapuram; On-site; INR 2,000,000 - 4,200,000; Full time

Job summary

Experion in India is seeking an experienced data engineer to design and implement scalable analytics platforms on AWS. You will build robust data models, data pipelines, and lakehouse architectures using PySpark and Python, translating business needs into reliable data solutions.

Join a collaborative team that emphasizes governance, security, and agile delivery, with opportunities to work on real-time streaming, AWS Step Functions, Lambda, and Iceberg-based data stores.

Qualifications

• 5+ years of professional software development experience focused on analytics platforms.
• 3+ years hands-on with data platforms and big data ecosystems.
• Proficient in Python and Spark; strong SQL skills.
• Experience designing data lakehouse architectures on AWS.
• Hands-on with AWS services like Step Functions, Lambda, EventBridge.
• Experience in Agile development environments.
• Familiar with additional AWS services (Kinesis Firehose, SQS, Bedrock, SageMaker).
• Nice-to-have ML data pipeline experience.

Responsibilities

• Design, develop, and maintain analytical data models and metrics.
• Architect data lakehouse solutions on AWS.
• Build and optimize data pipelines using PySpark and Python.
• Ensure high performance, reliability, and scalability of data systems.
• Collaborate with stakeholders to translate requirements into data solutions.
• Work with architecture and operations to enforce governance and standards.
• Monitor data workflows and troubleshoot data quality and efficiency.
• Adhere to Information Security Management policies.

Skills

Python

Spark

Big data

Data lakehouse

AWS

DynamoDB

PostgreSQL

Aurora

DynamoDB

Snowflake

Tools

AWS EMR

AWS ECS

Step Functions

Lambda

EventBridge

Apache Spark

Iceberg

PostgreSQL

Aurora

SageMaker

Job description

• Design, develop, and maintain analytical data models and metrics using modern, scalable data platform technologies.
• Architect and implement data lakehouse solutions leveraging AWS services.
• Build, optimize, and maintain robust data pipelines using PySpark, Python, and AWS Glue.
• Ensure high performance, reliability, and scalability of data systems.
• Collaborate closely with business stakeholders to translate requirements into data solutions.
• Partner with architecture and operations teams to enforce best practices, standards, and governance.
• Monitor and troubleshoot data workflows, ensuring data quality and system efficiency.
• To adhere to the Information Security Management policies and procedures.

• 5+ years of professional experience in software development, with a focus on analytical platform development.
• 3+ years of hands-on experience with data platforms and big data ecosystems.

• Strong expertise in big data technologies, including:

AWS EMR AWS ECS Step Functions Apache Spark PostgreSQL DynamoDB Apache Iceberg Proficiency in Python and Spark programming. Experience building highly scalable, low-latency data systems (e.g., DynamoDB, Aurora, PostgreSQL, Redshift).

Proven experience designing and implementing data lakehouse architectures on AWS.

Hands-on experience with AWS services such as Step Functions, Lambda, and EventBridge.

Experience working in Agile development environments. Preferred Qualifications Strong knowledge of additional AWS services such as: Kinesis Firehose SQS Bedrock SageMaker Experience with real-time data streaming and event-driven architectures.

Familiarity with machine learning data pipelines is a plus.

Soft Skills & Competencies

Strong ability to learn and adapt to new technologies quickly. Excellent problem-solving and analytical thinking skills.

Ability to work independently and collaboratively in a team environment.

Strong multitasking skills in a fast-paced, dynamic setting. Excellent verbal and written communication skills.