Senior AWS Data Engineer _ Exp: 5 to 7 Years

Atyeti Inc; India; Sur place; INR 1 800 000 - 2 600 000; Plein temps

Résumé du poste

Atyeti Inc is seeking a Data Engineer with 5-7 years of experience to design and optimize data pipelines using PySpark, advanced SQL, Snowflake, and Spark. The role requires hands-on expertise with Snowflake architecture, internal/external tables, stages, masking policies, and real-time data streaming with Kafka.

Unix shell scripting and JSON/XML/Parquet handling are part of daily tasks. You will contribute to scalable, secure data solutions and collaborate with cross-functional teams to deliver

Qualifications

• 5-7 years of experience as a Data Engineer with PySpark and advanced SQL.
• Hands-on experience with Snowflake architecture, tables, stages and masking policies.
• Experience on Big Data platforms with Spark, JSON/Parquet/XML formats.
• Proficient in UNIX shell scripting and data modeling.
• Familiar with data governance and agile software practices.

Responsabilités

• Design and build scalable data pipelines using PySpark and Snowflake.
• Implement real-time data streaming with Kafka and manage data formats.
• Collaborate in an agile team to optimize performance and governance.

Connaissances

PySpark

Advanced SQL

Spark

Snowflake

Airflow

Python

Big Data

JSON/Parquet/XML

Kafka

Unix Shell

Glue

Formation

Bachelor's degree in CS/IT

Outils

Snowflake

SQL

Kafka

Unix Shell

Glue

Airflow

Description du poste

PySpark and Advanced SQL is a must. We are looking for DE with excellent SQL skills and advance python and pyspark along with other required skillset.

• 5-7 years of experience as a Data engineer with extensive working experience usingPyspark, Advanced SQL, Snowflake, complex understanding of SQL and Skills, Spark, Snowflake, Glue and Airflow .
• Experience in working Snowflake and understanding of Snowflake architecture along with concepts like internal, external tables, stages, masking policies etc.
• Experienced on Big Data platforms
• Used SQL, PL/SQL and similar languages, UNIX shell scripting.
• Experienced with data modeling, design patterns, building highly scalable and secured solutions. Knowledge of agile software development process and familiarity with performance metric tools.
• Experience working with JSON, yaml, parquet, xml etc.
• Data Governance (Knowledge about meta data management, data catalog/access)
• Event based and real time data streaming using Kafka

Hands-on experience with AI-assisted development tools (e.g., Cursor, Claude, GitHub Copilot) for spec-driven development, code generation, debugging, SQL optimization, code reviews, and validating AI-generated code.