VS01502 - Java AWS Developer (python)

E4 Software Services Pvt Ltd.; Bengaluru; On-site; INR 1,200,000 - 1,500,000; Full time

Job summary

A leading software services company in India seeks a talented data engineer with over 6 years of experience to design and optimize data pipelines using PySpark on AWS Databricks. The ideal candidate will have strong expertise in Python, working knowledge of SQL, and a solid grasp of cloud-native architectures. Responsibilities include implementing data workflows, ensuring data quality, and collaborating cross-functionally. This role promises substantial growth in a dynamic environment.

Qualifications

• 6+ years of professional experience in the software industry.
• Strong hands-on expertise in PySpark for distributed data processing.
• Proven experience with Python for production-grade code.

Responsibilities

• Design and optimize data processing pipelines using PySpark on AWS Databricks.
• Implement data ingestion and transformation workflows.
• Collaborate with teams to build cloud-native data solutions.

Skills

PySpark

Python programming

AWS Databricks

SQL

Cloud-native architectures

Data engineering best practices

Job description

Responsibilities

• Design, develop, and optimize data processing pipelines using PySpark on AWS Databricks, ensuring scalability and high performance.
• Implement data ingestion, transformation, and aggregation workflows to support advanced analytics and enterprise reporting needs.
• Collaborate with cross‑functional teams to build cloud‑native data solutions leveraging AWS services such as S3 and EC2.
• Write efficient Python code for automation, orchestration, and integration of data workflows across environments.
• Ensure data quality, reliability, and governance through the implementation of best practices, including error handling, auditing, and logging.
• Participate in performance tuning of PySpark jobs, cloud cost optimization, and continuous improvement of the data platform.

Mandatory Skills:

• 6+ years of professional and relevant experience in software industry.
• Strong hands‑on expertise in PySpark for distributed data processing and transformation.
• Proven experience with Python programming, including implementing reusable, production‑grade code.
• Practical knowledge of AWS Databricks for building and orchestrating large‑scale data pipelines.
• Demonstrated experience in processing structured and unstructured data using Spark clusters and cloud data platforms.
• Ability to apply data engineering best practices including version control, CI/CD for data pipelines, and performance tuning.

Preferred Skills:

• Working knowledge of SQL for data querying, analysis, and troubleshooting.
• Experience using AWS S3 for object storage and EC2 for compute orchestration in cloud environments.
• Understanding of cloud‑native data architectures and principles of data security and governance.
• Exposure to BFSI domain use cases and familiarity with handling sensitive financial data.
• Familiarity with CI/CD tools and cloud‑native deployment practices.
• Familiarity with ETL scripting and data pipeline automation.
• Knowledge of big data ecosystems and distributed computing.