Senior Associate

AXTRIA; Arequipa; In loco; PEN 58.000 - 83.000; Tempo pieno

Descrizione del lavoro

AXTRIA is seeking a hands-on Project Lead with 4-6 years of experience to design, develop, and deploy data integration pipelines using Python/PySpark, AWS Glue, and Databricks on AWS.

The role requires deep Spark-based distributed computing expertise, proficiency in Databricks components, and the ability to mentor 1-2 teammates while delivering production-grade data platforms. You will collaborate with onshore/offshore leads and drive data ingestion and transformation across diverse formats.

Competenze

• 4–6 years of hands-on experience in designing and deploying data integration pipelines.
• Strong skills in Python, PySpark, Spark and SQL.
• Expertise with AWS Glue, Databricks and Spark-based distributed computing.
• Ability to mentor 1–2 teammates and lead technical delivery.
• Familiarity with data modelling (CRISP-DM) and data governance.

Mansioni

• Lead analytics and development tasks within large delivery teams.
• Develop, test, and deploy data integration processes on AWS/Azure.
• Provide PoC solutions and transition them to production.
• Create LLD/STM documents and translate business rules into data transformations.
• Manage Databricks notebooks, security, and workspace components.
• Oversee ingestion, processing, and data lake transformations at scale.

Conoscenze

Python

PySpark

Spark

Scala

SQL

R

Mentoring

Data modeling

CRISP-DM

ML concepts

AWS

Formazione

BE/B.Tech

MCA

Strumenti

AWS Glue

Databricks

EC2

S3

IAM

Athena

VPC

Notebook

MLFlow

Descrizione del lavoro

Bangalore, Block C2 (Floor 4), Brigade Tech Gardens

We are seeking a hands-on Project Lead with 4-6 years of experience in designing, developing, and deploying data integration pipelines using Python/PySpark, AWS Glue, and Databricks, with strong proficiency across AWS services including EC2, S3, IAM, Athena, and VPC. The ideal candidate brings deep expertise in Spark-based distributed computing, Databricks components (Databricks certified woudl be preferred) and the ability to code across R/Python/Scala/SQL while performing performance tuning and executor management. This role requires someone who can lead by example — independently driving technical delivery while mentoring and supervising 1-2 team members, with awareness of ML algorithms, data modelling frameworks (CRISP-DM), and a pharma domain background being a strong added advantage.

Job Responsibilities

• Be an Individual Contributor in the Analytics and Development team and solve real-world problems using cutting-edge capabilities and emerging technologies
• Be a part of large delivery teams working on advanced projects when expert assistance is required.
• Deliver advanced Data Science capabilities to businesses in a meaningful manner through successful proof-of-concept solutions, and later smoothly transition the proof-of-concept into production.
• Create LLD/STM documents, develop, test, and deploy data integration processes using Python/PySpark on AWS/Azure platform preferably using AWS Glue /Databricks - Collaborate with onshore/offshore lead designers, analyze LLD/STM, translate & apply business rules to data transformations
• Expertise Data Management principles. Experience in setting up ingestion, data processing- versioning, labelling, curating.
• Building ETL / data warehouse transformation processes and implementing Data lake project. Solutioning with large data sets, data processing at scale involving structured, semi-structured, and unstructured data formats.
• Data connectors for APIs, Kafka, batch processing.
• Managing Descopes involving Databricks, and various AWS components

Education

BE/B.Tech

Master of Computer Application

Work Experience

• Minimum of 4-6 years develop, test, and deploy data integration processes using Python/PySpark on AWS platform preferably using AWS Glue Databricks.
• Deep understanding of architecture and work experience on AWS Databricks, hands-on programming with Data Frames, Python, Scala.
• Good experience of Databrick components like Notebook, workspace, MLFlow and security management, creating dashboards, importing packages, DBFS, ML Flow, widgets and jobs in Databricks
• Familiar with cluster management, capacity & cost control in Databricks. Familiar with user & group access management and control
• Is aware of the overall architecture and UI
• Good experience of the notebook access management,
• Can code in R/Python/Scala/SQL
• 2-3 years of experience on Spark with Scala/Python/Java. Good experience of the distributed computing paradigm and various concepts in Spark
• Thorough knowledge of EC2, IAM, S3, VPC, Athena and Glue
• Is aware of the security and process design in AWS environment
• Is aware of the different phases involved in data modelling (CRISP-DM Model)
• Is aware of relevant modelling exercise to be performed (Forecasting, Clustering, Association, Classification etc.)

Ownership

Cultural Fit

Motivation to Learn and Grow

Databricks

Decision Making

Delivery Management- BIM/ Cloud Info Management

Problem Solving

Lifescience Knowledge

Communication

Capability Building / Thought Leadership