Python Data Engineer

ESR Healthcare; Bengaluru; On-site; INR 1,138,353 - 4,991,243; Full time

Job summary

A forward-thinking company is looking for a skilled Data Engineer to enhance its data infrastructure. In this exciting role, you will develop scalable data pipelines and integrate diverse data sources, ensuring quality and accessibility. Your expertise in Amazon Redshift, Python, and AWS will be crucial as you collaborate with cross-functional teams to optimize data architecture. This position offers the chance to work on innovative projects that drive business intelligence and analytics, making a significant impact on the organization. If you're passionate about data and eager to contribute to a dynamic team, this opportunity is perfect for you.

Qualifications

• 3+ years of experience in data engineering with expertise in AWS.
• Strong experience with ETL/ELT pipelines and data integration.

Responsibilities

• Design and maintain ETL/ELT pipelines using Python and Apache Airflow.
• Collaborate with teams to optimize data architecture and ensure data quality.

Skills

Python

Amazon Redshift

Apache Airflow

dbt

API Integration

AWS

SQL

Data Modeling

Education

Bachelor's degree in Computer Science

Master's degree in Data Science

Tools

AWS S3

AWS Lambda

AWS EC2

AWS CloudWatch

Git

Job description

Python Data Engineer

Python Data Engineer

Computer Software
India
In-office
Full-time
$13K - $57K per annum
Share Job
Apply
Description
About the Role:

We are seeking a skilled Data Engineer to build and maintain our data infrastructure, enabling business intelligence, reporting, and analytics. You will be responsible for developing scalable data pipelines, integrating data from diverse sources, and ensuring data quality and accessibility. Expertise in Amazon Redshift, Python, Apache Airflow, dbt, API integration, and AWS is essential. You will collaborate with cross-functional teams to optimize our data architecture and deliver clean, high-quality data to the business.

Key Responsibilities:

Data Pipeline Development: Design, build, and maintain robust ETL/ELT pipelines using Python, Apache Airflow, and dbt to ingest, transform, and load data into Amazon Redshift.
Amazon Redshift Management: Design, optimize, and maintain Amazon Redshift clusters for efficient large-scale data handling.
API Integration: Develop solutions to integrate external APIs for data ingestion, ensuring proper data extraction, transformation, and integration.
Data Modeling: Design and maintain scalable data models in Redshift (star/snowflake schemas) to support analytics and reporting.
AWS Infrastructure Management: Utilize AWS services (S3, Lambda, EC2, CloudWatch) to build and maintain a scalable and cost-efficient data architecture.
dbt (Data Build Tool): Leverage dbt to manage and automate SQL transformations, ensuring modularity, reusability, and documentation.
Workflow Orchestration: Use Apache Airflow to orchestrate and automate data workflows and scheduled jobs.
Data Quality & Testing: Implement data validation checks and testing frameworks to ensure data integrity, accuracy, and compliance.
Collaboration: Work closely with data scientists, analysts, and product teams to understand data needs and provide technical solutions.
Performance Optimization: Tune SQL queries and manage Redshift cluster performance for efficient data access and analysis.
Data Governance: Enforce data governance policies to ensure compliance with security, privacy, and data quality standards.
Key Skills & Qualifications:

Bachelor's/Master's degree in Computer Science, Engineering, Data Science, or a related field.
3+ years of experience in data engineering with expertise in Amazon Redshift, Python, and AWS.
Strong experience with Apache Airflow.
Hands-on experience with dbt.
Proficiency in API development and integration (RESTful APIs).
Extensive experience with AWS services (S3, Lambda, EC2, RDS, CloudWatch).
Expertise in data modeling (star/snowflake schemas).
Advanced SQL skills for querying and optimizing large datasets in Redshift.
Experience building ETL/ELT pipelines and integrating data from multiple sources (structured/unstructured).
Familiarity with Git and best practices for code management and deployment automation.
Knowledge of data governance principles.
Preferred Qualifications:

Experience with real-time data processing tools (Kafka, Kinesis).
Familiarity with data visualization tools (Tableau, Looker, Power BI).
Knowledge of other data warehousing solutions (Snowflake, BigQuery).
Experience with DevOps practices (Docker, Kubernetes).
Understanding of machine learning pipelines.
Soft Skills:

Strong analytical and problem-solving skills.
Ability to work independently and collaboratively.
Excellent written and verbal communication skills.
Detail-oriented, proactive, and self-motivated.
Strong organizational and project management skills.

Requirements
Where are you located?
What's your nationality?
What is your expected salary for this job?
What is your notice period; when can you engage this job?
Please only apply and press yes if you have read the requirements, education, experience and certifications, and match these.
Location
Bengaluru, Karnataka, India