Lead Python Developer New IN - Jaipur

iCapital Network, Inc.; Jaipur; Hybrid; INR 3,500,000 - 5,500,000; Full time

Benefits offered by this job

Salary plus annual bonus

Equity

Health and dental insurance

Generous PTO

Job summary

iCapital Network, Inc. is seeking an experienced Python developer specializing in automation and web scraping to join our data engineering team in Jaipur, India.

You will design and implement scraping solutions using Playwright, AWS, and serverless architectures, build scalable pipelines, and contribute to best practices. This role blends hands-on coding with architectural decisions and code reviews.

Qualifications

• Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent professional experience.
• 10+ years of experience in Python development, with a strong focus on automation and data extraction.
• Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager.
• Comfortable working with APIs, JSON/XML, and data transformation.
• Experience with CI or CD pipelines, Docker, and infrastructure as code (i.e. CloudFormation, Terraform).
• Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions.
• Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal).
• Background in data engineering or ETL workflows is a plus.
• Exposure to Python testing frameworks and writing unit or integration tests.

Responsibilities

• Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
• Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
• Integrate and manage complex workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.
• Responsible for Code reviews and deployments.
• Work in the office four days, with the flexibility to work remotely one day (Friday).
• iCapital offers a comprehensive benefits package that includes total compensation with bonus and equity, and health/dental insurance.

Skills

Python development

Automation

Data extraction

APIs/JSON/XML

CI/CD

Testing

Code reviews

Education

Bachelor’s or Master’s degree in CS/SE or related field

Tools

Playwright

Selenium

Scrapy

BeautifulSoup

requests

Docker

CloudFormation

Terraform

Airflow

AWS Lambda

S3

IAM

CloudWatch

Secrets Manager

Job description

Responsibilities

• Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
• Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
• Integrate and manage complex workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.
• Responsible for Code reviews and deployments.

Qualifications

• Bachelor’s or master’s degree in computer science, software engineering, or related field, or equivalent professional experience
• 10+ years of experience in Python development, with a strong focus on automation and data extraction
• Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests
• Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager
• Comfortable working with APIs, JSON/XML, and data transformation
• Experience with CI or CD pipelines, Docker, and infrastructure as code (i.e. CloudFormation, Terraform)
• Familiar with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions
• Prior experience scraping from secure or enterprise-level portals (i.e. financial, healthcare, legal)
• Background in data engineering or ETL workflows is a plus
• Exposure to Python testing frameworks and writing unit or integration tests

We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office four days, with the flexibility to work remotely one day (Friday).

iCapital offers a comprehensive benefits package that includes

• total compensation program consisting of competitive salary, annual performance bonus, and equity for all full-time employees
• healthcare with 100% employer-paid health and dental insurance
• generous paid time off (PTO)