Python Developer - Associate New IN - Jaipur

iCapital Network, Inc.; Jaipur; Hybrid; INR 1,800,000 - 2,600,000; Full time

Job summary

iCapital Network, Inc. is seeking a skilled Python developer to architect robust scraping solutions that interact with complex web applications and secured portals.

You will design and implement scalable data extraction pipelines using Python and AWS. Responsibilities include building serverless pipelines with Lambda, Step Functions, S3, CloudWatch, and Secrets Manager; ensuring fault tolerance, retries, and comprehensive logging; integrating multiple portals and APIs; contributing to tooling

Qualifications

• 5+ years of experience in Python development, with a strong focus on automation and data extraction.
• Proven expertise in web scraping using Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager.
• Comfortable working with APIs, JSON/XML, and data transformation.

Responsibilities

• Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
• Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
• Integrate and manage complex workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Skills

Python development

automation

data extraction

web scraping

Playwright

Selenium

Scrapy

BeautifulSoup

requests

AWS

Lambda

S3

IAM

CloudWatch

Secrets Manager

CI/CD pipelines

Docker

infrastructure as code

data transformation

APIs

JSON/XML

Education

Bachelor’s or Master’s degree in Computer Science, Software Engineering

Tools

Terraform

CloudFormation

Apache Airflow

Job description

You will assist in architecting and building robust, secure, and scalable scraping solutions that interact with complex web applications and secured portals.

Responsibilities:

• Assist with the design and implementation of advanced scraping solutions using Python, Playwright, and AWS services.
• Architect scraping pipelines using serverless AWS components such as Lambda, Step Functions, S3, CloudWatch, and Secrets Manager.
• Build systems that scale to support high volumes of data extraction with fault tolerance, retries, and intelligent logging.
• Integrate and manage complex workflows across multiple portals, APIs, and data sources.
• Contribute to architectural decisions, tooling, and best practices.

Qualifications:

• 5+ years of experience in Python development, with a strong focus on automation and data extraction.
• Proven expertise in web scraping using tools like Playwright, Selenium, Scrapy, BeautifulSoup, and requests.
• Proficient in deploying and managing scraping workloads in AWS, particularly Lambda, S3, IAM, CloudWatch, and Secrets Manager.
• Comfortable working with APIs, JSON/XML, and data transformation.

Preferred Qualifications:

• Experience with CI/CD pipelines, Docker, and infrastructure as code (e.g., CloudFormation, Terraform).
• Familiarity with data pipeline orchestration tools such as Apache Airflow or AWS Step Functions.
• Prior experience scraping from secure or enterprise-level portals (e.g., financial, healthcare, legal).
• Background in data engineering or ETL workflows is a plus.
• Exposure to Python testing frameworks and writing unit/integration tests.

Education:

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, or related field, or equivalent professional experience.

We believe the best ideas and innovation happen when we are together. Employees in this role will work in the office four days, with the flexibility to work remotely one day (Friday).

iCapital offers a comprehensive benefits package that includes a total compensation program consisting of competitive salary, annual performance bonus, and equity for all full-time employees; healthcare with 100% employer-paid health and dental insurance; and generous paid time off (PTO).