ETL Developer

Infinite Computer Solutions; Maryland; On-site; USD 110,000 - 160,000; Full time

Job summary

Infinite Computer Solutions is seeking an experienced ETL Developer with strong Python, PySpark, and FastAPI skills to design, build, and maintain scalable data integration solutions across enterprise platforms. You will work with MongoDB, Elasticsearch, Snowflake, MSSQL, and cloud services in AWS and GCP to ingest, transform, and load data, and expose RESTful APIs for data access.

You will collaborate with data engineers, architects, and analytics teams to deliver robust data pipelines,

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Equivalent work experience may be considered.
• 5+ years of experience in Data Engineering, ETL Development, or related roles.
• Strong experience developing ETL pipelines using Python and PySpark.
• Experience working in cloud-based data environments.
• Proven experience building REST APIs using FastAPI.

Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark.
• Extract, transform, validate, and load data from multiple sources.
• Build reusable data ingestion frameworks and automation utilities.
• Develop batch and near-real-time data processing solutions.
• Ensure data integrity, quality, and consistency across enterprise systems.
• Develop integrations with MongoDB, Elasticsearch, Snowflake, MSSQL.
• Create optimized data movement solutions between operational databases, cloud storage platforms, and analytical environments.
• Design and implement efficient data models and data loading strategies.
• Optimize SQL queries and database performance.
• Design and develop RESTful APIs using FastAPI for data ingestion/extraction/validation.

Skills

Python

PySpark / Apache Spark

SQL

Data transformation

Data quality

FastAPI

REST APIs

OpenAPI/Swagger

API security

Analytical thinking

Education

Bachelor’s degree in Computer Science/IT/Engineering

Tools

MongoDB

Elasticsearch

Snowflake

MSSQL

AWS

GCP

S3

GCS

Apache Spark

FastAPI (framework)

Job description

Job Description

ETL Developer – Python, PySpark, FastAPI, AWS & GCP

Position Summary

We are seeking a highly skilled ETL Developer with expertise in Python, PySpark, and FastAPI to design, develop, and maintain scalable data integration solutions across enterprise data platforms. The ideal candidate will have hands‑on experience building high‑performance ETL/ELT pipelines that extract, transform, and load data into and from MongoDB, Elasticsearch, Snowflake, and Microsoft SQL Server (MSSQL). This role requires strong cloud experience in both AWS and Google Cloud Platform (GCP), including the ability to read, process, and write data to Amazon S3 and Google Cloud Storage (GCS). The candidate will also develop RESTful APIs using FastAPI to support data ingestion, integration, and microservices‑based architectures.

Key Responsibilities

ETL Development & Data Integration

• Design, develop, and maintain scalable ETL/ELT pipelines using Python and PySpark.
• Extract, transform, validate, and load large volumes of structured and semi-structured data from multiple data sources.
• Build reusable data ingestion frameworks and automation utilities.
• Develop batch and near-real-time data processing solutions.
• Ensure data integrity, quality, and consistency across enterprise systems.

Database & Data Platform Integration

• Develop integrations with:
  • MongoDB
  • Elasticsearch
  • Snowflake
  • Microsoft SQL Server (MSSQL)
• Create optimized data movement solutions between operational databases, cloud storage platforms, and analytical environments.
• Design and implement efficient data models and data loading strategies.
• Optimize SQL queries and database performance.

API Development

• Design and develop RESTful APIs using FastAPI for:
  • Data ingestion services
  • Data extraction services
  • Data validation and transformation services
  • Internal data access and integration layers
• Implement authentication, authorization, logging, and monitoring for APIs.

Cloud Data Engineering

• Develop and maintain solutions for reading and writing data to:
  • Amazon S3
  • Google Cloud Storage (GCS)
• Build cloud-native data processing workflows.
• Implement secure data transfer mechanisms across cloud platforms.
• Support hybrid cloud and multi-cloud data architectures.

Performance Optimization

• Tune and optimize Spark jobs for large-scale distributed processing.
• Improve ETL throughput and reduce processing latency.
• Troubleshoot pipeline failures and implement robust error handling.
• Monitor system performance and recommend improvements.

DevOps & Automation

• Implement CI/CD pipelines for ETL and API deployments.
• Automate infrastructure and deployment processes where applicable.
• Participate in code reviews and maintain development standards.
• Create technical documentation and operational runbooks.

Collaboration

• Work closely with Data Architects, Data Engineers, Business Analysts, and Application Development teams.
• Collaborate with stakeholders to understand business and technical requirements.
• Participate in architecture reviews and solution design discussions.

Required Qualifications

Education

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
• Equivalent work experience may be considered.

Experience

• 5+ years of experience in Data Engineering, ETL Development, or related roles.
• Strong experience developing ETL pipelines using Python and PySpark.
• Experience working in cloud-based data environments.
• Proven experience building REST APIs using FastAPI.

Required Technical Skills

Programming & Data Processing

• Python (Advanced)
• PySpark / Apache Spark
• SQL (Advanced)
• Data Transformation and Data Quality Frameworks

Databases & Data Platforms

• MongoDB
• Elasticsearch
• Snowflake
• Microsoft SQL Server (MSSQL)

API Development

• FastAPI
• RESTful Services
• OpenAPI/Swagger
• API Security (OAuth2, JWT, API Keys)

Cloud Platforms

AWS

• Amazon S3

Google Cloud Platform (GCP)

• Google Cloud Storage (GCS)

Preferred Qualifications

• Experience with Agentic AI.

Key Competencies

• Strong analytical and problem-solving skills.
• Ability to work independently and collaboratively within cross-functional teams.
• Excellent verbal and written communication skills.
• Strong attention to detail and commitment to data quality.
• Ability to manage multiple priorities in a fast-paced environment.
• Passion for building scalable and reliable data solutions.

Qualifications

Bachelor Degree

Range Of Year Experience-Min Year

5

Range Of Year Experience-Max Year

5