Python Big Data Engineer

Veriipro; Hartford (CT); On-site; USD 120,000 - 150,000; Full time

Job summary

Veriipro is seeking a Python Big Data Engineer in Hartford, Connecticut, to design and develop scalable data engineering solutions. The candidate will have a strong background in Databricks, Python, and AWS, focusing on creating cloud-native data pipelines and implementing ETL processes.

The role involves automating infrastructure deployment and ensuring data quality. Interested candidates with 8-10 years of experience and strong analytical skills are encouraged to apply.

Qualifications

• 8–10 years of overall IT experience focusing on Data Engineering and Big Data technologies.
• Hands‑on experience with Databricks, Python, and AWS services.
• Strong knowledge of ETL development and data modeling.

Responsibilities

• Design and develop scalable data pipelines using Databricks and Python.
• Build and maintain cloud‑based data solutions on AWS.
• Develop and enhance ETL/ELT workflows.

Skills

Databricks

Python

AWS Cloud Services

Tools

Terraform

Node.js

Job description

We are seeking an experienced Python Big Data Engineer with strong expertise in Databricks, Python, and AWS to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have hands‑on experience building cloud‑native data pipelines, implementing ETL processes, and developing robust data platforms in an AWS environment. Experience with infrastructure automation and modern application development is highly desirable.

Roles and Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks and Python.
• Build and maintain cloud‑based data solutions on AWS.
• Develop and enhance ETL/ELT workflows for processing large‑scale datasets.
• Implement data integration, transformation, and validation processes.
• Collaborate with cross‑functional teams to understand business and technical requirements.
• Ensure data quality, reliability, security, and performance across platforms.
• Automate cloud infrastructure provisioning and deployment using Terraform.
• Support software engineering initiatives, including API and application development where required.
• Troubleshoot production issues and optimize existing data processing workflows.

Required Qualifications

• 8–10 years of overall IT experience with a strong focus on Data Engineering and Big Data technologies.
• Hands‑on experience with Databricks, Python, and AWS services.
• Strong knowledge of ETL development, data modeling, and data pipeline architecture.
• Experience working with large‑scale structured and unstructured datasets.
• Understanding of cloud‑native architecture and distributed data processing.
• Experience with version control systems and CI/CD practices.
• Strong analytical, problem‑solving, and communication skills.

Preferred Qualifications

• Experience with Node.js development.
• Hands‑on experience with Terraform for Infrastructure as Code (IaC).
• Familiarity with Agile/Scrum development methodologies.

Required Skills

Must‑Have Skills

• Databricks
• Python
• AWS Cloud Services

Preferred Skills

• Node.js
• Terraform