Big Data Engineer

BCforward; Wilmington (DE); On-site; USD 90,000 - 120,000; Full time

Job summary

A leading IT consulting firm is looking for a Big Data Engineer in Wilmington, Delaware to design and maintain scalable data processing pipelines. The ideal candidate will have strong experience in cloud data services and programming languages such as Python. Responsibilities include optimizing data models, collaborating with teams, and driving continuous improvement in data engineering processes. This contract position demands a proactive approach to tackle data challenges efficiently.

Qualifications

• Experience in analytical problem solving and critical thinking.
• Proficiency in at least one programming language (Python, Java, or Scala).
• Experience with distributed data processing frameworks (like Spark).
• Experience with cloud data Lakehouse platforms (such as AWS Data Lake services).
• Familiarity with scheduling/orchestration tools (like Airflow).
• Knowledge of relational and NoSQL databases.
• Understanding of data structures and serialization formats.
• Experience with Agile methodologies and CI/CD practices.

Responsibilities

• Design and maintain scalable data processing pipelines on the cloud.
• Develop and optimize data models for large-scale datasets.
• Collaborate with teams to translate business requirements into data solutions.
• Identify opportunities for enhancing data infrastructure and processing capabilities.

Skills

Analytical problem solving

Python

Spark

AWS Data Lake services

Airflow

NoSQL databases

JSON, AVRO, Protobuf

Agile methodology

Test-driven development

Job description

Presently we are unable to sponsor and request applicants to apply who are authorized to work without sponsorship*

Employment TYPE: Contract W2 - NO C2C

Job Title: Big Data Engineer

About BCforward

BCforward is a leading global IT consulting and workforce solutions firm providing services and support to Fortune 500 and government clients. Founded in 1998, BCforward has grown with our customers needs into a full-service business solutions provider. With delivery centers and offices across North America and India, we take pride in building long-term relationships and delivering excellence through innovation, collaboration, and integrity.

Job Description

We are seeking a Big Data Engineer to join our team. The ideal candidate will have strong experience in Java/J2EE development, Agile delivery, and CI/CD practices and a proven ability to design, build, test, deploy, and maintain secure, resilient applications.

Responsibilities

• Design, develop and maintain scalable and large-scale data processing pipelines and infrastructure on the cloud following engineering standards, governance standards and technology best practices
• Develop and optimize data models for large-scale datasets, ensuring efficient storage, retrieval, and analytics while maintaining data integrity and quality.
• Collaborate with cross-functional teams to translate business requirements into scalable and effective data engineering solutions.
• Demonstrate a passion for innovation and continuous improvement in data engineering, proactively identifying opportunities to enhance data infrastructure, data processing and analytics capabilities.

Required Skills & Qualifications

• Strong analytical problem solving and critical thinking skills
• Proficiency in at least one programming language ( Python, if not Java or Scala)
• Proficiency in at least one distributed data processing framework ( Spark or similar)
• Proficiency in at least one cloud data Lakehouse platforms (AWS Data lake services or Databricks, alternatively Hadoop)
• Proficiency in at least one scheduling/orchestration tools ( Airflow, if not AWS Step Functions or similar)
• Proficiency with relational and NoSQL databases.
• Proficiency in data structures, data serialization formats (JSON, AVRO, Protobuf, or similar), and big-data storage formats (Parquet, Iceberg, or similar)
• Experience working in teams following Agile methodology
• Experience with test-driven development (TDD) or behavior-driven development (BDD) practices, as well as working with continuous integration and continuous deployment (CI/CD) tools.

Preferred Skills

• Experience with cloud platforms and related services.
• Exposure to test automation and performance engineering.