Sr. Big Data Engineer

Rapinno Tech; New Boston (TX); Remote; USD 120,000 - 150,000; Full time

Job summary

A technology firm is seeking a Sr. Big Data Engineer to join their data platform team. In this role, you will be responsible for designing and maintaining scalable big data solutions. You should have significant experience in data engineering, integrating multiple data sources, and building data models. The ideal candidate will also have hands-on experience with containerization and API design. This position offers the flexibility of remote work, especially for candidates located in the EST region.

Qualifications

• Overall 10+ years of experience in software design and development.
• 5+ years of experience in data engineering.
• Hands-on knowledge of containers is a must.

Responsibilities

• Design, develop, and maintain data pipelines and platforms.
• Use big data technologies for scalable data solutions.
• Monitor performance and tune platform components.

Skills

Data engineering

Big data technologies

API design and implementation

NoSQL databases

ETL techniques

Agile methodology

Education

Bachelor’s Degree in Computer Science

Tools

Data visualization tools

Graph databases

Time series databases

Job description

Job Title: Sr. Big Data Engineer
Location: Boston region , USA ( look for EST Locals, can work remote initially).
Duration: 06+ Months Contract

Job Description

Overall 13+ year experience in software design and development. GC and Citizens preferred.

If H1 Candidates , Look for 13+ years minimum experience (check all the documents and share Genuine profiles only, as BGV is required )

Job purpose

Client is looking for a Sr. big data engineer to join data platform team located in Boston region, MA.

As part of the big data platform team, you will be responsible for designing, developing, testing , deploying and maintaining data platform components , big data solutions. As a Sr. big data engineer, you’ll work on collecting, storing, processing, and analyzing of huge sets of data and make this data easily accessible across the company and usable in multiple different projects.

Duties and responsibilities

• Use big data technologies to develop distributed, fault-tolerant scalable data solutions.
• Participate in discussions with customers, along with the product team, to understand their data requirements.
• Translate the business requirements in to corresponding data requirements.
• Collect and process data at scale from a variety of sources for different project needs.
• Participate in identifying, evaluating, selecting, and integrating big data frameworks and tools required for the big data platform.
• Design, develop, and maintain data pipelines , data platforms using selected frameworks and tools based on requirements from different projects.
• Convert structured and unstructured data into the form that is suitable for processing. Provide support to different teams in analysing data.
• Design, develop and maintain data API’s.
• Integrate data from variety of data sources using federation / virtualization techniques.
• Develop solutions independently based on high-level design and architecture with minimal supervision.
• Monitor the performance of the data platform on a regular basis and tune the infrastructure and platform components accordingly to ensure the best performance.
• Maintain a high level of expertise in data technologies and stay current on latest data technologies.

Qualifications

• Overall 10+ year experience in software design and development.
• 5+ years of experience in data engineering.
• Prior experience with implementing big data platform components that are scalable, high performing, and lower in operations cost.
• Proven experience with integration of data from multiple heterogeneous and distributed data sources.
• Experience with processing large amounts of data (structured and unstructured), building data models, data cleaning, data visualization and reporting.
• Experience in production support and troubleshooting.
• Hands-on knowledge of containers, API designing and implementing is a must.
• Experience with NoSQL databases, Graph databases, relataional databases, time series databases.
• Excellent knowledge of various ETL techniques and frameworks, various messaging systems, stream-processing systems, Big data ML toolkits, big data querying tools
• Good knowledge of Agile software development methodology.
• Excellent interpersonal, communication (verbal and written) skills.
• Proven experience in managing and working with teams based in multiple geograhies.

Bachelor’s Degree or higher in Computer Science or a related field