Data Engineer

Straive; Dadri; Remote; INR 1,200,000 - 1,800,000; Full time

Job summary

A data technology company is looking for a Data Engineer to build and maintain modern data platforms. This role involves developing cloud-centric data processing capabilities and ensuring data pipelines are scalable. The ideal candidate should have extensive experience in software engineering with a strong focus on data. Knowledge of AWS services and proficiency in Python and SQL are essential.

Qualifications

• 6+ years of experience in software engineering with strong focus on data.
• Experience working with Cloud data platforms, preferably AWS.
• Strong understanding of data modelling concepts and best practices.

Responsibilities

• Develop, test, and deploy software to generate data assets for downstream use.
• Work with big data and cloud technologies such as EC2, Lambda, AWS Glue.
• Help improve data management processes for massive volumes of data.

Skills

Python

SQL

Cloud platforms (AWS)

CI workflows

Data modeling

Terraform

Airflow

dbt

Education

University degree in Computer Science, Mathematics, Engineering, or related field

Tools

AWS Glue

Redshift

EC2

Lambda

Job description

Direct message the job poster from Straive

Building teams with brave minds at Straive!

Job Title: Data Engineer

Location: Remote

Job Type: Full time

Years of Experience: 6-9 years

About Straive:

Straive is a market leading Content and Data Technology company providing data services, subject matter expertise, & technology solutions to multiple domains.

Data Analytics & AI Solutions, Data AI Powered Operations and Education & Learning form the core pillars of the company’s long-term vision. The company is a specialized solutions provider to business information providers in finance, insurance, legal, real estate, life sciences and logistics. Straive continues to be the leading content services provider to research and education publishers.

Data Analytics & AI Services: Our Data Solutions business has become critical to our client's success. We use technology and AI with human experts-in loop to create data assets that our clients use to power their data products and their end customers' workflows. As our clients expect us to become their future-fit Analytics and AI partner, they look to us for help in building data analytics and AI enterprise capabilities for them.

With a client-base scoping 30 countries worldwide, Straive’s multi-geographical resource pool is strategically located in eight countries - India, Philippines, USA, Nicaragua, Vietnam, United Kingdom, and the company headquarters in Singapore.

Website: https://www.straive.com/

Role Overview:

We is looking for a Data Engineer with experience in building modern data platforms from the ground up. The successful candidate will build and maintain cloud-centric data processing capabilities that unleash the value of the League’s data assets to gain competitive advantage in the marketplace. They will be a hands-on contributor in the design and implementation of our cloud data platform that powers advanced analytics workloads.

The Data Engineer will work in an agile environment and will be responsible for building and maintaining data integration, ingestion, curation and pipeline orchestration capabilities. They are comfortable challenging assumptions to improve existing solutions and ensure the team is building the best scalable and cost-efficient product.

Responsibilities:

• Develop, test, and deploy software to generate data assets (relational, non-relational) for use by downstream BI engineers and data scientists
• Work with big data and cloud technologies such as EC2, Lambda, AWS Glue, Airflow, dbt, Redshift etc.
• Work closely with stakeholders to ensure successful data asset design and development
• Create software artifacts and patterns for reuse within the Data Engineering team.
• Ensure data pipelines are scalable, resilient and produced with the highest quality standards, metadata and validated for completeness and accuracy
• Work on a cross-functional Agile team responsible for end-to-end delivery of business needs
• Help improve data management processes - acquiring, transforming and storing massive volumes of structured and unstructured data
• Work closely with development teams to learn about needs, current processes and to promote best practices.

Required Qualifications & Experience:

• University degree in Computer Science, Mathematics, Engineering, or related field.
• 6+ years of experience in software engineering with strong focus on data.
• Experience working with Cloud data platforms, preferably AWS (Lambda, Step Functions, S3, AWS Glue, Athena, Redshift).
• An expert in Python and SQL, including query optimization for relational, NoSQL and columnar databases.
• Sound knowledge of CI workflows and build/test/deploy automation.
• Strong understanding of data modelling concepts and best practices.
• Relevant experience with IaC (Terraform, Cloud Formation)
• Relevant experience with modern big data processing and orchestration tools such as dbt and Airflow.
• A great teammate and self-starter, strong detail orientation is critical in this role.

Straive is an Equal Opportunity Employer. Our policy is clear: there shall be no discrimination based on age, disability, sex, race, religion or belief, gender reassignment, marriage/civil partnership, pregnancy/maternity, or sexual orientation.

We are an inclusive organization and actively promote equality of opportunity for all with the right mix of talent, skills and potential. We welcome all applications from a wide range of candidates. Selection for roles will be based on individual merit alone.

Referrals increase your chances of interviewing at Straive by 2x