Remote Senior Data Engineer (ETL Data Modeling)

Remotestar; Bengaluru; Remote; INR 1,500,000 - 2,500,000; Full time

Benefits offered by this job

Dynamic working environment

International exposure

Flexible working hours

Job summary

A dynamic tech company is looking for a Remote Senior Data Engineer (ETL Data Modeling) to develop and maintain ETL/ELT pipelines and support data integration from various sources. The ideal candidate has over 5 years of IT experience, deep knowledge in Python and SQL, and the ability to work in a fast-paced startup environment. This fully remote position offers flexible working hours and a chance to play a key role in the company's growth.

Qualifications

• 5+ years IT experience with 3+ years in data integration and ETL.
• Broad expertise with data engineering tools and systems.
• Deep knowledge of Python, SQL, and dimensional modeling.

Responsibilities

• Implement ETL/ELT pipelines using Python and Snowflake.
• Support data warehouse migration to Snowflake.
• Develop and maintain data solutions with AWS big data stack.

Skills

ETL/ELT development

Data Integration

Python

SQL

Data governance

Kafka

AWS Glue

Kubernetes

Apache Airflow

Tools

Snowflake

Redshift

DBT

Bitbucket/Git

Job description

RemoteStar is looking to hire aRemote Senior Data Engineer (ETL Data Modeling)on behalf of our client based in the UK with a fully remote work policy.

About Client

The client building, the B2B marketplace for diamonds. It’s an industry-leading B2B diamond and gemstones marketplace, connecting jewellery retailers to gemstone supplies They have a presence in London, Hong Kong, Amsterdam, and as well in Mumbai and now in New York in 2001.

About the role

TheRemote Senior Data Engineer (ETL Data Modeling) plays a pivotal role in growing our externally facing technical platform, supporting our customers' needs, and driving technical excellence within the team.

RESPONSIBILITIES

• Implementing ETL/ELT pipelines within and outside of a data warehouse using Python, Pyspark and Snowflakes Snow SQL.
• Support Redshift DWH to Snowflake Migration.
• Design, implement, and support data warehouse/data lake infrastructure using AWS big data stack, Python, Redshift, Snowflake, Glue/lake formation, EMR/Spark/Scala etc.
• Work with data analysts to scale value-creating capabilities, including data integrations and transformations, model features, and statistical and machine learning models.
• Work with Product Managers, Finance, Service Engineering Teams and Sales Teams on a day-to-day basis to support their new analytics requirements.
• Implement data quality and data governance measures and execute data profiling and data validation procedures
• Implement and uphold data governance practices to maintain data quality, integrity, and security throughout the data lifecycle.
• Leverage open-source technologies to build robust and cost-effective data solutions.
• Develop and maintain streaming pipelines using technologies like Apache Kafka etc.

Skills and Qualifications

• Must have total 5+ yrs. of IT experience and 3+ years' experience in data Integration, ETL/ETL development, and database design or Data Warehouse design
• Broad expertise and experience with distributed systems, streaming systems, and data engineering tools, such as Kubernetes, Kafka, Airflow, Dagster, etc.
• Experience in data transformation, ETL/ELT tool and technologies such as AWS Glue, DBTetc for transforming structured/semi structured and unstructured datasets.Experience in ingesting and integrating data from APIs/JDBC/CDC sources.
• Deep knowledge of Python, SQL, relational/ non-relational database design, and master data strategies.
• Experience defining, architecting, and rolling out data products, including ownership of data products through their entire lifecycle.
• Deep understanding of Star and Snowflake dimensional modeling. Experience with relational databases, including SQL queries, database definition, and schema design.
• Experience with data warehouses, distributed data platforms, and data lakes.
• Strong proficiency in SQL and at least one programming language (e.g., Python,Scala, JS).
• Familiarity with data orchestration tools, such as Apache Airflow, and the ability to design and manage complex data workflows.
• Familiarity with agile methodologies, sprint planning, and retrospectives.
• Proficiency with version control systems, Bitbucket/Git.
• Ability to work in a fast-paced startup environment and adapt to changing requirements with several ongoing concurrent projects.
• Excellent verbal and written communication skills.

Preferred/bonus skills

• Redshift to Snowflake migration experience.
• Experience with DevOps technologies such as Terraform, CloudFormation, and Kubernetes.
• While not mandatory, experience or knowledge in machine learning techniques is highly preferable, enriching our data engineering capabilities.
• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases

WHAT THEY OFFER

• Dynamic working environment in an extremely fast-growing company
• Work in an international environment
• Work in a pleasant environment with very little hierarchy
• Intellectually challenging, play a massive role in client’s success and scalability
• Flexible working hours