Senior Data Engineer

Pattern Inc; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Pattern Inc is seeking a Data Engineer to tackle business challenges impacting its competitive edge. The role involves developing and supporting scalable data streams, ensuring data quality, and collaborating with technology teams.

Candidates should have over 7 years of experience, with strong skills in SQL and Python, and a bachelor's degree in a relevant field. Knowledge of data architectures and cloud technologies is a plus, alongside leadership capabilities.

Qualifications

• 7+ years overall experience in data engineering or related field.
• 3+ years of experience with SQL and Python.
• Experience in implementing data pipelines using modern data architectures.

Responsibilities

• Develop, deploy, and support automated data streams.
• Implement data auditing strategies to ensure quality.
• Collaborate with teams to specify data requirements.

Skills

SQL

Python

Data Quality

Data Governance

Data Pipeline Implementation

Collaboration

Education

Bachelor's Degree in Data Science or related field

Tools

Redshift

BigQuery

Snowflake

Spark

Hive

Trino/Presto

Job description

As a Data Engineer at Pattern, you will be working on business problems that have a huge impact on how the company maintains its competitive edge.

Roles and Responsibilities

• Develop, deploy, and support automated, scalable real-time and batch data streams from a variety of sources into the lakehouse.
• Develop and implement data auditing strategies and processes to ensure data quality; identify and resolve problems associated with large scale data processing workflows; implement technical solutions to maintain data pipeline processes and troubleshoot failures
• Collaborate with technology teams and partners to specify data requirements and provide access to data
• Tune application and query performance using profiling tools and SQL or other relevant query language
• Translate business and analytics requirements for data to a comprehensive data model and pipelines
• Foster data expertise and own data quality for assigned areas of ownership
• Work with data infrastructure to triage issues and drive to resolution

Required Qualifications

• Bachelor’s Degree in Data Science, Data Analytics, Information Management, Computer Science, Information Technology, related field, or equivalent professional experience
• Overall experience should be more than 7 + years
• 3+ years of experience working with SQL and Python
• 3+ years of experience in implementing data pipelines using modern data architectures
• 2+ years of experience working with data warehouses such as Redshift, BigQuery, Snowflake, or similar
• Experience with open-source based data architectures: Spark, Hive, Trino / Presto or similar
• Excellent software engineering and scripting knowledge
• Strong communication skills (both in presentation and comprehension) along with the aptitude for cross-collaboration across data management and analytics domains
• Expertise with data systems working with massive data volumes from various data sources
• Ability to lead a team of Data Engineers

Preferred Qualifications

• Experience working with time series databases
• Advanced knowledge of SQL, including the ability to write stored procedures, triggers, analytic/windowing functions, and tuning
• Advanced knowledge of Snowflake, including the ability to write and orchestrate streams and tasks
• Background in Big Data, non-relational databases, Machine Learning and Data Mining is a plus
• Experience with cloud-based technologies including SNS, SQS, SES, S3, Lambda, and Glue
• Experience with modern data platforms like Redshift, Cassandra, DynamoDB, Apache Airflow, Spark, or ElasticSearch
• Expertise in Data Quality and Data Governance