Sr Data Engineer

OpenKrill; United States; Remote; USD 140,000 - 190,000; Full time

Job summary

OpenKrill, based in the United States, seeks a Senior Data Engineer to design, build, and maintain scalable data pipelines for ETL from diverse sources. You will enable analysts to access clean, reliable data quickly.

The role requires leading complex SQL, developing AI/ML tools for data processing, and collaborating with data scientists, engineers, and product teams to translate business needs into robust data solutions.

Qualifications

• Bachelor's or master’s degree in a related field.
• 6–8 years of data engineering experience, SaaS/financial systems preferred.
• Experience designing robust data pipelines and ETL processes.
• Strong SQL writing and data modeling capabilities.
• Familiarity with cloud data platforms (AWS/Azure) and data warehouses.

Responsibilities

• Design, build, implement, and maintain ETL pipelines from diverse data sources.
• Lead the writing of complex SQL queries to support analytics.
• Develop tools and programming for AI/ML and big-data techniques to cleanse and transform data.
• Evaluate tools/technologies for data infrastructure; collaborate with cross-functional teams.
• Work with Python, SQL, Spark, Databricks, APIs; handle structured and unstructured data.

Skills

Python

SQL

Distributed processing

Data governance

Problem solving

Security awareness

Stakeholder collaboration

Education

Bachelor's degree

Master's degree

Tools

Apache Spark

Databricks

Snowflake

Redshift

BigQuery

Airflow

dbt

Job description

The Senior Data Engineer will design, build, implement, and maintain data processing pipelines for the extraction, transformation, and loading (ETL) of data from a variety of data sources. The role will develop robust and scalable solutions that transform data into a useful format for analysis, enhance data flow, and enable end-users to consume and analyze data faster and easier.

Expected Duties:

• Senior Data Engineers will design, build, implement, and maintain data processing pipelines for the extraction, transformation, and loading (ETL) of data from a variety of data sources
• Expected to lead the writing of complex SQL queries to support analytics needs
• Responsible for developing technical tools and programming that leverage artificial intelligence, machine learning, and big-data techniques to cleanse, organize and transform data and to maintain, defend and update data structures and integrity on an automated basis
• Senior Data Engineers will evaluate and recommend tools and technologies for data infrastructure and processing. Collaborate with engineers, data scientists, data analysts, product teams, and other stakeholders to translate business requirements to technical specifications and coded data pipelines
• The role will work with tools, languages, data processing frameworks, and databases such as R, Python, SQL, Databricks, Spark, Delta, APIs. Work with structured and unstructured data from a variety of data stores, such as data lakes, relational database management systems, and/or data warehouses

Qualifications: Knowledge, Skills, and Abilities:

The role will include work on problems of diverse scopes where analysis of information requires evaluation of identifiable factors. Work is expected to be done independently through independent judgment.

• Ability to assess unusual circumstances and use sophisticated analytical and problem-solving techniques to identify the cause
• Working with people, processes facing Enterprise a plus, improvement of processes/reduction of costs in the enterprise processes leveraging data is preferable.
• Ability to enhance relationships and networks with senior internal partners who are not familiar with the subject often requires persuasion
• Architect and scale our modern data platform to support real-time and batch processing for financial forecasting, risk analytics, and customer insights
• Enforce high standards for data governance, quality, lineage, and compliance
• Partner with stakeholders across engineering, finance, sales, and compliance to translate business requirements into reliable data models and workflows.
• Evaluate emerging technologies and lead POCs that shape the future of our data stack.
• Champion a culture of security, automation, and continuous delivery in all data workflows

Technical Qualifications:

• Deep expertise in Python, SQL, and distributed processing frameworks like Apache Spark, Databricks, Snowflake, Redshift, BigQuery.
• Proven experience with cloud-based data platforms (preferably AWS or Azure).
• Hands-on experience with data orchestration tools (e.g., Airflow, dbt) and data warehouses (e.g., Databricks, Snowflake, Redshift, BigQuery).
• Strong understanding of data security, privacy, and compliance within a financial services context.
• Experience working with structured and semi-structured data (e.g., Delta, JSON, Parquet, Avro) at scale.
• Familiarity with modelling datasets in Salesforce, Netsuite and Anaplan to solve business use cases required.
• Previous experience Democratizing data at scale for the enterprise a huge plus.

Educational Qualifications and Work experience

• Bachelor's or master's degree in computer science, Engineering, or a related field.
• 6-8 years of experience in data engineering, with a strong focus on financial systems on SaaS platforms.