Senior Data Engineer - Data Lake / AI

Onward Legal Innovation; Washington (District of Columbia); Remote; USD 170,000 - 200,000; Full time

Job summary

Onward Legal Innovation is seeking a Senior Data Engineer to build and evolve an AWS data lake/lakehouse, ingest data from enterprise systems, and prepare analytics-ready datasets for AI/ML in a regulated environment.

This hands-on role requires shaping solutions amid ambiguity, collaborating with internal teams and vendors, and delivering scalable data platforms during a major data and AI transformation.

Qualifications

• 10+ years of enterprise data engineering experience.
• Experience with large-scale data transformation or cloud modernization.
• Strong hands-on AWS data platforms and services.
• Experience building data lakes, warehouses, lakehouse architectures.
• Strong ETL/ELT, SQL and Python skills.
• Experience with Airflow orchestration.
• Knowledge of Bronze / Silver / Gold architectures.
• Data quality, lineage, governance and metadata.
• Ability to turn ambiguity into scalable solutions.
• Experience with third-party vendors and technology partners.
• Background in enterprise, consulting, professional services or financial services.

Responsibilities

• Design and build scalable AWS-based data platforms and other data stacks.
• Build distributed processing workloads with EMR, Spark and Databricks.
• Consolidate data from multiple enterprise applications into central platforms.
• Build and optimize scalable ETL/ELT pipelines.
• Create curated, model-ready datasets for analytics, reporting and AI/ML.
• Implement data quality, lineage, metadata and governance standards.
• Collaborate with internal stakeholders and external vendors to shape solutions.
• Navigate changing requirements and bring structure to ambiguity.
• Help shape technical standards and mentor other data engineers.

Skills

10+ years experience

AWS data platforms

Data lakes / lakehouse

Databricks

Apache Spark

Python

SQL

ETL/ELT

Airflow

Data governance

Vendor collaboration

Tools

Amazon EMR

Databricks

Apache Spark

Job description

We are seeking a Senior Data Engineer with strong experience across AWS or Azure data, data lakes, and AI/ML platform ingestion to join a newly formed data engineering team at one of the world’s largest and most prestigious law firms as it undergoes a major data and AI transformation.

This is a senior, hands-on role focused on building an AWS data lake/lakehouse and evolving the enterprise data platform, ingesting data from complex enterprise business systems and creating high-quality datasets for analytics, reporting and AI/ML in a regulated environment. You’ll need to be comfortable with ambiguity, shape solutions, and collaborate closely with internal teams, vendors, and technology partners.

The role is primarily remote, but you must live within approximately 1.5 hours of Washington DC

Salary $170-200k plus Generous Benefits + Bonus

Candidates must already be eligible to work (Green Card or US Citizen), as there are no H-1B transfers or new visa sponsorship.

Key skills required for the Senior Data Engineer

• Circa 10+ years of enterprise data engineering experience.
• Experience working through large-scale data transformation or cloud modernisation programmes.
• Strong hands-on experience with AWS data platforms and services.
• Experience building data lakes, data warehouses and lakehouse architectures.
• Strong experience with technologies like Amazon EMR, Databricks and Apache Spark.
• Amazon Kinesis experience is desirable, particularly for real-time or streaming data ingestion.
• Exposure to data pipelines supporting AI/ML, embeddings, vector databases and RAG.
• Strong ETL/ELT, SQL and Python skills.
• Experience with orchestration tools such as Airflow.
• Knowledge of Bronze / Silver / Gold data architectures.
• Experience with data quality, lineage, governance and metadata.
• Ability to work effectively with ambiguity and turn loosely defined requirements into scalable technical solutions.
• Strong experience collaborating with third-party vendors, consultancies and technology partners.
• Background in large enterprise, consulting, professional services or financial services environments is highly desirable.

What you’ll do as a Senior Data Engineer

• Design and build scalable AWS-based data platforms, and potentially other data stacks.
• Build distributed processing workloads with tools like Amazon EMR, Spark and Databricks.
• Consolidate data from multiple enterprise applications into centralised data platforms.
• Build and optimise scalable ETL/ELT pipelines.
• Create curated, model-ready datasets for analytics, reporting and AI/ML.
• Implement data quality, lineage, metadata and governance standards.
• Work with internal stakeholders and external vendors to shape technical solutions and integrations.
• Navigate changing requirements and bring structure to complex, ambiguous problems.
• Help shape technical standards and mentor other data engineers.

Why this role?

This is a chance to join a globally recognised organisation at a key stage in a major data and AI transformation.

You’ll work to modernise a complex enterprise data estate and build the data foundations for analytics, machine learning and generative AI.

The role is particularly suited to someone who combines strong hands-on engineering with the ability to operate in ambiguity, influence vendors and stakeholders, and help drive a large-scale transformation forward.