Senior Data Engineer – Data Lake / AI

Onward Legal Innovation; Columbia (SC); Remote; USD 170,000 - 200,000; Full time

Job summary

Onward Legal Innovation seeks a Senior Data Engineer to advance a global law firm's data and AI transformation. You will design AWS-based data lakes/lakehouses, ingest from enterprise systems, and create high‑quality datasets for analytics and AI/ML in a regulated environment.

The role is primarily remote, but you must reside within about 1.5 hours of Washington DC and be eligible to work in the US. Competitive salary and bonus.

Qualifications

• 10+ years enterprise data engineering experience.
• Experience with large-scale data transformation or cloud modernisation.
• Hands-on with AWS data platforms and services.
• Experience building data lakes, warehouses and lakehouse architectures.
• Experience with data pipelines for AI/ML, embeddings, vector databases and RAG.

Responsibilities

• Design and build scalable AWS-based data platforms, and potentially other data stacks.
• Build distributed processing workloads with tools like Amazon EMR, Spark and Databricks.
• Consolidate data from multiple enterprise applications into centralised data platforms.
• Build and optimise scalable ETL/ELT pipelines.
• Create curated, model-ready datasets for analytics, reporting and AI/ML.
• Implement data quality, lineage, metadata and governance standards.
• Work with internal stakeholders and external vendors to shape technical solutions and integrations.
• Navigate changing requirements and bring structure to complex, ambiguous problems.
• Help shape technical standards and mentor other data engineers.

Skills

Data engineering 10+ yrs

Cloud data platforms

AI/ML data ingestion

Python

SQL

ETL/ELT

Data governance

Tools

Amazon EMR

Databricks

Apache Spark

Airflow

Kinesis

Job description

Job description We are seeking a Senior Data Engineer with strong experience across AWS or Azure data, data lakes, and AI/ML platform ingestion to join a newly formed data engineering team at one of the world’s largest and most prestigious law firms as it undergoes a major data and AI transformation.

This is a senior, hands-on role focused on building an AWS data lake/lakehouse and evolving the enterprise data platform, ingesting data from complex enterprise business systems and creating high-quality datasets for analytics, reporting and AI/ML in a regulated environment. You’ll need to be comfortable with ambiguity, shape solutions, and collaborate closely with internal teams, vendors, and technology partners.

The role is primarily remote, but you must live within approximately 1.5 hours of Washington DC

Salary $170-200k plus Generous Benefits + Bonus

Candidates must already be eligible to work (Green Card or US Citizen), as there are no H-1B transfers or new visa sponsorship.

Key skills required for the Senior Data Engineer

• Circa 10+ years of enterprise data engineering experience.
• Experience working through large-scale data transformation or cloud modernisation programmes.
• Strong hands-on experience with AWS data platforms and services.
• Experience building data lakes, data warehouses and lakehouse architectures.
• Strong experience with technologies like Amazon EMR, Databricks and Apache Spark.
• Amazon Kinesis experience is desirable, particularly for real-time or streaming data ingestion.
• Exposure to data pipelines supporting AI/ML, embeddings, vector databases and RAG.
• Strong ETL/ELT, SQL and Python skills.
• Experience with orchestration tools such as Airflow.
• Knowledge of Bronze / Silver / Gold data architectures.
• Experience with data quality, lineage, governance and metadata.
• Ability to work effectively with ambiguity and turn loosely defined requirements into scalable technical solutions.
• Strong experience collaborating with third-party vendors, consultancies and technology partners.
• Background in large enterprise, consulting, professional services or financial services environments is highly desirable.

What you’ll do as a Senior Data Engineer

• Design and build scalable AWS-based data platforms, and potentially other data stacks.
• Build distributed processing workloads with tools like Amazon EMR, Spark and Databricks.
• Consolidate data from multiple enterprise applications into centralised data platforms.
• Build and optimise scalable ETL/ELT pipelines.
• Create curated, model-ready datasets for analytics, reporting and AI/ML.
• Implement data quality, lineage, metadata and governance standards.
• Work with internal stakeholders and external vendors to shape technical solutions and integrations.
• Navigate changing requirements and bring structure to complex, ambiguous problems.
• Help shape technical standards and mentor other data engineers.

Why this role?

This is a chance to join a globally recognised organisation at akey stage in a major data and AI transformation.

You’ll work to modernise a complex enterprise data estate and build the data foundations for analytics, machine learning and generative AI.

The role is particularly suited to someone who combines strong hands-on engineering with the ability to operate in ambiguity, influence vendors and stakeholders, and help drive a large-scale transformation forward.