Lead Data Engineer

Smart IT Frame LLC; United States; On-site; USD 100,000 - 720,000; Full time

Job summary

A leading company in data solutions is seeking a Data Engineer to lead project implementations using AWS and Snowflake. The role requires expertise in ETL, data management, and strong problem-solving skills. Candidates should have a proven track record in data engineering, ideally within the insurance industry, and experience with cloud technologies is essential for success in this position.

Qualifications

• 10+ years of relevant experience in Data Engineering and delivery.
• Strong understanding of data warehousing concepts.
• Proficiency in SQL, Python, and other relevant programming languages.

Responsibilities

• Lead the design and implementation of data solutions using AWS and Snowflake.
• Develop and maintain data pipelines, ensuring data quality and security.
• Provide technical leadership and mentorship to junior data engineers.

Skills

Snowflake expertise

ETL/Data Management

SQL

Python

Pyspark

Data ingestion

Agile Methodology

Education

Bachelor's or Master's degree in Computer Science

Job description

2 weeks ago Be among the first 25 applicants

Direct message the job poster from Smart IT Frame LLC

Smart IT Frame Transforms Visions into Tech Realities | - Talent Aquisition Group

Responsibilities:

• Lead the design, development, and implementation of data solutions using AWS and Snowflake.
• Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
• Develop and maintain data pipelines, ensuring data quality, integrity, and security.
• Optimize data storage and retrieval processes to support data warehousing and analytics.
• Provide technical leadership and mentorship to junior data engineers.
• Work closely with stakeholders to gather requirements and deliver data-driven insights.
• Ensure compliance with industry standards and best practices in data engineering.
• Utilize knowledge of insurance, particularly claims and loss, to enhance data solutions.

Must have:

• Snowflake expertise (hands-on & architectural understanding) , challenging situation in the project(most common question)this must be data related, or implementation related only
• ETL/Data Management proficiency, mastery in: Query optimization, data quality, performance tuning, big data problems, RBAC, row/column-level security, CDC, Incremental vs Batch, Unix commands, job scheduling, CI/CD pipeline
• Team leadership (8-10+ members) , End-to-end project implementation with clarity, complex challenges in the project, describe the specific problem and its solution
• Stakeholder management & cross-functional collaboration (knowledge on data quality checks, profiling, STTM, data integration from multiple sources, reusable framework)
• Release planning, change management, training, KT, L2/L3 support
• Apache Iceberg + AWS glue good to have
• 10+ years of relevant experience in Data Engineering and delivery.
• 10+ years of relevant work experience in Big Data Concepts. Worked on cloud implementations.
• Strong experience with SQL, python and Pyspark
• Good understanding of Data ingestion and data processing frameworks
• Good experience in Snowflake, SQL, AWS (glue, EMR, S3, Aurora, RDS, AWS architecture)
• Good aptitude, strong problem-solving abilities, analytical skills, and ability to take ownership as appropriate.
• Should be able to do coding, debugging, performance tuning, and deploying the apps to the Production environment.
• Experience working in Agile Methodology

Requirements:

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer, with a focus on AWS and Snowflake.
• Strong understanding of data warehousing concepts and best practices.
• Excellent communication skills, with the ability to convey complex technical concepts to non-technical stakeholders.
• Experience in the insurance industry, preferably with knowledge of claims and loss processes.
• Proficiency in SQL, Python, and other relevant programming languages.
• Strong problem-solving skills and attention to detail.
• Ability to work independently and as part of a team in a fast-paced environment.

Seniority level

• Seniority level

  Not Applicable

Employment type

• Employment type

  Contract

Job function

• Job function

  Information Technology

• Industries

  Software Development

Referrals increase your chances of interviewing at Smart IT Frame LLC by 2x

Get notified about new Data Engineer jobs in United States.

United States $170,000.00-$720,000.00 6 days ago

San Francisco, CA $126,100.00-$186,800.00 3 days ago

San Francisco, CA $120,000.00-$150,000.00 5 days ago

United States $170,000.00-$720,000.00 1 day ago

Irving, TX $80,000.00-$100,000.00 6 days ago

United States $100,000.00-$120,000.00 8 hours ago

McLean, VA $80,000.00-$100,000.00 6 days ago

United States $120,000.00-$130,000.00 4 days ago

United States $114,000.00-$171,000.00 2 weeks ago

Fairfax, VA $100,000.00-$115,000.00 1 hour ago

Manhattan, NY $110,000.00-$160,000.00 5 days ago

North Carolina, United States $125,000.00-$145,000.00 3 weeks ago

Irvine, CA $130,000.00-$150,000.00 2 days ago

Western, WV $123,000.00-$185,000.00 1 day ago

New York, NY $139,500.00-$271,566.94 1 week ago

Washington DC-Baltimore Area $110,000.00-$120,000.00 1 week ago

United States $135,900.00-$153,000.00 1 day ago

United States $184,500.00-$217,100.00 1 week ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.