Data Engineer
Akaasa Technologies
Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.
Job summary
Akaasa Technologies is seeking a Senior Data Engineer to design, develop, and optimize scalable data platforms powering analytics, ML, and AI-driven solutions. You will work with Databricks, Spark, Delta Lake, Snowflake, Kafka, and AWS, contributing to GenAI, LLMs, RAG, and AI-assisted engineering.
The role emphasizes data quality, governance, and enterprise-scale data engineering across batch and streaming pipelines, with cloud-native, cost-efficient architectures.
Qualifications
- Bachelor's or Master's degree in Computer Engineering, Computer Science, Information Systems, or a related field.
- 8+ years of experience in software engineering, data engineering, or big data platform development.
- Strong experience designing and implementing enterprise-scale data pipelines.
- Hands-on expertise with Python, PySpark, Spark SQL, SQL, Kafka, Databricks, Delta Lake, Snowflake, Hive.
Responsibilities
- Design, build, and maintain scalable batch and real-time data pipelines.
- Develop and optimize ingestion, transformation, and processing frameworks for diverse datasets.
- Implement modern Lakehouse architectures using Databricks, Delta Lake, and Bronze–Silver–Gold patterns.
- Build data solutions for analytics, reporting, and machine learning.
- Ensure data quality, governance, lineage, security, and compliance.
Skills
Education
Tools
Job description
Job Location - NY, NY
Rate: $85/hr Contract
Interview: In-person
We are seeking an experienced Senior Data Engineer to design, develop, and optimize scalable data platforms that power enterprise analytics, machine learning, and AI-driven solutions. The ideal candidate will bring deep expertise in modern data engineering practices, cloud-native architectures, Lakehouse platforms, and distributed data processing technologies. This role will play a critical part in building reliable, high-performance data ecosystems leveraging Databricks, Spark, Delta Lake, Snowflake, Kafka, and AWS, while also contributing to the adoption of Generative AI, Large Language Models (LLMs), Retrieval Augmented Generation (RAG), and AI-assisted engineering solutions.
- Design, build, and maintain scalable batch and real-time data pipelines.
- Develop and optimize data ingestion, transformation, and processing frameworks for structured, semi-structured, and unstructured datasets.
- Implement modern Lakehouse architectures utilizing Databricks, Delta Lake, and Medallion (Bronze, Silver, Gold) design patterns.
- Build data solutions that support enterprise analytics, reporting, and machine learning initiatives.
- Ensure data quality, governance, lineage, security, and compliance across data ecosystems.
- Develop distributed data processing applications using PySpark and Spark SQL.
- Build and maintain streaming pipelines using Kafka, Spark Structured Streaming, and AWS Kinesis.
- Design fault-tolerant, scalable systems capable of processing large data volumes with low latency.
- Optimize workload performance through partitioning strategies, clustering, caching, and query tuning.
- Architect and implement cloud-based data solutions on AWS.
- Utilize AWS services including S3, EMR, EC2, Athena, Redshift, RDS, Lambda, IAM, SNS, and SQS.
- Design data storage and processing strategies that maximize reliability while minimizing operational costs.
- Support migration initiatives from traditional Hadoop and EMR environments to modern cloud-native platforms.
- Develop orchestration and scheduling frameworks using Airflow and Databricks Workflows.
- Build CI/CD pipelines and automation frameworks for deployment, monitoring, and data platform operations.
- Collaborate closely with architects, analysts, data scientists, and business stakeholders to deliver enterprise-grade solutions.
- Implement Generative AI-powered solutions for engineering productivity and operational excellence.
- Develop applications leveraging Large Language Models (LLMs), Retrieval Augmented Generation (RAG), Vector Databases, and Model Context Protocol (MCP).
- Build AI-assisted documentation, developer productivity tooling, and intelligent platform capabilities.
- Evaluate emerging AI technologies and identify opportunities for adoption within data engineering processes.
- Bachelor's or Master's degree in Computer Engineering, Computer Science, Information Systems, or a related field.
- 8+ years of experience in software engineering, data engineering, or big data platform development.
- Strong experience designing and implementing enterprise-scale data pipelines.
- Hands-on expertise with:
- Python
- PySpark
- Spark SQL
- SQL
- Kafka
- Databricks
- Delta Lake
- Snowflake
- Hive
- Experience building data solutions on AWS cloud platforms.
- Strong understanding of distributed computing, data modeling, and large-scale data processing.
- Experience with Git-based development workflows and CI/CD practices.
- Excellent analytical, troubleshooting, and problem-solving skills.
- Experience with real-time streaming architectures and event-driven systems.
- Knowledge of data governance, metadata management, and data quality frameworks.
- Experience with generative AI technologies including:
- LLMs
- RAG
- Vector Databases
- AI Agents
- MCP integrations
- Experience developing developer productivity tools and AI-assisted engineering workflows.
- Exposure to enterprise supply chain, retail, healthcare, or manufacturing data domains.
- AWS certifications are highly preferred.
Programming Languages
- Python
- Java
- SQL
- Shell Scripting
- C/C++
Big Data & Data Engineering
- PySpark
- Spark SQL
- Hive
- Databricks
- Delta Lake
- Snowflake
- Kafka
- HBase
- Sqoop
Workflow & Orchestration
- Apache Airflow
- Databricks Workflows
- Oozie
Cloud Technologies
- AWS S3
- EMR
- EC2
- Athena
- Redshift
- RDS
- IAM
- Lambda
- SNS
- SQS
AI & Modern Engineering
- Generative AI
- Large Language Models (LLMs)
- Retrieval Augmented Generation (RAG)
- Agentic AI Systems
- Model Context Protocol (MCP)
- Vector Databases
Visualization & Tools
- Tableau
- Git
- Docker
- Splunk
- IntelliJ IDEA
- PyCharm
- Cursor
Preferred Certifications
- AWS Certified Solutions Architect Associate
- AWS Certified Cloud Practitioner
- Databricks Certifications (preferred)
- Deliver highly scalable and reliable data pipelines.
- Improve platform performance, efficiency, and cost optimization.
- Enable enterprise-wide analytics and AI initiatives through trusted data products.
- Drive modernization of data platforms and adoption of cloud-native architectures.
- Leverage AI technologies to enhance engineering efficiency, automation, and innovation.
A senior-level data engineer with extensive experience in Databricks, Spark, AWS, Kafka, Snowflake, and Lakehouse architectures, who is equally passionate about modern AI technologies and building intelligent data platforms for the future.
- Apache Spark
- Data Bricks
- - Hands-on experience with Azure AI, Microsoft Copilot Studio, Gemini Enterprise, Spark AI development platforms, frameworks, and tools
- Programing in python
- Snowflake Data Warehouse
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
