Experience with: Python, Airflow, Snowflake
- Python: Strong coding skills; capable of developing robust, scalable applications and libraries.
- SQL: Ability to write complex queries, design/optimize schemas, and ensure high-performance data retrieval.
- Performance Optimization: Track record of profiling and tuning code (Python/SQL) for efficiency.
- AWS (Primary): Hands-on experience with key services such as S3 (data storage), Lambda (serverless execution), Step Functions (workflow automation), ECS/EKS (container orchestration), etc.
- Other Cloud Platforms: Familiarity with Azure, Google Cloud, or emerging services like AWS Bedrock.
- Pipeline Design & Migration: Capable of designing and scaling ML/data pipelines in the cloud, migrating on-premise/local workflows to AWS or other cloud environments with a focus on security and scalability.
Data Engineering & Workflow Orchestration:
- Workflow Orchestration: Familiarity with tools like Apache Airflow.
- Pandas & Local Data Processing: Comfort with local batch processing and data wrangling.
- Data Lakes & Warehouses: Experience with Snowflake for optimized data storage and retrieval.
Additional Considerations:
- ETL/ELT Pipeline Experience: Experience designing and maintaining data ingestion/processing pipelines; comfortable with tasks like data cleaning, transformation, and integration.
- Security & Compliance: Understanding of best practices for data security, governance, and compliance in cloud environments (IAM policies, data encryption, etc.).
- Collaboration & Communication: Ability to work cross-functionally with product teams, communicate findings, and translate data insights into actionable recommendations.
Nice to Have:
- RAG (Retrieval-Augmented Generation) & Vector Databases: Demonstrated familiarity with retrieval-augmented generation workflows, vector database solutions (e.g., Pinecone, Faiss, Milvus), and embedding models.
- LLMs & Prompt Engineering: Experience building or fine-tuning Large Language Models, crafting effective prompts.
Seniority level
Not Applicable
Employment type
Contract
Job function
Information Technology
Industries
Staffing and Recruiting
Salary Range: $100,000.00 - $720,000.00