Data Engineer

A Data Engineer role focused on designing, building, and maintaining scalable data management systems and pipelines to support AI/ML applications, with a strong emphasis on cloud platforms and data quality.

Location: Remote (India)

Responsibilities

• Design, construct, install, test, and maintain highly scalable data management systems and robust data pipelines.
• Ensure data quality, reliability, and accessibility for AI/ML engineers and LLM applications.
• Leverage cloud platforms and modern data engineering practices, including workflow orchestration.
• Design, build, and optimize scalable ETL/ELT data pipelines using Python and cloud-native tools (AWS, Azure, GCP).
• Develop data models and schemas optimized for analytical and AI/ML workloads.
• Implement data quality checks and monitoring frameworks.
• Manage and administer data warehouses, data lakes, and databases (SQL/NoSQL).
• Implement and manage workflow orchestration tools (Airflow, Prefect, Dagster).
• Collaborate with AI/ML Engineers and LLM Engineers to understand data requirements.
• Ensure data security and compliance standards.
• Optimize data storage and processing costs on Hyperscaler platforms.
• Write efficient and maintainable Python code for data processing.
• Troubleshoot and resolve data-related issues.

Requirements

• Proficiency in Python for data manipulation and pipeline development (e.g., Pandas, PySpark).
• Expertise in SQL and experience with relational and NoSQL databases.
• Hands-on experience with cloud-based data services on at least one Hyperscaler (AWS, Azure, GCP).
• Experience building and managing data pipelines and ETL/ELT processes.
• Familiarity with data warehousing concepts and data modeling.
• Understanding of data quality principles.
• Ability to work independently and take ownership of data infrastructure components.

Benefits

• Please review the Benefits & Perks section in our Team Member Handbook for comprehensive information.