Tasks:
- * Build scalable data-to-training pipelines
- * Collaborate with data and algorithm teams
- * Design distributed training frameworks
- * Develop training infrastructure
- * Enable rapid model validation
- * Ensure training pipeline stability
- * Implement data, tensor, pipeline, and sequence parallelism
- * Implement mixed precision training
- * Improve training throughput and resource utilization
- * Optimize kernels and data I/O
- * Optimize memory and communication
- * Optimize multimodal model training performance
- * Optimize thousand-GPU training workloads
- * Profile large-scale model training
- * Research advances in model training systems
Perks/Benefits:
Skills/Tech stack required:
[C++] [Communication optimization] [CUDA] [Data parallelism] [Deep learning] [DeepSpeed] [Distributed Training] [FSDP] [I/O] [I/O Optimization] [Kernel optimization] [Large-scale] [Large-scale model] [Large-scale Model Training] [Linux] [LLM] [Machine Learning] [Megatron-LM] [Memory Optimization] [Mixed Precision] [Model Training] [Multimodal model] [Multimodal model training] [NCCL] [Network Analysis] [Performance Profiling] [Pipeline parallelism] [Python] [PyTorch] [Ray] [Sequence parallelism] [Tensor Parallelism] [VLA] [VLM]
Educational requirements:
N/A
Role(s):
[Engineer] [Infrastructure Engineer] [Large-Scale Training Infrastructure Engineer] [Training Infrastructure Engineer]