Tasks:
- * Accelerate long-context and high-concurrency inference
- * Implement KV cache compression and quantization
- * Implement layer skipping and speculative decoding
- * Optimize context compression and special tokens
- * Optimize inference frameworks for throughput and TTFT
- * Optimize LLM inference architecture
- * Optimize memory bandwidth and GPU memory usage
- * Tune compute kernels across hardware platforms
Perks/Benefits:
Skills/Tech stack required:
[Attention Mechanisms] [C++] [Cache optimization] [Context compression] [CUDA] [FlashAttention] [FlashDecoding] [GPU Kernel] [GPU kernel optimization] [Inference Optimization] [Kernel optimization] [KV cache] [KV cache optimization] [KV Cache Quantization] [Layer skipping] [LLM Inference] [LLM Inference Optimization] [Parallel Computing] [Python] [PyTorch] [Special token optimization] [Speculative decoding] [TensorRT-LLM] [Token optimization] [Transformer Architecture] [VLLM]
Educational requirements:
[Master's Degree]
Role(s):
[Engineer] [Inference Optimization Engineer] [LLM Inference Optimization Engineer] [Optimization Engineer]