Llm Training

sPipe: Hybrid GPU and CPU Pipeline for Training LLMs Under Memory Pressure

How sPipe combines GPU and CPU resources in a hybrid pipeline to train large language models when GPU memory is limited. Architecture, offloading strategies, and practical benchmarks.