Welcome to the tutorials for vLLM Production Stack! This series of tutorials is designed to guide you through setting up and utilizing the vLLM production stack efficiently. Whether you're new to Kubernetes, Helm, or vLLM, or looking to deepen your understanding of advanced features like multi-model management and KV cache offloading, this series has you covered.
-
Install Kubernetes Environment Learn how to set up a Kubernetes environment as the foundation for running vLLM Production Stack.
-
Minimal Helm Installation A step-by-step guide for deploying vLLM Production Stack using Helm with minimal configuration.
-
Basic vLLM Configuration Learn how to customize vLLM options when using vLLM Production Stack.
-
Load Model from Persistent Volume Discover how to load models from a persistent volume to ensure efficient resource usage.
-
Launch Multiple Models Learn how to deploy and manage multiple models simultaneously in your vLLM environment.
-
Offload KV Cache Understand how to offload the KV cache to CPU to improve the performance in production use cases.
These tutorials are designed to be followed sequentially for beginners, but you can also jump to a specific tutorial based on your needs. Each tutorial includes:
- Prerequisites
- Detailed steps
- Commands to execute
- Expected outputs
- Explanations to enhance your understanding
If you encounter any issues or have suggestions for improving these tutorials, feel free to contribute by opening a pull request or an issue on our GitHub repository.
Happy learning!