Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Deploy WIP] Modified the EKS tutorial #144

Merged
merged 12 commits into from
Feb 17, 2025
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ servingEngineSpec:
pvcStorage: "40Gi"
pvcAccessMode:
- ReadWriteMany
storageClass: "efs_static_storage_class_indicator" #This is to let helm pvc know it should be empty string
storageClass: "" #This is to let helm pvc know it should be empty string
2 changes: 1 addition & 1 deletion helm/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ type: application
# This is the chart version. This version number should be incremented each time you make changes
# to the chart and its templates, including the app version.
# Versions are expected to follow Semantic Versioning (https://semver.org/)
version: 0.0.6
version: 0.0.7

maintainers:
- name: apostac
8 changes: 2 additions & 6 deletions helm/templates/pvc.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,8 @@ spec:
resources:
requests:
storage: {{ $modelSpec.pvcStorage | default "20Gi" }} # Default to 40Gi if not set
{{- if $modelSpec.storageClass }}
{{- if eq $modelSpec.storageClass "efs_static_storage_class_indicator" }}
storageClassName: ""
{{- else }}
storageClassName: "{{ $modelSpec.storageClass }}"
{{- end }}
{{- if hasKey $modelSpec "storageClass" }}
storageClassName: "{{ $modelSpec.storageClass }}"
{{- end }}
{{- if not (empty $modelSpec.pvcMatchLabels) }}
selector:
Expand Down
17 changes: 16 additions & 1 deletion tutorials/deployments/01-AWS-EKS-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This guide walks you through the script that sets up a vLLM production-stack on top of EKS on AWS. It includes how the script configures Elastic File System (EFS) for persistent volume, setting the security groups, and deploying a production AI inference stack using Helm.

## Prerequisites
## Installing Prerequisites

Before running this setup, ensure you have:

Expand All @@ -11,6 +11,20 @@ Before running this setup, ensure you have:
3. Kubectl
4. Helm

## TLDR

To run the service

```bash
bash entry_point.sh YOUR_AWSREGION EXAMPLE_YAML_PATH
```

Clean up the service (not including VPC) with:

```bash
bash clean_up.sh production-stack YOUR_AWSREGION
```

## Step by Step Explanation

### Step 1: Deploy the EKS Cluster
Expand Down Expand Up @@ -202,6 +216,7 @@ servingEngineSpec:
pvcStorage: "40Gi"
pvcAccessMode:
- ReadWriteMany
storageClass: ""
```

### Step 7 Stopping the Helm Cluster
Expand Down