Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: opentelemetry POC vLLM span cannot be concatenated with HTTP spans. #14330

Open
1 task done
RichardoMrMu opened this issue Mar 6, 2025 · 2 comments
Open
1 task done
Labels
bug Something isn't working

Comments

@RichardoMrMu
Copy link

Your current environment

The output of `python collect_env.py`
Your output of `python collect_env.py` here

Collecting environment information...
PyTorch version: 2.5.1
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.5 LTS (aarch64)
GCC version: (Ubuntu 12.3.0-1ubuntu1~22.04) 12.3.0
Clang version: Could not collect
CMake version: version 3.31.4
Libc version: glibc-2.35

Python version: 3.10.12 (main, Jan 17 2025, 14:35:34) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-6.12.10-orbstack-00297-gf8f6e015b993-aarch64-with-glibc2.35
Is CUDA available: False
CUDA runtime version: No CUDA
CUDA_MODULE_LOADING set to: N/A
GPU models and configuration: No CUDA
Nvidia driver version: No CUDA
cuDNN version: No CUDA
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 12
On-line CPU(s) list: 0-11
Vendor ID: Apple
Model: 0
Thread(s) per core: 1
Core(s) per cluster: 12
Socket(s): -
Cluster(s): 1
Stepping: 0x0
CPU max MHz: 2000.0000
CPU min MHz: 2000.0000
BogoMIPS: 48.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 asimddp sha512 asimdfhm dit uscat ilrcpc flagm sb dcpodp flagm2 frint bf16 ecv
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Not affected
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] numpy==1.26.4
[pip3] pyzmq==26.2.1
[pip3] torch==2.5.1
[pip3] torchaudio==2.5.1
[pip3] torchvision==0.20.1
[pip3] transformers==4.48.3
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.7.3.dev134+g40932d7a
vLLM Build Flags:
CUDA Archs: Not Set; ROCm: Disabled; Neuron: Disabled
GPU Topology:
Could not collect

NCCL_CUMEM_ENABLE=0
TORCHINDUCTOR_COMPILE_THREADS=1

🐛 Describe the bug

I follow the https://github.com/vllm-project/vllm/blob/main/examples/online_serving/opentelemetry/Otel.md. But face this

Image

Image

HTTP span has different traceid with vllm llm_request. Why this happend?

Before submitting a new issue...

  • Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
@RichardoMrMu RichardoMrMu added the bug Something isn't working label Mar 6, 2025
@RichardoMrMu
Copy link
Author

and when i python3 dummy_client.py. it generated not only one span for llm_request.like this

Image
it is so different with https://github.com/vllm-project/vllm/blob/main/examples/online_serving/opentelemetry/Otel.md.

@RichardoMrMu
Copy link
Author

by the way, when i use vllm serve facebook/opt-125m, the vllm main process will start a subprocess MQLLMEngine to manager the model. The MQLLMEngine process communicates with the main process (MQLLMEngineClient) by zeroMQ. Unless we instrument zeroMQ's client&server method, traceId and spanId can not pass through process.
like this. The link will be broken between zeroMQ client and server.
Image

Image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant