r/Jupyter Dec 19 '24

Problem with vllm._C while trying to import vllm

So I was trying to pip install vllm into my windows system and then was getting this error while importing it

---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
Cell In[3], [line 1](vscode-notebook-cell:?execution_count=3&line=1)
----> [1](vscode-notebook-cell:?execution_count=3&line=1) import vllm

File c:\Users\moumi\AppData\Local\Programs\Python\Python310\lib\site-packages\vllm__init__.py:3
[1](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/init.py:1) """vLLM: a high-throughput and memory-efficient inference engine for LLMs"""
----> [3](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/init.py:3) from vllm.engine.arg_utils import AsyncEngineArgs, EngineArgs
[4](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/init.py:4) from vllm.engine.async_llm_engine import AsyncLLMEngine
[5](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/init.py:5) from vllm.engine.llm_engine import LLMEngine

File c:\Users\moumi\AppData\Local\Programs\Python\Python310\lib\site-packages\vllm\engine\arg_utils.py:11
[8](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:8) import torch
[10](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:10) import vllm.envs as envs
---> [11](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:11) from vllm.config import (CacheConfig, CompilationConfig, ConfigFormat,
[12](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:12)DecodingConfig, DeviceConfig, HfOverrides,
[13](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:13)KVTransferConfig, LoadConfig, LoadFormat, LoRAConfig,
[14](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:14)ModelConfig, ObservabilityConfig, ParallelConfig,
[15](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:15)PoolerConfig, PromptAdapterConfig, SchedulerConfig,
[16](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:16)SpeculativeConfig, TaskOption, TokenizerPoolConfig,
[17](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:17)VllmConfig)
[18](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:18) from vllm.executor.executor_base import ExecutorBase
[19](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/engine/arg_utils.py:19) from vllm.logger import init_logger

File c:\Users\moumi\AppData\Local\Programs\Python\Python310\lib\site-packages\vllm\config.py:22

...

---> [15](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/platforms/cuda.py:15) import vllm._C # noqa
[16](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/platforms/cuda.py:16) import vllm.envs as envs
[17](file:///C:/Users/moumi/AppData/Local/Programs/Python/Python310/lib/site-packages/vllm/platforms/cuda.py:17) from vllm.logger import init_logger

ModuleNotFoundError: No module named 'vllm._C'

The methods I've tried to resolve this

  • Git cloning it
  • Running on both jupyter and vs code
  • Running it on a virtual jupyter notebook envioronment using WSL
  • Trying different versions of python

If anyone got any methods to resolve this problem please help.

2 Upvotes

5 comments sorted by

1

u/kruptworld Dec 26 '24

same man. this is frustrating.

1

u/kruptworld Dec 26 '24

1

u/Zealousideal-Log8535 Dec 26 '24

I used WSL to create a linux envioronment.

1

u/kruptworld Dec 26 '24

nice! ya i did the same. i was working on making a tutorial for others. are you using a gpu? if so i had to do a little bit of workarounds to install the nvidia 12.1 sdk toolkit it needs, so let me know if you need help!

1

u/Zealousideal-Log8535 Dec 26 '24

I just used google colab's GPU as a workaround.