Error requiring "flash_attn"
I'm trying to run MiniCPM-V which according to docs supports VLLM (https://github.com/OpenBMB/MiniCPM-V/tree/main?tab=readme-ov-file#inference-with-vllm), but on run I'm getting
Any help on how to overcome this error? I was trying to use the webUI to configure serverless.
ImportError: This modeling file requires the following packages that were not found in your environment: flash_attn. Run pip install flash_attnAny help on how to overcome this error? I was trying to use the webUI to configure serverless.
GitHub
MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone - OpenBMB/MiniCPM-V
Solution
It looks like you need flash_attn python module. You need to uncomment the flash_attn line in requirements.txt. It currently looks like this:
It needs to look like this:
After making that change you will need to rebuild the image.
It needs to look like this:
After making that change you will need to rebuild the image.
