how can I check the logs to see if my request uses the lora model
I deployed the qwen2-7B model using serverless and want to load the adapter checkpoint.
My environment variable configuration is shown in the figure below, where
LORA_MODULES={"name": "cn_writer", "path": "sinmu/cn-writer-qwen-7B-25w", "base_model_name": "Qwen/Qwen2-7B"}
My questions are as follows
1. After serverless is deployed, how can I check the logs to see if my request uses the lora model
2. If the logs cannot be viewed, what should be done to verify
Note: Please ignore the method of checking model performance to verify whether the lora model is used
3 Replies
Unknown User•8mo ago
Message Not Public
Sign In & Join Server To View
Thanks for your suggestion
The serverless package is runpod packaged and not open to the public. So I can't modify the log printing logic
The model list returned by model list contains lora model
Requesting cn_writer can get the result smoothly.
It's just that the log does not record the use of the model of the request, which makes me unsure whether the lora part is being used.
When deployed in pod mode, each request will be printed and marked with the lora used
Unknown User•8mo ago
Message Not Public
Sign In & Join Server To View