R
RunPod•6mo ago
Daan

Problem with venv

Hi, This is my handler.py file: 
#!/usr/bin/env python3
""" Example handler file. """

import runpod
import subprocess

# If your handler runs inference on a model, load the model here.
# You will want models to be loaded into memory before starting serverless.


def handler(job):
    
    return run_shell_script('./entrypoint.sh') 

def run_shell_script(script_path):
    try:
        result = subprocess.run([script_path], check=True, text=True, capture_output=True)
        return result.stdout
    except subprocess.CalledProcessError as e:
        return e.stderr


runpod.serverless.start({"handler": handler})
#!/usr/bin/env python3
""" Example handler file. """

import runpod
import subprocess

# If your handler runs inference on a model, load the model here.
# You will want models to be loaded into memory before starting serverless.


def handler(job):
    
    return run_shell_script('./entrypoint.sh') 

def run_shell_script(script_path):
    try:
        result = subprocess.run([script_path], check=True, text=True, capture_output=True)
        return result.stdout
    except subprocess.CalledProcessError as e:
        return e.stderr


runpod.serverless.start({"handler": handler})
when entrypoint.sh has this code:
export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls
export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls
it works fine and gives 
{
  "delayTime": 3917,
  "executionTime": 1082,
  "id": "3d4277ce-31d8-442c-830b-8817be24ecae-e1",
  "output": "Activate.ps1\naccelerate\naccelerate-config\naccelerate-estimate-memory\naccelerate-launch\nactivate\nactivate.csh\nactivate.fish\nconvert-caffe2-to-onnx\nconvert-onnx-to-caffe2\ndistro\nf2py\nhttpx\nhuggingface-cli\nisympy\nnormalizer\nopenai\npip\npip3\npip3.11\npython\npython3\npython3.11\nspacy\ntorchrun\ntqdm\ntransformers-cli\nweasel\n",
  "status": "COMPLETED"
}
{
  "delayTime": 3917,
  "executionTime": 1082,
  "id": "3d4277ce-31d8-442c-830b-8817be24ecae-e1",
  "output": "Activate.ps1\naccelerate\naccelerate-config\naccelerate-estimate-memory\naccelerate-launch\nactivate\nactivate.csh\nactivate.fish\nconvert-caffe2-to-onnx\nconvert-onnx-to-caffe2\ndistro\nf2py\nhttpx\nhuggingface-cli\nisympy\nnormalizer\nopenai\npip\npip3\npip3.11\npython\npython3\npython3.11\nspacy\ntorchrun\ntqdm\ntransformers-cli\nweasel\n",
  "status": "COMPLETED"
}
Solution:
Finally works, thanks for all the help! The issue was the base image that was different.
Jump to solution
18 Replies
Daan
Daan•6mo ago
If it has this code: 
#!/bin/bash

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin
pip list
#!/bin/bash

export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin
pip list
it gives 
{
  "delayTime": 6319,
  "executionTime": 1100,
  "id": "6f292b30-3562-463d-8f66-26a5355d9cf4-e1",
  "output": "./entrypoint.sh: /workspace/.venv/bin/pip: /workspace/.venv/bin/python: bad interpreter: No such file or directory\n",
  "status": "COMPLETED"
}
{
  "delayTime": 6319,
  "executionTime": 1100,
  "id": "6f292b30-3562-463d-8f66-26a5355d9cf4-e1",
  "output": "./entrypoint.sh: /workspace/.venv/bin/pip: /workspace/.venv/bin/python: bad interpreter: No such file or directory\n",
  "status": "COMPLETED"
}
Any idea where it goes wrong? I understood that runpod already chose the mount path as "runpod-volume", that is why I did "rm -rf /workspace && ln -s /runpod-volume /workspace". Also, when executing pip list in a pod, ade with the same network volume, no errors.
ashleyk
ashleyk•6mo ago
Does your docker image for your serverless worker have the same Python version as the one you used to create the venv?
Daan
Daan•6mo ago
I think it is, this is the docker image used to create the venv (i made a pod with this templatte and manually created the venv): 
# Gebruik een basisimage met Python 3.11
FROM python:3.11

# Stel de werkdirectory in op de root om te beginnen
WORKDIR /

# Kopieer de entrypoint  in de container
COPY ./entrypoint.sh /entrypoint.sh

# Maak de scripts uitvoerbaar
RUN chmod +x /entrypoint.sh

# Installeer vim
#RUN apt-get update && apt-get install -y vim

# Stel het entrypoint script in als het entrypoint van de container
ENTRYPOINT ["/entrypoint.sh"]
# Gebruik een basisimage met Python 3.11
FROM python:3.11

# Stel de werkdirectory in op de root om te beginnen
WORKDIR /

# Kopieer de entrypoint  in de container
COPY ./entrypoint.sh /entrypoint.sh

# Maak de scripts uitvoerbaar
RUN chmod +x /entrypoint.sh

# Installeer vim
#RUN apt-get update && apt-get install -y vim

# Stel het entrypoint script in als het entrypoint van de container
ENTRYPOINT ["/entrypoint.sh"]
and this is the one to make serverless endpoint: 
# Base image -> https://github.com/runpod/containers/blob/main/official-templates/base/Dockerfile
# DockerHub -> https://hub.docker.com/r/runpod/base/tags
FROM runpod/base:0.4.0-cuda11.8.0

# The base image comes with many system dependencies pre-installed to help you get started quickly.
# Please refer to the base image's Dockerfile for more information before adding additional dependencies.
# IMPORTANT: The base image overrides the default huggingface cache location.


# --- Optional: System dependencies ---
# COPY builder/setup.sh /setup.sh
# RUN /bin/bash /setup.sh && \
#     rm /setup.sh


# Python dependencies
COPY builder/requirements.txt /requirements.txt
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \
    rm /requirements.txt

# NOTE: The base image comes with multiple Python versions pre-installed.
#       It is reccommended to specify the version of Python when running your code.


# Add src files (Worker Template)
ADD src .

CMD python3.11 -u /handler.py
# Base image -> https://github.com/runpod/containers/blob/main/official-templates/base/Dockerfile
# DockerHub -> https://hub.docker.com/r/runpod/base/tags
FROM runpod/base:0.4.0-cuda11.8.0

# The base image comes with many system dependencies pre-installed to help you get started quickly.
# Please refer to the base image's Dockerfile for more information before adding additional dependencies.
# IMPORTANT: The base image overrides the default huggingface cache location.


# --- Optional: System dependencies ---
# COPY builder/setup.sh /setup.sh
# RUN /bin/bash /setup.sh && \
#     rm /setup.sh


# Python dependencies
COPY builder/requirements.txt /requirements.txt
RUN python3.11 -m pip install --upgrade pip && \
    python3.11 -m pip install --upgrade -r /requirements.txt --no-cache-dir && \
    rm /requirements.txt

# NOTE: The base image comes with multiple Python versions pre-installed.
#       It is reccommended to specify the version of Python when running your code.


# Add src files (Worker Template)
ADD src .

CMD python3.11 -u /handler.py
ashleyk
ashleyk•6mo ago
Where are you from @Daan ? Netherlands?
Daan
Daan•6mo ago
I live in Belgium 🙂
ashleyk
ashleyk•6mo ago
Ah, thats why I can read it, I can't really understand Dutch 😆 Is the Docker image public? Or is it private?
Daan
Daan•6mo ago
it is private, but I can make it public, one second daana2003/orca-gpu-runner:no_vim_10 daana2003/orca-gpu-runner:serverless_60
ashleyk
ashleyk•6mo ago
Is the first one for GPU cloud and the second one for serverless?
Daan
Daan•6mo ago
Yes, indeed Made the first one to manually make a .venv in worspace directory
ashleyk
ashleyk•6mo ago
I am getting this error with the first image: 
2024-01-02T15:16:53.747731452Z Wachten op app.sh om te bestaan...
2024-01-02T15:16:53.747750310Z /entrypoint.sh: line 14: .venv/bin/activate: No such file or directory
2024-01-02T15:16:53.747731452Z Wachten op app.sh om te bestaan...
2024-01-02T15:16:53.747750310Z /entrypoint.sh: line 14: .venv/bin/activate: No such file or directory
Daan
Daan•6mo ago
strange, I didn't had that error. I will retry it
ashleyk
ashleyk•6mo ago
Maybe your .venv folder already existed in /workspace
Daan
Daan•6mo ago
oh, no it is because you do not use my network volume. I use a network volume that is mounted in /workspace
ashleyk
ashleyk•6mo ago
Yeah seems like it expects certain things to already exist on the network volume
Daan
Daan•6mo ago
Yes indeed, so I made manually a venv and app.sh file. My script works fine then if I execute it in the GPU. But in the serverless endpoint, with the other image it goes wrong. I really can't understand why 
export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls
export TRANSFORMERS_CACHE=/workspace



rm -rf /workspace && ln -s /runpod-volume /workspace
  
source /workspace/.venv/bin/activate


cd workspace/.venv/bin

ls
works, but then if i change ls by "pip list" it gives the error that pip is not found...
ashleyk
ashleyk•6mo ago
Yeah there seems to be an issue with the venv, which sometimes happens when the python versions are not the same
Daan
Daan•6mo ago
I can try to add "FROM python:3.11" to the dockerfile for teh sereverless endpoint,and delete "RUN python3.11 -m pip install --upgrade pip", then it certainly would be the same version, right? Although I would exspect both images use python3.11 with the provided code
Solution
Daan
Daan•6mo ago
Finally works, thanks for all the help! The issue was the base image that was different.
Want results from more Discord servers?
Add your server
More Posts
refer to the current running pod's id from environment variableis there any way to refer to the current running id of pods from env?Cannot connect to jupyterlab/web terminalI cant access them through browsersExperiencing huge execution time on ServerlessWe are currently experiencing huge execution time and jobs actually not being processed on ServerlesHow do I upload a model to GPU-Cloud Stable Diffusion?I want to upload a model (Anything V3) to my RunPod Stable Diffusion. Can you tell me the simplest wUnable to SSHI could SSH in with ssh y8z10r63w2hkzp-64410d61@ssh.runpod.io -i ~/.ssh/id_ed25519 but not when usMount gpu in containerHello, I'd like to use runpod to perform blender render tasks. I wish to use this docker image: http24 GB VRAM is not enough for simple kohya_ss LORA generation.How come 24GB VRAM is not enough for generating simple Lora in kohya_ss? I've tried running it witTrouble with SSH via PuTTYI want to use putty for X11 Forwarding. I asked ChatGPT for help figuring this out, showing it the s"Initializing" State DurationHi everyone, I've recently started exploring serverless but encountered an issue. When I try to depIssue with Dependencies Not Being Found in Serverless EndpointI am encountering an issue with a network volume I created: First,I created a network volume and us