solanotodeschini Posts - Answer Overflow

solanotodeschini

•Created by solanotodeschini on 4/2/2025 in #⛅｜pods-clusters

Axolotl Fine Tune Error (flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol)

Hi! I was using axolotl image for fine tuning successfully but now I'm getting this error: flash_attn_2_cuda.cpython-311-x86_64-linux-gnu.so: undefined symbol Everything was working normally until yesterday. I'm following the steps in the fine tuning tutorial: https://docs.runpod.io/tutorials/pods/fine-tune-llm-axolotl#using-a-hugging-face-dataset

1 replies

RRunPod

•Created by solanotodeschini on 12/19/2024 in #⚡｜serverless

Worker not executing job

Hi! I'm trying to deploy a simple serverless worker for PDF processing that requires GPUs. I've followed the docs and was able to build it. But, when I send a request, it's forever stuck and workers exit. I'm having a hard time debugging it since it takes more than half an hour to build and upload the image. Dockerfile:

FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04

ENV DEBIAN_FRONTEND=noninteractive
WORKDIR /

# Install system dependencies, Python 3.10, and set it as default
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        software-properties-common \
        wget \
        git \
        libgl1 \
        libglib2.0-0 && \
    add-apt-repository ppa:deadsnakes/ppa && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
        python3.10 \
        python3.10-venv \
        python3.10-distutils \
        python3-pip && \
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1 && \
    rm -rf /var/lib/apt/lists/*

...

COPY src .

# Set the entry point to activate the virtual environment and run the command line tool
CMD ["/bin/bash", "-c", "source /opt/mineru_venv/bin/activate && python3 -u /runpod_handler.py"]

FROM nvidia/cuda:12.4.1-runtime-ubuntu22.04

ENV DEBIAN_FRONTEND=noninteractive
WORKDIR /

# Install system dependencies, Python 3.10, and set it as default
RUN apt-get update && \
    apt-get install -y --no-install-recommends \
        software-properties-common \
        wget \
        git \
        libgl1 \
        libglib2.0-0 && \
    add-apt-repository ppa:deadsnakes/ppa && \
    apt-get update && \
    apt-get install -y --no-install-recommends \
        python3.10 \
        python3.10-venv \
        python3.10-distutils \
        python3-pip && \
    update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1 && \
    rm -rf /var/lib/apt/lists/*

...

COPY src .

# Set the entry point to activate the virtual environment and run the command line tool
CMD ["/bin/bash", "-c", "source /opt/mineru_venv/bin/activate && python3 -u /runpod_handler.py"]

3 replies

Gaming

Programming