Railway•9mo ago

Trying to confirm my volume implementation

I noticed that when making changes to how data is stored in volumes, then redeploying, my data was lost. From looking at other posts with similar issues, it seems the problem might be my mount path in the Railway settings, but I'm hoping to get confirmation of that. Currently, my mount path is /data This is the code for saving the data:

const DATA_DIR = path.join(__dirname, '..', 'data');

const ensureDirectoryExists = (dirPath) => {
    if (!fs.existsSync(dirPath)) {
        fs.mkdirSync(dirPath, { recursive: true });
    }
};

const appendToCSV = async (filename, data) => {
    ensureDirectoryExists(DATA_DIR);
    const csvPath = path.join(DATA_DIR, `${filename}.csv`);
    
    try {
        await new Promise((resolve, reject) => {
            fs.appendFile(csvPath, `${data}\n`, (err) => {
                if (err) {
                    console.error('Error appending to CSV:', err);
                    reject(err); // Reject the promise on error
                } else {
                    console.log('Data appended to CSV:', csvPath);
                    console.log('Data:', data);
                    resolve(); // Resolve the promise on success
                }
            });
        });
    } catch (error) {
        console.error("Error appending to CSV:", error);
    }
};

const logActionToCSV = async (fid, uniqueId, product, page) => {
    const now = new Date().toISOString();
    const productName = product.title.replace(/,/g, '');
    const data = `${fid},${now},${productName},${page}`;

    appendToCSV(uniqueId, data);
};

const DATA_DIR = path.join(__dirname, '..', 'data');

const ensureDirectoryExists = (dirPath) => {
    if (!fs.existsSync(dirPath)) {
        fs.mkdirSync(dirPath, { recursive: true });
    }
};

const appendToCSV = async (filename, data) => {
    ensureDirectoryExists(DATA_DIR);
    const csvPath = path.join(DATA_DIR, `${filename}.csv`);
    
    try {
        await new Promise((resolve, reject) => {
            fs.appendFile(csvPath, `${data}\n`, (err) => {
                if (err) {
                    console.error('Error appending to CSV:', err);
                    reject(err); // Reject the promise on error
                } else {
                    console.log('Data appended to CSV:', csvPath);
                    console.log('Data:', data);
                    resolve(); // Resolve the promise on success
                }
            });
        });
    } catch (error) {
        console.error("Error appending to CSV:", error);
    }
};

const logActionToCSV = async (fid, uniqueId, product, page) => {
    const now = new Date().toISOString();
    const productName = product.title.replace(/,/g, '');
    const data = `${fid},${now},${productName},${page}`;

    appendToCSV(uniqueId, data);
};

Solution:

surefire way to mount the volume to the path your code uses: 1. print the DATA_DIR variable when on railway. 2. mount volume to the path that was logged. 3. profit...

Jump to solution

22 Replies

Percy•9mo ago

Project ID: b624c17e-a6de-4867-bbb0-89bdb4c2eb04

Fragly•9mo ago

I'd recommend getting the volume path from the RAILWAY_VOLUME_MOUNT_PATH variable more info: https://docs.railway.app/reference/variables#railway-provided-variables

ManuelMaccou | Mosaic•9mo ago

b624c17e-a6de-4867-bbb0-89bdb4c2eb04

Fragly•9mo ago

the volume is mounted on the root of the container, while your app is located in /app I'm guessing you're assuming the volume is located in your app directory best practice is to just use the RAILWAY_VOLUME_MOUNT_PATH variable since it contains the to your volume and will update if the mount is ever changed if you need to have the volume mounted in your app's directory though then you can set the mount to be /app/data

Solution

Brody•9mo ago

surefire way to mount the volume to the path your code uses: 1. print the DATA_DIR variable when on railway. 2. mount volume to the path that was logged. 3. profit

Fragly•9mo ago

what's wrong with using RAILWAY_VOLUME_MOUNT_PATH ?

Brody•9mo ago

its platform specific, and whats going to happen when railway supports multiple volumes?

Fragly•9mo ago

fair enough

ManuelMaccou | Mosaic•9mo ago

I print the path including DATA_DIR here:

const csvPath = path.join(DATA_DIR, `${filename}.csv`);
console.log('Data appended to CSV:', csvPath);

const csvPath = path.join(DATA_DIR, `${filename}.csv`);
console.log('Data appended to CSV:', csvPath);

This is the output:

Data appended to CSV: /app/backend/data/0c596704-90c8-459d-b520-476a314a82c0.csv

Data appended to CSV: /app/backend/data/0c596704-90c8-459d-b520-476a314a82c0.csv

So mounting to /data seems right to me? But I could definitely be mistaken btw I'm able to see that saved data, but when I rebuild/redeploy, its lost

Brody•9mo ago

fragly's assumption was correct when he said "I'm guessing you're assuming the volume is located in your app directory" the volume is mounted at the container's root. you are mounting to /data but saving your files into /app/backend/data see how it doesnt quite line up, mount the volume to /app/backend/data since thats the full path where you are saving the files

ManuelMaccou | Mosaic•9mo ago

I see I see. Will the old data be there? Or is more like going forward, that's where it will be. Not toally sure what "mounting" really means so probably a beginner question

Brody•9mo ago

mounting is just you specifying the path where you want the volume to be attached to in the container, relative to the root of the container. since you have been saving files into the ephemeral storage, only files saved into the volume will persist going forward

ManuelMaccou | Mosaic•9mo ago

Got it. So I dont need to change my code, only the mount path, and then all is good?

Brody•9mo ago

from what im seeing, yes that should be correct

ManuelMaccou | Mosaic•9mo ago

Sweet. Thanks man

Brody•9mo ago

let me know if it works out! have your code save some of the csv files and then redeploy to check if the files are still there

ManuelMaccou | Mosaic•9mo ago

I'm using it for analytics and a lot of people are viewing it now so I'll wait till tonight so I can first capture what's recorded in ephemeral storage, then make the change I'll you know tho

Brody•9mo ago

does your app provide a way to export the csv files?

ManuelMaccou | Mosaic•9mo ago

yea

Brody•9mo ago

okay cool

ManuelMaccou | Mosaic•9mo ago

Ok looks like that worked. Thanks for all your help

Brody•9mo ago

no problem!

Gaming

Programming

Trying to confirm my volume implementation