How can I use Multiprocessing in Serverless ?

Hi I am trying to do something somewhat simple

def run(self):
        print("TRAINER: Starting training")
        train = Train()
        trainer = self.ctx.Process(target=train.train, args=(self.config.config_path,))
        trainer.start()
        print("TRAINER: Starting watcher")
        self.watch()
        trainer.join()

def run(self):
        print("TRAINER: Starting training")
        train = Train()
        trainer = self.ctx.Process(target=train.train, args=(self.config.config_path,))
        trainer.start()
        print("TRAINER: Starting watcher")
        self.watch()
        trainer.join()

I have a training script in a training loop, and I want a watcher to check in on it at times

It runs fine locally but as soon as I put it in the docker, I get

lora-trainer-1  | --- Starting Serverless Worker |  Version 1.7.5 ---
lora-trainer-1  | WARN   | test_input.json not found, exiting.

lora-trainer-1  | --- Starting Serverless Worker |  Version 1.7.5 ---
lora-trainer-1  | WARN   | test_input.json not found, exiting.

From the trainer thread, why is this happening ? it looks as if its launching a whole new job and request handler

Runpod•16mo ago•

4 replies

Xqua

How can I use Multiprocessing in Serverless ?

Hi I am trying to do something somewhat simple

def run(self):
        print("TRAINER: Starting training")
        train = Train()
        trainer = self.ctx.Process(target=train.train, args=(self.config.config_path,))
        trainer.start()
        print("TRAINER: Starting watcher")
        self.watch()
        trainer.join()

def run(self):
        print("TRAINER: Starting training")
        train = Train()
        trainer = self.ctx.Process(target=train.train, args=(self.config.config_path,))
        trainer.start()
        print("TRAINER: Starting watcher")
        self.watch()
        trainer.join()

I have a training script in a training loop, and I want a watcher to check in on it at times

It runs fine locally but as soon as I put it in the docker, I get

lora-trainer-1  | --- Starting Serverless Worker |  Version 1.7.5 ---
lora-trainer-1  | WARN   | test_input.json not found, exiting.

lora-trainer-1  | --- Starting Serverless Worker |  Version 1.7.5 ---
lora-trainer-1  | WARN   | test_input.json not found, exiting.

From the trainer thread, why is this happening ? it looks as if its launching a whole new job and request handler

How can I use Multiprocessing in Serverless ?

Similar Threads

How can I use Multiprocessing in Serverless ?

Similar Threads

Similar Threads

Similar Threads