> The Dreambooth implementations out there typically introduce generated images out of the model its

The Dreambooth implementations out there typically introduce generated images out of the model itself into the training set. I do not suggest that, and that's why EveryDream is not a Dreambooth implementation. I instead suggest you can gather other alternative "ground truth" data and "mix it in" with your training data to "preserve" the model and avoid "forgetting" or other artifacts. The tools repo has a Laion data scraper, or you can use things datasets like FFHQ photo set.

I recommend "ground truth" images as opposed to the Dreambooth implementations which use generated images out of the model itself for preservation. Dreambooth training on generated images can reinforce the error already present in the model. Ex. 6 fingers, bad limbs, etc.