[Help Needed]: Have large datasets to feed to models
Hello everyone! 
I'm currently working on a personal project and have hit a bit of a roadblock. I'm using text generation models to create a chat interface, but I've noticed that I need to specify the system role and user prompts with each query. Unfortunately, there's no option to store a large dataset in the model's memory that it can reference each time it generates an output.
What I'd like to do is store a large set of data in one place, which the model can then access whenever I ask it a question. However, I'm relatively new to the Cloudflare ecosystem and could really use some guidance. Is there a way to achieve this using Cloudflare's functionalities or products?
To elaborate, I'm working with the GitHub API and fetching large amount of json data, which I'd then like my model to reference everytime it generates an output.
Thanks in advance for your help!
I'm currently working on a personal project and have hit a bit of a roadblock. I'm using text generation models to create a chat interface, but I've noticed that I need to specify the system role and user prompts with each query. Unfortunately, there's no option to store a large dataset in the model's memory that it can reference each time it generates an output.
What I'd like to do is store a large set of data in one place, which the model can then access whenever I ask it a question. However, I'm relatively new to the Cloudflare ecosystem and could really use some guidance. Is there a way to achieve this using Cloudflare's functionalities or products?
To elaborate, I'm working with the GitHub API and fetching large amount of json data, which I'd then like my model to reference everytime it generates an output.
Thanks in advance for your help!
Welcome to the official Cloudflare Developers server. Here you can ask for help and stay updated with the latest news
86,942Members
Resources
Similar Threads
Was this page helpful?
