Problem with long running api endpoint
Hi, can someone advise me on a problem I'm having with a long running api endpoint that works like this:
1. api endpoint is scheduled to run every 90mins and is used to publish data from one database to another.
2. when it runs the endpoint is called and it picks up all database records marked with 'publish=1' field, does some processing, and then calls another api which updates the other database.
One thing I've noticed is that when there are a large number of records (e.g. 13k records) the process just gets slower and slower over until it stops running,
in this case it only processed about 1000 records and now doesn't seem to be picking up any new records, every time i query the database there are still 12k records marked publish=1
This is just a regular endpoint with a get request on a .net 6 api. Does anyone know why this would be happening? Should I be using hangfire or some kind of background process?
1. api endpoint is scheduled to run every 90mins and is used to publish data from one database to another.
2. when it runs the endpoint is called and it picks up all database records marked with 'publish=1' field, does some processing, and then calls another api which updates the other database.
One thing I've noticed is that when there are a large number of records (e.g. 13k records) the process just gets slower and slower over until it stops running,
in this case it only processed about 1000 records and now doesn't seem to be picking up any new records, every time i query the database there are still 12k records marked publish=1
This is just a regular endpoint with a get request on a .net 6 api. Does anyone know why this would be happening? Should I be using hangfire or some kind of background process?
