API limitations

Does Railway have an API call limit per service in the free account? I am running a cron job that involves a set of external API calls in a loop. However, it stops after executing 20 APIs. I have tested it locally, and it works fine. I would like to know the limitations of the Railway server. Project ID: dd417141-599f-4431-bbf6-538cbc507f1d
27 Replies
Percy
Percy13mo ago
Project ID: dd417141-599f-4431-bbf6-538cbc507f1d
Brody
Brody13mo ago
railway has no limitations on the amount of calls your app can make to external services, you are likely being rate limited by the external service
malindu.dev
malindu.dev13mo ago
Thank you I'll check with them.
Brody
Brody13mo ago
what errors do you see on the 21st api call?
malindu.dev
malindu.dev13mo ago
No errors, Just stop
malindu.dev
malindu.dev13mo ago
I have added logs to monitor the pages. 1 page = 1 API call
Brody
Brody13mo ago
you probably aren't logging errors when they occur would you mind sharing what this external service is?
malindu.dev
malindu.dev13mo ago
yes, I have missed to add logs to see errors. I'll add and test again. CommerceLayer is the external service which I am using.
Brody
Brody13mo ago
and why the need for making consecutive api calls?
malindu.dev
malindu.dev13mo ago
They have a page limit, and we need to make multiple calls to retrieve all the data. Ex: "meta": { "record_count": 598, "page_count": 24 },
Brody
Brody13mo ago
they have that limit for a reason 😉
malindu.dev
malindu.dev13mo ago
To get all 598 records, we have to call 24 times the API
Brody
Brody13mo ago
they obviously dont appreciate you scanning through all the pages and if they don't appreciate that kind of activity, railway wont appreciate that activity either
malindu.dev
malindu.dev13mo ago
I my business requirement, I have to gather all the data per 5min. 🙂 This task is already working with Vercel, There I have to use multiple cron jobs to handle the load. Because they are having time limitations for long running services. That's the reason to move with Railway I'm not sure this time CommerceLayer side also having any issues, I'll talk with them also.
Brody
Brody13mo ago
very good chance they just tell you to stop scraping their api and thats kinda what im telling you too
malindu.dev
malindu.dev13mo ago
Without looping their API, There is no way to get all the data. That's the problem.
Brody
Brody13mo ago
the problem is why do you need to scrape their api for the data every 5 minutes
malindu.dev
malindu.dev13mo ago
I have to run 320 times to gather our production data 😅
Brody
Brody13mo ago
it also sounds like you should be on the railway teams plan
malindu.dev
malindu.dev13mo ago
Sorry, I can't explain the whole business logic in this thread. By the way, this is an API for extracting product details. When the product stock levels change from CommerceLayer side, we need to update our inventory every 5 minutes. Because Our inventory DB is the middleman used to expose stock levels to external services. Can you explain a bit why I need to move for Railway Teams plan?
Brody
Brody13mo ago
please find a different way to gather this data without loops, you are abusing CommerceLayer's api and CommerceLayer could decide to ban railway cloud, thus blocking anyone else on railway from using CommerceLayer the bigger issue is the abuse of CommerceLayer, but you would need to be on the teams plan because it sounds like you have a commercial product
malindu.dev
malindu.dev13mo ago
yes, I'll talk with our team, But this is my research to find a suitable service for our problem.
Brody
Brody13mo ago
in the interest of not getting external services to block railway, i will be asking the team to look into your use of the railway platform, no hard feelings
malindu.dev
malindu.dev13mo ago
Okay, no problem. I'll contact you again further more assistant on this case.
angelo
angelo13mo ago
hey @Malindu - whats the purpose? Research, what university?
malindu.dev
malindu.dev13mo ago
I'm a kind of R&D developer, So, I'm researching a suitable service for the project.
luna
luna13mo ago
I would suggest looking at their rate limiting docs. You likely want to use an existing SDK or wrapper that handles the rate limiting headers and backs off accordingly. https://docs.commercelayer.io/core/rate-limits
Rate limits
Information about the restrictions on the number of times a user or client can access Commerce Layer API endpoints within a specified period of time.