Crawlee & Apify•12mo ago

Monitoring CPU and memory usage in actors

Hi, I'm develping an actor in rust and I'm trying to access resources utilisation (CPU and memory). I've seen that Crawlee uses os.cpus() from node, but I'm looking for a rust equivalent. I can make it work locally by mounting the docker socket on the container (docker run -v /var/run/docker.sock:/var/run/docker.sock <image_tag>), but it does not work on the Apify platform. Are there any resources/pointers I could check on how Apify runs an actor's container? And how I could read these resources utilisation? Any help would be appreciated.

5 Replies

MEE6•12mo ago

@Jerome just advanced to level 1! Thanks for your contributions! 🎉

correct-apricotOP•12mo ago

Turns out I had missed this page from the docs: https://docs.apify.com/platform/actors/development/programming-interface/system-events. The websocket sends 2 events (the second one is not documented though):

{
   "name":"cpuInfo",
   "data":{
      "isCpuOverloaded":false,
      "createdAt":"2024-06-05T09:16:37.916Z"
   }
}

{
   "name":"cpuInfo",
   "data":{
      "isCpuOverloaded":false,
      "createdAt":"2024-06-05T09:16:37.916Z"
   }
}

{
    "name": "systemInfo",
    "data": {
        "memAvgBytes": 127389235.71674345,
        "memCurrentBytes": 168525824,
        "memMaxBytes": 257417216,
        "cpuAvgUsage": 1.655466795311761,
        "cpuMaxUsage": 115.72125786163522,
        "cpuCurrentUsage": 7.616518557239937,
        "isCpuOverloaded": false,
        "createdAt": "2024-06-05T09:16:39.092Z"
    }
}

{
    "name": "systemInfo",
    "data": {
        "memAvgBytes": 127389235.71674345,
        "memCurrentBytes": 168525824,
        "memMaxBytes": 257417216,
        "cpuAvgUsage": 1.655466795311761,
        "cpuMaxUsage": 115.72125786163522,
        "cpuCurrentUsage": 7.616518557239937,
        "isCpuOverloaded": false,
        "createdAt": "2024-06-05T09:16:39.092Z"
    }
}

other-emerald•12mo ago

Hello @Jerome, thanks for pointing out the missing documentation. I will pass it on the team so we can fix it.

correct-apricotOP•12mo ago

Hi @vojtechmaslan , thanks that'd be great. What seems to be missing from these stats is the total amount of available CPU/memory, is there a way to get that? The goal would be to compute a usage % to be able to throttle the system. Also I have no idea what "cpuCurrentUsage": 7.616518557239937 means, what unit is that? Thanks for your help. Ok total amount of memory is given by env variable ACTOR_MEMORY_MBYTES, and number of CPU cores depends on the memory: "1 GB gives you 0.25 CPU cores". So the only question remaining is what unit is the cpu usage in?

optimistic-gold•10mo ago

how do you find total CPU/RAM usuage? found it: https://api.apify.com/v2/acts?token=xxxx

Gaming

Programming

Monitoring CPU and memory usage in actors

Did you find this page helpful?