App statistic recording strategies
Hi gang,
I'm building an app that does 'stuff'. Things in the app can be on or off. I need to create historical graphs of the totals of 'on' things over time. Simple enough. The app is built with Laravel (PHP/MySQL).
My strategy at the moment is to have a daily runner script that selects the count of rows in the tables I'm interested in where status=1 (1 being 'on'). That yields an integer which I dump in a stats table along with the current date. That stats table is then harvested for relevant data the results being passed to the graphing system (Apex) in the main app and a simple datetime graph is produced. All that works.
My issue is that the graph has a potential 24 hour latency before it's updated with correct figures. 24 hours could elapse with the graph being potentially incorrect. How is this dealt with in 'real' apps'?
As I see it, I could:
1/ Put a disclaimer on the graph saying they're updated daily at midnight. Provide the accurate number on-demand in a small panel keeping the historicals separate.
2/ Increase the frequency of the runner. This would increase the granularity of the graph but greatly increase the rows in (and load on) the database (the example I gave above is a simplified example of what I'm actually doing - there's potentially hundreds of these counters). As time rolls on, the volumes of data being lobbed about will increase exponentially.
3/ Implement some sort of RRD wizardry. Still potentially heavy on the recording side also granularity is lost over time.
4/ Something else I'm not aware of or thinking of?
5/ I've just had a thought - A bunch of the systems I use at work all have up-to-the-minute accurate graphs of the stuff they're report on. How do they do that? Do they do what I'm doing already but tack on a quick point-in-time count before the graph is displayed? That might be the way..
Well, I'll post this anyway - always keen to learn!
cheers!
e
I'm building an app that does 'stuff'. Things in the app can be on or off. I need to create historical graphs of the totals of 'on' things over time. Simple enough. The app is built with Laravel (PHP/MySQL).
My strategy at the moment is to have a daily runner script that selects the count of rows in the tables I'm interested in where status=1 (1 being 'on'). That yields an integer which I dump in a stats table along with the current date. That stats table is then harvested for relevant data the results being passed to the graphing system (Apex) in the main app and a simple datetime graph is produced. All that works.
My issue is that the graph has a potential 24 hour latency before it's updated with correct figures. 24 hours could elapse with the graph being potentially incorrect. How is this dealt with in 'real' apps'?
As I see it, I could:
1/ Put a disclaimer on the graph saying they're updated daily at midnight. Provide the accurate number on-demand in a small panel keeping the historicals separate.
2/ Increase the frequency of the runner. This would increase the granularity of the graph but greatly increase the rows in (and load on) the database (the example I gave above is a simplified example of what I'm actually doing - there's potentially hundreds of these counters). As time rolls on, the volumes of data being lobbed about will increase exponentially.
3/ Implement some sort of RRD wizardry. Still potentially heavy on the recording side also granularity is lost over time.
4/ Something else I'm not aware of or thinking of?
5/ I've just had a thought - A bunch of the systems I use at work all have up-to-the-minute accurate graphs of the stuff they're report on. How do they do that? Do they do what I'm doing already but tack on a quick point-in-time count before the graph is displayed? That might be the way..
Well, I'll post this anyway - always keen to learn!
cheers!
e
A friendly place for developers to meet other devs, ask questions, get help, and just have a good time 🙂.
36,263Members
Resources
Was this page helpful?
