Commit graph

60 commits

Author SHA1 Message Date
sparky8512
e16649fbf1 Change name of "current" to "end_counter"
Since "current" got added to the global data group returned from getting the history stats in non-bulk mode, it was being output by all 3 of the history scripts, and the name "current" was a little confusing when looking at prior output, since old values would no longer be current. The description of it in the start param of history_bulk_data was confusing, too.
2021-01-22 18:43:51 -08:00
sparky8512
9b96c5dcc6 Implement sample counter tracking in bulk mode
Add tracking of exactly which samples have already been sent off to InfluxDB so that samples are neither missed nor repeated due to minor time deltas in OS task scheduling. For now, this is only being applied to bulk mode.

Make the -s option only apply to the first loop iteration for bulk mode, since subsequent loops will want to pick up all samples since prior iteration.

Also, omit the latency field from the data point sent to InfluxDB for samples where the ping drop is 100%. The raw history data apparently just repeats prior value in this case, probably because it cannot just leave a hole in the data array and there is no good way to indicate invalid.

Related to issue #5
2021-01-18 13:30:34 -08:00
sparky8512
0663008be7 Add first pass of bulk data mode
Adds a new option, -b, that will write the individual sample data to InfluxDB server, instead of summary data.
2021-01-17 16:29:56 -08:00
sparky8512
3fafcea882 Fix remaining pylint and yapf nits 2021-01-15 19:27:10 -08:00
sparky8512
a589a75ce5 Revamp error printing
Closes #8
2021-01-12 19:51:38 -08:00
sparky8512
9ccfeb8181 Correct one more int/float mixage
Also, pull the change into the JSON parser for consistency.

Related to issue #12
2021-01-12 11:23:37 -08:00
sparky8512
fcbcbf4ef7 Make sure ints stay ints and floats stay floats
Specific history data patterns would sometimes lead to some of the stats switching between int and float type even if they were always whole numbers. This should ensure that doesn't happen.

I think this will fix #12, but will likely require deleting all the spacex.starlink.user_terminal.ping_stats data points from the database before the type conflict failure will go away.
2021-01-11 22:04:39 -08:00
sparky8512
f067f08952 Add InfluxDB and MQTT history stats scripts
Unlike the status info scripts, these include support for setting host and database parameters via command line options. Still to be added is support for HTTPS/SSL.

Add a get_id function to the grpc parser module, so it can be used for tagging purposes.

Minor cleanups in some of the other scripts to make them consistent with the newly added scripts.
2021-01-09 12:03:37 -08:00
sparky8512
0ee39f61fd Small change to the interface between parser and calling scripts
Move the "samples" stat out of the ping drop group of stats and into a new general stats group.

This way, it  will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 19:17:34 -08:00
sparky8512
170dd2daae Reorganize history parsing logic into a separate module
Moves the parsing logic that will be shared by some upcoming scripts out into a separate module.

There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic.

This also adds some proper documentation for what each of the stats means.
2021-01-06 11:46:50 -08:00