Commit graph

42 commits

Author SHA1 Message Date
sparky8512
35150803ef New validity check for prolonged obstruction data
Reintroduce validity check for prolonged obstruction duration and interval, but based on checking for 0.0 and NaN values instead of relying on the valid flag in the gRPC message. Empirical evidence suggests that the dish reports (0, NaN) for those 2 values when it has no obstruction data with which to work, such as when there actually are no obstructions or when the dish has recently rebooted.

This should fix #42.
2022-03-26 15:22:02 -07:00
sparky8512
9ca1a8a51c Remove valid check from obstruction status data
Remove the validity check that was gating the reporting of the obstruction_duration and obstruction_interval data items.

When that got added to the grpc protocol, I assumed that the valid flag meant whether the other 2 fields were useful data. That may actually have been the intent, for all I know, but I've never seen that flag report true. However, I also never saw the banner in the app telling me I would get obstructions every X minutes, either. That is, until a few weeks ago, probably due to a dish firmware change. Now I see the app reporting this data, so it must be ignoring that valid flag, or at least not using it for that purpose.

Probably nobody cares about this data (including me), given that nobody ever complained about it always reporting empty values, but I don't want to remove it and mess up db schemas, so it might as well be populated.
2022-02-19 16:16:54 -08:00
sparky8512
52dc5ec79d Document new alert added in most recent firmware 2022-01-20 14:07:40 -08:00
sparky8512
c35588d01f Fixes related to failed grpc network connection
Add timeouts to all gRPC remote calls and bump yagrc package requirement to a version that does same for the reflection service, as well as fixing a state issue around failed lazy import resolution.

This should address the script hang symptom on issue #36.
2021-11-06 20:16:50 -07:00
sparky8512
b0f40c7dd7 Remove usage of another obsolete field
last_24h_obstructed_s has been removed from the grpc service protocol, along with all the other deprecated fields. This renders the seconds_obstructed item in the status mode group useless.

This is the down side of using reflection at run time to pull the protocol definitions. If something gets removed from the protocol that is still being used, it will break the run time instead of just returning default values. Then again, if it gets removed from the protocol, it's no longer useful, anyway.

This should address issue #35.
2021-10-25 07:39:48 -07:00
sparky8512
3dddd95ff3 Bring back the state data item in status group
Derive connectivity state information from the "outage" field of the get_status response, which I hadn't noticed before because it only populates when the dish is not in a connected state. This restores the state data item in the status mode group, which had been rendered useless due to a grpc service change.

In addition to the previous possible state names, this adds a few more that pertain to outages while otherwise connected, which I think were just previously reported as "CONNECTED", as well as some special cases of offline.
2021-10-19 15:34:45 -07:00
sparky8512
79839c0b19 Remove usage of obsolete grpc fields
A recent firmware change has stopped populating a number of result message fields in the grpc service that had previously been marked as deprecated. This caused the script to start crashing in some use cases.

While those fields are still in the protocol definition for now, this change removes usage of them entirely, in case they get removed. As things are now, they are useless, anyway, since they will always just return default values.

This renders useless the state, snr, scheduled, obstructed, and unscheduled items in the status, bulk_history, and ping_drop mode groups. Those items now mostly return empty values.

See issue #32 for more detail.
2021-10-16 08:47:18 -07:00
sparky8512
9f726e71af Further tweak to how -o loop polling works
The prior 2 changes made the handling of the first set of polled loops inconsistent with subsequent sets with respect to maintaining data across dish reboot. This change makes them both work the same (and correctly).
2021-10-15 09:57:10 -07:00
sparky8512
afab7553e3 Fix -o option breakage from recent change
This restores the ability of -o to keep polled data across a reboot. It broke due to a simple issue in concatenate_history, but I realized that counter tracking for -o accumulated data was also a bit broken, so I fixed that, too.

Fix #31
2021-10-07 12:23:48 -07:00
sparky8512
e1070965f2 Initial cut of obstruction map support
Add a new command line script, dish_obstruction_map.py, that writes a PNG image based on the obstruction map data queried from the dish.

Supports color or greyscale output and either with or without alpha channel.

Does not yet support running in an interval loop, mostly because that will require templatizing the output filename in order to be useful.

Tracked on issue #27
2021-09-07 17:29:56 -07:00
sparky8512
af940a9727 Improvements to how the -o option works
Change the loop polling function (-o) to aggregate the history data each polling loop instead of just keeping the last polled history so it can be logged when reboot is detected. This allows for computing statistics across a longer period than the size of the dish's history buffer, which has been reduced to 15 minutes recently.

This change also makes it so data is not logged right away when dish reboot is detected, so the logging always happens at the specified interval whether there was a reboot or not.

Finally, change the poll loop counting so data is not emitted on the first loop when polling is configured. That made sense to do when the history buffer was large enough to have the entire period's worth of data, but now it just results in a short period in the log output every time the script is restarted.

Fixes #29
2021-09-07 12:02:14 -07:00
sparky8512
41caa76962 Add a few more fields to the status group
Add dish direction and "prolonged" obstruction info to the status mode group.

These were added to the grpc service at some point over the last several months.

Only lightly tested, given that my dish no longer reports significant periods of obstruction.

This is related to discussion in issue #27, although it doesn't address that issue in the slightest.
2021-09-05 17:41:13 -07:00
sparky8512
07389cb0d9 Remove dependence on Python 3.8 or later
statistics.quantiles was not present in Python 3.7 or earlier, which is a problem on Windows if you want to run a binary optimized version of the protobuf package, since those are not currently being posted for Python 3.8 or later.

This change switches to use the weighted median function just with equal weights. It's a bit of overkill, but it also cuts out the mess that was working around deficiencies of the statistics.quantiles implementation.
2021-03-16 13:19:06 -07:00
sparky8512
55ba411db8 Go back to using message number for alert bits
SpaceX has been using inconsistent field ordering when adding alerts, so field index cannot be used to consistently identify the specific alerts. Message number is more appropriate for that, anyway, but is not guaranteed to be a low enough number to fit into a bit field. Oh well, in the unlikely event that SpaceX switches to larger message numbers, they just won't show up in the alerts bit field (but will still show up in alert_detail).

This does make the bit ordering in alerts inconsistent with prior versions of these tools, but I've never actually seen one of these alerts report true, so hopefully this doesn't impact anyone.

The alerts are still sorted by index number in the alert_detail text output, which is a problem for CSV output, but I think ordering by message number instead would be pointlessly complex. alert_detail is not a great fit for CSV output anyway, due to its variable length, so just added a warning about that in the text script module doc.
2021-03-07 09:22:52 -08:00
sparky8512
e10c9dbb7f Mostly cosmetic changes
A few things I noticed while porting this code to the JSON script. The only real change here is fixing the bulk history output to print UTC time instead of local time.
2021-02-21 13:57:48 -08:00
sparky8512
38987054b9 Add option to poll history more frequently
This further complicates the code, for functionality that probably only I care about, but when computing stats for relatively long time intervals, it really hurts when the dish reboots and up to an entire time period's worth of data is lost at exactly the point where it may have been having interesting behavior.
2021-02-19 10:56:20 -08:00
sparky8512
18829bd5cb Allow force option when schema version matches
Since the alert types are determined dynamically from the protocol definition, the status schema may need to be updated even if nothing changed in the scripts, when the dish software adds a new alert type (which just happened, say hello to the "mast_not_near_vertical" alert). This allows the manual override for that case, not just schema version downgrade.
2021-02-15 19:23:37 -08:00
sparky8512
a4bf2d1625 Support for overriding dish IP and port
Probably not terribly useful unless someone needs to tunnel through a different network to get to their dish, but it makes testing the dish unreachable case a lot easier. This was complicated a bit by the fact that a channel (and therefor the dish IP and port) is needed to get the list of alert types via reflection due to prior changes.

This exposed some issues with the error message for dish unreachable, so fixed those.
2021-02-15 18:50:22 -08:00
sparky8512
1659133168 Switch reflect usage to yagrc's new lazy importer
This makes the normal imports a bit more readable.

Lazy import requires yagrc v1.1.0, so bumped requirements.txt entry for that.
2021-02-14 17:22:52 -08:00
sparky8512
30e4b27516 Fix handling of 0 history samples range
This can happen when polling the history buffer very frequently (<= 1 second).
2021-02-14 13:47:24 -08:00
sparky8512
2ac5944824 Fix statistics error
statistics.quantiles doesn't handle the case of having only one data sample.
2021-02-13 17:21:11 -08:00
sparky8512
80e752a510 Counter state tracking for non-bulk history data
with option to disable to get prior behavior of fixed number of samples per loop iteration.
2021-02-13 10:17:42 -08:00
sparky8512
67b0045ac8 Add the non-abs wedge_fraction_obstructed status
I had only added the one that the Starlink app uses to show obstructions, because it didn't seem like the other one was all that useful, but people seem to be interested in studying the difference between the 2, so might as well have it. This is in the obstruction_detail group, along with the other one. I'm kinda regretting naming the first one as I did, though, because it's now a little confusing between my naming and the naming in the grpc message.

Since this is a new field, also had to implement schema updates for the sqlite script.
2021-02-12 19:53:28 -08:00
sparky8512
ec61333710 Two more places that need the grpc imports 2021-02-12 13:48:55 -08:00
sparky8512
491227ddb4 Remove need to generate grpc modules via protoc
If the spacex grpc modules are not available in the import path, will now fall back to using reflection to get them dynamically. I'm not real happy with the mess this made of the import lines, though (and neither is pylint...), so I may hack on that a little further when I get the time.

Add a requirements.txt file to enable installation of all prerequisites so users don't have to follow the individual instructions for each dependency package. At some point, I'll really need to add proper Python packaging so the whole thing can just be installed via pip.

Rearrange the README a bit, since some of the sections have increased or decreased in relevance over time.
2021-02-12 13:11:54 -08:00
sparky8512
381cfbee00 Hedge a little more on a technical detail
Specifically, on what wedges_fraction_obstructed means exactly, because I suspect the max values may vary based on weighting of the raw numbers.
2021-02-11 18:47:26 -08:00
sparky8512
db83a7f042 Switch generated protobuf module imports around
The way I had them, it was hiding the fact that there was no explicit import for the spacex.api.device.dish_pb2 module.
2021-02-04 20:02:45 -08:00
sparky8512
549a46ae56 Add new dish_grpc script for sqlite output
I'm sure this isn't a particularly optimal implementation, but it's functional.

This required exporting knowledge about the types that will be returned per field from starlink_grpc and moving things around a little in dish_common.
2021-02-03 17:23:01 -08:00
sparky8512
ed2ef50581 Catch GrpcError in new example
Apparently, it was a little _too_ simple.

Also, update the description for seconds_to_first_nonempty_slot field to reflect some behavior this script was able to capture.
2021-02-02 09:07:42 -08:00
sparky8512
94114bfd59 Add latency and usage history stat groups
Add latency and usage stat groups to the stats computed from history samples. This includes an attempt at characterizing latency under network load, too, but I don't know how useful that's going to be, so I have marked that as experimental, in case it needs algorithmic improvements.

The new groups are enabled on the command line by use of the new mode names: ping_latency, ping_loaded_latency, and usage.

Add valid_s to the obstruction details status group. This was the only missing field from everything available in the status response (other than wedge_fraction_obstructed, which seems redundant to wedge_abs_fraction_obstructed), and I only skipped it because I don't know what it means exactly. Adding it now with my best guess at a description in order to avoid a compatibility breaking change later.

Closes #5
2021-02-01 19:09:34 -08:00
sparky8512
68c1413dbd Keep grpc channel open across RPC calls
This restores the functionality that the InfluxDB status polling script had whereby instead of using a new grpc Channel for each RPC call, it would keep one open and reuse it, retrying one time if it ever fails, which can happen if the connection is lost between calls. Now all the grpc scripts have this functionality.

Also, hedge a little bit in the descriptions for what the obstruction detail fields means, given that I'm not sure my assumptions there are correct.
2021-01-30 11:24:17 -08:00
sparky8512
45b563f91a Refactor to reduce the amount of duplicate code
Combined the history and status scripts for each data backend and moved some of the shared code into a separate module. Since the existing script names were not appropriate for the new combined versions, the main entry point scripts now have new names, which better conform with Python module name conventions: dish_grpc_text.py, dish_grpc_mqtt.py, and dish_grpc_influx.py. pylint seems happier with those names, at any rate.

Switched the argument parsing from getopt to argparse, since that better facilitates sharing the common bits. The whole command line interface is now different in that the selection of data groups to process must be made as required arg(s) rather than option flags, but for the most part, the scripts support choosing an arbitrary list of groups and will process them all.

Split the monster main() functions into a more reasonable set of functions.

Added new functions to starlink_grpc to support getting the status, which returns the data in a form similar to the history data functions. Reformatted the starlink_grpc module docstring to render better with pydoc. Also changed the way sequence data field names are reported so that the consuming scripts can name them correctly without resorting to hacky special casing based on specific field names. This would subtly break the old scripts that had been expecting the old naming, but those scripts are now gone.

The code is harder to follow now, IMO, but this should allow adding of new features and/or data backends without having to make the same change in 6 places as had been the case. To that end, dish_grpc_text now supports bulk history mode, since it was trivial to add once I had it implemented in order to support that feature for dish_grpc_influx.
2021-01-29 19:25:23 -08:00
sparky8512
e16649fbf1 Change name of "current" to "end_counter"
Since "current" got added to the global data group returned from getting the history stats in non-bulk mode, it was being output by all 3 of the history scripts, and the name "current" was a little confusing when looking at prior output, since old values would no longer be current. The description of it in the start param of history_bulk_data was confusing, too.
2021-01-22 18:43:51 -08:00
sparky8512
9b96c5dcc6 Implement sample counter tracking in bulk mode
Add tracking of exactly which samples have already been sent off to InfluxDB so that samples are neither missed nor repeated due to minor time deltas in OS task scheduling. For now, this is only being applied to bulk mode.

Make the -s option only apply to the first loop iteration for bulk mode, since subsequent loops will want to pick up all samples since prior iteration.

Also, omit the latency field from the data point sent to InfluxDB for samples where the ping drop is 100%. The raw history data apparently just repeats prior value in this case, probably because it cannot just leave a hole in the data array and there is no good way to indicate invalid.

Related to issue #5
2021-01-18 13:30:34 -08:00
sparky8512
0663008be7 Add first pass of bulk data mode
Adds a new option, -b, that will write the individual sample data to InfluxDB server, instead of summary data.
2021-01-17 16:29:56 -08:00
sparky8512
3fafcea882 Fix remaining pylint and yapf nits 2021-01-15 19:27:10 -08:00
sparky8512
a589a75ce5 Revamp error printing
Closes #8
2021-01-12 19:51:38 -08:00
sparky8512
9ccfeb8181 Correct one more int/float mixage
Also, pull the change into the JSON parser for consistency.

Related to issue #12
2021-01-12 11:23:37 -08:00
sparky8512
fcbcbf4ef7 Make sure ints stay ints and floats stay floats
Specific history data patterns would sometimes lead to some of the stats switching between int and float type even if they were always whole numbers. This should ensure that doesn't happen.

I think this will fix #12, but will likely require deleting all the spacex.starlink.user_terminal.ping_stats data points from the database before the type conflict failure will go away.
2021-01-11 22:04:39 -08:00
sparky8512
f067f08952 Add InfluxDB and MQTT history stats scripts
Unlike the status info scripts, these include support for setting host and database parameters via command line options. Still to be added is support for HTTPS/SSL.

Add a get_id function to the grpc parser module, so it can be used for tagging purposes.

Minor cleanups in some of the other scripts to make them consistent with the newly added scripts.
2021-01-09 12:03:37 -08:00
sparky8512
0ee39f61fd Small change to the interface between parser and calling scripts
Move the "samples" stat out of the ping drop group of stats and into a new general stats group.

This way, it  will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 19:17:34 -08:00
sparky8512
170dd2daae Reorganize history parsing logic into a separate module
Moves the parsing logic that will be shared by some upcoming scripts out into a separate module.

There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic.

This also adds some proper documentation for what each of the stats means.
2021-01-06 11:46:50 -08:00