From 46f65a62144b783af1c1857ae6b4380adbebf80d Mon Sep 17 00:00:00 2001 From: sparky8512 <76499194+sparky8512@users.noreply.github.com> Date: Fri, 15 Jan 2021 18:39:33 -0800 Subject: [PATCH] Implement periodic loop option Add an interval timing loop for all the grpc scripts that did not already have one. Organized some of the code into functions in order to facilitate this, which caused some problems with local variables vs global ones, so moved the script code into a proper main() function. Which didn't really solve the access to globals issue, so also moved the mutable state into a class instance. The interval timer should be relatively robust against time drift due to the loop function running time and/or OS scheduler delay, but is far from perfect. Retry logic is now in place for both InfluxDB scripts. Retry for dishStatusInflux.py is slightly changed in that a failed write to InfluxDB server will be retried on every interval, rather than waiting for another batch full of data points to write, but this only happens once there is at least a full batch (currently 6) of data points pending. This new behavior matches how the autocommit functionality on SeriesHelper works. Changed the default behavior of dishStatusInflux.py to not loop, in order to match the other scripts. To get the old behavior, add a '-t 30' option to the command line. Closes #9 --- README.md | 42 +++- dishHistoryInflux.py | 363 +++++++++++++++++++++-------------- dishHistoryMqtt.py | 253 ++++++++++++++---------- dishHistoryStats.py | 210 +++++++++++--------- dishStatusCsv.py | 204 ++++++++++++-------- dishStatusInflux.py | 448 ++++++++++++++++++++++--------------------- dishStatusMqtt.py | 253 ++++++++++++++---------- 7 files changed, 1028 insertions(+), 745 deletions(-) diff --git a/README.md b/README.md index 427bb43..6cee089 100644 --- a/README.md +++ b/README.md @@ -7,7 +7,7 @@ For more information on what Starlink is, see [starlink.com](https://www.starlin `parseJsonHistory.py` operates on a JSON format data representation of the protocol buffer messages, such as that output by [gRPCurl](https://github.com/fullstorydev/grpcurl). The command lines below assume `grpcurl` is installed in the runtime PATH. If that's not the case, just substitute in the full path to the command. -All the tools that pull data from the dish expect to be able to reach it at the dish's fixed IP address of 192.168.100.1, as do the Starlink [Android app](https://play.google.com/store/apps/details?id=com.starlink.mobile) and [iOS app](https://apps.apple.com/us/app/starlink/id1537177988). When using a router other than the one included with the Starlink installation kit, this usually requires some additional router configuration to make it work. That configuration is beyond the scope of this document, but if the Starlink app doesn't work on your home network, then neither will these scripts. That being said, you do not need the Starlink app installed to make use of these scripts. +All the tools that pull data from the dish expect to be able to reach it at the dish's fixed IP address of 192.168.100.1, as do the Starlink [Android app](https://play.google.com/store/apps/details?id=com.starlink.mobile), [iOS app](https://apps.apple.com/us/app/starlink/id1537177988), and the browser app you can run directly from http://192.168.100.1. When using a router other than the one included with the Starlink installation kit, this usually requires some additional router configuration to make it work. That configuration is beyond the scope of this document, but if the Starlink app doesn't work on your home network, then neither will these scripts. That being said, you do not need the Starlink app installed to make use of these scripts. The scripts that don't use `grpcurl` to pull data require the `grpcio` Python package at runtime and generating the necessary gRPC protocol code requires the `grpcio-tools` package. Information about how to install both can be found at https://grpc.io/docs/languages/python/quickstart/ @@ -17,6 +17,10 @@ The scripts that use [InfluxDB](https://www.influxdata.com/products/influxdb/) f ## Usage +Of the 3 groups below, the grpc scripts are really the only ones being actively developed. The others are mostly by way of example of what could be done with the underlying data. + +### The JSON parser script + `parseJsonHistory.py` takes input from a file and writes its output to standard output. The easiest way to use it is to pipe the `grpcurl` command directly into it. For example: ``` grpcurl -plaintext -d {\"get_history\":{}} 192.168.100.1:9200 SpaceX.API.Device.Device/Handle | python parseJsonHistory.py @@ -28,7 +32,11 @@ python parseJsonHistory.py -h When used as-is, `parseJsonHistory.py` will summarize packet loss information from the data the dish records. There's other bits of data in there, though, so that script (or more likely the parsing logic it uses, which now resides in `starlink_json.py`) could be used as a starting point or example of how to iterate through it. Most of the data displayed in the Statistics page of the Starlink app appears to come from this same `get_history` gRPC response. See the file `get_history_notes.txt` for some ramblings on how to interpret it. -The other scripts can do the gRPC communication directly, but they require some generated code to support the specific gRPC protocol messages used. These would normally be generated from .proto files that specify those messages, but to date (2020-Dec), SpaceX has not publicly released such files. The gRPC service running on the dish appears to have [server reflection](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md) enabled, though. `grpcurl` can use that to extract a protoset file, and the `protoc` compiler can use that to make the necessary generated code: +The one bit of functionality this script has over the grpc scripts is that it supports capturing the grpcurl output to a file and reading from that, which may be useful if you're collecting data in one place but analyzing it in another. Otherwise, it's probably better to use `dishHistoryStats.py`, described below. + +### The grpc scripts + +This set of scripts can do the gRPC communication directly, but they require some generated code to support the specific gRPC protocol messages used. These would normally be generated from .proto files that specify those messages, but to date (2020-Dec), SpaceX has not publicly released such files. The gRPC service running on the dish appears to have [server reflection](https://github.com/grpc/grpc/blob/master/doc/server-reflection.md) enabled, though. `grpcurl` can use that to extract a protoset file, and the `protoc` compiler can use that to make the necessary generated code: ``` grpcurl -plaintext -protoset-out dish.protoset 192.168.100.1:9200 describe SpaceX.API.Device.Device mkdir src @@ -41,29 +49,47 @@ python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/wifi.proto python3 -m grpc_tools.protoc --descriptor_set_in=../dish.protoset --python_out=. --grpc_python_out=. spacex/api/device/wifi_config.proto ``` -Then move the resulting files to where the Python scripts can find them. +Then move the resulting files to where the Python scripts can find them in its import path, such as in the same directory as the scripts themselves. -Once those are available, the `dishHistoryStats.py` script can be used in place of the `grpcurl | parseJsonHistory.py` pipeline, with most of the same command line options. +Once those are available, the `dishHistoryStats.py` script can be used in place of the `grpcurl | parseJsonHistory.py` pipeline, with most of the same command line options. For example: +``` +python3 parseHistoryStats.py +``` -To collect and record summary stats every hour, you can put something like the following in your user crontab: +By default, `parseHistoryStats.py` (and `parseJsonHistory.py`) will output the stats in CSV format. You can use the `-v` option to instead output in a (slightly) more human-readable format. + +To collect and record summary stats at the top of every hour, you could put something like the following in your user crontab (assuming you have moved the scripts to ~/bin and made them executable): ``` 00 * * * * [ -e ~/dishStats.csv ] || ~/bin/dishHistoryStats.py -H >~/dishStats.csv; ~/bin/dishHistoryStats.py >>~/dishStats.csv ``` `dishHistoryInflux.py` and `dishHistoryMqtt.py` are similar, but they send their output to an InfluxDB server and a MQTT broker, respectively. Run them with `-h` command line option for details on how to specify server and/or database options. -`dishDumpStatus.py` is even simpler. Just run it as: +`dishStatusCsv.py`, `dishStatusInflux.py`, and `dishStatusMqtt.py` output the status data instead of history data, to various data backends. The information they pull is mostly what appears related to the dish in the Debug Data section of the Starlink app. As with the corresponding history scripts, run them with `-h` command line option for usage details. + +By default, all of these scripts will pull data once, send it off to the specified data backend, and then exit. They can instead be made to run in a periodic loop by passing a `-t` option to specify loop interval, in seconds. For example, to capture status information to a InfluxDB server every 30 seconds, you could do something like this: +``` +python3 dishStatusInflux.py -t 30 [... probably other args to specifiy server options ...] +``` + +Some of the scripts (currently only the InfluxDB ones) also support specifying options through environment variables. See details in the scripts for the environment variables that map to options. + +### Other scripts + +`dishDumpStatus.py` is a simple example of how to use the grpc modules (the ones generated by protoc, not `starlink_grpc.py`) directly. Just run it as: ``` python3 dishDumpStatus.py ``` -and revel in copious amounts of dish status information. OK, maybe it's not as impressive as all that. This one is really just meant to be a starting point for real functionality to be added to it. The information this script pulls is mostly what appears related to the dish in the Debug Data section of the Starlink app. +and revel in copious amounts of dish status information. OK, maybe it's not as impressive as all that. This one is really just meant to be a starting point for real functionality to be added to it. -`dishStatusCsv.py`, `dishStatusInflux.py`, and `dishStatusMqtt.py` output the same status data, but to various data backends. As with the corresponding history scripts, run them with `-h` command line option for usage details. +Possibly more simple examples to come, as the other scripts have started getting a bit complicated. ## To Be Done (Maybe) There are `reboot` and `dish_stow` requests in the Device protocol, too, so it should be trivial to write a command that initiates dish reboot and stow operations. These are easy enough to do with `grpcurl`, though, as there is no need to parse through the response data. For that matter, they're easy enough to do with the Starlink app. +Proper Python packaging, since some of the scripts are no longer self-contained. + ## Other Tidbits The Starlink Android app actually uses port 9201 instead of 9200. Both appear to expose the same gRPC service, but the one on port 9201 uses an HTTP/1.1 wrapper, whereas the one on port 9200 uses HTTP/2.0, which is what most gRPC tools expect. diff --git a/dishHistoryInflux.py b/dishHistoryInflux.py index d757901..b1a9b3e 100644 --- a/dishHistoryInflux.py +++ b/dishHistoryInflux.py @@ -10,169 +10,234 @@ # ###################################################################### +import getopt import datetime +import logging import os import sys -import getopt -import logging - +import time import warnings + from influxdb import InfluxDBClient import starlink_grpc -arg_error = False -try: - opts, args = getopt.getopt(sys.argv[1:], "ahn:p:rs:vC:D:IP:R:SU:") -except getopt.GetoptError as err: - print(str(err)) - arg_error = True +def main(): + arg_error = False -# Default to 1 hour worth of data samples. -samples_default = 3600 -samples = samples_default -print_usage = False -verbose = False -run_lengths = False -host_default = "localhost" -database_default = "starlinkstats" -icargs = {"host": host_default, "timeout": 5, "database": database_default} -rp = None - -# For each of these check they are both set and not empty string -influxdb_host = os.environ.get("INFLUXDB_HOST") -if influxdb_host: - icargs["host"] = influxdb_host -influxdb_port = os.environ.get("INFLUXDB_PORT") -if influxdb_port: - icargs["port"] = int(influxdb_port) -influxdb_user = os.environ.get("INFLUXDB_USER") -if influxdb_user: - icargs["username"] = influxdb_user -influxdb_pwd = os.environ.get("INFLUXDB_PWD") -if influxdb_pwd: - icargs["password"] = influxdb_pwd -influxdb_db = os.environ.get("INFLUXDB_DB") -if influxdb_db: - icargs["database"] = influxdb_db -influxdb_rp = os.environ.get("INFLUXDB_RP") -if influxdb_rp: - rp = influxdb_rp -influxdb_ssl = os.environ.get("INFLUXDB_SSL") -if influxdb_ssl: - icargs["ssl"] = True - if influxdb_ssl.lower() == "secure": - icargs["verify_ssl"] = True - elif influxdb_ssl.lower() == "insecure": - icargs["verify_ssl"] = False - else: - icargs["verify_ssl"] = influxdb_ssl - -if not arg_error: - if len(args) > 0: + try: + opts, args = getopt.getopt(sys.argv[1:], "ahn:p:rs:t:vC:D:IP:R:SU:") + except getopt.GetoptError as err: + print(str(err)) arg_error = True - else: - for opt, arg in opts: - if opt == "-a": - samples = -1 - elif opt == "-h": - print_usage = True - elif opt == "-n": - icargs["host"] = arg - elif opt == "-p": - icargs["port"] = int(arg) - elif opt == "-r": - run_lengths = True - elif opt == "-s": - samples = int(arg) - elif opt == "-v": - verbose = True - elif opt == "-C": - icargs["ssl"] = True - icargs["verify_ssl"] = arg - elif opt == "-D": - icargs["database"] = arg - elif opt == "-I": - icargs["ssl"] = True - icargs["verify_ssl"] = False - elif opt == "-P": - icargs["password"] = arg - elif opt == "-R": - rp = arg - elif opt == "-S": - icargs["ssl"] = True - icargs["verify_ssl"] = True - elif opt == "-U": - icargs["username"] = arg -if "password" in icargs and "username" not in icargs: - print("Password authentication requires username to be set") - arg_error = True + # Default to 1 hour worth of data samples. + samples_default = 3600 + samples = None + print_usage = False + verbose = False + default_loop_time = 0 + loop_time = default_loop_time + run_lengths = False + host_default = "localhost" + database_default = "starlinkstats" + icargs = {"host": host_default, "timeout": 5, "database": database_default} + rp = None + flush_limit = 6 -if print_usage or arg_error: - print("Usage: " + sys.argv[0] + " [options...]") - print("Options:") - print(" -a: Parse all valid samples") - print(" -h: Be helpful") - print(" -n : Hostname of InfluxDB server, default: " + host_default) - print(" -p : Port number to use on InfluxDB server") - print(" -r: Include ping drop run length stats") - print(" -s : Number of data samples to parse, default: " + str(samples_default)) - print(" -v: Be verbose") - print(" -C : Enable SSL/TLS using specified CA cert to verify server") - print(" -D : Database name to use, default: " + database_default) - print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") - print(" -P : Set password for authentication") - print(" -R : Retention policy name to use") - print(" -S: Enable SSL/TLS using default CA cert") - print(" -U : Set username for authentication") - sys.exit(1 if arg_error else 0) - -logging.basicConfig(format="%(levelname)s: %(message)s") - -try: - dish_id = starlink_grpc.get_id() -except starlink_grpc.GrpcError as e: - logging.error("Failure getting dish ID: " + str(e)) - sys.exit(1) - -timestamp = datetime.datetime.utcnow() - -try: - g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(samples, verbose) -except starlink_grpc.GrpcError as e: - logging.error("Failure getting ping stats: " + str(e)) - sys.exit(1) - -all_stats = g_stats.copy() -all_stats.update(pd_stats) -if run_lengths: - for k, v in rl_stats.items(): - if k.startswith("run_"): - for i, subv in enumerate(v, start=1): - all_stats[k + "_" + str(i)] = subv + # For each of these check they are both set and not empty string + influxdb_host = os.environ.get("INFLUXDB_HOST") + if influxdb_host: + icargs["host"] = influxdb_host + influxdb_port = os.environ.get("INFLUXDB_PORT") + if influxdb_port: + icargs["port"] = int(influxdb_port) + influxdb_user = os.environ.get("INFLUXDB_USER") + if influxdb_user: + icargs["username"] = influxdb_user + influxdb_pwd = os.environ.get("INFLUXDB_PWD") + if influxdb_pwd: + icargs["password"] = influxdb_pwd + influxdb_db = os.environ.get("INFLUXDB_DB") + if influxdb_db: + icargs["database"] = influxdb_db + influxdb_rp = os.environ.get("INFLUXDB_RP") + if influxdb_rp: + rp = influxdb_rp + influxdb_ssl = os.environ.get("INFLUXDB_SSL") + if influxdb_ssl: + icargs["ssl"] = True + if influxdb_ssl.lower() == "secure": + icargs["verify_ssl"] = True + elif influxdb_ssl.lower() == "insecure": + icargs["verify_ssl"] = False else: - all_stats[k] = v + icargs["verify_ssl"] = influxdb_ssl -points = [{ - "measurement": "spacex.starlink.user_terminal.ping_stats", - "tags": {"id": dish_id}, - "time": timestamp, - "fields": all_stats, -}] + if not arg_error: + if len(args) > 0: + arg_error = True + else: + for opt, arg in opts: + if opt == "-a": + samples = -1 + elif opt == "-h": + print_usage = True + elif opt == "-n": + icargs["host"] = arg + elif opt == "-p": + icargs["port"] = int(arg) + elif opt == "-r": + run_lengths = True + elif opt == "-s": + samples = int(arg) + elif opt == "-t": + loop_time = float(arg) + elif opt == "-v": + verbose = True + elif opt == "-C": + icargs["ssl"] = True + icargs["verify_ssl"] = arg + elif opt == "-D": + icargs["database"] = arg + elif opt == "-I": + icargs["ssl"] = True + icargs["verify_ssl"] = False + elif opt == "-P": + icargs["password"] = arg + elif opt == "-R": + rp = arg + elif opt == "-S": + icargs["ssl"] = True + icargs["verify_ssl"] = True + elif opt == "-U": + icargs["username"] = arg -if "verify_ssl" in icargs and not icargs["verify_ssl"]: - # user has explicitly said be insecure, so don't warn about it - warnings.filterwarnings("ignore", message="Unverified HTTPS request") + if "password" in icargs and "username" not in icargs: + print("Password authentication requires username to be set") + arg_error = True -influx_client = InfluxDBClient(**icargs) -try: - influx_client.write_points(points, retention_policy=rp) - rc = 0 -except Exception as e: - logging.error("Failed writing to InfluxDB database: " + str(e)) - rc = 1 -finally: - influx_client.close() -sys.exit(rc) + if print_usage or arg_error: + print("Usage: " + sys.argv[0] + " [options...]") + print("Options:") + print(" -a: Parse all valid samples") + print(" -h: Be helpful") + print(" -n : Hostname of InfluxDB server, default: " + host_default) + print(" -p : Port number to use on InfluxDB server") + print(" -r: Include ping drop run length stats") + print(" -s : Number of data samples to parse, default: loop interval,") + print(" if set, else " + str(samples_default)) + print(" -t : Loop interval in seconds or 0 for no loop, default: " + + str(default_loop_time)) + print(" -v: Be verbose") + print(" -C : Enable SSL/TLS using specified CA cert to verify server") + print(" -D : Database name to use, default: " + database_default) + print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") + print(" -P : Set password for authentication") + print(" -R : Retention policy name to use") + print(" -S: Enable SSL/TLS using default CA cert") + print(" -U : Set username for authentication") + sys.exit(1 if arg_error else 0) + + if samples is None: + samples = int(loop_time) if loop_time > 0 else samples_default + + logging.basicConfig(format="%(levelname)s: %(message)s") + + class GlobalState: + pass + + gstate = GlobalState() + gstate.dish_id = None + gstate.points = [] + + def conn_error(msg): + # Connection errors that happen in an interval loop are not critical + # failures, but are interesting enough to print in non-verbose mode. + if loop_time > 0: + print(msg) + else: + logging.error(msg) + + def flush_points(client): + try: + client.write_points(gstate.points, retention_policy=rp) + if verbose: + print("Data points written: " + str(len(gstate.points))) + gstate.points.clear() + except Exception as e: + conn_error("Failed writing to InfluxDB database: " + str(e)) + return 1 + + return 0 + + def loop_body(client): + if gstate.dish_id is None: + try: + gstate.dish_id = starlink_grpc.get_id() + if verbose: + print("Using dish ID: " + gstate.dish_id) + except starlink_grpc.GrpcError as e: + conn_error("Failure getting dish ID: " + str(e)) + return 1 + + timestamp = datetime.datetime.utcnow() + + try: + g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(samples, verbose) + except starlink_grpc.GrpcError as e: + conn_error("Failure getting ping stats: " + str(e)) + return 1 + + all_stats = g_stats.copy() + all_stats.update(pd_stats) + if run_lengths: + for k, v in rl_stats.items(): + if k.startswith("run_"): + for i, subv in enumerate(v, start=1): + all_stats[k + "_" + str(i)] = subv + else: + all_stats[k] = v + + gstate.points.append({ + "measurement": "spacex.starlink.user_terminal.ping_stats", + "tags": { + "id": gstate.dish_id + }, + "time": timestamp, + "fields": all_stats, + }) + if verbose: + print("Data points queued: " + str(len(gstate.points))) + + if len(gstate.points) >= flush_limit: + return flush_points(client) + + return 0 + + if "verify_ssl" in icargs and not icargs["verify_ssl"]: + # user has explicitly said be insecure, so don't warn about it + warnings.filterwarnings("ignore", message="Unverified HTTPS request") + + influx_client = InfluxDBClient(**icargs) + try: + next_loop = time.monotonic() + while True: + rc = loop_body(influx_client) + if loop_time > 0: + now = time.monotonic() + next_loop = max(next_loop + loop_time, now) + time.sleep(next_loop - now) + else: + break + finally: + if gstate.points: + rc = flush_points(influx_client) + influx_client.close() + + sys.exit(rc) + + +if __name__ == '__main__': + main() diff --git a/dishHistoryMqtt.py b/dishHistoryMqtt.py index e9267cc..1e7b855 100644 --- a/dishHistoryMqtt.py +++ b/dishHistoryMqtt.py @@ -10,9 +10,10 @@ # ###################################################################### -import sys import getopt import logging +import sys +import time try: import ssl @@ -24,111 +25,161 @@ import paho.mqtt.publish import starlink_grpc -arg_error = False -try: - opts, args = getopt.getopt(sys.argv[1:], "ahn:p:rs:vC:ISP:U:") -except getopt.GetoptError as err: - print(str(err)) - arg_error = True +def main(): + arg_error = False -# Default to 1 hour worth of data samples. -samples_default = 3600 -samples = samples_default -print_usage = False -verbose = False -run_lengths = False -host_default = "localhost" -mqargs = {"hostname": host_default} -username = None -password = None - -if not arg_error: - if len(args) > 0: + try: + opts, args = getopt.getopt(sys.argv[1:], "ahn:p:rs:t:vC:ISP:U:") + except getopt.GetoptError as err: + print(str(err)) arg_error = True - else: - for opt, arg in opts: - if opt == "-a": - samples = -1 - elif opt == "-h": - print_usage = True - elif opt == "-n": - mqargs["hostname"] = arg - elif opt == "-p": - mqargs["port"] = int(arg) - elif opt == "-r": - run_lengths = True - elif opt == "-s": - samples = int(arg) - elif opt == "-v": - verbose = True - elif opt == "-C": - mqargs["tls"] = {"ca_certs": arg} - elif opt == "-I": - if ssl_ok: - mqargs["tls"] = {"cert_reqs": ssl.CERT_NONE} - else: - print("No SSL support found") - sys.exit(1) - elif opt == "-P": - password = arg - elif opt == "-S": - mqargs["tls"] = {} - elif opt == "-U": - username = arg -if username is None and password is not None: - print("Password authentication requires username to be set") - arg_error = True + # Default to 1 hour worth of data samples. + samples_default = 3600 + samples = None + print_usage = False + verbose = False + default_loop_time = 0 + loop_time = default_loop_time + run_lengths = False + host_default = "localhost" + mqargs = {"hostname": host_default} + username = None + password = None -if print_usage or arg_error: - print("Usage: " + sys.argv[0] + " [options...]") - print("Options:") - print(" -a: Parse all valid samples") - print(" -h: Be helpful") - print(" -n : Hostname of MQTT broker, default: " + host_default) - print(" -p : Port number to use on MQTT broker") - print(" -r: Include ping drop run length stats") - print(" -s : Number of data samples to parse, default: " + str(samples_default)) - print(" -v: Be verbose") - print(" -C : Enable SSL/TLS using specified CA cert to verify broker") - print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") - print(" -P: Set password for username/password authentication") - print(" -S: Enable SSL/TLS using default CA cert") - print(" -U: Set username for authentication") - sys.exit(1 if arg_error else 0) - -logging.basicConfig(format="%(levelname)s: %(message)s") - -try: - dish_id = starlink_grpc.get_id() -except starlink_grpc.GrpcError as e: - logging.error("Failure getting dish ID: " + str(e)) - sys.exit(1) - -try: - g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(samples, verbose) -except starlink_grpc.GrpcError as e: - logging.error("Failure getting ping stats: " + str(e)) - sys.exit(1) - -topic_prefix = "starlink/dish_ping_stats/" + dish_id + "/" -msgs = [(topic_prefix + k, v, 0, False) for k, v in g_stats.items()] -msgs.extend([(topic_prefix + k, v, 0, False) for k, v in pd_stats.items()]) -if run_lengths: - for k, v in rl_stats.items(): - if k.startswith("run_"): - msgs.append((topic_prefix + k, ",".join(str(x) for x in v), 0, False)) + if not arg_error: + if len(args) > 0: + arg_error = True else: - msgs.append((topic_prefix + k, v, 0, False)) + for opt, arg in opts: + if opt == "-a": + samples = -1 + elif opt == "-h": + print_usage = True + elif opt == "-n": + mqargs["hostname"] = arg + elif opt == "-p": + mqargs["port"] = int(arg) + elif opt == "-r": + run_lengths = True + elif opt == "-s": + samples = int(arg) + elif opt == "-t": + loop_time = float(arg) + elif opt == "-v": + verbose = True + elif opt == "-C": + mqargs["tls"] = {"ca_certs": arg} + elif opt == "-I": + if ssl_ok: + mqargs["tls"] = {"cert_reqs": ssl.CERT_NONE} + else: + print("No SSL support found") + sys.exit(1) + elif opt == "-P": + password = arg + elif opt == "-S": + mqargs["tls"] = {} + elif opt == "-U": + username = arg -if username is not None: - mqargs["auth"] = {"username": username} - if password is not None: - mqargs["auth"]["password"] = password + if username is None and password is not None: + print("Password authentication requires username to be set") + arg_error = True -try: - paho.mqtt.publish.multiple(msgs, client_id=dish_id, **mqargs) -except Exception as e: - logging.error("Failed publishing to MQTT broker: " + str(e)) - sys.exit(1) + if print_usage or arg_error: + print("Usage: " + sys.argv[0] + " [options...]") + print("Options:") + print(" -a: Parse all valid samples") + print(" -h: Be helpful") + print(" -n : Hostname of MQTT broker, default: " + host_default) + print(" -p : Port number to use on MQTT broker") + print(" -r: Include ping drop run length stats") + print(" -s : Number of data samples to parse, default: loop interval,") + print(" if set, else " + str(samples_default)) + print(" -t : Loop interval in seconds or 0 for no loop, default: " + + str(default_loop_time)) + print(" -v: Be verbose") + print(" -C : Enable SSL/TLS using specified CA cert to verify broker") + print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") + print(" -P: Set password for username/password authentication") + print(" -S: Enable SSL/TLS using default CA cert") + print(" -U: Set username for authentication") + sys.exit(1 if arg_error else 0) + + if samples is None: + samples = int(loop_time) if loop_time > 0 else samples_default + + if username is not None: + mqargs["auth"] = {"username": username} + if password is not None: + mqargs["auth"]["password"] = password + + logging.basicConfig(format="%(levelname)s: %(message)s") + + class GlobalState: + pass + + gstate = GlobalState() + gstate.dish_id = None + + def conn_error(msg): + # Connection errors that happen in an interval loop are not critical + # failures, but are interesting enough to print in non-verbose mode. + if loop_time > 0: + print(msg) + else: + logging.error(msg) + + def loop_body(): + if gstate.dish_id is None: + try: + gstate.dish_id = starlink_grpc.get_id() + if verbose: + print("Using dish ID: " + gstate.dish_id) + except starlink_grpc.GrpcError as e: + conn_error("Failure getting dish ID: " + str(e)) + return 1 + + try: + g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(samples, verbose) + except starlink_grpc.GrpcError as e: + conn_error("Failure getting ping stats: " + str(e)) + return 1 + + topic_prefix = "starlink/dish_ping_stats/" + gstate.dish_id + "/" + msgs = [(topic_prefix + k, v, 0, False) for k, v in g_stats.items()] + msgs.extend([(topic_prefix + k, v, 0, False) for k, v in pd_stats.items()]) + if run_lengths: + for k, v in rl_stats.items(): + if k.startswith("run_"): + msgs.append((topic_prefix + k, ",".join(str(x) for x in v), 0, False)) + else: + msgs.append((topic_prefix + k, v, 0, False)) + + try: + paho.mqtt.publish.multiple(msgs, client_id=gstate.dish_id, **mqargs) + if verbose: + print("Successfully published to MQTT broker") + except Exception as e: + conn_error("Failed publishing to MQTT broker: " + str(e)) + return 1 + + return 0 + + next_loop = time.monotonic() + while True: + rc = loop_body() + if loop_time > 0: + now = time.monotonic() + next_loop = max(next_loop + loop_time, now) + time.sleep(next_loop - now) + else: + break + + sys.exit(rc) + + +if __name__ == '__main__': + main() diff --git a/dishHistoryStats.py b/dishHistoryStats.py index 683f490..f08ac84 100644 --- a/dishHistoryStats.py +++ b/dishHistoryStats.py @@ -11,105 +11,141 @@ ###################################################################### import datetime -import sys import getopt import logging +import sys +import time import starlink_grpc -arg_error = False -try: - opts, args = getopt.getopt(sys.argv[1:], "ahrs:vH") -except getopt.GetoptError as err: - print(str(err)) - arg_error = True +def main(): + arg_error = False -# Default to 1 hour worth of data samples. -samples_default = 3600 -samples = samples_default -print_usage = False -verbose = False -print_header = False -run_lengths = False - -if not arg_error: - if len(args) > 0: + try: + opts, args = getopt.getopt(sys.argv[1:], "ahrs:t:vH") + except getopt.GetoptError as err: + print(str(err)) arg_error = True - else: - for opt, arg in opts: - if opt == "-a": - samples = -1 - elif opt == "-h": - print_usage = True - elif opt == "-r": - run_lengths = True - elif opt == "-s": - samples = int(arg) - elif opt == "-v": - verbose = True - elif opt == "-H": - print_header = True -if print_usage or arg_error: - print("Usage: " + sys.argv[0] + " [options...]") - print("Options:") - print(" -a: Parse all valid samples") - print(" -h: Be helpful") - print(" -r: Include ping drop run length stats") - print(" -s : Number of data samples to parse, default: " + str(samples_default)) - print(" -v: Be verbose") - print(" -H: print CSV header instead of parsing file") - sys.exit(1 if arg_error else 0) + # Default to 1 hour worth of data samples. + samples_default = 3600 + samples = None + print_usage = False + verbose = False + default_loop_time = 0 + loop_time = default_loop_time + run_lengths = False + print_header = False -logging.basicConfig(format="%(levelname)s: %(message)s") + if not arg_error: + if len(args) > 0: + arg_error = True + else: + for opt, arg in opts: + if opt == "-a": + samples = -1 + elif opt == "-h": + print_usage = True + elif opt == "-r": + run_lengths = True + elif opt == "-s": + samples = int(arg) + elif opt == "-t": + loop_time = float(arg) + elif opt == "-v": + verbose = True + elif opt == "-H": + print_header = True -g_fields, pd_fields, rl_fields = starlink_grpc.history_ping_field_names() + if print_usage or arg_error: + print("Usage: " + sys.argv[0] + " [options...]") + print("Options:") + print(" -a: Parse all valid samples") + print(" -h: Be helpful") + print(" -r: Include ping drop run length stats") + print(" -s : Number of data samples to parse, default: loop interval,") + print(" if set, else " + str(samples_default)) + print(" -t : Loop interval in seconds or 0 for no loop, default: " + + str(default_loop_time)) + print(" -v: Be verbose") + print(" -H: print CSV header instead of parsing history data") + sys.exit(1 if arg_error else 0) -if print_header: - header = ["datetimestamp_utc"] - header.extend(g_fields) - header.extend(pd_fields) - if run_lengths: - for field in rl_fields: - if field.startswith("run_"): - header.extend(field + "_" + str(x) for x in range(1, 61)) - else: - header.append(field) - print(",".join(header)) - sys.exit(0) + if samples is None: + samples = int(loop_time) if loop_time > 0 else samples_default -timestamp = datetime.datetime.utcnow() + logging.basicConfig(format="%(levelname)s: %(message)s") -try: - g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(samples, verbose) -except starlink_grpc.GrpcError as e: - logging.error("Failure getting ping stats: " + str(e)) - sys.exit(1) + g_fields, pd_fields, rl_fields = starlink_grpc.history_ping_field_names() -if verbose: - print("Parsed samples: " + str(g_stats["samples"])) - print("Total ping drop: " + str(pd_stats["total_ping_drop"])) - print("Count of drop == 1: " + str(pd_stats["count_full_ping_drop"])) - print("Obstructed: " + str(pd_stats["count_obstructed"])) - print("Obstructed ping drop: " + str(pd_stats["total_obstructed_ping_drop"])) - print("Obstructed drop == 1: " + str(pd_stats["count_full_obstructed_ping_drop"])) - print("Unscheduled: " + str(pd_stats["count_unscheduled"])) - print("Unscheduled ping drop: " + str(pd_stats["total_unscheduled_ping_drop"])) - print("Unscheduled drop == 1: " + str(pd_stats["count_full_unscheduled_ping_drop"])) - if run_lengths: - print("Initial drop run fragment: " + str(rl_stats["init_run_fragment"])) - print("Final drop run fragment: " + str(rl_stats["final_run_fragment"])) - print("Per-second drop runs: " + ", ".join(str(x) for x in rl_stats["run_seconds"])) - print("Per-minute drop runs: " + ", ".join(str(x) for x in rl_stats["run_minutes"])) -else: - csv_data = [timestamp.replace(microsecond=0).isoformat()] - csv_data.extend(str(g_stats[field]) for field in g_fields) - csv_data.extend(str(pd_stats[field]) for field in pd_fields) - if run_lengths: - for field in rl_fields: - if field.startswith("run_"): - csv_data.extend(str(substat) for substat in rl_stats[field]) - else: - csv_data.append(str(rl_stats[field])) - print(",".join(csv_data)) + if print_header: + header = ["datetimestamp_utc"] + header.extend(g_fields) + header.extend(pd_fields) + if run_lengths: + for field in rl_fields: + if field.startswith("run_"): + header.extend(field + "_" + str(x) for x in range(1, 61)) + else: + header.append(field) + print(",".join(header)) + sys.exit(0) + + def loop_body(): + timestamp = datetime.datetime.utcnow() + + try: + g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(samples, verbose) + except starlink_grpc.GrpcError as e: + logging.error("Failure getting ping stats: " + str(e)) + return 1 + + if verbose: + print("Parsed samples: " + str(g_stats["samples"])) + print("Total ping drop: " + str(pd_stats["total_ping_drop"])) + print("Count of drop == 1: " + str(pd_stats["count_full_ping_drop"])) + print("Obstructed: " + str(pd_stats["count_obstructed"])) + print("Obstructed ping drop: " + str(pd_stats["total_obstructed_ping_drop"])) + print("Obstructed drop == 1: " + str(pd_stats["count_full_obstructed_ping_drop"])) + print("Unscheduled: " + str(pd_stats["count_unscheduled"])) + print("Unscheduled ping drop: " + str(pd_stats["total_unscheduled_ping_drop"])) + print("Unscheduled drop == 1: " + str(pd_stats["count_full_unscheduled_ping_drop"])) + if run_lengths: + print("Initial drop run fragment: " + str(rl_stats["init_run_fragment"])) + print("Final drop run fragment: " + str(rl_stats["final_run_fragment"])) + print("Per-second drop runs: " + + ", ".join(str(x) for x in rl_stats["run_seconds"])) + print("Per-minute drop runs: " + + ", ".join(str(x) for x in rl_stats["run_minutes"])) + if loop_time > 0: + print() + else: + csv_data = [timestamp.replace(microsecond=0).isoformat()] + csv_data.extend(str(g_stats[field]) for field in g_fields) + csv_data.extend(str(pd_stats[field]) for field in pd_fields) + if run_lengths: + for field in rl_fields: + if field.startswith("run_"): + csv_data.extend(str(substat) for substat in rl_stats[field]) + else: + csv_data.append(str(rl_stats[field])) + print(",".join(csv_data)) + + return 0 + + next_loop = time.monotonic() + while True: + rc = loop_body() + if loop_time > 0: + now = time.monotonic() + next_loop = max(next_loop + loop_time, now) + time.sleep(next_loop - now) + else: + break + + sys.exit(rc) + + +if __name__ == '__main__': + main() diff --git a/dishStatusCsv.py b/dishStatusCsv.py index c8b7968..55443b5 100644 --- a/dishStatusCsv.py +++ b/dishStatusCsv.py @@ -1,111 +1,147 @@ #!/usr/bin/python3 ###################################################################### # -# Output get_status info in CSV format. +# Output Starlink user terminal status info in CSV format. # -# This script pulls the current status once and prints to stdout. +# This script pulls the current status and prints to stdout either +# once or in a periodic loop. # ###################################################################### import datetime -import sys import getopt import logging +import sys +import time import grpc import spacex.api.device.device_pb2 import spacex.api.device.device_pb2_grpc -arg_error = False -try: - opts, args = getopt.getopt(sys.argv[1:], "hH") -except getopt.GetoptError as err: - print(str(err)) - arg_error = True +def main(): + arg_error = False -print_usage = False -print_header = False - -if not arg_error: - if len(args) > 0: + try: + opts, args = getopt.getopt(sys.argv[1:], "ht:H") + except getopt.GetoptError as err: + print(str(err)) arg_error = True - else: - for opt, arg in opts: - if opt == "-h": - print_usage = True - elif opt == "-H": - print_header = True -if print_usage or arg_error: - print("Usage: " + sys.argv[0] + " [options...]") - print("Options:") - print(" -h: Be helpful") - print(" -H: print CSV header instead of parsing file") - sys.exit(1 if arg_error else 0) + print_usage = False + default_loop_time = 0 + loop_time = default_loop_time + print_header = False -logging.basicConfig(format="%(levelname)s: %(message)s") + if not arg_error: + if len(args) > 0: + arg_error = True + else: + for opt, arg in opts: + if opt == "-h": + print_usage = True + elif opt == "-t": + loop_time = float(arg) + elif opt == "-H": + print_header = True -if print_header: - header = [ - "datetimestamp_utc", - "hardware_version", - "software_version", - "state", - "uptime", - "snr", - "seconds_to_first_nonempty_slot", - "pop_ping_drop_rate", - "downlink_throughput_bps", - "uplink_throughput_bps", - "pop_ping_latency_ms", - "alerts", - "fraction_obstructed", - "currently_obstructed", - "seconds_obstructed" - ] - header.extend("wedges_fraction_obstructed_" + str(x) for x in range(12)) - print(",".join(header)) - sys.exit(0) + if print_usage or arg_error: + print("Usage: " + sys.argv[0] + " [options...]") + print("Options:") + print(" -h: Be helpful") + print(" -t : Loop interval in seconds or 0 for no loop, default: " + + str(default_loop_time)) + print(" -H: print CSV header instead of parsing file") + sys.exit(1 if arg_error else 0) -try: - with grpc.insecure_channel("192.168.100.1:9200") as channel: - stub = spacex.api.device.device_pb2_grpc.DeviceStub(channel) - response = stub.Handle(spacex.api.device.device_pb2.Request(get_status={})) -except grpc.RpcError: - logging.error("Failed getting status info") - sys.exit(1) + logging.basicConfig(format="%(levelname)s: %(message)s") -timestamp = datetime.datetime.utcnow() + if print_header: + header = [ + "datetimestamp_utc", + "hardware_version", + "software_version", + "state", + "uptime", + "snr", + "seconds_to_first_nonempty_slot", + "pop_ping_drop_rate", + "downlink_throughput_bps", + "uplink_throughput_bps", + "pop_ping_latency_ms", + "alerts", + "fraction_obstructed", + "currently_obstructed", + "seconds_obstructed", + ] + header.extend("wedges_fraction_obstructed_" + str(x) for x in range(12)) + print(",".join(header)) + sys.exit(0) -status = response.dish_get_status + def loop_body(): + timestamp = datetime.datetime.utcnow() -# More alerts may be added in future, so rather than list them individually, -# build a bit field based on field numbers of the DishAlerts message. -alert_bits = 0 -for alert in status.alerts.ListFields(): - alert_bits |= (1 if alert[1] else 0) << (alert[0].number - 1) + try: + with grpc.insecure_channel("192.168.100.1:9200") as channel: + stub = spacex.api.device.device_pb2_grpc.DeviceStub(channel) + response = stub.Handle(spacex.api.device.device_pb2.Request(get_status={})) -csv_data = [ - timestamp.replace(microsecond=0).isoformat(), - status.device_info.id, - status.device_info.hardware_version, - status.device_info.software_version, - spacex.api.device.dish_pb2.DishState.Name(status.state) -] -csv_data.extend(str(x) for x in [ - status.device_state.uptime_s, - status.snr, - status.seconds_to_first_nonempty_slot, - status.pop_ping_drop_rate, - status.downlink_throughput_bps, - status.uplink_throughput_bps, - status.pop_ping_latency_ms, - alert_bits, - status.obstruction_stats.fraction_obstructed, - status.obstruction_stats.currently_obstructed, - status.obstruction_stats.last_24h_obstructed_s -]) -csv_data.extend(str(x) for x in status.obstruction_stats.wedge_abs_fraction_obstructed) -print(",".join(csv_data)) + status = response.dish_get_status + + # More alerts may be added in future, so rather than list them individually, + # build a bit field based on field numbers of the DishAlerts message. + alert_bits = 0 + for alert in status.alerts.ListFields(): + alert_bits |= (1 if alert[1] else 0) << (alert[0].number - 1) + + csv_data = [ + timestamp.replace(microsecond=0).isoformat(), + status.device_info.id, + status.device_info.hardware_version, + status.device_info.software_version, + spacex.api.device.dish_pb2.DishState.Name(status.state), + ] + csv_data.extend( + str(x) for x in [ + status.device_state.uptime_s, + status.snr, + status.seconds_to_first_nonempty_slot, + status.pop_ping_drop_rate, + status.downlink_throughput_bps, + status.uplink_throughput_bps, + status.pop_ping_latency_ms, + alert_bits, + status.obstruction_stats.fraction_obstructed, + status.obstruction_stats.currently_obstructed, + status.obstruction_stats.last_24h_obstructed_s, + ]) + csv_data.extend(str(x) for x in status.obstruction_stats.wedge_abs_fraction_obstructed) + rc = 0 + except grpc.RpcError: + if loop_time <= 0: + logging.error("Failed getting status info") + csv_data = [ + timestamp.replace(microsecond=0).isoformat(), "", "", "", "DISH_UNREACHABLE" + ] + rc = 1 + + print(",".join(csv_data)) + + return rc + + next_loop = time.monotonic() + while True: + rc = loop_body() + if loop_time > 0: + now = time.monotonic() + next_loop = max(next_loop + loop_time, now) + time.sleep(next_loop - now) + else: + break + + sys.exit(rc) + + +if __name__ == '__main__': + main() diff --git a/dishStatusInflux.py b/dishStatusInflux.py index 98c5d01..0c00b41 100644 --- a/dishStatusInflux.py +++ b/dishStatusInflux.py @@ -8,249 +8,263 @@ # ###################################################################### -import time -import os -import sys import getopt import logging +import os +import sys +import time import warnings +import grpc from influxdb import InfluxDBClient from influxdb import SeriesHelper -import grpc - import spacex.api.device.device_pb2 import spacex.api.device.device_pb2_grpc -arg_error = False -try: - opts, args = getopt.getopt(sys.argv[1:], "hn:p:t:vC:D:IP:R:SU:") -except getopt.GetoptError as err: - print(str(err)) - arg_error = True +def main(): + arg_error = False -print_usage = False -verbose = False -host_default = "localhost" -database_default = "starlinkstats" -icargs = {"host": host_default, "timeout": 5, "database": database_default} -rp = None -default_sleep_time = 30 -sleep_time = default_sleep_time - -# For each of these check they are both set and not empty string -influxdb_host = os.environ.get("INFLUXDB_HOST") -if influxdb_host: - icargs["host"] = influxdb_host -influxdb_port = os.environ.get("INFLUXDB_PORT") -if influxdb_port: - icargs["port"] = int(influxdb_port) -influxdb_user = os.environ.get("INFLUXDB_USER") -if influxdb_user: - icargs["username"] = influxdb_user -influxdb_pwd = os.environ.get("INFLUXDB_PWD") -if influxdb_pwd: - icargs["password"] = influxdb_pwd -influxdb_db = os.environ.get("INFLUXDB_DB") -if influxdb_db: - icargs["database"] = influxdb_db -influxdb_rp = os.environ.get("INFLUXDB_RP") -if influxdb_rp: - rp = influxdb_rp -influxdb_ssl = os.environ.get("INFLUXDB_SSL") -if influxdb_ssl: - icargs["ssl"] = True - if influxdb_ssl.lower() == "secure": - icargs["verify_ssl"] = True - elif influxdb_ssl.lower() == "insecure": - icargs["verify_ssl"] = False - else: - icargs["verify_ssl"] = influxdb_ssl - -if not arg_error: - if len(args) > 0: + try: + opts, args = getopt.getopt(sys.argv[1:], "hn:p:t:vC:D:IP:R:SU:") + except getopt.GetoptError as err: + print(str(err)) arg_error = True - else: - for opt, arg in opts: - if opt == "-h": - print_usage = True - elif opt == "-n": - icargs["host"] = arg - elif opt == "-p": - icargs["port"] = int(arg) - elif opt == "-t": - sleep_time = int(arg) - elif opt == "-v": - verbose = True - elif opt == "-C": - icargs["ssl"] = True - icargs["verify_ssl"] = arg - elif opt == "-D": - icargs["database"] = arg - elif opt == "-I": - icargs["ssl"] = True - icargs["verify_ssl"] = False - elif opt == "-P": - icargs["password"] = arg - elif opt == "-R": - rp = arg - elif opt == "-S": - icargs["ssl"] = True - icargs["verify_ssl"] = True - elif opt == "-U": - icargs["username"] = arg -if "password" in icargs and "username" not in icargs: - print("Password authentication requires username to be set") - arg_error = True + print_usage = False + verbose = False + default_loop_time = 0 + loop_time = default_loop_time + host_default = "localhost" + database_default = "starlinkstats" + icargs = {"host": host_default, "timeout": 5, "database": database_default} + rp = None + flush_limit = 6 -if print_usage or arg_error: - print("Usage: " + sys.argv[0] + " [options...]") - print("Options:") - print(" -h: Be helpful") - print(" -n : Hostname of InfluxDB server, default: " + host_default) - print(" -p : Port number to use on InfluxDB server") - print(" -t : Loop interval in seconds or 0 for no loop, default: " + - str(default_sleep_time)) - print(" -v: Be verbose") - print(" -C : Enable SSL/TLS using specified CA cert to verify server") - print(" -D : Database name to use, default: " + database_default) - print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") - print(" -P : Set password for authentication") - print(" -R : Retention policy name to use") - print(" -S: Enable SSL/TLS using default CA cert") - print(" -U : Set username for authentication") - sys.exit(1 if arg_error else 0) + # For each of these check they are both set and not empty string + influxdb_host = os.environ.get("INFLUXDB_HOST") + if influxdb_host: + icargs["host"] = influxdb_host + influxdb_port = os.environ.get("INFLUXDB_PORT") + if influxdb_port: + icargs["port"] = int(influxdb_port) + influxdb_user = os.environ.get("INFLUXDB_USER") + if influxdb_user: + icargs["username"] = influxdb_user + influxdb_pwd = os.environ.get("INFLUXDB_PWD") + if influxdb_pwd: + icargs["password"] = influxdb_pwd + influxdb_db = os.environ.get("INFLUXDB_DB") + if influxdb_db: + icargs["database"] = influxdb_db + influxdb_rp = os.environ.get("INFLUXDB_RP") + if influxdb_rp: + rp = influxdb_rp + influxdb_ssl = os.environ.get("INFLUXDB_SSL") + if influxdb_ssl: + icargs["ssl"] = True + if influxdb_ssl.lower() == "secure": + icargs["verify_ssl"] = True + elif influxdb_ssl.lower() == "insecure": + icargs["verify_ssl"] = False + else: + icargs["verify_ssl"] = influxdb_ssl -logging.basicConfig(format="%(levelname)s: %(message)s") + if not arg_error: + if len(args) > 0: + arg_error = True + else: + for opt, arg in opts: + if opt == "-h": + print_usage = True + elif opt == "-n": + icargs["host"] = arg + elif opt == "-p": + icargs["port"] = int(arg) + elif opt == "-t": + loop_time = int(arg) + elif opt == "-v": + verbose = True + elif opt == "-C": + icargs["ssl"] = True + icargs["verify_ssl"] = arg + elif opt == "-D": + icargs["database"] = arg + elif opt == "-I": + icargs["ssl"] = True + icargs["verify_ssl"] = False + elif opt == "-P": + icargs["password"] = arg + elif opt == "-R": + rp = arg + elif opt == "-S": + icargs["ssl"] = True + icargs["verify_ssl"] = True + elif opt == "-U": + icargs["username"] = arg -def conn_error(msg): - # Connection errors that happen while running in an interval loop are - # not critical failures, because they can (usually) be retried, or - # because they will be recorded as dish state unavailable. They're still - # interesting, though, so print them even in non-verbose mode. - if sleep_time > 0: - print(msg) - else: - logging.error(msg) + if "password" in icargs and "username" not in icargs: + print("Password authentication requires username to be set") + arg_error = True -class DeviceStatusSeries(SeriesHelper): - class Meta: - series_name = "spacex.starlink.user_terminal.status" - fields = [ - "hardware_version", - "software_version", - "state", - "alert_motors_stuck", - "alert_thermal_throttle", - "alert_thermal_shutdown", - "alert_unexpected_location", - "snr", - "seconds_to_first_nonempty_slot", - "pop_ping_drop_rate", - "downlink_throughput_bps", - "uplink_throughput_bps", - "pop_ping_latency_ms", - "currently_obstructed", - "fraction_obstructed"] - tags = ["id"] - retention_policy = rp + if print_usage or arg_error: + print("Usage: " + sys.argv[0] + " [options...]") + print("Options:") + print(" -h: Be helpful") + print(" -n : Hostname of InfluxDB server, default: " + host_default) + print(" -p : Port number to use on InfluxDB server") + print(" -t : Loop interval in seconds or 0 for no loop, default: " + + str(default_loop_time)) + print(" -v: Be verbose") + print(" -C : Enable SSL/TLS using specified CA cert to verify server") + print(" -D : Database name to use, default: " + database_default) + print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") + print(" -P : Set password for authentication") + print(" -R : Retention policy name to use") + print(" -S: Enable SSL/TLS using default CA cert") + print(" -U : Set username for authentication") + sys.exit(1 if arg_error else 0) -if "verify_ssl" in icargs and not icargs["verify_ssl"]: - # user has explicitly said be insecure, so don't warn about it - warnings.filterwarnings("ignore", message="Unverified HTTPS request") + logging.basicConfig(format="%(levelname)s: %(message)s") -influx_client = InfluxDBClient(**icargs) + class GlobalState: + pass -rc = 0 -try: - dish_channel = None - last_id = None - last_failed = False + gstate = GlobalState() + gstate.dish_channel = None + gstate.dish_id = None + gstate.pending = 0 - pending = 0 - count = 0 - while True: + class DeviceStatusSeries(SeriesHelper): + class Meta: + series_name = "spacex.starlink.user_terminal.status" + fields = [ + "hardware_version", + "software_version", + "state", + "alert_motors_stuck", + "alert_thermal_throttle", + "alert_thermal_shutdown", + "alert_unexpected_location", + "snr", + "seconds_to_first_nonempty_slot", + "pop_ping_drop_rate", + "downlink_throughput_bps", + "uplink_throughput_bps", + "pop_ping_latency_ms", + "currently_obstructed", + "fraction_obstructed", + ] + tags = ["id"] + retention_policy = rp + + def conn_error(msg): + # Connection errors that happen in an interval loop are not critical + # failures, but are interesting enough to print in non-verbose mode. + if loop_time > 0: + print(msg) + else: + logging.error(msg) + + def flush_pending(client): try: - if dish_channel is None: - dish_channel = grpc.insecure_channel("192.168.100.1:9200") - stub = spacex.api.device.device_pb2_grpc.DeviceStub(dish_channel) - response = stub.Handle(spacex.api.device.device_pb2.Request(get_status={})) - status = response.dish_get_status - DeviceStatusSeries( - id=status.device_info.id, - hardware_version=status.device_info.hardware_version, - software_version=status.device_info.software_version, - state=spacex.api.device.dish_pb2.DishState.Name(status.state), - alert_motors_stuck=status.alerts.motors_stuck, - alert_thermal_throttle=status.alerts.thermal_throttle, - alert_thermal_shutdown=status.alerts.thermal_shutdown, - alert_unexpected_location=status.alerts.unexpected_location, - snr=status.snr, - seconds_to_first_nonempty_slot=status.seconds_to_first_nonempty_slot, - pop_ping_drop_rate=status.pop_ping_drop_rate, - downlink_throughput_bps=status.downlink_throughput_bps, - uplink_throughput_bps=status.uplink_throughput_bps, - pop_ping_latency_ms=status.pop_ping_latency_ms, - currently_obstructed=status.obstruction_stats.currently_obstructed, - fraction_obstructed=status.obstruction_stats.fraction_obstructed) - pending += 1 - last_id = status.device_info.id - last_failed = False - except grpc.RpcError: - if dish_channel is not None: - dish_channel.close() - dish_channel = None - if last_failed: - if last_id is None: - conn_error("Dish unreachable and ID unknown, so not recording state") - # When not looping, report this as failure exit status - rc = 1 - else: + DeviceStatusSeries.commit(client) + if verbose: + print("Data points written: " + str(gstate.pending)) + gstate.pending = 0 + except Exception as e: + conn_error("Failed writing to InfluxDB database: " + str(e)) + return 1 + + return 0 + + def get_status_retry(): + """Try getting the status at most twice""" + + channel_reused = True + while True: + try: + if gstate.dish_channel is None: + gstate.dish_channel = grpc.insecure_channel("192.168.100.1:9200") + channel_reused = False + stub = spacex.api.device.device_pb2_grpc.DeviceStub(gstate.dish_channel) + response = stub.Handle(spacex.api.device.device_pb2.Request(get_status={})) + return response.dish_get_status + except grpc.RpcError: + gstate.dish_channel.close() + gstate.dish_channel = None + if channel_reused: + # If the channel was open already, the connection may have + # been lost in the time since prior loop iteration, so after + # closing it, retry once, in case the dish is now reachable. if verbose: - print("Dish unreachable") - DeviceStatusSeries(id=last_id, state="DISH_UNREACHABLE") - pending += 1 + print("Dish RPC channel error") + else: + raise + + def loop_body(client): + try: + status = get_status_retry() + DeviceStatusSeries(id=status.device_info.id, + hardware_version=status.device_info.hardware_version, + software_version=status.device_info.software_version, + state=spacex.api.device.dish_pb2.DishState.Name(status.state), + alert_motors_stuck=status.alerts.motors_stuck, + alert_thermal_throttle=status.alerts.thermal_throttle, + alert_thermal_shutdown=status.alerts.thermal_shutdown, + alert_unexpected_location=status.alerts.unexpected_location, + snr=status.snr, + seconds_to_first_nonempty_slot=status.seconds_to_first_nonempty_slot, + pop_ping_drop_rate=status.pop_ping_drop_rate, + downlink_throughput_bps=status.downlink_throughput_bps, + uplink_throughput_bps=status.uplink_throughput_bps, + pop_ping_latency_ms=status.pop_ping_latency_ms, + currently_obstructed=status.obstruction_stats.currently_obstructed, + fraction_obstructed=status.obstruction_stats.fraction_obstructed) + gstate.dish_id = status.device_info.id + except grpc.RpcError: + if gstate.dish_id is None: + conn_error("Dish unreachable and ID unknown, so not recording state") + return 1 else: if verbose: - print("Dish RPC channel error") - # Retry once, because the connection may have been lost while - # we were sleeping - last_failed = True - continue + print("Dish unreachable") + DeviceStatusSeries(id=gstate.dish_id, state="DISH_UNREACHABLE") + + gstate.pending += 1 if verbose: - print("Samples queued: " + str(pending)) - count += 1 - if count > 5: - try: - if pending: - DeviceStatusSeries.commit(influx_client) - rc = 0 - if verbose: - print("Samples written: " + str(pending)) - pending = 0 - except Exception as e: - conn_error("Failed to write: " + str(e)) - rc = 1 - count = 0 - if sleep_time > 0: - time.sleep(sleep_time) - else: - break -finally: - # Flush on error/exit + print("Data points queued: " + str(gstate.pending)) + if gstate.pending >= flush_limit: + return flush_pending(client) + + return 0 + + if "verify_ssl" in icargs and not icargs["verify_ssl"]: + # user has explicitly said be insecure, so don't warn about it + warnings.filterwarnings("ignore", message="Unverified HTTPS request") + + influx_client = InfluxDBClient(**icargs) try: - if pending: - DeviceStatusSeries.commit(influx_client) - rc = 0 - if verbose: - print("Samples written: " + str(pending)) - except Exception as e: - conn_error("Failed to write: " + str(e)) - rc = 1 - influx_client.close() - if dish_channel is not None: - dish_channel.close() + next_loop = time.monotonic() + while True: + rc = loop_body(influx_client) + if loop_time > 0: + now = time.monotonic() + next_loop = max(next_loop + loop_time, now) + time.sleep(next_loop - now) + else: + break + finally: + # Flush on error/exit + if gstate.pending: + rc = flush_pending(influx_client) + influx_client.close() + if gstate.dish_channel is not None: + gstate.dish_channel.close() + sys.exit(rc) + + +if __name__ == '__main__': + main() diff --git a/dishStatusMqtt.py b/dishStatusMqtt.py index e91763f..ce84ab6 100644 --- a/dishStatusMqtt.py +++ b/dishStatusMqtt.py @@ -3,14 +3,15 @@ # # Publish Starlink user terminal status info to a MQTT broker. # -# This script pulls the current status once and publishes it to the -# specified MQTT broker. +# This script pulls the current status and publishes it to the +# specified MQTT broker either once or in a periodic loop. # ###################################################################### -import sys import getopt import logging +import sys +import time try: import ssl @@ -18,116 +19,170 @@ try: except ImportError: ssl_ok = False -import paho.mqtt.publish - import grpc +import paho.mqtt.publish import spacex.api.device.device_pb2 import spacex.api.device.device_pb2_grpc -arg_error = False -try: - opts, args = getopt.getopt(sys.argv[1:], "hn:p:C:ISP:U:") -except getopt.GetoptError as err: - print(str(err)) - arg_error = True +def main(): + arg_error = False -print_usage = False -host_default = "localhost" -mqargs = {"hostname": host_default} -username = None -password = None - -if not arg_error: - if len(args) > 0: + try: + opts, args = getopt.getopt(sys.argv[1:], "hn:p:t:vC:ISP:U:") + except getopt.GetoptError as err: + print(str(err)) arg_error = True - else: - for opt, arg in opts: - if opt == "-h": - print_usage = True - elif opt == "-n": - mqargs["hostname"] = arg - elif opt == "-p": - mqargs["port"] = int(arg) - elif opt == "-C": - mqargs["tls"] = {"ca_certs": arg} - elif opt == "-I": - if ssl_ok: - mqargs["tls"] = {"cert_reqs": ssl.CERT_NONE} - else: - print("No SSL support found") - sys.exit(1) - elif opt == "-P": - password = arg - elif opt == "-S": - mqargs["tls"] = {} - elif opt == "-U": - username = arg -if username is None and password is not None: - print("Password authentication requires username to be set") - arg_error = True + print_usage = False + verbose = False + default_loop_time = 0 + loop_time = default_loop_time + host_default = "localhost" + mqargs = {"hostname": host_default} + username = None + password = None -if print_usage or arg_error: - print("Usage: " + sys.argv[0] + " [options...]") - print("Options:") - print(" -h: Be helpful") - print(" -n : Hostname of MQTT broker, default: " + host_default) - print(" -p : Port number to use on MQTT broker") - print(" -C : Enable SSL/TLS using specified CA cert to verify broker") - print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") - print(" -P: Set password for username/password authentication") - print(" -S: Enable SSL/TLS using default CA cert") - print(" -U: Set username for authentication") - sys.exit(1 if arg_error else 0) + if not arg_error: + if len(args) > 0: + arg_error = True + else: + for opt, arg in opts: + if opt == "-h": + print_usage = True + elif opt == "-n": + mqargs["hostname"] = arg + elif opt == "-p": + mqargs["port"] = int(arg) + elif opt == "-t": + loop_time = float(arg) + elif opt == "-v": + verbose = True + elif opt == "-C": + mqargs["tls"] = {"ca_certs": arg} + elif opt == "-I": + if ssl_ok: + mqargs["tls"] = {"cert_reqs": ssl.CERT_NONE} + else: + print("No SSL support found") + sys.exit(1) + elif opt == "-P": + password = arg + elif opt == "-S": + mqargs["tls"] = {} + elif opt == "-U": + username = arg -logging.basicConfig(format="%(levelname)s: %(message)s") + if username is None and password is not None: + print("Password authentication requires username to be set") + arg_error = True -try: - with grpc.insecure_channel("192.168.100.1:9200") as channel: - stub = spacex.api.device.device_pb2_grpc.DeviceStub(channel) - response = stub.Handle(spacex.api.device.device_pb2.Request(get_status={})) -except grpc.RpcError: - logging.error("Failed getting status info") - sys.exit(1) + if print_usage or arg_error: + print("Usage: " + sys.argv[0] + " [options...]") + print("Options:") + print(" -h: Be helpful") + print(" -n : Hostname of MQTT broker, default: " + host_default) + print(" -p : Port number to use on MQTT broker") + print(" -t : Loop interval in seconds or 0 for no loop, default: " + + str(default_loop_time)) + print(" -v: Be verbose") + print(" -C : Enable SSL/TLS using specified CA cert to verify broker") + print(" -I: Enable SSL/TLS but disable certificate verification (INSECURE!)") + print(" -P: Set password for username/password authentication") + print(" -S: Enable SSL/TLS using default CA cert") + print(" -U: Set username for authentication") + sys.exit(1 if arg_error else 0) -status = response.dish_get_status + if username is not None: + mqargs["auth"] = {"username": username} + if password is not None: + mqargs["auth"]["password"] = password -# More alerts may be added in future, so rather than list them individually, -# build a bit field based on field numbers of the DishAlerts message. -alert_bits = 0 -for alert in status.alerts.ListFields(): - alert_bits |= (1 if alert[1] else 0) << (alert[0].number - 1) + logging.basicConfig(format="%(levelname)s: %(message)s") -topic_prefix = "starlink/dish_status/" + status.device_info.id + "/" -msgs = [(topic_prefix + "hardware_version", status.device_info.hardware_version, 0, False), - (topic_prefix + "software_version", status.device_info.software_version, 0, False), - (topic_prefix + "state", spacex.api.device.dish_pb2.DishState.Name(status.state), 0, False), - (topic_prefix + "uptime", status.device_state.uptime_s, 0, False), - (topic_prefix + "snr", status.snr, 0, False), - (topic_prefix + "seconds_to_first_nonempty_slot", status.seconds_to_first_nonempty_slot, 0, False), - (topic_prefix + "pop_ping_drop_rate", status.pop_ping_drop_rate, 0, False), - (topic_prefix + "downlink_throughput_bps", status.downlink_throughput_bps, 0, False), - (topic_prefix + "uplink_throughput_bps", status.uplink_throughput_bps, 0, False), - (topic_prefix + "pop_ping_latency_ms", status.pop_ping_latency_ms, 0, False), - (topic_prefix + "alerts", alert_bits, 0, False), - (topic_prefix + "fraction_obstructed", status.obstruction_stats.fraction_obstructed, 0, False), - (topic_prefix + "currently_obstructed", status.obstruction_stats.currently_obstructed, 0, False), - # While the field name for this one implies it covers 24 hours, the - # empirical evidence suggests it only covers 12 hours. It also resets - # on dish reboot, so may not cover that whole period. Rather than try - # to convey that complexity in the topic label, just be a bit vague: - (topic_prefix + "seconds_obstructed", status.obstruction_stats.last_24h_obstructed_s, 0, False), - (topic_prefix + "wedges_fraction_obstructed", ",".join(str(x) for x in status.obstruction_stats.wedge_abs_fraction_obstructed), 0, False)] + class GlobalState: + pass -if username is not None: - mqargs["auth"] = {"username": username} - if password is not None: - mqargs["auth"]["password"] = password + gstate = GlobalState() + gstate.dish_id = None -try: - paho.mqtt.publish.multiple(msgs, client_id=status.device_info.id, **mqargs) -except Exception as e: - logging.error("Failed publishing to MQTT broker: " + str(e)) - sys.exit(1) + def conn_error(msg): + # Connection errors that happen in an interval loop are not critical + # failures, but are interesting enough to print in non-verbose mode. + if loop_time > 0: + print(msg) + else: + logging.error(msg) + + def loop_body(): + try: + with grpc.insecure_channel("192.168.100.1:9200") as channel: + stub = spacex.api.device.device_pb2_grpc.DeviceStub(channel) + response = stub.Handle(spacex.api.device.device_pb2.Request(get_status={})) + + status = response.dish_get_status + + # More alerts may be added in future, so rather than list them individually, + # build a bit field based on field numbers of the DishAlerts message. + alert_bits = 0 + for alert in status.alerts.ListFields(): + alert_bits |= (1 if alert[1] else 0) << (alert[0].number - 1) + + gstate.dish_id = status.device_info.id + topic_prefix = "starlink/dish_status/" + gstate.dish_id + "/" + msgs = [ + (topic_prefix + "hardware_version", status.device_info.hardware_version, 0, False), + (topic_prefix + "software_version", status.device_info.software_version, 0, False), + (topic_prefix + "state", spacex.api.device.dish_pb2.DishState.Name(status.state), 0, False), + (topic_prefix + "uptime", status.device_state.uptime_s, 0, False), + (topic_prefix + "snr", status.snr, 0, False), + (topic_prefix + "seconds_to_first_nonempty_slot", status.seconds_to_first_nonempty_slot, 0, False), + (topic_prefix + "pop_ping_drop_rate", status.pop_ping_drop_rate, 0, False), + (topic_prefix + "downlink_throughput_bps", status.downlink_throughput_bps, 0, False), + (topic_prefix + "uplink_throughput_bps", status.uplink_throughput_bps, 0, False), + (topic_prefix + "pop_ping_latency_ms", status.pop_ping_latency_ms, 0, False), + (topic_prefix + "alerts", alert_bits, 0, False), + (topic_prefix + "fraction_obstructed", status.obstruction_stats.fraction_obstructed, 0, False), + (topic_prefix + "currently_obstructed", status.obstruction_stats.currently_obstructed, 0, False), + # While the field name for this one implies it covers 24 hours, the + # empirical evidence suggests it only covers 12 hours. It also resets + # on dish reboot, so may not cover that whole period. Rather than try + # to convey that complexity in the topic label, just be a bit vague: + (topic_prefix + "seconds_obstructed", status.obstruction_stats.last_24h_obstructed_s, 0, False), + (topic_prefix + "wedges_fraction_obstructed", ",".join(str(x) for x in status.obstruction_stats.wedge_abs_fraction_obstructed), 0, False), + ] + except grpc.RpcError: + if gstate.dish_id is None: + conn_error("Dish unreachable and ID unknown, so not recording state") + return 1 + if verbose: + print("Dish unreachable") + topic_prefix = "starlink/dish_status/" + gstate.dish_id + "/" + msgs = [(topic_prefix + "state", "DISH_UNREACHABLE", 0, False)] + + try: + paho.mqtt.publish.multiple(msgs, client_id=gstate.dish_id, **mqargs) + if verbose: + print("Successfully published to MQTT broker") + except Exception as e: + conn_error("Failed publishing to MQTT broker: " + str(e)) + return 1 + + return 0 + + next_loop = time.monotonic() + while True: + rc = loop_body() + if loop_time > 0: + now = time.monotonic() + next_loop = max(next_loop + loop_time, now) + time.sleep(next_loop - now) + else: + break + + sys.exit(rc) + + +if __name__ == '__main__': + main()