jeff/starlink-grpc-tools
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity
starlink-grpc-tools/dishHistoryStats.py

115 lines
4 KiB
Python
Raw Normal View History

Last of initial set of scripts
2020-12-22 22:16:33 -06:00
#!/usr/bin/python3
######################################################################
#
# Equivalent script to parseJsonHistory.py, except integrating the
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
# gRPC calls, instead of relying on separate invocation of grpcurl.
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
#
# This script examines the most recent samples from the history data
# and computes several different metrics related to packet loss. By
# default, it will print the results in CSV format.
#
######################################################################
import datetime
import sys
import getopt
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
import starlink_grpc
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
arg_error = False
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
try:
Add (optional) run length stats Add reporting of ping drop runs per length of run. Useful(?) for characterizing how long a typical "outage" is.
2020-12-24 16:58:28 -06:00
opts, args = getopt.getopt(sys.argv[1:], "ahrs:vH")
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
except getopt.GetoptError as err:
print(str(err))
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
arg_error = True
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
# Default to 1 hour worth of data samples.
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
samples_default = 3600
samples = samples_default
print_usage = False
verbose = False
parse_all = False
print_header = False
run_lengths = False
if not arg_error:
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
if len(args) > 0:
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
arg_error = True
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
else:
for opt, arg in opts:
if opt == "-a":
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
parse_all = True
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
elif opt == "-h":
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
print_usage = True
Add (optional) run length stats Add reporting of ping drop runs per length of run. Useful(?) for characterizing how long a typical "outage" is.
2020-12-24 16:58:28 -06:00
elif opt == "-r":
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
run_lengths = True
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
elif opt == "-s":
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
samples = int(arg)
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
elif opt == "-v":
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
verbose = True
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
elif opt == "-H":
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
print_header = True
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
if print_usage or arg_error:
print("Usage: " + sys.argv[0] + " [options...]")
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
print("Options:")
print(" -a: Parse all valid samples")
print(" -h: Be helpful")
Add (optional) run length stats Add reporting of ping drop runs per length of run. Useful(?) for characterizing how long a typical "outage" is.
2020-12-24 16:58:28 -06:00
print(" -r: Include ping drop run length stats")
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
print(" -s <num>: Parse <num> data samples, default: " + str(samples_default))
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
print(" -v: Be verbose")
print(" -H: print CSV header instead of parsing file")
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
sys.exit(1 if arg_error else 0)
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
Small change to the interface between parser and calling scripts Move the "samples" stat out of the ping drop group of stats and into a new general stats group. This way, it will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 21:17:34 -06:00
g_fields, pd_fields, rl_fields = starlink_grpc.history_ping_field_names()
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
if print_header:
header = ["datetimestamp_utc"]
Small change to the interface between parser and calling scripts Move the "samples" stat out of the ping drop group of stats and into a new general stats group. This way, it will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 21:17:34 -06:00
header.extend(g_fields)
header.extend(pd_fields)
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
if run_lengths:
for field in rl_fields:
if field.startswith("run_"):
header.extend(field + "_" + str(x) for x in range(1, 61))
else:
header.append(field)
print(",".join(header))
sys.exit(0)
Add (optional) run length stats Add reporting of ping drop runs per length of run. Useful(?) for characterizing how long a typical "outage" is.
2020-12-24 16:58:28 -06:00
timestamp = datetime.datetime.utcnow()
Small change to the interface between parser and calling scripts Move the "samples" stat out of the ping drop group of stats and into a new general stats group. This way, it will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 21:17:34 -06:00
g_stats, pd_stats, rl_stats = starlink_grpc.history_ping_stats(-1 if parse_all else samples,
verbose)
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
Small change to the interface between parser and calling scripts Move the "samples" stat out of the ping drop group of stats and into a new general stats group. This way, it will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 21:17:34 -06:00
if g_stats is None:
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
# verbose output already happened, so just bail.
sys.exit(1)
if verbose:
Small change to the interface between parser and calling scripts Move the "samples" stat out of the ping drop group of stats and into a new general stats group. This way, it will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 21:17:34 -06:00
print("Parsed samples: " + str(g_stats["samples"]))
print("Total ping drop: " + str(pd_stats["total_ping_drop"]))
print("Count of drop == 1: " + str(pd_stats["count_full_ping_drop"]))
print("Obstructed: " + str(pd_stats["count_obstructed"]))
print("Obstructed ping drop: " + str(pd_stats["total_obstructed_ping_drop"]))
print("Obstructed drop == 1: " + str(pd_stats["count_full_obstructed_ping_drop"]))
print("Unscheduled: " + str(pd_stats["count_unscheduled"]))
print("Unscheduled ping drop: " + str(pd_stats["total_unscheduled_ping_drop"]))
print("Unscheduled drop == 1: " + str(pd_stats["count_full_unscheduled_ping_drop"]))
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
if run_lengths:
print("Initial drop run fragment: " + str(rl_stats["init_run_fragment"]))
print("Final drop run fragment: " + str(rl_stats["final_run_fragment"]))
print("Per-second drop runs: " + ", ".join(str(x) for x in rl_stats["run_seconds"]))
print("Per-minute drop runs: " + ", ".join(str(x) for x in rl_stats["run_minutes"]))
Last of initial set of scripts
2020-12-22 22:16:33 -06:00
else:
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
csv_data = [timestamp.replace(microsecond=0).isoformat()]
Small change to the interface between parser and calling scripts Move the "samples" stat out of the ping drop group of stats and into a new general stats group. This way, it will make more sense if/when additional stat groups are added, since that stat will apply to all of them.
2021-01-08 21:17:34 -06:00
csv_data.extend(str(g_stats[field]) for field in g_fields)
csv_data.extend(str(pd_stats[field]) for field in pd_fields)
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
if run_lengths:
for field in rl_fields:
if field.startswith("run_"):
csv_data.extend(str(substat) for substat in rl_stats[field])
Add (optional) run length stats Add reporting of ping drop runs per length of run. Useful(?) for characterizing how long a typical "outage" is.
2020-12-24 16:58:28 -06:00
else:
Reorganize history parsing logic into a separate module Moves the parsing logic that will be shared by some upcoming scripts out into a separate module. There's still just as much duplication between the JSON parser and the grpc parser as there was before, but this should at least prevent further duplication of this logic. This also adds some proper documentation for what each of the stats means.
2021-01-06 13:46:50 -06:00
csv_data.append(str(rl_stats[field]))
print(",".join(csv_data))
Reference in a new issue Copy permalink