Commit graph

19 commits

Author SHA1 Message Date
sparky8512
a692f8930a
Add another potential TODO 2021-01-27 16:26:23 -08:00
sparky8512
2e045ade16 Add tracking of counter across script invocations
Write the sample counter value corresponding with the last recorded data point into the database along with the rest of the sample data so that it can be read out on next invocation of the script and data collection resumed where it left off.

Switch default sample count to all samples when in bulk mode, which now really means all samples since the last one recorded already.

Switch the time precision to be 1 second. Data points are only written one per second, anyway, and this way if there is any overlap due to counter tracking failure, the existing data will just get overwritten instead of creating duplicates.

Add a maximum queue length, so the script doesn't just keep using more memory if it persistently (>10 days) fails writing to the InfluxDB server.

Hack around some issues I ran into with the influxdb-python client library, especially with respect to running queries against InfluxDB 2.0 servers.

This concludes the functionality related to bulk collection of history data discussed on issue #5
2021-01-21 20:39:37 -08:00
sparky8512
9a57c93d73 Back out changing the default command options
Per review feedback, this could have interfered with the ability to set this option via environment variable. It was a bit messy, anyway.
2021-01-16 20:12:22 -08:00
sparky8512
9b04e8387c Minor changes based on thorough proof read 2021-01-16 10:33:22 -08:00
sparky8512
2e71acbbdb Changes to work better with Docker containers
Handle SIGTERM to enable graceful script shutdown when a container is stopped. This currently only matters for the InfluxDB scripts, and only when they run in a loop, since if the script is hard-terminated, it won't flush out any queued data points to the InfluxDB server. This also required changing the entrypoint script to exec python instead of running it as a child process of the shell running entrypoint.sh, since Docker will only deliver SIGTERM to the parent process it started directly.

Also, add -t 30 to the default Docker command to match the script default behavior prior to the changes in 46f65a6214
2021-01-16 10:17:32 -08:00
sparky8512
46f65a6214 Implement periodic loop option
Add an interval timing loop for all the grpc scripts that did not already have one. Organized some of the code into functions in order to facilitate this, which caused some problems with local variables vs global ones, so moved the script code into a proper main() function. Which didn't really solve the access to globals issue, so also moved the mutable state into a class instance.

The interval timer should be relatively robust against time drift due to the loop function running time and/or OS scheduler delay, but is far from perfect.

Retry logic is now in place for both InfluxDB scripts. Retry for dishStatusInflux.py is slightly changed in that a failed write to InfluxDB server will be retried on every interval, rather than waiting for another batch full of data points to write, but this only happens once there is at least a full batch (currently 6) of data points pending. This new behavior matches how the autocommit functionality on SeriesHelper works.

Changed the default behavior of dishStatusInflux.py to not loop, in order to match the other scripts. To get the old behavior, add a '-t 30' option to the command line.

Closes #9
2021-01-15 18:39:33 -08:00
Leigh Phillips
0ed3e074db
Update README.md 2021-01-11 17:24:10 -08:00
Leigh Phillips
eecc65a5ba
Update README.md 2021-01-11 00:01:50 -08:00
Leigh Phillips
d21b196f78
Update README.md 2021-01-11 00:00:49 -08:00
Leigh Phillips
ff2d0eacb1
Merge pull request #2 from sparky8512/main
SSL/TLS support for InfluxDB and MQTT scripts
2021-01-10 21:53:32 -08:00
sparky8512
ce44f3c021 SSL/TLS support for InfluxDB and MQTT scripts
SSL/TLS support for InfluxDB and MQTT scripts

Copy the command line option handling into the status scripts to facilitate this. Also copy the setting from env from dishStatusInflux_cron.py.

Better error handling for failures while writing to the data backend. Error printing verbosity is now a bit inconsistent, but I'll address that separately.

Still to be done is dishStatusInflux_cron.py, pending a decision on what to do with that script, given that dishStatusInflux.py can now be run in one-shot mode.

This is related to issue #2.
2021-01-10 21:36:44 -08:00
Leigh Phillips
27ce46cb3c
Update README.md
Name & run docker in daemon mode.
2021-01-09 13:22:55 -08:00
sparky8512
f067f08952 Add InfluxDB and MQTT history stats scripts
Unlike the status info scripts, these include support for setting host and database parameters via command line options. Still to be added is support for HTTPS/SSL.

Add a get_id function to the grpc parser module, so it can be used for tagging purposes.

Minor cleanups in some of the other scripts to make them consistent with the newly added scripts.
2021-01-09 12:03:37 -08:00
sparky8512
253d6e9250
Fix spelling error 2021-01-09 11:43:25 -08:00
Leigh Phillips
c28e025893
Update README.md 2021-01-08 22:41:57 -08:00
sparky8512
f206a3ad91 Add a short blurb for the recently added scripts 2020-12-30 11:47:03 -08:00
sparky8512
61e9abf7d4 Add header comments and update README
Also, add a timestamp to the CSV output, similar to what is in the get_history equivalent.
2020-12-29 20:45:30 -08:00
sparky8512
8d3aadad41
Fill out the README 2020-12-22 20:05:25 -08:00
sparky8512
33e14d2073
Initial commit 2020-12-22 14:40:56 -08:00