Mycodo (web interface) crashing

jacobguldbergdk · April 9, 2021, 7:33pm

Just started setting up a hydroponic system on a raspberry pi 4 running mycodo on a freshly installed SD card setup using the official raspberry pi imager.

Im not running the system yet, but if I leave the pi plugged in for some time, im unable to access the webfront. Some times I get a nginx error (I think it was 502), sometimes its just not loading.
I can still access the pi via putty. Next time it crashes, I have to check if the daemon is still running.

There some other curiosities that might be related

When creating or deleting a widget in a dashboard, it often times out after I hit create / delete. A reload usually helps, but sometimes I just loose it.
Every time I access the webfront for the first time after a reboot I get a warning saying that its unsafe and the certificate is bad

Looking through the log files, the only one that seems to hold some valuable information is the web server error log:

2021/04/07 12:33:41 [crit] 4932#4932: *1 connect() to unix:/var/run/mycodoflask.sock failed (2: No such file or directory) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock:/", host: "localhost"
2021/04/07 14:11:36 [error] 599#599: *799 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "POST /dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:27:53 [error] 600#600: *942 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:28:43 [error] 600#600: *941 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:28:43 [error] 599#599: *943 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:28:53 [error] 599#599: *944 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /daemonactive?_=1617801707273 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617801707273", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:28:53 [error] 599#599: *924 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:33:53 [error] 600#600: *942 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:34:43 [error] 600#600: *941 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:34:43 [error] 599#599: *943 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:34:53 [error] 599#599: *924 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:34:53 [error] 599#599: *944 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /daemonactive?_=1617801707274 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617801707274", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:45:58 [error] 599#599: *1320 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "POST /save_dashboard_layout HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/save_dashboard_layout", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/07 14:46:13 [error] 599#599: *1171 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "POST /dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 08:51:46 [error] 621#621: *230 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /static/img/favicon.png HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/static/img/favicon.png", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 08:51:46 [error] 622#622: *211 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /video_feed/030a771c-92d5-418c-9492-8d0245497c5f HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/video_feed/030a771c-92d5-418c-9492-8d0245497c5f", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 08:52:01 [error] 622#622: *250 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /graph-async HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/graph-async", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:11:08 [error] 620#620: *32 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /video_feed/030a771c-92d5-418c-9492-8d0245497c5f HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/video_feed/030a771c-92d5-418c-9492-8d0245497c5f", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:11:38 [error] 620#620: *72 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:11:38 [error] 620#620: *71 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:12:08 [error] 620#620: *80 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /daemonactive?_=1617869103965 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617869103965", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:12:08 [error] 620#620: *79 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:12:28 [error] 620#620: *85 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:17:08 [error] 620#620: *32 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:17:38 [error] 620#620: *71 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:17:38 [error] 620#620: *72 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:18:08 [error] 620#620: *79 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /daemonactive?_=1617869103966 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617869103966", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:18:08 [error] 620#620: *80 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:18:28 [error] 620#620: *85 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:23:08 [error] 620#620: *32 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:23:38 [error] 620#620: *71 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:23:38 [error] 620#620: *72 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:24:08 [error] 620#620: *80 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /daemonactive?_=1617869103967 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617869103967", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:24:08 [error] 620#620: *79 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:24:28 [error] 620#620: *85 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:29:08 [error] 620#620: *32 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:29:38 [error] 620#620: *72 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:29:38 [error] 620#620: *71 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/4fc3f3cb-c15b-4bea-ab88-2aacd6a19e0e/input/cc693c01-3a08-47ff-b6dc-47b95407a732/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:30:08 [error] 620#620: *80 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /daemonactive?_=1617869103968 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617869103968", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:30:08 [error] 620#620: *79 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/08 09:30:28 [error] 620#620: *85 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET /last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/last/8ee0f0eb-b4f7-4ea4-88bf-a8f865050666/input/51b18180-c40f-4d35-b0e2-843d84e0efb8/120", host: "192.168.1.53", referrer: "https://192.168.1.53/dashboard/2426fa1a-d34d-4e72-a1c9-2c8900a37e60"
2021/04/09 07:28:32 [error] 619#619: *132 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/", host: "192.168.1.53"
2021/04/09 08:14:41 [error] 620#620: *165 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.29, server: , request: "GET / HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/", host: "192.168.1.53"
2021/04/09 19:36:50 [error] 619#619: *21 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /outputstate HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/outputstate", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:36:50 [error] 619#619: *20 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /daemonactive?_=1617796808135 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617796808135", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:36:50 [error] 619#619: *133 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:40:53 [error] 619#619: *183 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:40:53 [error] 619#619: *185 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /daemonactive?_=1617796808137 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617796808137", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:40:53 [error] 619#619: *186 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /outputstate HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/outputstate", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:42:50 [error] 619#619: *20 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:42:50 [error] 619#619: *21 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /daemonactive?_=1617796808138 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617796808138", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:42:50 [error] 619#619: *133 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /outputstate HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/outputstate", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:46:53 [error] 619#619: *183 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:46:53 [error] 619#619: *186 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /daemonactive?_=1617796808139 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617796808139", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:46:53 [error] 619#619: *185 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /daemonactive?_=1617796808136 HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/daemonactive?_=1617796808136", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:48:50 [error] 619#619: *133 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /outputstate HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/outputstate", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:48:50 [error] 619#619: *21 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /outputstate HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/outputstate", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 19:48:50 [error] 619#619: *20 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 192.168.1.27, server: , request: "GET /time HTTP/1.1", upstream: "http://unix:/var/run/mycodoflask.sock/time", host: "192.168.1.53", referrer: "https://192.168.1.53/output"
2021/04/09 20:01:02 [alert] 619#619: *257 open socket #14 left in connection 5
2021/04/09 20:01:02 [alert] 619#619: *256 open socket #13 left in connection 14
2021/04/09 20:01:02 [alert] 619#619: aborting

KyleGabriel · April 9, 2021, 8:59pm

This indicates you are using a camera to stream video to a dashboard.

Do you happen to have a video feed on your dashboard? Video is very prone to issues and I would recommend deleting this widget and if you must have a feed from the camera, use the photo option, not video.

This is expected. Mycodo uses a self-signed SSL certificate. Your browser is warning you that it is self-signed and it is safe if you expect it to be self-signed. If you whitelist the address you don’t have to see this warning.

KyleGabriel · April 9, 2021, 9:18pm

As a test, can you remove the camera widget or at least change it from video to image, then see if the issue persists?

The issue I believe comes down to the ability of the Pi to keep up with serving images to the browser. The default video stream width and height setting is 1024x768 and if we look in camera_picamera.py that serves the stream using the picamera library, we can see a 0.05 second wait period between frames:

github.com

kizniche/Mycodo/blob/15dbcb75c768d7f9612618f4d012abd35299e39b/mycodo/mycodo_flask/camera/camera_picamera.py#L50-L55


for _ in camera.capture_continuous(
        stream, 'jpeg', use_video_port=True):
    # return current frame
    stream.seek(0)
    time.sleep(0.05)
    yield stream.read()

When you start a video stream, you may have noticed that the video begins to lag, progressively getting longer and longer. This is likely due to the Pi acquiring images faster than it can serve them at the set 1 / 0.05 = 20 frames per second. This backup I believe is what’s causing timeouts and other resource issues.

There are a few things that can be done to help this. First, you can reduce the video stream resolution, which sends less data for each frame. However, I just now discovered an even better solution which involves changing the frames per second from 20 down to a smaller value, such as 2. I haven’t experimented enough to know at what point streaming becomes unstable, but it’s likely somewhere between 2 and 20, which can also be affected by the stream resolution.

I’ll go ahead and add a camera option to set the frames per second and set the default somewhat low and the user can decide to increase it and see how their system can handle it.

KyleGabriel · April 9, 2021, 10:55pm

I went ahead and added a Stream Frames Per Second option for streamable cameras and set the default to 5 fps. This will be available in the next release, but if you want to test before then, you can upgrade to master.

jacobguldbergdk · April 10, 2021, 6:22am

You might be right. I was just playing around and added a cam to the dashboard. Its a pi high res, probably doesnt make it better.

I deleted the dashboard entirely and so far I havent seen any issues. Fingers crossed:)

Good idea with the frame cap. I dont see many systems, where a high framerate is needed.

KyleGabriel · April 13, 2021, 1:19am

Just following up after a few days to see if your issue was resolved.

jacobguldbergdk · April 13, 2021, 11:19am

Yup. So I’m sure it was related to the camera as you suggested

jacobguldbergdk · May 21, 2021, 3:44pm

Got around to setup the camera again, but im struggeling to get it to work. If I up the resolution to the full 4056x3040 it doesnt take a image at all (its not in the folder either). I can grab an image via putty, so I know it works. Also it works if I lower the resolution to the default 1024x768.

In putty it takes around 7 sec to take a full res image, so maybe mycodo is not patient enough to wait for the image.

Im not trying to stream, I just want a image every 60 sec on the dashboard and a timelapse image every 2 hour or so.

Also if I try to add it as a dashboard widget I get the following error, even if the resolution is at 1024x768

Error: This camera type is not capable of streaming

Here is the web log

May 21 17:39:19 raspberrypi gunicorn[571]: mmal: mmal_vc_component_enable: failed to enable component: ENOSPC
May 21 17:39:19 raspberrypi gunicorn[571]: 2021-05-21 17:39:19,278 The camera is already open by picamera. Retrying 4 times.
May 21 17:39:21 raspberrypi gunicorn[571]: mmal: mmal_vc_component_enable: failed to enable component: ENOSPC
May 21 17:39:21 raspberrypi gunicorn[571]: 2021-05-21 17:39:21,119 The camera is already open by picamera. Retrying 4 times.
May 21 17:39:22 raspberrypi gunicorn[571]: mmal: mmal_vc_component_enable: failed to enable component: ENOSPC
May 21 17:39:22 raspberrypi gunicorn[571]: 2021-05-21 17:39:22,957 The camera is already open by picamera. Retrying 4 times.
May 21 17:39:24 raspberrypi gunicorn[571]: mmal: mmal_vc_component_enable: failed to enable component: ENOSPC
May 21 17:39:24 raspberrypi gunicorn[571]: 2021-05-21 17:39:24,766 The camera is already open by picamera. Retrying 4 times.
May 21 17:39:26 raspberrypi gunicorn[571]: mmal: mmal_vc_component_enable: failed to enable component: ENOSPC
May 21 17:39:26 raspberrypi gunicorn[571]: 2021-05-21 17:39:26,601 The camera is already open by picamera. Retrying 4 times.
May 21 17:39:27 raspberrypi gunicorn[571]: 2021-05-21 17:39:27,603 Exception raised in 'camera_record' when setting user grp: [Errno 2] No such file or directory: '/home/pi/Mycodo/cameras/e7751835-a8eb-4338-90b0-e5abeb7c9ae4/still/Still-1-PiHighResCam-2021-05-21_17-39-15.jpg'
May 21 17:39:27 raspberrypi gunicorn[571]: Traceback (most recent call last):
May 21 17:39:27 raspberrypi gunicorn[571]:   File "/home/pi/Mycodo/mycodo/devices/camera.py", line 413, in camera_record
May 21 17:39:27 raspberrypi gunicorn[571]:     set_user_grp(path_file, 'mycodo', 'mycodo')
May 21 17:39:27 raspberrypi gunicorn[571]:   File "/home/pi/Mycodo/mycodo/utils/system_pi.py", line 674, in set_user_grp
May 21 17:39:27 raspberrypi gunicorn[571]:     os.chown(filepath, uid, gid)
May 21 17:39:27 raspberrypi gunicorn[571]: FileNotFoundError: [Errno 2] No such file or directory: '/home/pi/Mycodo/cameras/e7751835-a8eb-4338-90b0-e5abeb7c9ae4/still/Still-1-PiHighResCam-2021-05-21_17-39-15.jpg'
May 21 17:39:27 raspberrypi gunicorn[571]: 2021-05-21 17:39:27,605 Exception raised in 'camera_record' when setting user grp: [Errno 2] No such file or directory: '/home/pi/Mycodo/cameras/e7751835-a8eb-4338-90b0-e5abeb7c9ae4/still/Still-1-PiHighResCam-2021-05-21_17-39-15.jpg'
May 21 17:39:27 raspberrypi gunicorn[571]: Traceback (most recent call last):
May 21 17:39:27 raspberrypi gunicorn[571]:   File "/home/pi/Mycodo/mycodo/devices/camera.py", line 424, in camera_record
May 21 17:39:27 raspberrypi gunicorn[571]:     set_user_grp(path_file, 'mycodo', 'mycodo')
May 21 17:39:27 raspberrypi gunicorn[571]:   File "/home/pi/Mycodo/mycodo/utils/system_pi.py", line 674, in set_user_grp
May 21 17:39:27 raspberrypi gunicorn[571]:     os.chown(filepath, uid, gid)
May 21 17:39:27 raspberrypi gunicorn[571]: FileNotFoundError: [Errno 2] No such file or directory: '/home/pi/Mycodo/cameras/e7751835-a8eb-4338-90b0-e5abeb7c9ae4/still/Still-1-PiHighResCam-2021-05-21_17-39-15.jpg'

KyleGabriel · May 21, 2021, 3:58pm

There are multiple libraries. Some do not work well with large amounts of data with high resolutions, some do. Test them all.

Make sure you are running the latest code from master, as there have been more recent changes (fixes) to the camera widget. I suspect you are not using the latest code.

jacobguldbergdk · May 21, 2021, 4:11pm

Got it running with the raspistill library. Hadent noticed there were multiple:)

My version is the master from a couple of days ago. I’ll try and update to the latest and see what happens