I am running the latest version 8.16.0 on a Rpi 3+ with bookworm. I set up the system 3 times with a new installation because I thought I have made a mistake. I only aquire data from a mosquitto server on another Rpi installation in the same network. After I have restarted the backend data rush in for a few minutes and than the live messurement will stop with “no data last 30 sec”.
What is the interval that the MPPT device is sending data to your MQTT server?
Are your MPPT devices running Tasmota?
Have you tried removing all the inputs and just trying the MPPT inputs one at a time to see if they work?
Have you setup a graph in a Dashboard that tracks the data from these inputs to see if you are actually getting data even though you see a 0 in the Live Measurements?
Have you tried enabling the “Log Level:Debug Interface” check boxes in the input setups so that there is more detailed information in the logs?
Sometimes with MQTT inputs, you will see a reading of 0 in the Live Measurements if the interval that the data is being sent from the device is longer than 30 seconds. I’m not talking about the “15.0 second interval” that you see in the Live Measurements, that interval is just how often Mycodo checks if there is a message waiting in queue on the MQTT server… if no data is actually being sent to the MQTT server by the MPPT device, you’ll end up with a reading of “0” in the Live Measurements until the MPPT device sends a new data message.
It may be helpful to use an MQTT diagnostic client like MQTT Explorer so you can actually view what’s going on inside your MQTT server.
the interval that the MPPT device is sending data to my MQTT server = 5 sec
Are your MPPT devices running Tasmota? Yes
I removed all inputs and activated one by one. They all work and I get data. The first 4 inputs are running constantly and have no blackouts. Activating MPPT 3 (Input 0446d5bc) after a while all inputs going stale.
I have enabled “Log Level:Debug Interface” and the data are fine until the the data go stale.
The data are available in MQTT Explorer.
I will continue testing and send the log file if I can catch the blackout situation again.
If your MPPT devices are running Tasmota they can’t possibly be sending MQTT topic messages at 5 second intervals since Tasmota’s minimum teleperiod is 10 seconds. Check your Tasmota devices and confirm what the TelePeriod is set to by using the Console.
When you say “the data goes stale” what behavior are you seeing in the Mycodo log? Are the messages no longer being received by Mycodo, or are the messages still being received but contain no data?
If some of your inputs seem to be working, and then stop when you activate “MPPT 3” then you might want to check the MQTT client names on all your Tasmota devices and in the Mycodo Input settings. All of the client names must be unique, none of them can use the same client name, or you will end up with conflicts that may cause the behavior you are seeing.
Thx for the reply. To be 100% sure I have changed the power supply and the sd card. After the system is starting all values are shown. I am sure that I have no problems with client names. After some minutes the the frontend is showing no data but the log file is still reporting incoming data. The system is starting to be very unresponsive. If I run ownly two values it seems to be stable. But more MQTT data leads to the described problem. Attacheg you find the latest log file.
for some reason I can’t access that log file. Please attach files directly to the post, you should be able to attach at least one simple text file or just copy and paste the relevant error lines directly into your post.
What model of Raspberry Pi are you using?
Do you mean you replaced the Raspberry Pi power supply or the MPPT power supply?
If your model of Raspberry Pi has a RED power light… do you ever see it blinking on and off?
If you ever see the RED power light randomly blinking on and off it means the Pi’s power supply isn’t providing enough voltage or the USB cable you are using is causing too much voltage drop.
Are you using an SD card with at least a “V30” rating?
Any SD card below a V30 rating may be too slow to keep up with the read/write throughput from the Pi.
I am using PI 3 B+. I have replace the Raspberry power supply because I had undervoltage errors. I am not using a V30 sd card. The system is on a Sandisk Ultra card. The relevant logfile is to long
You don’t need to post the entire log file, nor would I want to read through it, just post a couple of samples of any errors or odd behavior you are seeing…a few lines should suffice.
Without knowing more specific details of your setup, it’s going to be difficult to continue troubleshooting these issues… but it sounds like you might have multiple problems happening simultaneously.
I can tell you that running a Pi 3B+, with an SD card that you haven’t benchmarked for write speed, and a possible power supply problem, are the main things you should focus on solving first.
I’ve had brand new Sandisk “Ultra” cards only benchmark at 12 MB/s write speeds… that will severely impact the Pi’s overall performance as the SD card can’t keep up with the read/write operations of the Pi.
I’ve also had power supplies and USB cables that were just not getting enough voltage or current to the Pi and causing crashing or SD card corruption.
The Pi 3B+ SD card slot maximum write speed is only about 20 MB/s, and that’s for sequential writes, random write speeds will be much slower, probably more like 10 MB/s.
If you have a lot of inputs, and they are all trying to acquire data every 5 seconds, while trying to read and write that data to a slow SD card in a slow SD card slot, you will eventually reach a “bottleneck” in the throughput and the system will become unresponsive. You can try increasing the read interval on your inputs to 10 or 15 seconds and see if that eases up some of the data traffic.
I had a similar problem running a Pi 3B+ as well as a Pi 4 8GB, and the only thing that solved the problem was getting rid of the SD card altogether and running the operating system and Mycodo from a SSD plugged into the Pi 4’s USB3 port, which has a write speed of around 620 MB/s.
You might be able to do the same thing with the Pi 3B+ since it’s USB2 ports can reach about 60 MB/s write speeds with an SSD drive.
I recommend using M.2 drives in a USB-C enclosure since they are small and fairly low priced.
Here is something that will be very helpful for any Raspberry Pi troubleshooting.
I have attached a bash script called “checkthrottle.sh”.
I did not write this script, there are many different versions of this script floating around on Pi forums all over the web if you go looking.
What this script does is use the CLI command “vcgencmd” to query the Pi about it’s past and present power, temperature, and CPU throttle state.
Basically it will tell you if there is a problem with the power supply coming into your Pi and if the CPU has been throttled because of a present or past voltage drop or if the CPU temperature has risen above the throttling point.
Simply copy the checkthrottle.txt file into your Raspberry Pi’s home folder,
change the file extension from .txt to .sh (checkthrottle.sh),
and run it by typing sudo bash checkthrottle.sh
I install this file on all my Pis now, it is a simple but very useful diagnostic tool.
Thx for your very helpful tips. I currently order a V30 SD card and will making checks with your tools. I also guess there is a bottleneck because some comments lead to an error message “database is locked”.
Yes… “database is locked” is the exact same problem I kept having due to the SD card and the Pi’s SD card slot writing too slow to keep up with all of the operations I had Mycodo performing.
In my case, even the faster V30 and V60 SD cards still did not solve the problem.
The only way to solve the problem was to increase the sensor read interval to reduce the data traffic or use an external SSD that could keep up with the read/write cycles.
My recommendation is to buy a Pi 4 and an SSD drive and run the OS and Mycodo from the SSD drive plugged into one of the Pi 4’s USB 3 ports. And use a good passive heat-sink case with the Pi 4 or even a fan case to keep it cool. If the Pi throttles the CPU due to over-heating or under-voltage you will see a drop in performance.
Some of these issues are not the fault of Mycodo but rather the underlying operating system… the operating systems are being updated and optimized to run on the newer, faster chip-sets like the Pi 4 and Pi 5 and beyond, the Pi 3B+ was released almost 7 years ago and is now considered an “old” piece of hardware by most standards.
I made a test with a SBC Rock64 1GB and a standard class 10 sd card. I increased the number of mqtt requests from 2 to 5. The numbers are coming in all 3 seconds. The system is running with dietpi bookworm. Until now the system is running stable and the processor laod is much better than under a Pi 3 B+. Again thanks for your helpfull input on this particular case.
You really do need a V30 or V60 or even a V90 rated SD card for running an operating system on a newer SBC, and that still may not be fast enough for some newer SBCs.
After all, SD cards were designed for data storage, they were never really meant to be used as live system drives… that’s what SSDs are for.
Did you actually put your SD card in a USB 3 card reader, plug it into a USB 3 port on your computer, and benchmark it for actual real-world read/write performance?
If not, then you are just asking for more trouble down the road.
I still recommend not using SD cards for “critical systems” and switching to M.2 SSD. MUCH faster and more reliable, especially for an automation system that could be handling critical things around your home.
