InfluxDB causes extreme CPU load

Hi,

I am on Mycodo 8.15.12 and everything worked fine after initial installation issues. Interface was always quite slow though. All of a sudden, without changing anything, my RasPi 3 A is basically not responding anymore due to InfluxDB process utizilizing the whole CPU power. As soon as I stop the influxdb process, Mycodo is responding again and is also much more responive than initially.
Anyone who knows this problem? Seems to have been a problem fromm the start which deteriorated over time.There wasn’t much data in the database…
Thanks.

Yup, been seeing the same excruciating slowdown on both a Pi 3B+ and a Pi4 using InfluxDB v2. The web interface has become so slow it’s practically non-responsive, it can take upwards of 2 minutes for any settings saves or function activation/deactivation to happen, even longer to fully load a dashboard.
Even re-creating the database with a fresh empty file doesn’t seem to help.
Trying to open more than one Mycodo web page causes both pages to hang and never load.
No difference in speed between running the OS and Mycodo from an SD card or a USB drive either.

Using htop from the CLI shows all 4 cores constantly pegged around 65% usage and memory almost all used. All of the files with the most CPU usage are “/usr/bin/influxd”.
And this is after disabling all but a few essential Inputs and Functions to keep my hydroponics system running.
Something has definitely changed recently with Influx, or one of the many other dependencies and packages Mycodo uses… because I’ve been using Mycodo for over 2 years, I haven’t changed my setup, and I’ve never had these problems until the past 7 months.

I’ve reinstalled Mycodo and it worked very responsive for a few days. Now it is unfortunately unusable again. All it had to process was one temperature sensor input with a measurement every 30s or so.
:frowning:

Well, the Pi 3 and the Pi 3B+ are actually 64-bit SOC.
Which OS are you using… 32 or 64 bit?
If you are running the 32-bit OS, you need to use the InfluxDB V1.
If you are using the 64-bit OS, you might want to run the InfluxDB V2.

Also, on the Pi 3 models, the WiFi and the SD card share the same I/O bus, so it should noticeably improve performance if you disable the WiFi on the Pi in the config.txt file and use the Ethernet connection to hard-wire the Pi to your router.

dtoverlay=disable-wifi

Make sure you are using a quality brand-name SD card with at least a U3 class speed rating, the faster the better.
The speed of the SD card can greatly effect the overall I/O speed of the Pi.

If you are sure you are using a good ultra-speed SD card, and you are using a Raspberry Pi 3 or 3B+, then you can overclock the SD card slot by adding this line to the end of your /boot/config.txt file…

dtparam=sd_overclock=100

This will set the Pi’s SD card reader to it’s maximum speed to be compatible with the ultra-speed SD card.
It may offer a little improvement in performance, but a Pi 3A is really not much better than the Pi 2. I really recommend a Pi4… or even a new Pi 5 maybe.

The latest unreleased version of Mycodo upgrades influxdb 2.x to the latest version. You can upgrade to the master branch and see if this fixes your issue.

Thanks for your replies. Unfortunately I’ve installed a 32 bit OS. Therefore it seems I cannot use InfluxDB V2. Disabling WiFi is no option, since the Pi doesn’t have an Ethernet port. The SD card I use only has a U1 classification.
I’ll try with a 64 bit OS and reinstall everything on a U3 card. I hope it will be resolved by using InfluxDB V2.

Hi again,

I bought a new high speed U3 class SD card and installed RasPi OS Lite 64 bit and Mycodo from scratch. Unfortunately nothing changed. The system ran great for about a day, now it’s back to the initial issue. Behaviour thefore just like before on Influx v1.
I currently don’t have access to the terminal. However, from what I understand a load average of about 6 queued processes is not good and likely the reason the system is responding extremely slow.

I am running Mycodo 8.15.13 and a Mosquitto server on that machine - nothing more. Mycodo is set to use InfluxDB v2 and processed measurement data of a few MQTT subscriptions. Measurements are maybe a couple of houndred numbers. With the latest 8.15.13 release, the influxdb should be updated to the latest version according to the changelog.
I seem to have something in my specific configuration which causes the system to fail. But given the little functions I use, I have no idea what it may be :frowning:
Any help very appreciated, thanks.

At this point, the Pi 3A is a pretty “old” piece of hardware (2018), and it has pretty limited I/O options… only 1 USB-2 port, no on-board Ethernet, and only 512MB of RAM.

The only thing you can do at this point is buy a better Pi… like a Pi 4 or Pi 5. I’d recommend one with 8GB of RAM.

The Pi 4 has on-board USB-3 which is fast enough for connecting an external SSD drive.

The Pi 5 has on-board PCIe 2.0 which would be ideal for connecting an external SSD.

Improving read/write I/O speeds is the only real way to get faster performance from a Raspberry Pi… and that is what the main problem is here… lack of an I/O bus that can keep up with all the sensor and database read/write cycles. I think the other issue here is that there have been changes made to the Raspberry Pi OS in anticipation of the Pi 5 release. Unfortunately, I think these recent changes may have lead to a decrease in performance of the older Pi models when running the latest PI OS. :frowning:

As soon as I have some extra cash, I plan on upgrading to a Pi 5 and a NVMe SSD drive.

Fair enough. Might be worth a try. I’m a little worried tho, that it will just be the same with a more powerful Pi. I mean, I assume other people use Mycodo with dozens of Inputs, Outputs, Functions, Controllers etc. also on Pi’s, no? All I did was importing a few measurements (like four data points a minute) and nothing else. It’s hard to imagine for me that it could overburden the machine already. :thinking:
What can this old Pi do then? :sweat_smile:
But yeah, happy to spend the cash if it works then.
Thanks.

This issue appears in a very low minority of installs. None of my dozen or so installs of Mycodo suffer from this issue, and range from Pi 3s to Pi 5s. I do tend to use exclusively SSDs these days because they are so cheap and provide the best reliability and performance boost when compared to using a micro SD. I’d suggest first try using an SSD (with USB to SATA adapter) before upgrading to a new Pi. But I can say the Pi 5 is significantly faster than the Pi 4, and the Pi 4 is significantly faster than the Pi 3.

UPDATE:

So I recently moved Mydoco from the Pi 3B+ I was running it on to my Pi 4 8gb.
I spent some more time researching SD cards and bought one of these…

https://www.amazon.com/SanDisk-Extreme-microSDTM-Adapter-SDSQXCU-064G-GN6MA/dp/B09X7BYSFG/ref=sr_1_17?crid=1QWJ0BZKQT2IK&dib=eyJ2IjoiMSJ9.t14xbQTFzn97KXs0Z2YRwT5Yb-CjmOtFo6h6KlarJQVUf4RKAFZL_hf1LSpgdYR2mkKzR22HG9dMqb-1XbhDINyDNLmEA9g5p-5BI42CFl78mboUXkPM03OsMd2T3pCtkDGHdfl5HKuY2qhCQxVm7z7ksvn1XIonOAGz52ON-St88HuAU1S0NCk0w-xnKLY8.Te5hTwsXO9cCMz6Lan4KyVSWaGptj-XKh8Vaji8xcOE&dib_tag=se&keywords=sandisk+extreme+pro&qid=1708815146&sprefix=sandisk+extreme+pro%2Caps%2C163&sr=8-17

It’s the “V30” rating you need to look for.
This rating means the card is capable of a minimum of 30 MB/s write speeds, usually much higher. (They also have V60 and V90 rated cards that are even faster, but they are not cheap).
These cards are specifically designed for 4k and 8k ultra-high-def video recording on professional AV equipment. Thus they need to have very high throughput speeds.
Most SD cards and USB flash drives are advertised based on their read speeds. However, most of them have some horrendously slow write speeds (as I discovered after bench-marking all of my SD cards and USB flash drives)… but it is the write speeds that are much more important when using the card to run an operating system due to the constant high I/O throughput and random-access reads.

I actually bench-marked this V30 card on my desktop using Linux disk tools and a USB-3 card reader…
Read speeds were in excess of 300 MB/s.
More importantly, the write speeds were in excess of 80 MB/s.
Random access speeds were a mere 0.7 ms

All these specs WAY out-performed even a spinning hard drive connected with a USB-3 drive enclosure.

Since the Pi 4’s SD card reader is capable of the same speeds as a USB-3 port, it seems that this ultra-fast card is definitely making a very noticeable difference so far in how fast and smooth Mycodo now seems to be running on my Pi 4. I still haven’t enabled all of my Functions yet, but will be soon.
Ill post more updates as I continue testing.

1 Like