Trouble restoring from a backup

Hi,

I’m new here but not new to Mycodo. I have always been able to figure things out on my own before but I’m coming up a bit stumped and time is rather relevant. I have posted in support because I don’t think it’s a bug, at the moment I’m assuming that Problem Exists Between Chair And Keyboard.

I had an SD card corrupt when I was rebooting my PI that runs my rather complex hydroponic greenhouse setup. I have built multiple Mycodo based controllers and I have always made it a practice to produce a full disk image of my SD card once I have my system effectively setup. In the past when I have had this problem I just swap out the SD with a freshly written image and my system is only down for half an hour.

Unfortunately I was just putting the finishing touches on a major rebuild and hadn’t made an image yet. I have been backing up regularly and managed to salvage a recent backup, so I figured I should be good. I put a fresh install on a new SD card and checked that it was working fine. I then copied my backup directory over from Mycodo-backups, then restored from it from inside the Mycodo web interface. It says it has started restoring succesfully, but after the daemon shuts down during the restore I can’t get it to come back. The web server just returns a 502 error: Bad gateway after the restore.

Reboots have not helped, nor have my attempts to restart the daemon or the web server manually from the terminal. Of possible relevance my backup was taken from Mycodo 8.40 and I am trying to restore with the latest release, 8.10.1.

I wanted to include all the command line attempts, error messages, and log portions I thought would be relevant below, but the forum won’t let me post them. It keeps telling me that new users can only post two links, so i assume it’s confusing file paths in the logs for hyperlinks and rejecting me. I’d be happy to post them if I can.

I would be incredibly grateful for any help, as would my many plants that are currently stuck in a less than optimal hodgepodge of stopgap solutions.

Oh, and also, Kyle you are amazing, Mycodo is astoundingly great, and I hope your shipping containers full of mushrooms are doing well. Just really great work across the board, thank you so much for sharing it with the world.

1 Like

Hi. I just upped your trust level to unlock more posting abilities. Try this command and see if it fixes the issue you’re seeing.

sudo bash ~/Mycodo/mycodo/scripts/upgrade_post.sh
1 Like

Wow, thanks for the quick response!

Unfortunately that doesn’t seem to have solved it, still getting that 502 Bad Gateway. The command you gave produced quite a lot of output though.

Again, many thanks for all of your hard work!

So the result of that command was: upgrade_post_sh log.txt (32.0 KB)

Here is the log file output from my original attempt to restore from backup: daemon_start.txt (17.0 KB)

What is the output from these commands?

sudo service mycodoflask stop
cd ~/Mycodo/mycodo
sudo /var/mycodo-root/env/bin/gunicorn \
--workers 1 \
--worker-class gthread \
--threads 2 \
--timeout 300 \
--pid /var/run/mycodoflask.pid \
--bind unix:/var/run/mycodoflask.sock start_flask_ui:app

Also, use attachments for long text, to make it easier to read the thread.

1 Like

Ack! Sorry about the wall of log text, kinda obvious I should be using attachments now that you mention it. I’ve attached the output for you.

Thanks!
mycodo-error-msg4.txt (2.2 KB)

I can’t make sense of your logs. There’s something messed up and it could take a lot of time to diagnose it. What I would do is start fresh by flashing Raspberry Pi OS, installing the latest version of Mycodo, upgrade to master, then do this:

Stop Mycodo:

sudo service mycodo stop
sudo service mycodoflask stop

Copy your backed up mycodo.db to the currently installed system:

cp /path/to/backed_up/Mycodo/databases/mycodo.db ~/Mycodo/databases

Upgrade the database, run the upgrade script, and start the services:

cd ~/Mycodo/databases
sudo ../env/bin/alembic upgrade head
sudo ../env/bin/python ./alembic_post.py
sudo bash ../mycodo/scripts/upgrade_post.sh
sudo service mycodo restart
sudo service mycodoflask restart
1 Like

Ok, I’ll nuke the install and start fresh as you recommend. I’ll post up my results when I get through all that.

Thanks again, I really appreciate everything.

1 Like

Did you have any custom inputs/outputs/functions/widgets in use that you imported?

1 Like

I didn’t have any custom inputs or outputs, and I don’t think any of my functions or widgets would count as custom. I had a lot of functions that I had set up but they were all types of functions already present in the options Mycodo makes available. Same thing with my widgets. I had a few python outputs but they used the existing Python Command output.

So I’m fairly certain this is a no… I didn’t ever custom code anything and import it through the settings section.

Well it looks like I’m a difficult case I’m afraid.

I tried starting fresh and following your procedure above, but still no luck. It throws errors at various points along the way. I’ve got separate log files from the upgrade, alembic upgrade, alembic pot, upgrade_post, and just for good measure the rest of the logs at /var/log/mycodo.

Sorry I’m such a hassle… boy do I wish I had imaged that SD card before all this!
Is it possible to install an earlier version instead of the most recent? I’m wondering if I could just try putting 8.4 back on in hopes that the backup restoration will play nicely, but thats just pure speculation on my part.

So here are the logs… and again, thanks.

alembic_post_log.txt (10.0 KB)
database_upgrade_head_log.txt (5.3 KB)
database_upgrade_post_log.txt (53.6 KB)
login.log (112 Bytes)
mycodo.log (1.4 MB)
mycodokeepup.log (539 Bytes)

The database upgrade failed, and because the database couldn’t upgrade, nothing after that could work. I just pushed a fix for the database upgrade. Try the process again.

I thought that was what you initially tried and it didn’t work.

In any case, even if you got the older version installed, you would run into the same issue if you attempted to upgrade because the database upgrade would have failed anyway (as you just found out).

1 Like

Honestly, you have got to be the most responsive dev on the planet!

I’ve got a fresh install of the latest version going in right now, but no, I never tried putting 8.4 back on, I’ve been trying with the latest version this whole time. I can give the process another go with the current version and that fresh update you just pushed. I have another SD card I’ll put a fresh raspbian image on and I can try it with 8.4 if that doesn’t work. I’m just not finding how to put an earlier version on instead of the current release, i’ve just been installing Mycodo with:

curl -L https://kizniche.github.io/Mycodo/install | bash

1 Like

You can install any of the backups by simply moving/renaming the backup directory to “/home/pi/Mycodo”, then running:

sudo bash ~/Mycodo/mycodo/scripts/upgrade_post.sh

But we still need to fix the issue with the upgrade otherwise you’ll run into the same issue as you would with my other procedure, above.

1 Like

Aha! Victory!

It looks like that update you pushed to the database upgrade script did it. The first time I tried it still went sideways on me, but then I realized that I had begun my Mycodo install literally a minute before you committed that change, so I think it must have still pulled down the previous version. Cleaned it off, ran it again and everything completed without errors this time.

Thank you so much for your help, my plants and I are truly grateful! I could have reconfigured my Mycodo setup from scratch by going back to my build notes and schematics but it would have been a very long and tedious process. I have my system back up and running just like before, and I fully imaged the SD card and burned a backup card so if I get another corruption I can just swap the SD card out and keep things running.

So again, thanks for your help, and thanks for developing Mycodo in the first place, it’s a great piece of software and you just keep on improving it.

1 Like

I have found one thing that has been off since I restored the update. When I load up the dashboard page it has been throwing a color index error, and the graphs don’t always load right. Other than that everything seems to be working fine.

I’ll try deleting my graph widgets and rebuild them, see if that gets rid of the error. Just thought you should know.