HDD Sleeping Disconnecting and Connecting Too Often!

Kernel, Main, Utilities & Applications, Miscellaneous Devices.
TLoMatt
Posts: 23
Joined: Sat Sep 03, 2022 6:21 pm
Has thanked: 10 times
Been thanked: 3 times

HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by TLoMatt »

Hi I read about an answer from jca link here viewtopic.php?t=1335 about how he managed to run a script in order to make a portable hdd with no power supply run all the time preventing it from sleeping , problem is that I don't know about linux what i first did is connect my Mister wireless with a software called "putty" from what I understand once connected to mister running commands from putty would be the same as running commands from mister's black terminal (by pressing the F9 key)

After that I did not understand very well how to run the sh scripts at first I though that I had to run the commands from putty copy and pasting this (using my UUID string with the help of the blkid command)

Code: Select all

#!/bin/bash
while :
do
dd if=/dev/disk/by-uuid/fd9bdf29-f845-4b26-9aa8-0502d0cd9792 of=/dev/null count=1 skip=$RANDOM >/dev/null 2>&1
sleep 25m
Lo-que-hice.jpg
Lo-que-hice.jpg (104.39 KiB) Viewed 1723 times

I forgot the "done" command as I though that wasn't a command (keep in mind again I don't know about linux)
After pasting that code immediately my hdd stopped being recognized by my mister

I didn't know how to fix this or even what exactly happened so I formatted my mister main 8 gb micro sd set it up like a new installation and everything and after doing so now my external hdd is recognized again but I've noticed my hdd is doing a sound like stopping and restarting very often

First of all I want to know if those linux commands messed up something in my external hdd and how to restore it like it was before

And second I really want so badly to understand and be able to apply correctly those scripts so my external hdd does not stop spinning anymore

I noticed about the spinning stopping cause everytime I tried to play some PSX game after some seconds or minutes my hdd makes a noise like it would stopped and then wakes up again but that it enough to make my entire mister freeze cause I guess it makes sense the data streaming abruptly stops cause of the external hdd stopping and btw this is something I didn't experienced before with psx booted from a micro sd (I guess micro sds never sleeps compared to hdds )
Flandango
Core Developer
Posts: 459
Joined: Wed May 26, 2021 9:35 pm
Has thanked: 59 times
Been thanked: 383 times

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by Flandango »

From the looks of it, the commands you pasted into the terminal didn't do anything destructive.
The script, is meant to continously randomly read from the hard drive in order to keep it spinning.
Most likely you are experiencing the same issue I have with external non-powered hard drives that it's not getting enough power from the USB and it's spinning up and down. You could try a higher amperage (not voltage) power supply (see the other numerous threads about power supplies) or you can try using a powered USB hub for the external drive.
User avatar
PistolsAtDawn
Posts: 359
Joined: Fri Feb 18, 2022 7:29 pm
Has thanked: 307 times
Been thanked: 100 times

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by PistolsAtDawn »

I used to have to do this on a home server on a single board computer with an external hard drive. I would not personally use "disk destroyer" for this task even though the code you have above is not destructive. I don't think you harmed your disk, but you played with fire! A mistyped dd command can wreck the contents of a drive. Again, that particular command as you had it should be harmless.

The hdparm utility would be the right way to disable sleep mode on an internal hard drive on a normal Linux distro, but external drives use cheap USB bridges that don't support the low level functions needed for hdparm to fix the sleep issue. That's been my experience anyway.

Flandango is probably correct about power usage, so check that first, but it still might be worth your time to use the touch command scheduled as a job instead of manually running a script from PuTTY.

Touch creates or updates a file with the current time when run. It's just enough write activity to keep a drive from sleeping. You can have it run on an interval using the cron daemon. Cron jobs don't work "out of the box" on MiSTer, but see the end of this message for a work around.

The touch command would be like this:

Code: Select all

touch /path/to/externalDrive/keepAwake
Where "keepAwake" is the empty file created and updated by this command. It can be named anything (it's best practice to avoid spaces in the file name though because sometimes it's hard for scripts to properly parse that file name).

If you did want to use the original script anyway, you can't paste it into the terminal like that and have it work. You will want to copy that text and paste it into an empty text file (Notepad++ if you're using Windows. Don't use the built-in Notepad program because it does not format the file correctly for Linux to be able to use it).

You could also create this file through PuTTY. Use the program nano to create the new text file where the script commands above will live. You should be able to copy and paste it in. Ctrl + s to save and Ctrl + x to exit.

Code: Select all

nano nameofscript.sh
The text file would be saved as the file "nameofscript.sh". The .sh means it's a shell script. You would then execute it by entering into the terminal:

Code: Select all

bash nameofscript.sh
or

Code: Select all

./nameofscript.sh
assuming that the terminal is in the correct directory where the script is located. If not you can change to that directory using the "cd" command first or type out the full path and name of the script together:

Code: Select all

./media/fat/Scripts/nameofscript.sh
That will run it assuming you put it on the micro SD card with the other MiSTer scripts. If you place it there, then you should be able to run it from the main MiSTer menu like the other scripts, although I think you wouldn't want to do that as it might lock the screen to the terminal while that runs.

You can also schedule that script to run using the cron daemon although it's cleaner to just replace the touch command within the quotation marks below with the dd command from the script.

I'm putting the following in bold for everyone, not yelling at you, OP.

I've seen this come up a few times. Cron jobs are possible but not setup to work by default on the MiSTer Linux distro. I got cron jobs working in a pretty hacky way, but it does work.

The folders /var/spool/cron and /var/spool/cron/crontabs need to be created after each startup as well as the cron job/s in the file at /var/spool/cron/crontabs/root

The cron daemon then needs to be started.

You could use crontab to create the commands in the cron job file, however it is destroyed upon shutdown.

That missing file with the desired command/s can be recreated in different ways, but the below is easier for new Linux users than explaining how to use and exit the vi editor with crontab or how to change vi to nano first and then how to move the crontab output to a different location so it's not destroyed at shutdown.

I know there are probably better ways to do this.
"Don't @ me!" :mrgreen:


Okay TLoMatt,

Edit /media/fat/linux/user-startup.sh and add the following to the end of the file:

Code: Select all

mkdir -p /var/spool/cron/crontabs
echo "*/25 * * * * touch /path/to/externalDrive/keepAwake" > /var/spool/cron/crontabs/root
crond
Save and reboot. It should work going forward. You can replace the touch command in the above text with whatever command you want at whatever interval.

The above code block will run touch once every 25 minutes to update the keepAwake file's properties with the current time which will also keep the hard drive spinning.

The echo command is putting the text in quotations (the command you want automated) in the file after the > symbol. If you want to add additional cron jobs then you will still use the echo command but you will need to use >> before each file name.

> is "save this command's output to this file name and overwrite it if it exists."
>> is "add this output at the end of the existing text in this file or create the file with this text if it doesn't exist."

So it would look like this:

Code: Select all

echo "*/25 * * * * touch /path/to/externalDrive/keepAwake" > /var/spool/cron/crontabs/root
echo "*/25 * * * * otherCommand" >> /var/spool/cron/crontabs/root
Actually since the file that contains the cron job commands gets destroyed upon power off, you could use >> with every echo command. There's no chance we're adding to an existing file accidentally when we meant to start fresh. I just wanted you to understand the difference in case you do want to use multiple jobs at some point.

This site can help you format new cron job lines properly and easily: https://crontab.guru/

If you want to hide the keepAwake file just use a period at the beginning of the file name. You can do the same with folders.

You can disable these commands from running by editing the file at /media/fat/linux/user-startup.sh and placing a # at the beginning of each line.

The way above would not be necessary on a standard Linux distro. Linux isn't as complicated as you might think if you're used to Windows or MacOS, it's just different. Things are more open to modifications so that you get to have the final say about what your computer does instead of having to accept that being decided for you by a corporation. The new Steam Deck is running on Linux, and it's a great example of how malleable Linux is and how easy it can be to use when a great team gets behind a distribution. The MiSTer is also another great example, of course, but we all know the FPGA part is the real magic with the Linux distro being tailored for speed and small size.
TLoMatt
Posts: 23
Joined: Sat Sep 03, 2022 6:21 pm
Has thanked: 10 times
Been thanked: 3 times

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by TLoMatt »

PistolsAtDawn wrote: Mon Oct 31, 2022 4:10 pm
I'm putting the following in bold for everyone, not yelling at you, OP.

I've seen this come up a few times. Cron jobs are possible but not setup to work by default on the MiSTer Linux distro. I got cron jobs working in a pretty hacky way, but it does work.

The folders /var/spool/cron and /var/spool/cron/crontabs need to be created after each startup as well as the cron job/s in the file at /var/spool/cron/crontabs/root

The cron daemon then needs to be started.

You could use crontab to create the commands in the cron job file, however it is destroyed upon shutdown.

That missing file with the desired command/s can be recreated in different ways, but the below is easier for new Linux users than explaining how to use and exit the vi editor with crontab or how to change vi to nano first and then how to move the crontab output to a different location so it's not destroyed at shutdown.

I know there are probably better ways to do this.
"Don't @ me!" :mrgreen:


Okay TLoMatt,

Edit /media/fat/linux/user-startup.sh and add the following to the end of the file:

Code: Select all

mkdir -p /var/spool/cron/crontabs
echo "*/25 * * * * touch /dev/disk/by-uuid/fd9bdf29-f845-4b26-9aa8-0502d0cd9792/keepAwake" > /var/spool/cron/crontabs/root
crond
Save and reboot. It should work going forward. You can replace the touch command in the above text with whatever command you want at whatever interval.

The above code block will run touch once every 25 minutes to update the keepAwake file's properties with the current time which will also keep the hard drive spinning.

The echo command is putting the text in quotations (the command you want automated) in the file after the > symbol. If you want to add additional cron jobs then you will still use the echo command but you will need to use >> before each file name.

> is "save this command's output to this file name and overwrite it if it exists."
>> is "add this output at the end of the existing text in this file or create the file with this text if it doesn't exist."

So it would look like this:

Code: Select all

echo "*/25 * * * * touch /dev/disk/by-uuid/fd9bdf29-f845-4b26-9aa8-0502d0cd9792/keepAwake" > /var/spool/cron/crontabs/root
echo "*/25 * * * * otherCommand" >> /var/spool/cron/crontabs/root
Actually since the file that contains the cron job commands gets destroyed upon power off, you could use >> with every echo command. There's no chance we're adding to an existing file accidentally when we meant to start fresh. I just wanted you to understand the difference in case you do want to use multiple jobs at some point.

This site can help you format new cron job lines properly and easily: https://crontab.guru/

If you want to hide the keepAwake file just use a period at the beginning of the file name. You can do the same with folders.

You can disable these commands from running by editing the file at /media/fat/linux/user-startup.sh and placing a # at the beginning of each line.

The way above would not be necessary on a standard Linux distro. Linux isn't as complicated as you might think if you're used to Windows or MacOS, it's just different. Things are more open to modifications so that you get to have the final say about what your computer does instead of having to accept that being decided for you by a corporation. The new Steam Deck is running on Linux, and it's a great example of how malleable Linux is and how easy it can be to use when a great team gets behind a distribution. The MiSTer is also another great example, of course, but we all know the FPGA part is the real magic with the Linux distro being tailored for speed and small size.
First of all I thank you very much PistolsAtDawn !!! For taking time to do tests until figuring out how to make cron jobs work in Mister and explaining it I'm aware you also edit your reply like 3 times in order to make it more understandable thanks man you're awesome !!! :D

After reading your guide and the commands at first I couldn't make it work I realized that for whatever reason trying to create the KeepAwake file using /by-uuid/ and the hdd uuid was not working , in FTP I tried to open my hdd using this directory /dev/disk/by-uuid/ in that directory I can see the hdd uuid is a shortcut access (I guess to the root of the external hdd) by clicking into it Winscp doesn't let me go through trying with ssh (in putty) it prompt me a message saying something like I don't have permissions so instead of that directory I though "why don't I try using the direct directory of my hdd?" which is media/usb0 if i'm not wrong and that did the trick , the keepAwake file was created and it was updated according to the minutes written with the script working I realized my 5 TB western digital elements was awake indeed but sadly from time to time playing Ps1 games it was (again) loosing connection which means like you guys told me that my external hdd is needing more electricity in order to be turn on (at least with some ps1 games)

I would like to know 2 things , 1 is why I can't access my external hdd by using /dev/disk/by-uuid/fd9bdf29-f845-4b26-9aa8-0502d0cd9792
Is this something related to to the misswritten command I wrote the other time ?

And with this code

Code: Select all

echo "*/25 * * * * touch /dev/disk/by-uuid/fd9bdf29-f845-4b26-9aa8-0502d0cd9792/keepAwake"
At first I though the */25 was maybe miswritten cause according from the crontab syntaxis from the site you provide me crontab uses 5 asterisks but in your code there are 6 if we take into account the 25 number also from https://crontab.guru/ there's no

Code: Select all

/*
before the number and even though your code works so I was just curious to know why is that :mrgreen: (I don't know about linux commands but i'm willing to learn :) )
User avatar
aberu
Core Developer
Posts: 1192
Joined: Tue Jun 09, 2020 8:34 pm
Location: Longmont, CO
Has thanked: 247 times
Been thanked: 411 times
Contact:

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by aberu »

*/25 means it will run at every 25th minute. Every 25 minutes that command will run.

https://crontab.guru/#*/25_*_*_*_*

the parsing of crontab in this instance for each time is delimited by a space
birdybro~
User avatar
PistolsAtDawn
Posts: 359
Joined: Fri Feb 18, 2022 7:29 pm
Has thanked: 307 times
Been thanked: 100 times

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by PistolsAtDawn »

It occurred to me too late that you likely needed to use a different path to use the drive, but I see you figured that out.

In Unix/Linux/BSD every device in a computer is treated as though it's a file. That's just how they're represented on these systems. Browsing through /dev/ you'll see the hardware on the MiSTer. Your hard drive is also this way, but you're not trying to interact with the properties of the drive (like it's filesystem labels or UUID which are also shown as files). You're actually trying to access a partition and filesystem on it, so it must be mounted to a different filesystem location that you as a user can interact with. /media/usb0 is that place for your drive.

The /media/ folder exists on your micro SD card, but the filesystem on your external drive is now usable at /media/usb0/. Why? Because a user (root in our case) is able to read and write to a folder on the SD card, and the external drive's filesystem is no exception now that it's mounted there. But it doesn't make any sense that we would try to create a new file on the drive's UUID, so even as the root user you will get a "you don't have permission to do that" type message if you try.

I'm actually a little embarrassed that I forgot that. I've been a happy Linux user since the late 90s, and I should know better. I was overly focused on getting cron working.

Do a "cat /etc/fstab" or "fdisk -l" and you'll see the drives on the MiSTer and where they're mounted as well as what options are used in mounting.

aberu I think answered your other question. Play with the site we linked about cron time formatting. You can have them scheduled for some really ridiculously-specific times. You'll get a better understanding from it than another wall of text from me.

https://crontab.guru/#1-20/2,21-40/3_10_*_5_*/2
MrMartian
Posts: 21
Joined: Wed Nov 24, 2021 4:04 pm
Has thanked: 18 times
Been thanked: 36 times

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by MrMartian »

I was wrong...
User avatar
PistolsAtDawn
Posts: 359
Joined: Fri Feb 18, 2022 7:29 pm
Has thanked: 307 times
Been thanked: 100 times

Re: HDD Sleeping Disconnecting and Connecting Too Often!

Unread post by PistolsAtDawn »

MrMartian wrote: Wed Nov 02, 2022 12:35 pm I was wrong...
About what? Just curious.
Post Reply