Difference between revisions of "USBDisks"

From SME Server
Jump to navigationJump to search
m (Gave better formatting to echo output in script.)
 
(19 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
===USB Disks===
 
===USB Disks===
 
{{Warning box|NTFS partitions are supported if you install ntfs-3g and ntfsprogs, please use either fat32/vfat or ext2/ext3/ext4}}
 
{{Warning box|NTFS partitions are supported if you install ntfs-3g and ntfsprogs, please use either fat32/vfat or ext2/ext3/ext4}}
====Introduction====
 
=====The Problem=====
 
SME will automatically mount connected USB disk drives into the /media folder.
 
Auto-mounting will be performed based both the specific USB port used to connect your drives and the order in which they are connected unless you use another method to specify mount points.
 
 
Imagine this example:
 
* Connect USB 'A' to your system.  It is automatically assigned to /dev/sdc and auto-mounted to /media/usbdisk
 
* Disconnect USB 'A' and connect USB 'B' (to the same or to a different USB port).  It will be assigned to /dev/sdc and auto-mounted to /media/usbdisk.
 
* Re-connect USB 'A' while USB 'B' is still connected, and it will be assigned to /dev/sdd and auto-mounted to /media/usbdisk1.
 
 
Or this example:
 
* Connect USB 'A' to the second USB port on your SME.  It is assigned to /dev/sdc
 
* Connect USB 'B' to the first USB port without disconnecting USB 'A'.  USB 'B' is assigned to /dev/sdd
 
* Leave both drives connected and re-boot.  USB 'B' is now connected to /dev/sdc, while USB 'A' is connected to /dev/sdd!
 
 
In one case, SME automatically mounted my USB drive in /media/DiskLabel, but, since the resulting automatic entry in /etc/fstab depended on the assigned device, the next drive I connected was also mounted in /media/DiskLabel, even though it had a completely different label!
 
 
=====The Solution=====
 
The easiest way to avoid this behavior and provide reliable mount points for your USB drives is to label each drive, then specify the desired mount point in /etc/fstab based on the drive label.  This will allow you to:
 
* Reliably mount your USB drives to drive-specific directories regardless of what device is assigned to the drive by SME.  This would allow you to specify separate USB disks for specific ibays by using lines like these in /etc/fstab:<br>
 
LABEL=Disk1 /home/e-smith/files/ibays/ibay-1/html ext3 defaults
 
LABEL=Disk2 /home/e-smith/files/ibays/ibay-2/html ext3 defaults
 
* Reliably mount multiple disks in the one location, in case you want to use a rotating set of disk drives for backups, by using lines like these in /etc/fstab (just don't connect two of these drives at the same time!):<br>
 
LABEL=Disk1 /media/backup ext3 defaults
 
LABEL=Disk2 /media/backup ext3 defaults
 
LABEL=Disk3 /media/backup ext3 defaults
 
 
 
====Supported FileSystems====
 
====Supported FileSystems====
 
=====ext2/ext3/ext4=====
 
=====ext2/ext3/ext4=====
Line 53: Line 26:
  
 
====Identifying your USB drive====
 
====Identifying your USB drive====
 +
See [[USBDisks#USB_on_SME_8|USB_on_SME_8]] for SME 8 servers.
 +
 
After connecting your USB drive, execute the command
 
After connecting your USB drive, execute the command
 
  mount
 
  mount
Line 76: Line 51:
 
You'll most likely find an existing vfat dos partition, which you have to delete first. Refer to http://wiki.contribs.org/AddExtraHardDisk#Partitioning_the_disk for steps to create a partition. In the following we assume, that you have created a single partition ''/dev/sdc1''.
 
You'll most likely find an existing vfat dos partition, which you have to delete first. Refer to http://wiki.contribs.org/AddExtraHardDisk#Partitioning_the_disk for steps to create a partition. In the following we assume, that you have created a single partition ''/dev/sdc1''.
 
</li><li>Now format the drive with an ext3 filesystem
 
</li><li>Now format the drive with an ext3 filesystem
  mkfs.ext3 -L MyLabel /dev/sdc1
+
  mkfs ext3 -L MyLabel /dev/sdc1
 
</li><li>or format the drive with an ext4 filesystem
 
</li><li>or format the drive with an ext4 filesystem
  mkfs.ext4 -L MyLabel /dev/sdc1
+
  mkfs ext4 -L MyLabel /dev/sdc1
 
</li><li>Make the mount point
 
</li><li>Make the mount point
 
  mkdir -p /media/usbdevice
 
  mkdir -p /media/usbdevice
Line 172: Line 147:
 
* Add multiple entries as described in [[USBDisks#The_Solution]] to meet your specific requirements.
 
* Add multiple entries as described in [[USBDisks#The_Solution]] to meet your specific requirements.
  
or for ntfs
+
Or for ntfs
  LABEL=MyLabel /media/affadevice ntfs-3g defaults
+
  /dev/NTFS-partition /media/affadevice ntfs-3g defaults   0      0
  
 
====Mount your disk automatically at boot-up====
 
====Mount your disk automatically at boot-up====
Line 185: Line 160:
  
 
Once the script exists you can add additional mount commands by editing S95mount_USB using:
 
Once the script exists you can add additional mount commands by editing S95mount_USB using:
  pico -w /etc/e-smith/events/local/S95mount_USB
+
  nano -w /etc/e-smith/events/local/S95mount_USB
  
 
The 'local' event will run each time your server boots up.  If you connect or disconnect a drive and need to re-mount it you can re-run your auto-mount commands using:
 
The 'local' event will run each time your server boots up.  If you connect or disconnect a drive and need to re-mount it you can re-run your auto-mount commands using:
 
  signal-event local
 
  signal-event local
 +
 
====Mount your disks automatically when connected====
 
====Mount your disks automatically when connected====
 
{{incomplete}}
 
{{incomplete}}
Line 208: Line 184:
 
We use hal for the first and manually create and mount for the second
 
We use hal for the first and manually create and mount for the second
  
 +
=====Find the Drive Name=====
 
To find  the device name, label, filesystem and the uid
 
To find  the device name, label, filesystem and the uid
 +
Create the file hal-find-by-property.sh
 +
 +
nano hal-find-by-property.sh
 +
 +
paste the text below into the file and save it.
  
 
  #! /bin/bash
 
  #! /bin/bash
Line 218: Line 200:
 
         fs=$(hal-get-property --udi $udi --key volume.fstype)
 
         fs=$(hal-get-property --udi $udi --key volume.fstype)
 
         lb=$(hal-get-property --udi $udi --key volume.label)
 
         lb=$(hal-get-property --udi $udi --key volume.label)
         echo $dev": label "$lb",file system: "$fs ",uid: "$udi
+
         echo device: $dev", label: "$lb", file system: "$fs", uid: "$udi
 
     fi
 
     fi
 
  done
 
  done
  
To check a mount point exists and mount
+
Make the file executable.
 +
chmod +x hal-find-by-property.sh
  
 +
Run the file with the USB drive attached and read the output of the file from the terminal.
 +
sh hal-find-by-property.sh
 +
 +
The output will tell you the device, volume label, file system type, and the uid. If this is a new drive you have to run fdisk to create a partition and format the drive before you can mount it.
 +
=====Run fdisk on the Drive=====
 +
 +
To create a partition with fdisk, use the device output from the hal-find-by-property.sh script as the argument to the fdisk command. If the output from hal-find-by-property.sh is /dev/sdb1 the fdisk command will be
 +
fdisk /dev/sdb1
 +
 +
You can list the partitions on the disk by entering the p command when prompted. It's a good idea to see what partitions exist before you start. If the drive is pre-formatted for fat or ntfs it's good practice to delete the factory partitions.
 +
 +
If this is a new drive, you will need to create a new primary partition. Do this by entering an n at the prompt followed by p for primary. Enter a partition number of 1. fdisk will then ask for the first and last sector. If you are formatting a new disk, hit enter at both questions. This will use the entire disk. When the process ends, hit p to verify you have the disk setup like you want it. Once you are happy with the partition table, hit w to write the partition table and exit fdisk.
 +
 +
=====Format The Drive=====
 +
 +
You now need to format the drive. To format the drive with an ext3 filesystem located at /dev/sdb1 and give it a label of usbdrive, issue the following command:
 +
mkfs.ext3 -L usbdrive /dev/sdb1
 +
 +
=====Mount The Drive=====
 +
 +
Finally you mount the drive. As mentioned before, the mount point has to exist before you issue the mount command. A typical place for USB drives to be mounted is /media. If there is already a mount point in /media you can check it by listing the directory contents of media.
 
  ll /media
 
  ll /media
mount /dev/sda1 /media/120Gbfat32 (ie. mount /media/120Gbfat32 no longer works)
+
 
  ls -lh /media/120Gbfat32/
+
Once you have a suitable location, make sure nothing is mounted there by issuing the mount command from the terminal with no arguments. If you wanted to mount the drive /dev/sdb1 at /media/usbdrive, you would issue the following command
  umount /dev/sda1 (when finished)
+
 
 +
  mount /dev/sdb1 /media/usbdrive
 +
  ls -lh /media/usbdrive/
 +
 
 +
=====UnMount the Drive=====
 +
 
 +
Unmount the drive before unplugging it with
 +
  umount /dev/sdb1
  
 
Note:  When configuring a USB drive to be used by the built in [[SME_Server:Documentation:Administration_Manual:Chapter10|Workstation Backup To USB]] you must ensure that the drive is not mounted automatically.  When configuring the Workstation Backup, it will look for unmounted removable drives and allow you to select what you want.  If the drive is already mounted when you configure it then you will get an error saying "No Removable disk available".
 
Note:  When configuring a USB drive to be used by the built in [[SME_Server:Documentation:Administration_Manual:Chapter10|Workstation Backup To USB]] you must ensure that the drive is not mounted automatically.  When configuring the Workstation Backup, it will look for unmounted removable drives and allow you to select what you want.  If the drive is already mounted when you configure it then you will get an error saying "No Removable disk available".

Latest revision as of 14:29, 8 March 2016

USB Disks

Warning.png Warning:
NTFS partitions are supported if you install ntfs-3g and ntfsprogs, please use either fat32/vfat or ext2/ext3/ext4


Supported FileSystems

ext2/ext3/ext4
  1. recommended for USB drives intended for use only on SME
  2. works great under Linux
  3. supports hard links, symlinks
  4. can be read under Windows using explore2fs from http://www.chrysocome.net/explore2fs
  5. File & Disk size limits depend on the block size used when formatting the disk. SME defaults to a 1K block size.
    1K block size: 16GB max file size, 2TB max disk size
    2K block size: 256GB max file size, 8 TB max disk size.
    4K block size: 2TB max file size, 16 TB max disk size
  6. You can determine your current block size using (replace /dev/sdc1 with the correct value for your USB drive)
    dumpe2fs -h /dev/sdc1
NTFS

You can format and use the NTFS if you install from epel ntfs-3g and ntfsprogs See the complete features of ntfs-3g

yum install ntfs-3g ntfsprogs --enablerepo=epel
fat32/vfat
  1. pre-configured on most USB hard drives
  2. supported natively by Windows
  3. (2^32)-1 byte ( 4GB - 1 byte) maximum file size
  4. Requires occasional defragmentation for optimal speed
  5. Maximum disk size dependent on cluster size. 32KB Clusters would allow a disk of approximately 8TB

Identifying your USB drive

See USB_on_SME_8 for SME 8 servers.

After connecting your USB drive, execute the command

mount

and look for anything mounted in /media/???.

If your drive did not auto-mount, search /var/log/messages for the kernel commands related to your device using:

egrep "(kernel|fstab|scsi).*(usb|USB)" /var/log/messages

You should have a line logged by fstab-sync giving the auto-mount folder created and the device name for your new device that looks like this:

Sep 23 17:11:14 office fstab-sync[32193]: added mount point /media/usbdisk1 for /dev/sdd1

You should now be able to mount your USB drive using the values found in /var/log/messages. In our example, that would mean:

mount /media/usbdisk1

or

mount /dev/sdd1

Formatting your USB drive

ext2/ext3/ext4
  1. Copied with slight changes from Affa#Alternatively_setup_a_USB_drive
  2. Connect a USB hard disk to the USB Bus. Now you must determine what device the kernel has assigned to the drive. View the /var/log/message and search for Initializing USB Mass Storage driver. A few lines below you'll find the name of the device. In this example it is sdc. Replace /dev/sdc by your device in following instructions. Use the fdisk program to create a linux partition fdisk /dev/sdc You'll most likely find an existing vfat dos partition, which you have to delete first. Refer to http://wiki.contribs.org/AddExtraHardDisk#Partitioning_the_disk for steps to create a partition. In the following we assume, that you have created a single partition /dev/sdc1.
  3. Now format the drive with an ext3 filesystem mkfs ext3 -L MyLabel /dev/sdc1
  4. or format the drive with an ext4 filesystem mkfs ext4 -L MyLabel /dev/sdc1
  5. Make the mount point mkdir -p /media/usbdevice
  6. Customize /etc/fstab as shown here: Customizing fstab
  7. Mount the drive mount reads /etc/fstab to look for instructions on what to do if not specifically told on the command line. After updating /etc/fstab as recommended, you can "mount" your USB disk reliably by specifying either the target directory: mount /media/usbdevice or the disk label: mount LABEL=MyLabel
  8. Crosscheck your work using df -h
fat32/vfat

You can format your drive from your SME server using

mkfs.vfat -n MyLabel /dev/sdd1

Since this command will destroy irretrievably all information on the specified device (/dev/sdd1 in this case), be very sure you know exactly which device is the drive you want to format!

NTFS

You can format and use the NTFS if you install from epel ntfs-3g and ntfsprogs

yum install ntfs-3g ntfsprogs --enablerepo=epel

then

 mkfs.ntfs -n MyLabel /dev/sdd1

Since this command will destroy irretrievably all information on the specified device (/dev/sdd1 in this case), be very sure you know exactly which device is the drive you want to format!

labeling your USB drive

Only required if you didn't label your drive using the previous section.

ext2/ext3/ext4
e2label /dev/sdd1 MyLabel
fat32/vfat

mtools

SME includes 'mtools' which can be used to manage FAT, FAT32, and VFAT partitions.

'mtools' uses drive letters instead of device names to access devices. These drive letters must be defined in /etc/mtools.conf before any of the mtools will work.

You can add your USB drive to /etc/mtools.conf using a command like the one shown below. However,

  • be sure to replace /dev/sdd1 with the device name identified above for your USB drive
  • be aware that the assigned device may change each time you reconnect your drive or reboot!
echo 'drive e: file="/dev/sdd1"' >> /etc/mtools.conf

Once you have created the drive letter in mtools.conf you can view or edit the disk label using the following commands:

  • Show the current label:
mlabel -s e:
  • Clear the volume label:
mlabel -c e:
  • Assign a new volume label:
mlabel e:MyLabel

dosfslabel

SME also includes dosfslabel which appears to be related to labeling "dos" drives.

dosfslabel /dev/sdd1

produces the following output:

Warning: FAT32 support is still ALPHA.
MyLabel

While it is safe to use dosfslabel to check the label on a drive, you should not use it to create or change the label on any drive that contains data you care about.

Dual Partitions

A method to use both ext3 and vfat
eg. an ext3 partition the size of your server, for an rsync backup.
the rest as vfat for easier portability

Plug in your drive or check it is unmounted

fdisk /dev/sdd
p - print existing partitions
d - delete partitions
n - create new partitions
p (primary), 1, +160G (the size you want)
n - create 2nd partition
p (primary), 2, return (rest of the disk)
p - print and check
w - write

mkfs.ext3 -L MaxExt3 /dev/sdd1
mkfs.vfat -n MaxVfat /dev/sdd2

Customizing fstab

Add the following line to the /etc/fstab

LABEL=MyLabel /media/affadevice ext3 defaults
  • Replace 'ext3' with 'vfat' if your drive is formatted as fat32/vfat.
  • Replace 'ext3' with 'ext4' if your drive is formatted as ext4.
  • Replace /media/affadevice with the folder in which you want your USB drive mounted (here and in all "mount" commands)
  • Make sure that the mount directory (/media/affadevice in this example) exists and is empty! Linux will not mount a drive in a directory that is not empty.
  • Add multiple entries as described in USBDisks#The_Solution to meet your specific requirements.

Or for ntfs

/dev/NTFS-partition /media/affadevice ntfs-3g defaults	  0       0

Mount your disk automatically at boot-up

Use the following commands to create a script that will run at each re-boot if you want to re-mount your USB disk automatically:

cat <<EOF > /etc/e-smith/events/local/S95mount_USB
#! /bin/sh
mount LABEL=MyLabel
EOF

chmod +x /etc/e-smith/events/local/S95mount_USB

Once the script exists you can add additional mount commands by editing S95mount_USB using:

nano -w /etc/e-smith/events/local/S95mount_USB

The 'local' event will run each time your server boots up. If you connect or disconnect a drive and need to re-mount it you can re-run your auto-mount commands using:

signal-event local

Mount your disks automatically when connected

Incomplete.png Incomplete:
This article or section needs to be expanded. Please help to fill the gaps or discuss the issue on the talk page


|Please edit this section to specify how to get SME to automatically mount USB drives when they are connected

Format limitations when linking disk to an ibay

If you wish to mount an external USB drive, and then add a link in a ibay to that drive, then in order to change ownership and have write access you must format the drive as ext3 only. Refer this forum thread for info http://forums.contribs.org/index.php?topic=40240.new;topicseen#new

More Information

USB on SME 8

Things change on SME 8

It's difficult to work out what your device name is ie /dev/sda1
The mount point isn't automatically added in /etc/fstab

We use hal for the first and manually create and mount for the second

Find the Drive Name

To find the device name, label, filesystem and the uid Create the file hal-find-by-property.sh

nano hal-find-by-property.sh

paste the text below into the file and save it.

#! /bin/bash
hal-find-by-property --key volume.fsusage --string filesystem |
while read udi ; do
   # ignore optical discs
   if test "$(hal-get-property --udi $udi --key volume.is_disc)" == "false" ; then
       dev=$(hal-get-property --udi $udi --key block.device)
       fs=$(hal-get-property --udi $udi --key volume.fstype)
       lb=$(hal-get-property --udi $udi --key volume.label)
       echo device: $dev", label: "$lb", file system: "$fs", uid: "$udi
   fi
done

Make the file executable.

chmod +x hal-find-by-property.sh

Run the file with the USB drive attached and read the output of the file from the terminal.

sh hal-find-by-property.sh

The output will tell you the device, volume label, file system type, and the uid. If this is a new drive you have to run fdisk to create a partition and format the drive before you can mount it.

Run fdisk on the Drive

To create a partition with fdisk, use the device output from the hal-find-by-property.sh script as the argument to the fdisk command. If the output from hal-find-by-property.sh is /dev/sdb1 the fdisk command will be

fdisk /dev/sdb1

You can list the partitions on the disk by entering the p command when prompted. It's a good idea to see what partitions exist before you start. If the drive is pre-formatted for fat or ntfs it's good practice to delete the factory partitions.

If this is a new drive, you will need to create a new primary partition. Do this by entering an n at the prompt followed by p for primary. Enter a partition number of 1. fdisk will then ask for the first and last sector. If you are formatting a new disk, hit enter at both questions. This will use the entire disk. When the process ends, hit p to verify you have the disk setup like you want it. Once you are happy with the partition table, hit w to write the partition table and exit fdisk.

Format The Drive

You now need to format the drive. To format the drive with an ext3 filesystem located at /dev/sdb1 and give it a label of usbdrive, issue the following command:

mkfs.ext3 -L usbdrive /dev/sdb1
Mount The Drive

Finally you mount the drive. As mentioned before, the mount point has to exist before you issue the mount command. A typical place for USB drives to be mounted is /media. If there is already a mount point in /media you can check it by listing the directory contents of media.

ll /media

Once you have a suitable location, make sure nothing is mounted there by issuing the mount command from the terminal with no arguments. If you wanted to mount the drive /dev/sdb1 at /media/usbdrive, you would issue the following command

mount /dev/sdb1 /media/usbdrive
ls -lh /media/usbdrive/
UnMount the Drive

Unmount the drive before unplugging it with

umount /dev/sdb1 

Note: When configuring a USB drive to be used by the built in Workstation Backup To USB you must ensure that the drive is not mounted automatically. When configuring the Workstation Backup, it will look for unmounted removable drives and allow you to select what you want. If the drive is already mounted when you configure it then you will get an error saying "No Removable disk available".

The same applies for the Console Backup to USB