JIGDO - create SME Server ISOs

jigdo and SME 7

Introduction

jigdo is a method of downloading CD and DVD images over the Internet (and other large files) and is used by Debian and Ubuntu when new releases are created. jigdo files contain a list of the files that need to be included in the new CD or DVD ISO image and you can get the files from local sources and over the Internet. This means when you want to create ISO images for updated versions of Debian, you only need to download the changed files. This saves both bandwidth and and time.

jigdo's approach is unique due to the way it identifies the small pieces that the large file consists of: For instance, if the file is a CD image, the individual files on the CD are contained somewhere within the image file. jigdo-file is capable of finding these files, so if copies of the individual files on the CD are stored separately on the server, jigdo-lite can be made to download these files one by one, and assemble the CD image on the fly on the user's computer. This scheme has several advantages:

   * The large image does not need to be stored on the server, instead only the many small files contained in the image. This works with CD images, DVD images (both ISO9660 and UDF format), uncompressed zip files, tar archives... jigdo-file is format independent.
   * In spite of the above, jigdo creates a bit-exact copy of the original image on the user's machine. (To achieve this, the directory data, boot block etc. of the image is stored in a special .template file which is distributed alongside the .jigdo file.)
   * There is full control over where jigdo-lite will download the individual parts. It is possible to define mirrors so users can choose their nearest mirror.
   * jigdo relies on standard HTTP/FTP, no support for special protocols like rsync, and no installation of special software is required on the mirrors.
   * jigdo-lite supports resuming aborted downloads, or continuing the download with another mirror if the current one proves slow.
   * It is possible to "upgrade" a CD image: If a new version of an image is released, jigdo-lite can download only the data that has changed, the remaining data is read from the outdated version.
   * It is possible to release modified versions: If someone distributes a CD image and you have made a few small modifications, e.g. added some files, then you only need to upload the data for your modifications to your webspace. All the data from the original CD image is still fetched from the original site.

If you have read this far, you may be confused now, asking yourself what jigdo does and doesn't do. :-) It seems that the jigdo concept is a bit difficult to grasp at first. If you're still interested, have a look at the respective section of the HOWTO and at the Examples section of the jigdo-file manual.

Installing jigdo on CentOS and RHEL

You need to download and install the correct RPM file from http://dag.wieers.com/rpm/packages/jigdo/ . The choices currently available are for RHEL/CentOS 1 through 5 (they're the ones labelled eg jigdo-0.7.3-1.el5.rf.i386.rpm where the text in bold indicates Enterprise Linux 5), Red Hat 9, and Fedora 1 through 3. There are also source RPMs available.

Download the file first, and then use yum from the command line so install it running either as the root user, or using the sudo command:

1. Downloading files for CentOs 5 workstation:

wget http://dag.wieers.com/rpm/packages/jigdo/jigdo-0.7.3-1.el5.rf.i386.rpm yum install jigdo-0.7.3-1.el5.rf.i386.rpm

[Note: download the el4 files for CentOs 4 (SME7) if you do the job on this platform...]

ok

GET 7.5.1 27/6/10

Create JIGDO directory and move latest ISO there:

[root@canopus JIGDO]# pwd
/home/chris/JIGDO
[root@canopus JIGDO]# ls
smeserver-7.5rc2-i386.iso

SME latest jigdo files are here:

http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/testing/7.5.1/iso/i386/

Lets create a mount point:

[root@canopus /]# cd mnt/
[root@canopus mnt]# ls
[root@canopus mnt]# mkdir iso
[root@canopus mnt]# ls iso

and mount

mount -o loop smeserver-7.5rc2-i386.iso /mnt/iso

[root@canopus JIGDO]# jigdo-lite

Jigsaw Download "lite"
Copyright (C) 2001-2005  |  jigdo@
Richard Atterer          |  atterer.net

-----------------------------------------------------------------
To resume a half-finished download, enter name of .jigdo file.
To start a new download, enter URL of .jigdo file.
You can also enter several URLs/filenames, separated with spaces,
or enumerate in {}, e.g. `http://server/cd-{1_NONUS,2,3}.jigdo'

1. Provide URL when requested:

jigdo: http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/testing/7.5.1/iso/i386/smeserver- 7.5.1-i386.jigdo

Downloading .jigdo file --2010-06-27 10:31:52--

http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/testing/7.5.1/iso/i386/smeserver-7.5.1- i386.jigdo

Resolving distro.ibiblio.org... 152.46.7.109 Connecting to distro.ibiblio.org|152.46.7.109|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 45384 (44K) [text/plain] Saving to: `smeserver-7.5.1-i386.jigdo'

100%[=======================================================================================================================================>] 45,384 --.-K/s in 0.001s

2010-06-27 10:31:52 (46.0 MB/s) - `smeserver-7.5.1-i386.jigdo' saved [45384/45384]

---

Images offered by `http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/testing/7.5.1/iso/i386/smeserver-7.5.1-i386.jigdo':

 1: smeserver-7.5.1-i386.iso

---

If you already have a previous version of the CD you are downloading, jigdo can re-use files on the old CD that are also present in the new image, and you do not need to download them again. Mount the old CD ROM and enter the path it is mounted under (e.g. `/mnt/cdrom'). Alternatively, just press enter if you want to start downloading the remaining files.

1. Provide mount point information:

Files to scan: /mnt/iso

Downloading .template file --2010-06-27 10:33:37-- http://distro.ibiblio.org/pub/linux/distributions/smeserver/releases/testing/7.5.1/iso/i386/smeserver-7.5.1-i386.template Resolving distro.ibiblio.org... 152.46.7.109 Connecting to distro.ibiblio.org|152.46.7.109|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 4622904 (4.4M) [text/plain] Saving to: `smeserver-7.5.1-i386.template'

100%[=======================================================================================================================================>] 4,622,904 422K/s in 12s

2010-06-27 10:33:50 (362 KB/s) - `smeserver-7.5.1-i386.template' saved [4622904/4622904]

Found 504 of the 595 files required by the template Copied input files to temporary file `smeserver-7.5.1-i386.iso.tmp' - repeat command and supply more files to continue

---

If you already have a previous version of the CD you are downloading, jigdo can re-use files on the old CD that are also present in the new image, and you do not need to download them again. Mount the old CD ROM and enter the path it is mounted under (e.g. `/mnt/cdrom'). Alternatively, just press enter if you want to start downloading the remaining files.

You can also enter a single digit from the list below to select the respective entry for scanning:

 1: /mnt/iso

Files to scan:

[Press ENTER] Dowload of 91 files takes place against various sites.

---

Finished! The fact that you got this far is a strong indication that `smeserver-7.5.1-i386.iso' was generated correctly. I will perform an additional, final check, which you can interrupt safely with Ctrl-C if you do not want to wait.

OK: Checksums match, image is good!

[root@canopus JIGDO]# ls index.html jigdo-3DJYYty-KSM jigdo-file-cache.db smeserver-7.5.1-i386.iso smeserver-7.5.1-i386.jigdo smeserver-7.5.1-i386.template smeserver-7.5rc2-i386.iso [root@canopus JIGDO]# ls -lah total 1.3G drwxr-xr-x 3 root root 4.0K Jun 27 11:43 . drwx------ 59 chris chris 4.0K Jun 27 11:19 .. -rw-r--r-- 1 root root 1.2K Jun 27 10:08 index.html drwxr-xr-x 2 root root 4.0K Jun 27 10:22 jigdo-3DJYYty-KSM -rw-r--r-- 1 root root 332K Jun 27 10:34 jigdo-file-cache.db -rw-r--r-- 1 root root 650M Jun 27 11:43 smeserver-7.5.1-i386.iso < Here! -rw-r--r-- 1 root root 45K Jun 25 01:35 smeserver-7.5.1-i386.jigdo -rw-r--r-- 1 root root 4.5M Jun 25 01:35 smeserver-7.5.1-i386.template -rw-rw-r-- 1 chris chris 650M May 6 18:43 smeserver-7.5rc2-i386.iso