Diskless Linux boot over iSCSI

A step-by-step example of a Linux configuration to configure iSCSI diskless booting.
These steps can also be used for diskless NFS booting.
In our instructions, we will configure and load the IBM 8247-22L server, ppc64le architecture.
The OS we will be loading will be Ubuntu 14.04.
Fedora 18, x86 architecture will act as the OS on which the image connected via iSCSI (iscsi target) will live.
It is also possible to use Ubuntu 15.04 and any other new version of Fedora - 20, 21, 22, etc.
The instruction assumes that iscsi target will be on x86 architecture, and iscsi initiator on ppc64le,
it is better for you if you have everything the same.

We will follow these steps:
  1. Create a disk.
  2. Set up and run iSCSI target.
  3. Install Ubuntu on a USB flash drive.
  4. Let's take the data we need from the flash drive system.
  5. Install Ubuntu on a remote iSCSI drive.
  6. Create an initrd image with iSCSI support.
  7. Let's configure the PXE server as part of the DHCP server.
  8. Let's configure the PXE server in the TFTP server part.
  9. Finally, boot from PXE via iSCSI.

Configuring iSCSI target

We connect our server with a network cable to the machine where Fedora is installed, let the name of the network interface be em1 here.
Also, let the IP of this machine on em1 be 169.254.2.140.
On this machine, create a 10GiB container:
ifconfig em1 169.254.2.140/24 up
dd if=/dev/zero of=/disk1.iscsi bs=1M count=10240
Configuring, connecting and launching iSCSI target: 
tgtadm --lld iscsi --op new --mode target --tid 1 -T server1:disk1
tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /disk1.iscsi
tgtadm --lld iscsi --op bind --mode target --tid 1 -I 169.254.2.148
tgtadm --lld iscsi --op show --mode target #to check
The latter command allows access to the address 169.254.2.148, this is the address of the server we connected.
The tgtadm utility lives in the scsi-target-utils package.
If we use complex iptables rules, redirection, masquerade, do not forget to allow packet forwarding: 
vi /etc/sysctl.conf
net.ipv4.ip_forward = 1

Hint for loopback mount

After I have the data in this file:
parted /disk1.iscsi

(parted) u
Unit?  [compact]? B
(parted) print
Model:  (file)
Disk /disk1.iscsi: 4294967296B
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags: 

Number  Start        End          Size         File system  Name  Flags
 1      1048576B     8388607B     7340032B                        boot
 2      8388608B     4048551935B  4040163328B  ext4
 3      4048551936B  4293918719B  245366784B

mount -o loop,ro,offset=8388608 /disk1.iscsi /mnt/

Installing the OS on a USB flash drive

Download the Ubuntu 15.04 (or other) image for our server http://cdimage.ubuntu.com/releases/15.04/release/ -- use PowerPC64 Little-Endian server install image.
* For POWER8 Little-Endian computers, such as Power Systems S8xxL Linux-only servers.

We will write this image to a USB flash drive or to a CD-ROM.
Then install the image from the CD-ROM to FLASH, or from one FLASH to another, from CD-ROM or FLASH to the hard disk.

We will need this installation in order to:
  1. After loading into it to form initrd with iSCSI support.
  2. Also, get the kernel image from here.
  3. Also, get the pxelinux.0 file from here.

Create an initrd image with iSCSI support, copy the kernel image and the necessary data

mkdir /out/
cp -ia /boot/vmlinux-3.13.0-24-generic /out/ #copying kernel
aptitude install -y syslinux-common
cp -ai /usr/lib/syslinux/pxelinux.0 /out/
aptitude install sysv-rc-conf
echo "iscsi" >> /etc/initramfs-tools/modules
aptitude install open-iscsi
# This indicator file makes mkinitramfs copy the necessary
# scripts and utilities in the initrd image.
touch /etc/iscsi/iscsi.initramfs
aptitude install initramfs-tools
mkinitramfs -o /out/initrd.img-3.13.0-24-generic
# Delete the indicator file, otherwise our system will be from a flash drive
# next time try to boot via iSCSI.
rm /etc/iscsi/iscsi.initramfs
To install packages, you need to bring the OS to the Internet.

Installing Ubuntu on a Remote iSCSI Disk

In the "Ununut" installer, you can select the option to connect extended drives.
Having selected it, you need to specify the iSCSI target name as server1:disk1.
Specify 169.254.2.140 as the IP address, as the port 3260.
After which, as usual, partition the remote disk and install Ubuntu again.

Set up a PXE server

We need to configure DHCP and TFTP server.
For Fedora, the names of the packages we need are: 
yum install dhcp tftp-server
If for some reason you raise your iSCSI "target" to Ubuntu, then: 
aptitude install isc-dhcp-server tftpd-hpa

/bin/systemctl start  dhcpd.service
/bin/systemctl status  dhcpd.service

vi /etc/xinetd.d/tftp
change this:
        disable                 = yes
to this:
        disable                 = no

/bin/systemctl restart xinetd.service
/bin/systemctl start xinetd.service
/bin/systemctl status xinetd.service
You can listen to all the commands going to the TFTP server like this: tcpdump -nnn -i em1 port 69 
vi /etc/dhcp/dhcpd.conf

allow booting;
allow bootp;

subnet 169.254.2.0 netmask 255.255.255.0 {
    range 169.254.2.148 169.254.2.148;
    option broadcast-address 169.254.2.255;
    option routers 169.254.2.140;
    option domain-name-servers 8.8.8.8;
    filename "/pxelinux.0";
}

host client {
# There should be a MAC address of the client’s network card (interface) written through colons.
    hardware ethernet xx:xx:xx:xx:xx:xx;
    fixed-address 169.254.2.148;
}
Now PXE configuration: 
mkdir /var/lib/tftpboot/pxelinux.cfg
# Instead of xx-xx-xx-xx-xx-xx, it is necessary to substitute the MAC address of the network card of our server,
# written in lowercase, with a dash between octets.
# You need to take the MAC name of the standard interface (not the MGMT port) and add "01-" to it at the beginning of the file name

vi /var/lib/tftpboot/pxelinux.cfg/01-xx-xx-xx-xx-xx-xx

DEFAULT linux

label linux
    kernel vmlinux-3.13.0-24-generic
    append initrd=initrd.img-3.13.0-24-generic root=/dev/sda2 ip=dhcp iscsi_initiator=iqn.com.domain.client:default iscsi_target_name=server1:disk1 iscsi_target_ip=169.254.2.140 iscsi_target_port=3260 iscsi_username= iscsi_password=
    ipappend 2
In arguments, the adra may need to specify console=tty0 if you see the kernel loading, but you do not see initrd starting and loading the system.
You may need to change root=/dev/sda2 if you partitioned the drive using iSCSI differently.
Previously copied to flash drive into the /out/ files
initrd.img-3.13.0-24-generic pxelinux.0 vmlinux-3.13.0-24-generic
copy to: chmod 644 /var/lib/tftpboot/.
Then we set the world-readable rights: 
chmod 644 /var/lib/tftpboot/* /var/lib/tftpboot/pxelinux.cfg/*
chmod 755 /var/lib/tftpboot/pxelinux.cfg
Everything ready to boot from the PXE server.

We load the server on PXE from iSCSI disk

In the case of the IBM servers, when they start, a boot manager named Petitboot is loaded. It scans the network and local media for download sources, in our case our network boot with the name Linux will be displayed, here is what it will look like settings: 
Petitboot Option Editor
────────────────────────────────────────────────────────────────────────────── 
 Device:         (*) Specify paths/URLs manually

 Kernel:         tftp://169.254.2.140/vmlinux-3.13.0-24-generic                
 Initrd:         tftp://169.254.2.140/initrd.img-3.13.0-24-generic             
 Device tree:                                                                  
 Boot arguments: initrd=initrd.img-3.13.0-24-generic root=/dev/sda2 ip=dhcp iscsi_initiator=iqn.com.domain.client:default iscsi_target_name=server1:disk1 iscsi_target_ip=169.254.2.140 iscsi_target_port=3260 console=tty0  iscsi_username= iscsi_password= BOOTIF=xx:xx:xx:xx:xx:xx