Pete's Log: iSCSI Woes
Entry #2001, (Coding, Hacking, & CS stuff)(posted when I was 43 years old.)
All I wanted was to connect to an iSCSI disk from a Raspberry Pi and have it auto-mount on boot. All it's taken is two nap times.
The Pi in question is running Ubuntu, and the only results I seemed to be able to achieve yesterday were either a pi that didn't boot at all (at least not into a reachable via the network state) or a pi that booted fine but didn't auto-mount the volume. Today I finally got it working.
My NAS has three network interfaces (an internal one and two ethernet ports-I have one port on the "home" network and the other on my restricted IOT/HA network that the pis are on) and currently has two iSCSI targets. So when I initially went through the exercise to connect from the pi to the NAS via iSCSI, I ran a discovery, which I think was my mistake.
sudo iscsiadm -m discovery -t sendtargets -p 192.168.right.network
This discovered six items—two targets times three portals. I did not realize at the time that iscsiadm added all six to its database with node.startup = automatic. I then proceeded to log in to the one target I cared about:
sudo iscsiadm -m node -T iqn.that.i.cared.about -p 192.168.right.network:3260 --login
That worked great, since I specified the right target and the right portal. The target got added as /dev/sda, I happily ran fdisk and mkfs.ext4 and mounted the volume and everything looked great. Looked up the modern incantation to add to /etc/fstab and rebooted. And the disk wasn't mounted. Spent some quality time learning how systemd mounts network volumes and created a .mount file instead of an fstab entry, but that didn't get me anywhere. Except I do appreciate that I understand systemd just a tad bit more now.
This whole time I had been assuming it was a problem with systemd trying to mount the volume before the network was ready, but I finally paid enough attention to the logs:
Sep 11 14:50:38 rpi-01 iscsiadm[836]: iscsiadm: Could not log into all portals Sep 11 14:50:38 rpi-01 iscsiadm[836]: Logging in to [iface: default, target: iqn.i.dont.want.mounted.on.this.pi, portal: 192.168.wrong.network,3260] (multiple) Sep 11 14:50:38 rpi-01 iscsiadm[836]: Logging in to [iface: default, target: iqn.i.dont.want.mounted.on.this.pi, portal: 172.17.0.1,3260] (multiple) Sep 11 14:50:38 rpi-01 iscsiadm[836]: Logging in to [iface: default, target: iqn.i.dont.want.mounted.on.this.pi, portal: 192.168.right.network,3260] (multiple) Sep 11 14:50:38 rpi-01 iscsiadm[836]: Logging in to [iface: default, target: iqn.that.i.cared.about, portal: 192.168.wrong.network,3260] (multiple) Sep 11 14:50:38 rpi-01 iscsiadm[836]: Logging in to [iface: default, target: iqn.that.i.cared.about, portal: 172.17.0.1,3260] (multiple) Sep 11 14:50:38 rpi-01 iscsiadm[836]: Logging in to [iface: default, target: iqn.that.i.cared.about, portal: 192.168.right.network,3260] (multiple) Sep 11 14:50:38 rpi-01 iscsiadm[836]: Login to [iface: default, target: iqn.that.i.cared.about, portal: 192. 168.right.network,3260] successful. Sep 11 14:50:38 rpi-01 systemd[1]: open-iscsi.service: Main process exited, code=exited, status=8/n/a Sep 11 14:50:38 rpi-01 systemd[1]: open-iscsi.service: Failed with result 'exit-code'. Sep 11 14:50:38 rpi-01 systemd[1]: Failed to start Login to default iSCSI targets.
So we have six login attempts, of which only one (iqn.that.i.cared.about via 192. 168.right.network) succeeds. So open-iscsi never starts and my target never mounts.
Login to default iSCSI targets is the description defined in /lib/systemd/system/open-iscsi.service where we find that it launches the open-iscsi service via this command:
ExecStart=/sbin/iscsiadm -m node --loginall=automatic
loginall=automatic, you say? That's when I dug in and found that iscsiadm had added all the nodes it discovered with node.startup = automatic.
I suppose one option would have been to change node.startup to manual for those, but since I will never want to connect to any of them anyway, I just deleted them:
iscsiadm -m node -o delete -T iqn.i.dont.want.mounted.on.this.pi iscsiadm -m node -o delete -T iqn.that.i.cared.about -p 192.168.wrong.network:3260,1 iscsiadm -m node -o delete -T iqn.that.i.cared.about -p 172.17.0.1:3260,1
Quick reboot and my volume is finally mounted at boot. Phew!