Post by John FerlanPost by TomKPost by Martin KletzanderPost by TomKPost by John FerlanHaving/using a root squash via an NFS pool is "easy" (famous last words)
Create some pool XML (taking the example I have)
% cat nfs.xml
<pool type='netfs'>
<name>rootsquash</name>
<source>
<host name='localhost'/>
<dir path='/home/bzs/rootsquash/nfs'/>
<format type='nfs'/>
</source>
<target>
<path>/tmp/netfs-rootsquash-pool</path>
<permissions>
<mode>0755</mode>
<owner>107</owner>
<group>107</group>
</permissions>
</target>
</pool>
In this case 107:107 is qemu:qemu and I used 'localhost' as the
hostname, but that can be a fqdn or ip-addr to the NFS server.
You've already seen my /etc/exports
virsh pool-define nfs.xml
virsh pool-build rootsquash
virsh pool-start rootsquash
virsh vol-list rootsquash
Now instead of
<disk type='file' device='disk'>
<source file='/var/lib/one//datastores/0/38/disk.0'/>
<target dev='hda'/>
<driver name='qemu' type='qcow2' cache='none'/>
</disk>
<disk type='volume' device='disk'>
<driver name='qemu' type='qemu' cache='none'/>
<source pool='rootsquash' volume='disk.0'/>
<target dev='hda'/>
</disk>
The volume name may be off, but it's perhaps close. I forget how to do
the readonly bit for a pool (again, my focus is elsewhere).
Of course you'd have to adjust the nfs.xml above to suit your
environment and see what you see/get. The privileges for the pool and
volumes in the pool become the key to how libvirt decides to "request
access" to the volume. "disk.1" having read access is probably not an
issue since you seem to be using it as a CDROM; however, "disk.0" is
going to be used for read/write - thus would have to be appropriately
configured...
Thanks John! Appreciated again.
No worries, handle what's on the plate now and earmark this for checking
once you have some free cycles. I can temporarily hop on one leg by
using Martin Kletzander's workaround (It's a POC at the moment).
I'll have a look at your instructions further but wanted to find out if
that config nfs.xml is a one time thing correct? I'm spinning these up
at will via the OpenNebula GUI and if I have update for each VM, that
breaks the Cloud provisioning. I'll go over your notes again. I'm
optimistic. :)
The more I'm thinking about it, the more I am convinced that the
workaround is actually not a workaround. The only thing you need to do
is having execute for others (precisely for 'nobody' on the nfs share)
in the whole path on all directories. Without that even the pool won't
be usable from libvirt. However it does not pose any security issue as
it only allows others to check the path. When qemu is launched, it has
the proper "label", meaning uid:gid to access the file so it will be
able to read/write or whatever permissions you set there. It's just
that libvirt does some checks that the path exists for example.
Hope that's understandable and it will resolve your issue permanently.
Have a nice day,
Martin
_______________________________________________
libvirt-users mailing list
https://www.redhat.com/mailman/listinfo/libvirt-users
The only reason I said that this might be a 'workaround' is due to John
Farlan commenting that he'll look at this later on. Ideally the
opennebula community keeps the other permissions to nill and presumably
they work on NFSv3 per the forum topic I included earlier from them.
But if setting the permissions on nobody to allow for the functionality,
I would be comfortable with that.
Martin and I were taking different paths... But yes, it certainly makes
sense given your error message about canonical path and the need for
eXecute permissions... I think I started wondering about that first, but
then jumped into the NFS pool because that's what my reference point is
for root-squash. Since root squash essentially sends root requests as
"nfsnobody" (IOW: others not the user or group), then the "o+x" approach
is the solution if you're going directly at the file.
John
Yes, appears the o+x is the only way right now. It definitely tries to
access the share as root though, on CentOS 7 since I also tried to add
nfsnobody and nobody to the oneadmin group and that did not work
either. Seems OpenNebula doesn't have this issue with NFSv3 running on
Ubuntu:
[***@mdskvm-p01 ~]# rmdir /tmp/netfs-rootsquash-pool
[***@mdskvm-p01 ~]# cat nfs.xml
<pool type='netfs'>
<name>rootsquash</name>
<source>
<host name='opennebula01'/>
<dir path='/var/lib/one'/>
<format type='nfs'/>
</source>
<target>
<path>/tmp/netfs-rootsquash-pool</path>
<permissions>
<mode>0755</mode>
<owner>9869</owner>
<group>9869</group>
</permissions>
</target>
</pool>
[***@mdskvm-p01 ~]#
[***@mdskvm-p01 ~]#
[***@mdskvm-p01 ~]# virsh pool-define nfs.xml
Pool rootsquash defined from nfs.xml
[***@mdskvm-p01 ~]# virsh pool-build rootsquash
Pool rootsquash built
[***@mdskvm-p01 ~]# virsh pool-start rootsquash
error: Failed to start pool rootsquash
error: cannot open path '/tmp/netfs-rootsquash-pool': Permission denied
[***@mdskvm-p01 ~]# virsh vol-list rootsquash
error: Failed to list volumes
error: Requested operation is not valid: storage pool 'rootsquash' is
not active
[***@mdskvm-p01 ~]# ls -altri /tmp/netfs-rootsquash-pool
total 4
133 drwxrwxrwt. 14 root root 4096 Apr 14 00:05 ..
68785924 drwxr-xr-x 2 oneadmin oneadmin 6 Apr 14 00:05 .
[***@mdskvm-p01 ~]#
[***@mdskvm-p01 ~]# id oneadmin
uid=9869(oneadmin) gid=9869(oneadmin)
groups=9869(oneadmin),992(libvirt),36(kvm)
[***@mdskvm-p01 ~]# id nobody
uid=99(nobody) gid=99(nobody) groups=99(nobody),9869(oneadmin)
[***@mdskvm-p01 ~]# id nfsnobody
uid=65534(nfsnobody) gid=65534(nfsnobody)
groups=65534(nfsnobody),9869(oneadmin)
[***@mdskvm-p01 ~]# id root
uid=0(root) gid=0(root) groups=0(root)
[***@mdskvm-p01 ~]#
[***@mdskvm-p01 ~]# ps -ef|grep -i libvirtd
root 352 31058 0 00:31 pts/1 00:00:00 grep --color=auto -i
libvirtd
root 1459 1 0 Apr11 ? 00:07:40 /usr/sbin/libvirtd
--listen --config /etc/libvirt/libvirtd.conf
[***@mdskvm-p01 ~]#
[***@mdskvm-p01 ~]# umount /var/lib/one
[***@mdskvm-p01 ~]# mount --no-canonicalize /var/lib/one
[***@mdskvm-p01 ~]# umount /var/lib/one
[***@mdskvm-p01 ~]# mount /var/lib/one
[***@mdskvm-p01 ~]# mount|tail -n 1
192.168.0.70:/var/lib/one on /var/lib/one type nfs4
(rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.60,local_lock=none,addr=192.168.0.70)
[***@mdskvm-p01 ~]# umount /var/lib/one
[***@mdskvm-p01 ~]# mount --no-canonicalize /var/lib/one
[***@mdskvm-p01 ~]# mount|tail -n 1
192.168.0.70:/var/lib/one on /var/lib/one type nfs4
(rw,relatime,vers=4.0,rsize=8192,wsize=8192,namlen=255,soft,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=192.168.0.60,local_lock=none,addr=192.168.0.70)
[***@mdskvm-p01 ~]# su - oneadmin
Last login: Thu Apr 14 00:27:59 EDT 2016 on pts/0
[***@mdskvm-p01 ~]$ virsh -d 1 --connect qemu:///system create
/var/lib/one//datastores/0/47/deployment.0
create: file(optdata): /var/lib/one//datastores/0/47/deployment.0
error: Failed to create domain from
/var/lib/one//datastores/0/47/deployment.0
error: can't canonicalize path '/var/lib/one//datastores/0/47/disk.1':
Permission denied
[***@mdskvm-p01 ~]$
CONTROLLER ( NFS Server )
[***@opennebula01 one]$ ls -ld
/var{,/lib{,/one{,/datastores{,/0{,/47{,/disk.1}}}}}}
drwxr-xr-x. 19 root root 4096 Apr 4 21:26 /var
drwxr-xr-x. 28 root root 4096 Apr 13 03:30 /var/lib
drwxr-x---. 12 oneadmin oneadmin 4096 Apr 14 00:40 /var/lib/one
drwxrwxr-x 6 oneadmin oneadmin 46 Mar 31 02:44 /var/lib/one/datastores
drwxrwxr-x 8 oneadmin oneadmin 60 Apr 13 23:31
/var/lib/one/datastores/0
drwxrwxr-x 2 oneadmin oneadmin 68 Apr 13 23:32
/var/lib/one/datastores/0/47
-rw-r--r-- 1 oneadmin oneadmin 372736 Apr 13 23:32
/var/lib/one/datastores/0/47/disk.1
[***@opennebula01 one]$
NODE ( NFS Client )
[***@mdskvm-p01 ~]$ ls -ld
/var{,/lib{,/one{,/datastores{,/0{,/47{,/disk.1}}}}}}
drwxr-xr-x. 21 root root 4096 Apr 11 07:10 /var
drwxr-xr-x. 45 root root 4096 Apr 13 04:11 /var/lib
drwxr-x--- 12 oneadmin oneadmin 4096 Apr 14 00:39 /var/lib/one
drwxrwxr-x 6 oneadmin oneadmin 46 Mar 31 02:44 /var/lib/one/datastores
drwxrwxr-x 8 oneadmin oneadmin 60 Apr 13 23:31
/var/lib/one/datastores/0
drwxrwxr-x 2 oneadmin oneadmin 68 Apr 13 23:32
/var/lib/one/datastores/0/47
-rw-r--r-- 1 oneadmin oneadmin 372736 Apr 13 23:32
/var/lib/one/datastores/0/47/disk.1
[***@mdskvm-p01 ~]$
Cheers,
Tom K.
-------------------------------------------------------------------------------------
Living on earth is expensive, but it includes a free trip around the sun.