Repository navigation
NFS Export with xfs filesystem #20
Description
Hi,
With this configuration
$ linstor rg lp nfs_group -p
+-------------------------+
| Key | Value |
|=========================|
| FileSystem/Type | xfs |
+-------------------------+
$ linstor-gateway nfs create shares 10.20.117.254/32 10G --allowed-ips=10.20.117.0/24 --resource-group=nfs_group
Created export 'shares' at 10.20.117.254:/srv/gateway-exports/shares
linstor-gateway generate a drbd-reactor config file for an ext4 filesystem:
$ cat /etc/drbd-reactor.d/linstor-gateway-nfs-shares.toml | grep ext4
"ocf:heartbeat:Filesystem fs_cluster_private device=/dev/drbd/by-res/shares/0 directory=/srv/ha/internal/shares fstype=ext4 run_fsck=no",
"ocf:heartbeat:Filesystem fs_1 device=/dev/drbd/by-res/shares/1 directory=/srv/gateway-exports/shares fstype=ext4 run_fsck=no",
with FileSystem/Type ext4 setting all are fine
Activity
chrboe commentedon Mar 22, 2023
Hi, thanks for the report!
I agree that this is not very intuitive, but this is actually the intended behavior.
Because we set the
all_squashNFS option, the root of the file system must be owned bynobody:nobody(notroot:root, which is the default). Unfortunately, we cannot just do achown nobody:nobody /after the mkfs, because LINSTOR actually makes the mkfs call for us (on another machine). For ext4, we can accomplish this at mkfs time using theroot_ownerflag (see here, and its usage). Such an option is unfortunately not available formkfs.xfs.This is why, currently, only ext4 is supported (and I could have sworn that this fact is documented somewhere...)
Since linstor-gateway explicitly sets the
FileSystem/Typeproperty on the volume, it overrides the value you set on the resource group.I'll close this issue, but I'll try to figure out a way to more clearly hint the user towards this fact.
Of course, if you have an idea on how to implement the same logic for xfs, we are all ears 🙂
cduchenoy commentedon Mar 22, 2023
Hi, thanks for the technical explanation!
I followed the official doc at https://linbit.com/drbd-user-guide/linstor-guide-1_0-en/#s-linstor-gateway-create-nfs
Here are the points that I think need improvement:
Moreover, if the linstor API is secured by an ssl certificate, we have not found a way to pass the client certificate in the linstor-gateway configuration...
I will report an issue on this point with a hack (haproxy)
Instead of closing bug reports, could you classify them as improvements?
same as #9
chrboe commentedon Mar 23, 2023
I fully agree with the first two points. Especially the fact that you can only create one NFS export is a big pain point IMO, and we are working with the LINSTOR team to find solutions to improve this.
Regarding the systemd version: this is an upstream "issue" in drbd-reactor, but the ExecCondition warning can safely be ignored; it is not required for its operation. The oldest systemd version where I have personally successfully run linstor-gateway is 219.
I'm not exactly sure what you mean by the last two points... Could you elaborate? LINSTOR should automatically create these directories.
That is an interesting point regarding the SSL certificate; I will play around with that. Not many people use that LINSTOR feature, so it has never come up.
Sorry if closing the issue came across as dismissive. We are definitely working on improving these areas; I just don't like having many issues open for things that are not currently planned.
Yesterday I actually thought of another route we could take in order to possibly support XFS NFS shares, so I guess we can reopen this issue as a reminder.
cduchenoy commentedon Mar 23, 2023
We use the linstor-gateway go binaries and we have to manually create the directories for ocf* service to start
RumenBlack84 commentedon Jun 10, 2024
Do these same limitations apply to zfs? I was really hoping to use the linstor gateway to just export a zfs dataset that is being replicated using drbd.