Mounting Windows shares containing spaces

SNG · 16 April 2020 13:11

I’m looking for some help mounting a windows network samba share which contains spaces in the path. I can mount it if I edit /etc/fstab directly, using a successful mount of a path without spaces as a template and using \040 to replace the spaces in my desired path. However, Add network drive from DietPi-Drive_Manager will not accept the spaces in the network path dialogue and fails with the following output:-
[FAILED] Samba mount failed with the following error output:

mount error(95): Operation not supported
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(95): Operation not supported
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(2): No such file or directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(2): No such file or directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)
mount error(20): Not a directory
Refer to the mount.cifs(8) manual page (e.g. man mount.cifs)

I’ve tried using single and double quotes in various ways to enclose the path and I’ve tried replacing the spaces with the escape sequence \040 in the share name dialogue, all fail the same way.

Is there a way to make it work?
Regards
SNG

Joulinar · 16 April 2020 16:34

well I guess you found the workaround already be editing /etc/fstab

For the rest, I guess MichaIng would need to have a look.

WarHawk · 17 April 2020 06:19

I believe you have to add a / after each word

mount My/ Computer/ and the like

Ah correction…I googled and found that you put a \040 where the space would be

Like My\040Computer

Joulinar · 17 April 2020 21:05

Hi,

I guess the \040 syntax is for fstab, not for shell command line. A working syntax could be like this. Was working quite well on my test.

mount -t cifs '//192.168.0.10/test 123 test' /mnt/samba -o user=some,pass=user

I placed a small ‘echo’ inside drive manager script. This is the output. So \040 is already placed BUT using double backslash

mount -t cifs -o username=some,password=user,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=3.1.1 //192.168.0.10/test\\040123\\040test /mnt/samba

Anyway it would be better to use quotes.

SNG · 19 April 2020 13:08

Thanks for your efforts, Joulinar.

Following on I had a look at the drive manager script and line 1757 is:
“samba_clientshare=${G_WHIP_RETURNED_VALUE//[[:space:]]/\\040}”
which does seem to replace spaces with “\040”, so I changed it to:
samba_clientshare=${G_WHIP_RETURNED_VALUE//[[:space:]]/\040}

Still had the same problem though, which is strange 'cos it works fine if I edit fstab by hand! Looking further the script tries to mount the share using mount cifs versions from 3.1.1 down to 1.0 and on success updates fstab with that command.

My set-up is latest Buster dietpi running on a pi3 and the share is on an ancient version of a Windows 2008 server (WHS 2011). I did some tests and from the command line, versions above 2.1 do not work even with a non space path, so I cannot test further with versions 3.0 and 3.1.1.

Testing versions 1.0, 2.0 and 2.1, I tried to mount the the problem share from the command line on my system and found that I could not get “/040” to work as a substitute for a space using any of them. The command line I tested was:
mount -t cifs -o username=“MyUser”,password=“thisPswd”,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=2.1 //192.168.99.201/BTSync\040Share /mnt/sambaBTsync

However both enclosing the share path in double quotes or using a simple backslash space "\ ", to escape the space, worked just fine:
mount -t cifs -o username=“MyUser”,password=“thisPswd”,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=2.1 “//192.168.99.201/BTSync Share” /mnt/sambaBTsync
or
mount -t cifs -o username=“MyUser”,password=“thisPswd”,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=2.1 //192.168.99.201/BTSync\ Share /mnt/sambaBTsync.

My conclusion is that although the code in drive manager may work for versions 3.0 and abover (I cannot test it), for version 2.1 and below the script is broken because mount cifs does not support the \040 substitution for a space in the same way it is used in fstab.

Does my testing and conclusion seem correct, or have I misunderstood how it all works? Any comments welcome.

Regards
SNG

Joulinar · 19 April 2020 21:32

yes your observation is correct. mount cifs does not support the \040. I’m not really good on shell scripting but I tried die play around with drive manager script and was able to get the correct syntax which is working on manual execution. However drive manager still did not like it.

first one if failing with “mount.cifs: bad UNC (’//192.168.0.10/test 123 test’)”

mount -t cifs -o username=MyUser,password=MyPass,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=3.1.1 '//192.168.0.10/test 123 test' /mnt/samba

while the second one if failing with “mount error(2): No such file or directory”

mount -t cifs -o username=MyUser,password=MyPass,iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=3.1.1 //192.168.0.10/test\ 123\ test /mnt/samba

as I said, both commands are working if I put them directly into console

SNG · 21 April 2020 17:53

Got it working!
The two issues in the existing code are:

Spaces are replaced with \040 rather than \040 required by fstab. (line 1758)
The mount command does not accept \040 (line 1793) as a space substitute.

To fix these I changed line 1758 which gets the share directory from the user and replaces any and all spaces with \040, from:
samba_clientshare=${G_WHIP_RETURNED_VALUE//[[:space:]]/\\040}
to:

samba_clientshare=${G_WHIP_RETURNED_VALUE//[[:space:]]/\\040}

so that spaces are replaced with \040.

Then line 1793, which uses samba_clientshare, is modified so that \040 is replaced in-line with an escaped space (\ )

from:
if mount -t cifs -o username="$samba_clientuser",password="$samba_clientpassword",iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=$i //"$samba_clientname"/"${samba_clientshare}" “$samba_fp_mount_target” &>> $fp_tmp; then
to

if mount -t cifs -o username="$samba_clientuser",password="$samba_clientpassword",iocharset=utf8,uid=dietpi,gid=dietpi,file_mode=0770,dir_mode=0770,vers=$i //"$samba_clientname"/"${samba_clientshare//\\040/\ }" "$samba_fp_mount_target" &>> $fp_tmp; then

This line, 1793, is in a for loop which steps down through versions of cifs from 3.1.1 until it finds the highest supported version which works, before updating fstab. My change does a string in string replacement on samba_clientshare, like this
${samba_clientshare//\040/\ }
making it compatible with the mount command.

I tried putting the the escaped path into a variable and then using that variable in the mount command but I couldn’t find how to get that to work!! Anyway doing it in-line seems much neater.

Regards
SNG

Joulinar · 21 April 2020 21:46

yup that seems to be working, at least for the mount part. What I noticed, some strange visualisation within drive manager with the mount containing spaces. For testing purposes i did 2 mounts. First one is a normal one, the second a mount with spaces.

root@DietPi3:~# df -h|grep samba
//192.168.0.10/Share          2.7T  566G  2.1T  22% /mnt/samba
//192.168.0.10/test 123 test  2.7T  566G  2.1T  22% /mnt/samba1
root@DietPi3:~#

You can see on the picture attached that information for 2nd mount are “incorrect”

It seems drive manager did not detect the drive correctly even if it is correctly added to /etc/fstab

[ INFO ] DietPi-Drive_Manager |  - Detected mounted drive: //192.168.0.10/test > 2.1T

as well removing the mount is not possible

[.     ] DietPi-Drive_Manager | umount 2.1T
[FAILED] DietPi-Drive_Manager | umount 2.1T

SNG · 22 April 2020 10:15

Oh dear! It seems that dietpi’s mount detection code doesn’t handle spaces correctly either
I had scanned dietpi-drive_manager code to see where the variable samba_clientshare (a local variable) was being used to make sure my changes were not going to break anything else and all seems fine, so the detection code cannot rely on the that variable.

I’ve identified the detection code and it is way beyond me so, sadly, it looks as though I shall have to look for a solution for my application somewhere other than DietPi 'cos I don’t suppose this issue will get fixed any time soon:cry:

All the same, I’ll have a look to see how I go about raising a bug report.

Regards
SNG

Joulinar · 22 April 2020 10:41

I guess there isn’t a real issue for your system. You can always use /etc/fstab directly to setup/mount your drive during boot. And this is something you don’t do on regular basis, efforts are limited I would say. But yes it might be something to be checked by MichaIng . I was looking into code as well and I found the section where drives are detected and it seems using df command. Detection start around line 170. Looks like it’s going to separate the information by space and put values into an array. But it’s way above my shell scripting skill

Detection section

		# Detect mounted drives
		G_DIETPI-NOTIFY 2 'Detecting drives, please wait...'
		# - Only detect mounts with valid source path (word 1 contains "/")
		mawk '$1~/\//{print}' <<< $(df -Pha) > .df_out_tmp

Separate section

			aDRIVE_ISMOUNTED[$index]=1
			aDRIVE_MOUNT_SOURCE[$index]=$(mawk '{print $1}' <<< $line)
			aDRIVE_SIZE_TOTAL[$index]=$(mawk '{print $2}' <<< $line)
			aDRIVE_SIZE_USED[$index]=$(mawk '{print $3}' <<< $line)
			aDRIVE_SIZE_FREE[$index]=$(mawk '{print $4}' <<< $line)
			aDRIVE_SIZE_PERCENTUSED[$index]=$(mawk '{print $5}' <<< $line)
			aDRIVE_MOUNT_TARGET[$index]=$(mawk '{print $6}' <<< $line)

SNG · 27 April 2020 15:53

OK, I’ve hacked it some more and got the mount detection code working.

Dietpi’s file_manager uses a df command to return the files system description in human readable form and saves it in a temporary file. The code then uses MAWK to process each line of this file in turn, treating each line as a record and splitting each record into fields using space as the delimiter. Each ‘record’ normally returns six fields but when the share source name has spaces it results in more than six fields being returned, one extra field for each space.
The orginal code looks like this:

#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------" > $fp_fstab_tmp

		# Detect mounted drives
		G_DIETPI-NOTIFY 2 'Detecting drives, please wait...'
		# - Only detect mounts with valid source path (word 1 contains "/")
		mawk '$1~/\//{print}' <<< $(df -Pha) > .df_out_tmp

		# Remove misc items from list
		# - bind: https://github.com/MichaIng/DietPi/issues/2013#issuecomment-416394374
		[[ $misc_mounts ]] && while read line
		do

			local input_mount_target=$(mawk '{print $2}' <<< $line)
			sed -i "\#[[:blank:]]$input_mount_target$#d" .df_out_tmp
			# - target + source removal via $6 required for bind mounts: https://github.com/MichaIng/DietPi/issues/2013#issuecomment-417413867
			local input_mount_source=$(mawk '{print $1}' <<< $line)
			sed -i "\#[[:blank:]]$input_mount_source$#d" .df_out_tmp
			[[ $G_DEBUG == 1 ]] && G_DIETPI-NOTIFY 0 " - Detected misc mount and removed from df scrape: $input_mount_source > $input_mount_target"

		done <<< "$misc_mounts"

		# Process final df result
		while read line
		do

			Init_New_Device

			aDRIVE_ISMOUNTED[$index]=1
			aDRIVE_MOUNT_SOURCE[$index]=$(mawk '{print $1}' <<< $line)
			aDRIVE_SIZE_TOTAL[$index]=$(mawk '{print $2}' <<< $line)
			aDRIVE_SIZE_USED[$index]=$(mawk '{print $3}' <<< $line)
			aDRIVE_SIZE_FREE[$index]=$(mawk '{print $4}' <<< $line)
			aDRIVE_SIZE_PERCENTUSED[$index]=$(mawk '{print $5}' <<< $line)
			aDRIVE_MOUNT_TARGET[$index]=$(mawk '{print $6}' <<< $line)
			# Workaround for /dev/root under RPi, force physical location
			[[ ${aDRIVE_MOUNT_TARGET[$index]} == '/' ]] && aDRIVE_MOUNT_SOURCE[$index]=$FP_ROOTFS_SOURCE

			aDRIVE_SOURCE_DEVICE[$index]=$(Return_Drive_Without_Partitions ${aDRIVE_MOUNT_SOURCE[$index]})
			[[ ${aDRIVE_MOUNT_SOURCE[$index]} == /dev/${aDRIVE_SOURCE_DEVICE[$index]} ]] || aDRIVE_ISPARTITIONTABLE[$index]=1
			aDRIVE_UUID[$index]=$(blkid ${aDRIVE_MOUNT_SOURCE[$index]} -s UUID -o value)
			(( ${aDRIVE_ISPARTITIONTABLE[$index]} )) && aDRIVE_PART_UUID[$index]=$(blkid ${aDRIVE_MOUNT_SOURCE[$index]} -s PARTUUID -o value)

and etc.

I’ve used the AWK built in variable $NF to get the number of fields for each line and if there are more then six I intercept it and use a loop to reconstruct the proper source share name.

My code looks like this:

#----------------------------------------------------------------
# PHYSICAL DRIVES
#----------------------------------------------------------------" > $fp_fstab_tmp

		# Detect mounted drives
		G_DIETPI-NOTIFY 2 'Detecting drives, please wait...'
		# - Only detect mounts with valid source path (word 1 contains "/")
		mawk '$1~/\//{print}' <<< $(df -Pha) > .df_out_tmp

		# Remove misc items from list
		# - bind: https://github.com/MichaIng/DietPi/issues/2013#issuecomment-416394374
		[[ $misc_mounts ]] && while read line
		do

			local input_mount_target=$(mawk '{print $2}' <<< $line)
			sed -i "\#[[:blank:]]$input_mount_target$#d" .df_out_tmp
			# - target + source removal via $6 required for bind mounts: https://github.com/MichaIng/DietPi/issues/2013#issuecomment-417413867
			local input_mount_source=$(mawk '{print $1}' <<< $line)
			sed -i "\#[[:blank:]]$input_mount_source$#d" .df_out_tmp
			[[ $G_DEBUG == 1 ]] && G_DIETPI-NOTIFY 0 " - Detected misc mount and removed from df scrape: $input_mount_source > $input_mount_target"

		done <<< "$misc_mounts"
		
		# Process final df result
		while read line
		do
			Init_New_Device

			# Get the total number of fields retured for each line
			local number_fields=$(mawk '{print NF}' <<< $line)
			aDRIVE_ISMOUNTED[$index]=1

			# If there are more than 6 fields in the record then the source name has spaces!
			if [[ $number_fields -ge 7 ]]; then
				# 7 or more fields, source name has spaces
				# Get the first part of the share source name
				local source_parts=$(mawk '{print $1}' <<< $line)
				# Now add the remaining parts 
				for (( n=2; n<=($number_fields - 5); n++ ))
				do  
					source_parts="$source_parts "$(mawk "{print \$($n)}" <<< $line)
				done
				aDRIVE_MOUNT_SOURCE[$index]=$source_parts		
			else
				# Record must have 6 fields and is normal
				aDRIVE_MOUNT_SOURCE[$index]=$(mawk '{print $1}' <<< $line)
			fi

			aDRIVE_SIZE_TOTAL[$index]=$(mawk '{print $(2+NF-6)}' <<< $line)
			aDRIVE_SIZE_USED[$index]=$(mawk '{print $(3+NF-6)}' <<< $line)
			aDRIVE_SIZE_FREE[$index]=$(mawk '{print $(4+NF-6)}' <<< $line)
			aDRIVE_SIZE_PERCENTUSED[$index]=$(mawk '{print $(5+NF-6)}' <<< $line)
			aDRIVE_MOUNT_TARGET[$index]=$(mawk '{print $(6+NF-6)}' <<< $line)
			# Workaround for /dev/root under RPi, force physical location
			[[ ${aDRIVE_MOUNT_TARGET[$index]} == '/' ]] && aDRIVE_MOUNT_SOURCE[$index]=$FP_ROOTFS_SOURCE

			aDRIVE_SOURCE_DEVICE[$index]=$(Return_Drive_Without_Partitions ${aDRIVE_MOUNT_SOURCE[$index]})
			[[ ${aDRIVE_MOUNT_SOURCE[$index]} == /dev/${aDRIVE_SOURCE_DEVICE[$index]} ]] || aDRIVE_ISPARTITIONTABLE[$index]=1
			aDRIVE_UUID[$index]=$(blkid ${aDRIVE_MOUNT_SOURCE[$index]} -s UUID -o value)
			(( ${aDRIVE_ISPARTITIONTABLE[$index]} )) && aDRIVE_PART_UUID[$index]=$(blkid ${aDRIVE_MOUNT_SOURCE[$index]} -s PARTUUID -o value)

and etc.

I’m sure this can be done much more elegantly if I had a better understanding of AWK than I have garnered but, hey, it works.

My changes to the code are highlighted and explained here, changes highlighted in orange:

Get the total number of fields returned for each line

local number_fields=$(mawk ‘{print NF}’ <<< $line)
aDRIVE_ISMOUNTED[$index]=1

If there are more than 6 fields in the record then the sourece name has spaces!

if [[ $number_fields -ge 7 ]]; then

7 or more fields, source name has spaces

Get the first part of the share source name

local source_parts=$(mawk ‘{print $1}’ <<< $line)

Now add the remaining parts

for (( n=2; n<=($number_fields - 5); n++ ))
do
source_parts="$source_parts "$(mawk “{print $($n)}” <<< $line)
done
aDRIVE_MOUNT_SOURCE[$index]=$source_parts
else

Record must have 6 fields and is normal

aDRIVE_MOUNT_SOURCE[$index]=$(mawk ‘{print $1}’ <<< $line)
fi

The following change forces the allocations to be last five fields regardless

aDRIVE_SIZE_TOTAL[$index]=$(mawk ‘{print $(2+NF-6)}’ <<< $line)
aDRIVE_SIZE_USED[$index]=$(mawk ‘{print $(3+NF-6)}’ <<< $line)
aDRIVE_SIZE_FREE[$index]=$(mawk ‘{print $(4+NF-6)}’ <<< $line)
aDRIVE_SIZE_PERCENTUSED[$index]=$(mawk ‘{print $(5+NF-6)}’ <<< $line)
aDRIVE_MOUNT_TARGET[$index]=$(mawk ‘{print $(6+NF-6)}’ <<< $line)

Regards
SNG

Joulinar · 27 April 2020 22:42

cool, thx for your time to have it further investigated. I opened an issue on GitHub to follow up on the possible improvement.

https://github.com/MichaIng/DietPi/issues/3491

MichaIng · 28 April 2020 13:16

SNG
Many thanks for reporting the issue and providing a fix. I was thinking about a way to do this more failsafe, as other fields might contain spaces as well, e.g. the mount point:
tmpfs 2020572 0 2020572 0% /mnt/test test
I was hoping these are “tab” characters, but they are spaces, one up to multiple ones. The df headers are a mixture of left and right alignments, so scraping each line based on character counts from headers is not possible from what I found.

Sadly df does not provide any alternative output format. /proc/mounts and similar do not show space usage. findmnt is an alternative, with different output modes, however based on output mode, either spaces are still an issue or things like quotes are, or “dangerous” characters are masked by hex code or similar, which would break the script at other places.

The only failsafe method I can see is to run a dedicated findmnt -no on each mount for each value. To get the list of sources/mounts to loop through: df -a --output=source | mawk ‘///{print}’

Did I forget/overseen anything? I’ll add the above as PR and run some tests to see if it is much slower or acceptable.

SNG · 30 April 2020 14:04

Hi, MichaIng, thanks for looking at this. I must first confess that my efforts have all been down to google! I really don’t have any in depth experience with linux and had never used either df or AWK before. Sadly as I get older my memory is not what it used to be and it takes several iterations of anything to get it to stick these days As a result I’m finding it difficult to visualize what you are proposing.

SNG · 30 April 2020 14:07

I was only trying to ‘fix’ what I didn’t have control over, there is a built in share on Windows Home Server which contains spaces and they are pretty common on Windows paths anyway. On the dietpi box I’m creating the mount points so have direct control over it, but to do some error catching for spaces elsewhere I thought I could check the ‘percentage used’ field and make sure that it is the next to last one - it seems be always a number followed by ‘%’ for the ‘records’ we are looking at. Indeed if it is not where we expect it then this would imply that there are spaces in the mount point, which could then be processed appropriately. I don’t see why df would introduce any other extraneous spaces in the other four ‘fields’ without there being an error elsewhere.

So my sequence of events would be:
Test for number of fields greater than six and if so…
Check the position of ‘percentage used’ , if it is not fifth field from the beginning concatenate the first fields into the share path until it would be.
Check the position of ‘percentage used’ and if it is not next to last field concatenate any following fields into the mount point path.
Use the position of ‘percentage used’ to determine the position of the other four fields and extract them.

I wouldn’t have thought speed of execution in this bit of code is critical, it’s not run very often and doesn’t loop.

My code still leaves an error on the file_manager screen. The line above in the table still doesn’t have the correct path but I cannot see where it is getting it from.

│                         ●─ //192.168.99.201/BTSync ──────────────────────────────────────────────────          │
│       /mnt/sambaBTSync  : //192.168.99.201/BTSync Shares | Net | Capacity: 19T | Used: 11T (60%)               │

MichaIng · 1 May 2020 15:22

PR is up to fix white space handling, including \040 conversion of spaces from Samba shares for fstab only: https://github.com/MichaIng/DietPi/pull/3497
I’ll run some tests now. This is just a start, e.g. to fully support spaces, space to \040 conversion, also when scraping /etc/fstab for entries, must do done for all mount sources and targets. But this is something for a later release.

Joulinar · 1 May 2020 15:55

Ok I will do some testing once merged into dev

MichaIng · 1 May 2020 20:00

Okay works fine now according to my tests. Performance has even become slightly better since bind mount exclusion needed to be done differently, which is now more efficient: DietPi-Drive_Manager | Fix handling mount sources and targets with spaces by MichaIng · Pull Request #3497 · MichaIng/DietPi · GitHub

Only downside is that managing two mount points of the same file system is not possible now with drive manager. However adding a second mount of the same source was not possible before, and AFAIK managing those would have caused issues, at least when unmounting one it would have been lost on a new drive scan loop, etc.

Joulinar · 2 May 2020 00:25

MichaIng
ok seems working. However I have seen this message during umount

[.     ] DietPi-Drive_Manager | umount /mnt/samba
[  OK  ] DietPi-Drive_Manager | umount /mnt/samba
rmdir: failed to remove '/mnt/samba': Device or resource busy

SNG · 6 May 2020 16:15

I’ve been trying to test this and do manage to mount the offending windows share. Unfortunately I’m having problems testing further since using drive_manager seems to be setting the root file system read only and as a result it cannot update fstab with any changes once this happens. To move on I need to burn a backup copy of the system to my sd card and start over. I notice that another couple of threads have been started up with similar problems, most of which seem to be this same issue.

SNG