NFS data corruption problem (serious)

Summary:

Using nfs to transfer files to MyBook corrupts some of my files. From a small research I found out that some other low end NAS devices (NLSU2 and Thecus N2100 using Debian) exhibit the same problem as well.

NFS data corruption problem (serious)

mrsaccess 11 Oct 2008 22:09

I got today a MyBook World Edition 1TB.
First thing I did was to upgrade the firmware to 2.0.0.18, enable ssh and turn on nfs sharing since I am using Linux and it's more convenient for me.

I had four files of 1.2GB on my machine, each spanned on 12 rar archives. That makes 48 rar files, 100MB each. I transferred them to MyBook through nfs.
Since MyBook runs Linux I thought it would be fun to ssh to it, install unrar through optware and use MyBook to unrar them!

Here the problem begins. Three files (of 48 total) were corrupted and unrar gave me a CRC error. Since these belonged to different archives I was able to get back only one of my four 1.2GB files!

Rather alarmed I tried to uncompress the originals on my computer. No problem at all.
I transfered the "broken" files back to my computer just to make sure it wasn't the optware version of unrar the problem but the unrar program on my computer found them too corrupted.

After some more trials it was clear that some -random- files get corrupted during transfer TO MyBook (and not FROM MyBook).

I copied the files to an external usb hard disk, connected it to MyBook, ssh'd to MyBook and copied them to the internal hard disk. There was no problem there.

So the problem had to be the nfs or the network.

With a little search I bumped on some threads on Debian mailing list describing the same problem though it was about Linksys NLSU2 and Thecus N2100.
Here are some links, the first one is more clear:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=404447
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=445156

Notice that I also had the "nfsd: non-standard errno: -14" message on my dmesg.

It isn't still clear to me if the problem is in the nfs or the network implemantation but some people claim about a similar problem with SMB (samba). There is one old thread in this forum too.

There isn't a fix available but there are some advices to minimize the problem which I intend to try and inform you.

Please notice that data corruption is one of the most serious problems but it can go easily undetected.
If you mainly transfer movies or mp3s to your MyBook it is possible that an error may go unnoticed.
But in archives (zip, rar, etc) and other types of files the problem won't go unnoticed and you may loose your files.

So, has anyone else noticed?

Re: NFS data corruption problem (serious)

mrsaccess 13 Oct 2008 10:25

After many trials I wasn't able to get the kernel nfs server not to corrupt my files. :(

There is a solution though, to use the userspace NFS v3 server which can be installed from optware as unfs3.

This is slower than the kernel nfs server but it doesn't produce corrupted files.
My performance tests showed on a 100mbps network:
Kernel NFS server, read from MyBook: 7.1 MB/sec
Kernel NFS server, write to MyBook: 3.4 MB/sec
Userspace NFS server, read from MyBook: 5.0 MB/sec
Userspace NFS server, write to MyBook: 3.0 MB/sec

The performance hit on read is significant.

It may be the case that a gigabit ethernet solves the problem with the kernel server but I haven't currently a gigabit setup to test it.

For anyone who wants to try unfs3 you just need to run portmap before running unfsd.
Also you may want to run unfsd with a high priority:
$ nice -n -10 unfsd

Last edited on 13 Oct 2008 10:28 by mrsaccess Show more

Re: NFS data corruption problem (serious)

kevku 13 Oct 2008 11:16

well I mostly ftp to upload big files and it seems to be ok. hmm never actually noticed the nfs corruption, but if I think abut it, it possible

Re: NFS data corruption problem (serious)

mrsaccess 15 Oct 2008 16:03

If anyone wants to find out if this problem applies to him is relatively easy.

Find a big directory on your computer. I think there is 1 or 2 corruptions per 2GB of transfers, so find a directory bigger than 2GB, I would suggest 4GB.

Now open a terminal and go to the parent directory of the directory you found.
Let's assume you will use this directory:
/foo/bar/parentDirectory/bigDirectory

So you will cd to:
$ cd /foo/bar/parentDirectory

Now create a md5sum of the files of the big directory:
$ find bigDirectory ! -type d -print0 | xargs -0 md5sum » bigDirectory.md5

Transfer the bigDirectory and the bigDirectory.md5 to MyBook through nfs. Put them in the same place.
Then ssh to MyBook, go the directory you placed bigDirectory and bigDirectory.md5 and run:
$ md5sum -c bigDirectory.md5

This will check all the files transferred and inform you upon any problem.

Last edited on 15 Oct 2008 16:06 by mrsaccess Show more