Large file copy problem solved?

Summary:

I'm able to copy 30G of movies to Mybook without crash

Large file copy problem solved?

Pineapple 1209108311|%e %b %Y, %H:%M %Z|agohover

Here is what I did.
First check the share file system for error.
Login with SSH, get into superuser with su.
Unmount the share partition: umount /dev/md4
Run disc check: fsck.ext3 /dev/md4
It took a long time. Probably faster to delete the partition and reload it.
I had lots of errors. I ended up deleting all files after error check cleaned up the driver.
I then reboot Mybook, too lazy to find out how to properly mounting the partition again.
I add the following line to: /var/oxsemi/smb.conf
under [global] section:
oplocks = no
case sensitive = True
read size = 16384
read raw = no
write raw = no
load printers = no

I believe oplocks = no is what make the different.
read raw and write raw is for performance

Feel free to try and correct me if I'm wrong.
I'm still copying 68G of movies now. About 2hrs for 30G.
I don't care about speed now, I just want it to work.

Note: DO NOT use alt-web interface to edit the file. You WILL brick Mybook. Login with SSH and use vi or nano.

Update: 18-June-2008
Trying to copy 400GB from MyBook to another computer. the PC keep shutting down on its own. Add the following registry key if you have same problem:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters]
"EnableOplocks"=dword:00000000

last edited on 1213822814|%e %b %Y, %H:%M %Z|agohover by Pineapple + show more

Re: Large file copy problem solved?

etsme 1209494416|%e %b %Y, %H:%M %Z|agohover

Ive been reading up on optlocks for about an hour now, I was thinking I had to disable it in Vista.. but I just done your steps above and so far I have copied 3GB and still going.. I have 32GB in-line 3 hours it says over a 54mb wireless connection…. ill tell you all about it later

http://www.smallnetbuilder.com/content/view/30039/78/

thanks for your input Pineapple

last edited on 1209677803|%e %b %Y, %H:%M %Z|agohover by etsme + show more

Re: Large file copy problem solved?

Pineapple 1209502086|%e %b %Y, %H:%M %Z|agohover

Good to know it helps.
Oplocks works only when both side have it enabled. So as long the the one in MyBook is disabled, its good.

Re: Large file copy problem solved?

etsme 1209677726|%e %b %Y, %H:%M %Z|agohover

yep copied well over 90GB now not a hint of a glitch…………. you are now considered as a god in my house pineapple. top marks!
if it was'nt for you the kids would be playing football in the street with the drive right now.
cheers
john

Re: Large file copy problem solved?

Lorent 1209725621|%e %b %Y, %H:%M %Z|agohover

I wasn't aware there was a problem? Been backing up my 250GB hard disk with no problems? :)

Re: Large file copy problem solved?

psilo 1210510163|%e %b %Y, %H:%M %Z|agohover

The "straight out of the box" version didn't work. after+1gb my my book world edition 500gb freezes.
I really needed to format the /dev/md4 partition. and now it works really well!
I'm uploading now with a speed of 4 mb/s +8gb

Re: Large file copy problem solved?

Pineapple 1210626396|%e %b %Y, %H:%M %Z|agohover

Looks like some of them come with bad partition?

Re: Large file copy problem solved?

Johny_Airman 1217741086|%e %b %Y, %H:%M %Z|agohover

Well, I've got my drive yesterday. Reading a bits and pieces before, I never tried the stupid WD anywhere, just set static IP in my router (not even inside the My Book), typed that IP in my browser, set "backup" shared folder in that My book admin program. Then opened that folder in windows explorer and moved home made 5 DVDs into it in the row without hiccup. (regular VOB size of 1GB). I turn it off then (was too late), turned it back on today and moved 3 DVDs (actually the 3rd is copying as I write this), no problem again.

So - do you think WD actually fixed some of the bugs? Or where "should" be problem of copying large files? Is it only when using the WD anywhere? (but if I understand correctly, people here are not using it anyway..?)

Since I'm writing, can I have question which belong in some other post I guess? I have totally ZERO knowledge of linux apart from knowing it exists somewhere in the universe. How much knowledge do I have to get before I should try to hack WD and install FTP? (I'd love to share and access my own video/audio over the Internet and that is not possible using the stupid WD anywhere.)

Re: Large file copy problem solved?

teh_crash 1212052353|%e %b %Y, %H:%M %Z|agohover

If anyone gets error message after trying to fsck:

Error allocating icount link information: Memory allocation failed

Here is solution: make 1gb empty file, mount it as swap, then run fsck.

Re: Large file copy problem solved?

TorPram 1214199534|%e %b %Y, %H:%M %Z|agohover

Hi - first time on here and complete newbie on WDWE- I have the II 1TB.

I bought it last wk and formatted it RAID 1 as soon as I got home. Got 2 computers to see the drive (one XP, one Vista) even though I have to reset the router every time before I turn on the drive (haven't looked at fixing the ip yet). So far so good…

Now I have the same problem of the drive dropping off network disrupting large files transfers and reconnects by itself. To copy my 80GB of photos little bit at a time would take me all week - I'd rather buy a new drive which works straight away. TBH, I have no clue what you write above as I've never used linux nor all that computer literate.

Contacted support and they suggest updating firmware - but I believe I already have the latest. I've tried to search around before wasting people's time on forums but can't really find much help. Is there anywhere helps me solves the problem in laymen's terms?

I thought this product was aimed at consumers who can plug it in and follow included instructions - now I'm finding that the product is very poor in this regard - should I just give up and use it as a bookend? (I'm in Thailand and bought it at a computer fair so no return policy). It still frustrates me so much that such a big company with so much resource still make really shoddy, relatively simple products.

Thanks!

Re: Large file copy problem solved?

Furedetsu 1227542293|%e %b %Y, %H:%M %Z|agohover

***

last edited on 1227542823|%e %b %Y, %H:%M %Z|agohover by Furedetsu + show more

Re: Large file copy problem solved?

melman 1247451615|%e %b %Y, %H:%M %Z|agohover

#I am linux newbie.
#My PC is Ubuntu 8.10 and Windows XP Pro. and Home edition.

I used Pineapple's hack(thank you Pineapple!) but i cant solved large file copy problem.
so I read documents of samba system(URL:http://www.linuxtopia.org/online_books/network_administration_guides/using_samba_book/ch08_06_02.html)
and I added line in "/var/oxsemi/smb.conf".

/var/oxsemi/smb.conf
dfree command = /usr/local/samba/bin/dfree

and i maked new file "/usr/local/samba/bin/dfree".

/usr/local/samba/bin/dfree
#!/bin/sh
df $1 | tail -1 | awk '{print $2" "$4}'

and change permission.
chmod 755 /usr/local/samba/bin/dfree

reboot MyBook system.
reboot

it is maybe working. now i solved file copy problem.
i copyed over 900 big files(size:300MB-1.5GB,file type:iso,avi,mpg,divx..etc.)
and no problem.

Re: Large file copy problem solved?

sansen 1212046932|%e %b %Y, %H:%M %Z|agohover

Hello!!

This also fixed my issue: When I was writing many files, it said the disk was write protected and I wasn't able to write anymore unless I rebooted the disk.

You're great!!! Thanks a lot!!!!!!

/edit: ooooooops problem continues… I'll continue looking for a solution :(

/edit II: trying after reformating, it now seems to work! :)

last edited on 1212050787|%e %b %Y, %H:%M %Z|agohover by sansen + show more

Re: Large file copy problem solved?

sidee008 1213542720|%e %b %Y, %H:%M %Z|agohover

Hi,

ok, this will sound lame but would you be so kind and write down how you reformated? I'm asking because I'm hoping that you'll reply in the way which will let keep my things on a hdd. I have half of my 1TB MyBook full and no other place where to move everything. Is there any way?

Re: Large file copy problem solved?

Pineapple 1213649604|%e %b %Y, %H:%M %Z|agohover

Don't reformat if you want to keep all the files.
Run the disk check to see if any files are corrupted. If they're corrupted, you can use them anyway.

Re: Large file copy problem solved?

sidee008 1214233712|%e %b %Y, %H:%M %Z|agohover

Hi,

I've tried to run fsck.ext3 but when the disk check gets to the phase4 it says that there is incorrect size of: "i_node… is 938423948732 and should be 0". Than it asks if I want to fix it and if I say yes the fsck immediatelly quits without fixing my hdd :( Any ideas?

Re: Large file copy problem solved?

Pineapple 1214330602|%e %b %Y, %H:%M %Z|agohover

You got corrupted files. You'll need to delete the corrupted one to get it going again. Move as many as you can out of the drive. You can run fsck.ext3 with option to answer yes to all question. I gave up on mine and just deleted everything.

Re: Large file copy problem solved?

StreetUrchin 1215971236|%e %b %Y, %H:%M %Z|agohover

Hi All

Thanks for the invaluable advice on here, much appreciated.

My problem is that my MyBook deosn't have the /var/oxsemi/smb.conf file?

Any advice or am i missing something?

Many thanks

Re: Large file copy problem solved?

ikaros 1227657376|%e %b %Y, %H:%M %Z|agohover

Check /etc/smb.conf

Re: Large file copy problem solved?

Lakewater5 1215987579|%e %b %Y, %H:%M %Z|agohover

I also have the same problem as StreetUrchin.

No /var/oxsemi/smb.conf file.

Anyone have a sample smb.conf file I can load ?

Thanks…..

UPDATE ……………………………..!!!!!!!!!!!!!!!!!!!!!!!

After additional research I found the missing smb.conf file referenced in a line of code within /etc/inetd.conf.

I have edited: /var/oxsemi/smb.conf to add (per Pineapple's recommendation):

oplocks = no
case sensitive = True
read size = 16384
read raw = no
write raw = no
load printers = no

So far GREAT. Have now tranferred 46 GIG with no problems!

A BIG thanks to Pineapple!

BAD NEWS !!!!

Unit is back to old tricks and is failing again…

Seems to be time related and not related to size or quantity of files transferred.

Back to drawing board………..

last edited on 1215992901|%e %b %Y, %H:%M %Z|agohover by Lakewater5 + show more

Re: Large file copy problem solved?

StreetUrchin 1216002262|%e %b %Y, %H:%M %Z|agohover

Thanks for updating LakeWater, gets me a stage further also - let me know how you get on.

Re: Large file copy problem solved?

Pineapple 1216006162|%e %b %Y, %H:%M %Z|agohover

Add this in your XP registry to turn off oplock in XP:
[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters]
"EnableOplocks"=dword:00000000

Re: Large file copy problem NOT-solved (yet)?

Lakewater5 1216347143|%e %b %Y, %H:%M %Z|agohover

I have tried everything posted except formatting the partition
and I am concerned about trying this without being able to
back-up the entire drive first AND because I do not have any
other Linux machine up and running.

Anyone have a step-by-step on how to format ONLY the PUBLIC partition ? and
anyone know any way to do a binary backup using a windows platform ?

Thanx……LW5

Re: Large file copy problem NOT-solved (yet)?

StreetUrchin 1216473972|%e %b %Y, %H:%M %Z|agohover

I did format as described in the 'troubleshooting' section of this site and everything seems to be fine now. Sorry, can't help you with the step-by-step guide you want, but a format is the way.

Re: Large file copy problem FINALLY solved?

Lakewater5 1216487968|%e %b %Y, %H:%M %Z|agohover

I finally gave in and reformatted the partition containing "PUBLIC". So far this seems to have solved the problem. Over 50GB copied [with a few files that are over 5 GB each] with no problems.

As soon as sure - will post step-by-step.

Thanx for the GREAT help…LW5

Re: Large file copy problem solved?

Lakewater5 1216866181|%e %b %Y, %H:%M %Z|agohover

StreetUrchin you are 100% on the spot. I formatted the partition and am up to about 100G so far…No issues.
I will submitt an edit to this post in day or two with step by step. (the step by step is a copy from another post but I am not sure where)

Thanks to all for he help!!!!!!!!!!!

Re: Large file copy problem solved?

greenfish 1217321714|%e %b %Y, %H:%M %Z|agohover

i've tried all of the solutions mentioned in here, and im still getting I/O errors moving my 9GB files.

Seriously im begining to regret I bought this piece of junk in the first place

Re: Large file copy problem solved?

Johny_Airman 1217741780|%e %b %Y, %H:%M %Z|agohover

This is maybe totally dumb idea, but so far I have 8 DVDs on it (=cca 30GB) without any crash or problem.
But - reading few previous posts about PUBLIC partition - that was the one pre-created on the drive. Well, I did not like the name, so I ignored it and created "new shared folder" (named BACKUP) and it is in the list of folders side by side with PUBLIC. I never used that public one and I'm copying and copying all my data into that backup folder.

Apart from creating that one shared folder, the unit is straight out of the box standing here in front of me. No other hacking, setting, formating or anything else.

Anyone thinks, that simply creating a new, (untouched by bad WD guys) might have solved the problem in my case? Could someone WITH the problem of copying try it as a test and post the result? Or they really fixed it in WD, since I have it only since yesterday (have no clue, how long that lied at the seller's shelf, I bought it of eBay)

last edited on 1217741887|%e %b %Y, %H:%M %Z|agohover by Johny_Airman + show more

Re: Large file copy problem solved?

alephsmith 1217744906|%e %b %Y, %H:%M %Z|agohover

Not everyone has this problem so your experience is unlikely to indicate a solution but rather the plain fact that you have not been affected. Both the PUBLIC folder and your new BACKUP folder reside on the same partition and in addition this problem is not restricted to /dev/sda4 anyway.

I think it is quite clear that some drives are shipping/or develop over time corruption making it necessary to reformat the drive. Has anyone just tried asking for a replacement drive from WD?

Re: Large file copy problem solved?

greenfish 1218136175|%e %b %Y, %H:%M %Z|agohover

STILL having issues.. for crying out loud. :(

I just transfered 2GB = input/output error, give me a break.. i mean seriusly wtf

sorry for language guys but this is getting to my head

Re: Large file copy problem solved?

Pineapple 1218144990|%e %b %Y, %H:%M %Z|agohover

Your drive is probably corrupted. Delete all the files in the share and:
Login with SSH, get into superuser with su.
Unmount the share partition: umount /dev/md4
Run disc check: fsck.ext3 /dev/md4

Re: Large file copy problem solved?

greenfish 1219050817|%e %b %Y, %H:%M %Z|agohover

hi Pineapple sorry for the late reply. I can't run fsck hell i cant do anything anymore. for some odd reason I can't no matter what unmount my /dev/md4 volume, it's always the same thing:

sudo umount /dev/md4
umount: Couldn't umount /shares/internal: Device or resource busy

sudo umount /dev/md4
umount: Couldn't umount /shares/internal: Device or resource busy

I've tried killing my NFSD completly, removing /etc/exports completly etc etc… booting into kernel - user mode only …still it complains the unit is busy..

Heh…. any ideas friend?

Thanks

Re: Large file copy problem solved?

greenfish 1219052573|%e %b %Y, %H:%M %Z|agohover

I removed dev/md4 completly from /etc/fstab, rebooted the NAS, and then I run fsck.ext3 /dev/md4

fsck.ext3 /dev/md4
e2fsck 1.38 (30-Jun-2005)
/dev/md4: clean, 94/60555264 files, 16251787/121081888 blocks (check after next mount)

so I guess thats a good thing?

too bad I still cant transfer any large files without that "nice" I/O error

Re: Large file copy problem solved?

greenfish 1219955701|%e %b %Y, %H:%M %Z|agohover

bump

Re: Large file copy problem solved?

Pineapple 1221260031|%e %b %Y, %H:%M %Z|agohover

Everything looks good. Double check oplock is off in mybook and Windows PC.
I'll take a look at my smb.conf and see what I'm using now.

Re: Large file copy problem solved?

Venderwel 1222678843|%e %b %Y, %H:%M %Z|agohover

Hi there, not sure if I have the same problem, but I got the WD My Book World 500GB for 2 days now and I have problems writing files to it.
After a few minutes it goes in error and after that I can't write files in the directory anymore. Anyone knows what that is?

Re: Large file copy problem solved?

Aspirin 1222724553|%e %b %Y, %H:%M %Z|agohover

Venderwel, you may want to share the error message.

Re: Large file copy problem solved?

Venderwel 1222874413|%e %b %Y, %H:%M %Z|agohover

Darn, I can't even login with ssh to root anymore!! :-( HELP!!

Re: Large file copy problem solved?

nikp 1223288775|%e %b %Y, %H:%M %Z|agohover

Can anyone explain to me what this issue solves? It sounds like some issue i have. I have found that some AVI files on my MyBook cause the network connection to drop. But these avi's can be as 'small' as 175mb. Is this the same problem?

Re: Large file copy problem solved?

Darkhorse 1223448261|%e %b %Y, %H:%M %Z|agohover

nikp - Have a look at this thread as I think it might be more relevant: here

Re: Large file copy problem solved?

Gudis 1226317831|%e %b %Y, %H:%M %Z|agohover

Hi,

I'm new to this forum and dont have any experience with LINUX… Is there a way to fix this Network drive through windows ???

Re: Large file copy problem solved?

Pineapple 1227052906|%e %b %Y, %H:%M %Z|agohover

Nop. Windows can only read the disk but not formatting it. You need to SSH in the drive and use Linux commands.

Re: Large file copy problem solved?

Furedetsu 1227542478|%e %b %Y, %H:%M %Z|agohover

When I want to unmount or run disk check I get "Couldn't umount /shares/internal: Device or resource busy".

edit: I did a reboot of the MyBook and now its doing the disk check…
can someone explain step by step how to turn off Oplocks?
It doesn't find the /var/oxsemi/smb.conf file and I don't understand Lakewater's solution…

edit: After disk check I still got the file copy error.
After a format everything seems fine.
I just tranferred 40G to My Book.

last edited on 1227567676|%e %b %Y, %H:%M %Z|agohover by Furedetsu + show more

Re: Large file copy problem solved?

costelg 1245817728|%e %b %Y, %H:%M %Z|agohover

Will a firmware upgrade fix this large file limit problem?

What is SSH and where do I get it??

Rgds

George

Re: Large file copy problem solved?

boondoklife 1245856523|%e %b %Y, %H:%M %Z|agohover

what is the issue you are having? I am able to copy large volumes of data all day long with out an issue.

---

while ( love & passion ) {
for( fight = 0 ; rights < freedom ; rights++ )
fight = standup( rights );
free( babylon ); }