WHS Users - Drive Balancing tool now available... - Page 3 - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #61 of 82 Old 01-09-2010, 05:43 AM
Member
 
cranston's Avatar
 
Join Date: Feb 2005
Posts: 22
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Hi Beefcake:

WHS HP EX470, 1 x 500GB and 1 x 1TB disks. The 500gigger got about 80% filled, and then I added the 1TB. After a couple months, the current state is that the 500gigger is filled, and the 1TB is about 40% filled. So, on Thursday night (this is now Saturday morning), I started running the tool. Before starting the tool, I shut off all the backups to the WHS, and to my knowledge, none of the computers on the network have had interface with the server.

Upon start, the tool took up about 500 more GB of the remaining 600 GB on the 1TB drive, and the tool has been running ever since, approximately for the last 36 hours. The DOS window on the tool keeps saying:

---------------

Drive 0 is 75% done.
Going to sleep while data is being moved ...
Total Disk Throughput: x MB/s (where x ranges from 0 to as high as 12)
[it then cycles thru approximately 12 of the Total Disk Throughputs and then says]
DE Service being restarted due to low disk activity for 30 seconds ...

Restarting the Drive Extender Service ....

and then it starts again

-----------------

As I said, this has been going on for 36 hours or so. Is this normal, and if so, a) how much longer should I expect it to run and b) if I want to shut it off, can I safely do so without corrupting anything?

Thanks,
Cran
cranston is offline  
Sponsored Links
Advertisement
 
post #62 of 82 Old 01-09-2010, 01:27 PM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Sounds like you are having low bandwidth issues. I would suggest killing the process with a Ctrl-C and then deleting the "fake" directories as specified in the readme. I would also suggest rebooting the WHS server and trying again to make sure you are using the latest version (1.05). When running this version you have the option to set the threshold at which the tool will try to restart DE. I'd suggest lowering that number to something like 2 (for 2MB/sec).

I've had this issue in the past with low bandwidth and it is not directly related to my tool, but rather something in WHS.

Let me know how it goes for you.

-Brian
Beefcake is offline  
post #63 of 82 Old 02-01-2010, 03:50 PM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Hey! I finally fixed the program for you! Could you please download it from this link?
Beefcake is offline  
post #64 of 82 Old 02-23-2010, 01:06 PM
Member
 
sean.scott's Avatar
 
Join Date: May 2007
Location: Hyde Park, NY
Posts: 44
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
The last time I ran the drive balancer script, I was able to get all of the non system data off of the system drive and distribute the rest of the data on the non-system drives.

When answering the questions:

Do you want to include D: in the balancing effort (y/n)?
Do you want to clear out the LandingZone (y/n)?

How would I answer those to get the desired results of having no shared data on the system drive?

Thanks,
-Sean

Pioneer Kuro PRO-151FD | KRP-500M | PDP-5020FD | PDP-4280HD

Pioneer Elite SC-05 | VSX-01TXH | BDP-23FD

Bowers & Wilkins CM8 | CM-Center | M-1

http://sean-scott.com

sean.scott is offline  
post #65 of 82 Old 02-23-2010, 03:48 PM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Hi Sean,

I would like you to check the link (post #63) above for a newer version of the tool. I worked extensively on getting the algorithm for clearing off the system drive as good as possible. There should be no shared data after it is done and I think that just means backups of client PCs are still on the system drive.

Then, you would answer
Do you want to include D: in the balancing effort (y/n)? => n
Do you want to clear out the LandingZone (y/n)? => y

Let me know how it goes. I've had a tester or two and I'd like to release it, but will await feedback from you.

-Brian
Beefcake is offline  
post #66 of 82 Old 02-24-2010, 09:55 PM
Member
 
sean.scott's Avatar
 
Join Date: May 2007
Location: Hyde Park, NY
Posts: 44
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Brian, I ran the downloaded version from post #64 and it didn't seem to give me much different results.

I've attached some screen shots of before and after of disk usage. Let me know if there is anything else you'd like me to try.





Pioneer Kuro PRO-151FD | KRP-500M | PDP-5020FD | PDP-4280HD

Pioneer Elite SC-05 | VSX-01TXH | BDP-23FD

Bowers & Wilkins CM8 | CM-Center | M-1

http://sean-scott.com

sean.scott is offline  
post #67 of 82 Old 02-25-2010, 07:33 AM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
The fact is that Drive Balancer is telling WHS to move data and it won't move any more. The thing that comes to mind is backups and duplicated data. I see you have 130GB of backups, so that's part of it. Do you have folder duplication turned on? If so, and if you are comfortable with doing this for the run of Drive Balancer, I'd suggest turning off all folder duplication and then try running Drive Balancer again after disk activity stops from the turning off of folder duplication.
Beefcake is offline  
post #68 of 82 Old 03-15-2010, 11:58 AM
AVS Special Member
 
HTPCat's Avatar
 
Join Date: Dec 2007
Location: Citrus Heights, CA
Posts: 1,226
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 23
Quote:
Originally Posted by cranston View Post

Hi Beefcake:

WHS HP EX470, 1 x 500GB and 1 x 1TB disks. The 500gigger got about 80% filled, and then I added the 1TB. After a couple months, the current state is that the 500gigger is filled, and the 1TB is about 40% filled. So, on Thursday night (this is now Saturday morning), I started running the tool. Before starting the tool, I shut off all the backups to the WHS, and to my knowledge, none of the computers on the network have had interface with the server.

Upon start, the tool took up about 500 more GB of the remaining 600 GB on the 1TB drive, and the tool has been running ever since, approximately for the last 36 hours. The DOS window on the tool keeps saying:

---------------

Drive 0 is 75% done.
Going to sleep while data is being moved ...
Total Disk Throughput: x MB/s (where x ranges from 0 to as high as 12)
[it then cycles thru approximately 12 of the Total Disk Throughputs and then says]
DE Service being restarted due to low disk activity for 30 seconds ...

Restarting the Drive Extender Service ....

and then it starts again

-----------------

As I said, this has been going on for 36 hours or so. Is this normal, and if so, a) how much longer should I expect it to run and b) if I want to shut it off, can I safely do so without corrupting anything?

Thanks,
Cran

Quote:
Originally Posted by r.mcadams View Post

Beefcake, I used the tool once at it worked perfectly... a couple months later I used it again, and for some reason it didn't seem to complete (all drives were 98% full). Sooo, I manually deleted the files in the following directories:

C:\\fs\\*\\fake\\
D:\\fake\\

After doing this, I've had continuous "file conflicts" within WHS. When I open the "File Conflict" dialog box within WHS console, I get a long list of files with "The group or resource is not in the correct state to perform the requested operation". Based on what I could find online, that seems to indicates a "file handle" problem (I have no idea what that means... I'm sure you do) and that the only solution is to delete the files, which I have done multiple times. After doing so, another long list of files will show up next time, do the same thing... NEW long list of file conflicts.

Any help you can provide would be greatly appreciated! :-)

Thanks, Rich

So I used this tool last week on my WHS that is currently populated with 16 HDD's with 18TB allocated to the storage pool. When it first ran it was working fine up until I went to bed that night at which point it had almost balanced out my storage pool but had one more drive that was still far from equalling the others percentage wise so it was still working...so far so good then when I awoke I had the problem above where it is looping going to sleep and then re-starting with write speeds of 12 down to 0 then over again.

So I did a Crl C to kill the process and did the delete process like Rich in the above post and ended up with the same problem he had of file conflicts. I am pretty sure it has something I did when deleteing the folders

C:\\fs\\*\\fake\\
D:\\fake\\

Anyways there was no way to fix the file conflicts I tried disconnecting all clients and rebooting WHS to no avail and the files affected were video files that would no longer play - Ended up having to delete around 1.5 TBs of movies to fix all the conflicts. Not a huge deal since I can re-rip the movies. I would still like to run this app again but just not feeling comfortable at this point what I did wrong so I can avoid another batch of file conflicts . Any insight as to what may have gone wrong?

HTPCat
"Give Me More Audio & Video Toys"

HTPCat is offline  
post #69 of 82 Old 03-15-2010, 04:33 PM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Very interesting about the file conflicts. There is no way deleting the "fake" directories is causing it and I can assure you that it is not Drive Balancer itself causing the conflicts, but something underneath all the action. I think you would have eventually had those conflicts, and DB made the problem come to the surface.

The root of the problem is why are you seeing such low bandwidth when the tool is actively trying to move data. 12MB/s is horrible. With internal drives, you should be getting at least 30+MB/s. I regularly get 47-100MB/s per disk (up to two at a time) when I run my tool.

I would suggest looking in the event viewer to see if you are seeing any disk/controller errors occurring. Do you see anything in there?

LMK
Beefcake is offline  
post #70 of 82 Old 03-15-2010, 09:42 PM
Newbie
 
chucklesg's Avatar
 
Join Date: Dec 2006
Posts: 2
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
HELP
I used this program and now my 5 x 1.5Tb drives are nicely balanced BUT my 1Tb System drive is 98%full

How the heck do I Balance' the system drive? I did delete the D:\\Fake files but could not find and 'Fake' files or Dirs under C:\\fs\\*\\fake (btw what does the * mean? all subdirs under fs? - I looked under all Subdirs and found no fake files)...

I have 10+ images of my BluRay movies on my data drives, is it because the files are greater than 20GB the system drive is screwed up?

I've re-installed the server, repaired the database (which finally allowed me to access my backups from my 10 home computers which were inaccessible after I used the Drive Balancer.

Now I have to fix my system drive because it slows down my backup when the System drive is 98% full

Again please help...



Chuck G.
chucklesg is offline  
post #71 of 82 Old 03-15-2010, 09:46 PM
Newbie
 
chucklesg's Avatar
 
Join Date: Dec 2006
Posts: 2
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Update - I just relogged onto the WHS Console and looked under the Disk Mgt tab and the System drive is back to 2%...

Don't know what happened but REALLY Glad it did )

Chuck G.
chucklesg is offline  
post #72 of 82 Old 03-16-2010, 06:03 AM
Advanced Member
 
BobSalita's Avatar
 
Join Date: Jun 2001
Location: CHI, CLD, PAR, PHX, PDX, NKC, AUK, MEL, SYD, BUE, SCL
Posts: 936
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 12
Is it beneficial to run this on a WHS system which has no folder duplication enabled? My system drive is 53% full but the other three drives are 93-98%. I'd like to balance the drives because, I'm guessing here, that PerfectDisk 11 will have more possibilities to defragment. I find defragmenting to be extremely important for throughput.

Former Top 1000 Amazon reviewer bemoaning the loss of objective user reviews. Amazon, stop your complacency!
BobSalita is offline  
post #73 of 82 Old 03-16-2010, 09:54 AM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
BobSalita,
It really sounds like you need to get another drive and then run the balancing tool. I don't know if PD11 will run any better by emptying the drives. They claim you need 1% of the drive free to defrag.

Folder duplication definitely makes it harder for drive balancer to do a good job when the pool is nearing capacity. Shutting it off would enable you to free up space and have less road blocks when balancing.
Beefcake is offline  
post #74 of 82 Old 04-02-2010, 09:04 PM
Newbie
 
bitpicker1's Avatar
 
Join Date: Apr 2010
Posts: 1
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I'm having problems with an error.

Thanks,

Ken
Driver Balancer 1.5

Four drives
one 250 GB System Drive Western digital
Two 1.5 TB drives Seagate
one WD Green drive (4K sector, but jumpered) WD15EARS

I've a copy of the command problem screen text.


C:\\Temp>"drive balancer"
Do you want to include D: in the balancing effort (y/n)? n
Filling Up LandingZone so it doesn't get used
Do you want to clear out the LandingZone (y/n)? n

Getting free space information on each drive in pool...
Getting information for C:\\fs\\D ...
Getting information for C:\\fs\\F ...
Getting information for C:\\fs\\H ...

Please choose from the options:
1) Have an equal percentage of each disk used
2) Have an equal amount of data on each disk
3) Have an equal amount of free space on each disk
4) Enter your own values - CAUTION -
5) Quit - if no option works for you

Choice (1-5): 1

Making Calculations for drive balancing...
Stats - Drive: 0 TotalCap: 1397GB NewFreeSpace: 558GB
Stats - Drive: 1 TotalCap: 1397GB NewFreeSpace: 558GB
Stats - Drive: 2 TotalCap: 1397GB NewFreeSpace: 558GB
Does this seem appropriate (y/n)? y
Enter the time interval in secs for checking disk activity (Normal = 5):

Unhandled Exception: System.FormatException: Input string was not in a correct f
ormat.
at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffe
r& number, NumberFormatInfo info, Boolean parseDecimal)
at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo in
fo)
at System.Convert.ToInt32(String value)
at Drive_Balancer.DriveBalancer.CalculateFreeSpace(Int32& IntervalTime, Int32
& DiskBandwidth, UInt64[]& newFreeSpace, Double[]& totalGBytes, Double[]& freeGB
ytes, List`1& Folders)
at Drive_Balancer.Program.Main()
bitpicker1 is offline  
post #75 of 82 Old 04-05-2010, 03:05 PM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
you have to actually type a number in.... hitting in enter will result in the error. I am going to fix this is a future release.

sorry about this oversight.
Beefcake is offline  
post #76 of 82 Old 06-11-2010, 11:28 PM
AVS Special Member
 
palehorse's Avatar
 
Join Date: Nov 2005
Posts: 2,393
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
I'm getting an average between 96 and 110 MB/s for "Total Disk throughput" during this process. Where does that sit in terms of average/expected speeds for this script? (High/average/low/abysmal?)

I'm using 2x 2TB EARS drives that I just re-added using the 7/8 jumper method, so I'm trying to figure out if they're performing properly. (In fact, this balance run is a result of my shuffling their data around in order to add said jumpers since my dumb ass missed that necessity when I first built the system a few weeks ago... woops! )

EDIT: As it approaches the end of Drive 0 (90% done), the speeds have jumped to an average of 165 MB/s, or so... once again, how does this compare to others?
palehorse is offline  
post #77 of 82 Old 06-12-2010, 06:29 AM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Well, it can only ever move 1 file at a time, so take the total and divide by 2. You are averaging about 50MB/s, so that's not bad if you have lots of smaller files. If it's one giant file, then you should be able to get 120-150+ total (60-75 per drive)

-Brian
Beefcake is offline  
post #78 of 82 Old 06-12-2010, 08:17 AM
AVS Special Member
 
palehorse's Avatar
 
Join Date: Nov 2005
Posts: 2,393
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by Beefcake View Post

Well, it can only ever move 1 file at a time, so take the total and divide by 2. You are averaging about 50MB/s, so that's not bad if you have lots of smaller files. If it's one giant file, then you should be able to get 120-150+ total (60-75 per drive)

-Brian

excellent, thank you for the prompt response and good info! And, of course, thanks for such a great piece of software!

In this case, it was about 100 bluray mkv's, their associated metadata, 100+ GB of mp3's, and 50GB of photos that were getting shuffled around... so I guess that would qualify as lots of small files... and you're saying that's "not bad."

Cool!

Thanks again!
palehorse is offline  
post #79 of 82 Old 08-05-2010, 06:55 PM
Newbie
 
embeeuu's Avatar
 
Join Date: Jul 2010
Posts: 2
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
What a great little program. It did exactly what I wanted. Only thing was I had to delete the D:\\fake directory at the end. Thanks
embeeuu is offline  
post #80 of 82 Old 12-13-2010, 06:12 AM
Newbie
 
Mr Bogus's Avatar
 
Join Date: Feb 2010
Posts: 1
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Yea I know this is an old thread but...

As WHS is based on Server 03, can this be run on a W2003 server? (I would try it but all my 03 boxes are production and do not have a test box up right now)..


Thanks!
Mr Bogus is offline  
post #81 of 82 Old 12-13-2010, 09:18 AM
Senior Member
 
gcoupe's Avatar
 
Join Date: Jan 2009
Location: Netherlands
Posts: 304
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by Mr Bogus View Post

As WHS is based on Server 03, can this be run on a W2003 server?

As far as I am aware: Nope. It uses the Drive Extender technology that is unique to WHS, and which Microsoft in their infinite wisdom have just axed. There goes the Unique Selling Point of WHS...

Geoff Coupe
gcoupe is offline  
post #82 of 82 Old 12-13-2010, 08:29 PM - Thread Starter
Advanced Member
 
Beefcake's Avatar
 
Join Date: Sep 2001
Location: Charlton, MA USA
Posts: 933
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
As the author, this tool will not work on anything but WHS v1.
Beefcake is offline  
Reply Home Theater Computers

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off