Building a low power, high perf NAS with iSCSI - Page 2 - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #31 of 51 Old 01-10-2011, 11:22 AM - Thread Starter
AVS Special Member
 
rajdude's Avatar
 
Join Date: May 2005
Location: Woodbridge, VA
Posts: 2,102
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by aaronwt View Post

With such a small amount of critical data, why not just get a 2TB hard drive(under$100) to put the info on for a backup.

That is what I currently do, but not too happy with it. Reliability of those backups is the issue.


Quote:
Originally Posted by aaronwt View Post

For me I use a WHS with duplication, an INtel NAS with RAID 5, a software RAID 5 solution in a networked PC(TiVo Server) , a couple of DLink NAS boxes with RAID 1, and a 2TB eSATA/USB box with rebit software for backup of my main PC which uses a couple of RAID 0 arrays.(so i use reBit and also image the system periodically since if any of the five drives in the arrays dies, it's toast.)

May I ask why so many storage nodes?
Why not consolidate all that data into one server [be it a NAS or a WHS] ??
Then back it up........ no?

-Rajiv
rajdude is offline  
Sponsored Links
Advertisement
 
post #32 of 51 Old 01-10-2011, 11:39 AM
AVS Club Gold
 
aaronwt's Avatar
 
Join Date: Aug 2001
Location: Northern VA(Woodbridge)
Posts: 20,801
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 259 Post(s)
Liked: 593
Quote:
Originally Posted by rajdude View Post

That is what I currently do, but not too happy with it. Reliability of those backups is the issue.




May I ask why so many storage nodes?
Why not consolidate all that data into one server [be it a NAS or a WHS] ??
Then back it up........ no?

I already have 44TB in my WHS which I use mostly for movies/pictures. My Intel NAS is for music/pictures and the RAID in my TiVo server is for my TV recordings. The Dlinks backup my downloaded Zune music, pictures and some of my critical files.
And my WHS can handle some temporray overflow since I have around 15TB of free space left.

But all my Network storage equals close to 60TB of storage. I thought the WHS limit is around 50TB(not sure off hand). But I also don't want to put all my eggs in one basket. So if there are any issues with any device it will not affect everything. My movie watching, Tv watching, Music listening etc. (for instance, I'm upgrading some of my 1.5TB drives in my WHS with 2TB drives. To remove one of my USB drives can take 8 to 12 hours to move the duplicated data to other drives. During that time the network shares are not accessible. By having stored content in other network devices I will still have access to that content)

39TB unRAID1--53TB unRAID2--36TB unRAID3
LED DLP
XBL/PSN: WormholeXtreme

aaronwt is offline  
post #33 of 51 Old 01-10-2011, 12:08 PM - Thread Starter
AVS Special Member
 
rajdude's Avatar
 
Join Date: May 2005
Location: Woodbridge, VA
Posts: 2,102
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
BTW, I just posted some observations on OpenFiler above in this thread:

http://www.avsforum.com/avs-vb/showt...7#post19775987

-Rajiv
rajdude is offline  
post #34 of 51 Old 01-11-2011, 07:28 AM - Thread Starter
AVS Special Member
 
rajdude's Avatar
 
Join Date: May 2005
Location: Woodbridge, VA
Posts: 2,102
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Wait a minute......are you sure you have 18TB or is it really 18 GB?

18 TeraBytes is WAAAAAAAY too big!

How did you manage to send 18TB over a WAN to crashplan?
Your ISP must be pissed, to say the least


Quote:
Originally Posted by mlknez View Post

Use crashplan http://www.crashplan.com I have over 18TB on there right now.


-Rajiv
rajdude is offline  
post #35 of 51 Old 01-11-2011, 07:33 AM
AVS Special Member
 
mlknez's Avatar
 
Join Date: Jul 2005
Posts: 1,842
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 17
Quote:
Originally Posted by rajdude View Post

Wait a minute......are you sure you have 18TB or is it really 18 GB?

18 TeraBytes is WAAAAAAAY too big!

How did you manage to send 18TB over a WAN to crashplan?
Your ISP must be pissed, to say the least

Terabytes.

I now have 18.2TB as of this morning and climbing. My upload speed is 5Mbps but I throttle crashplan at 3.5.
mlknez is offline  
post #36 of 51 Old 01-11-2011, 07:45 AM
AVS Special Member
 
mlknez's Avatar
 
Join Date: Jul 2005
Posts: 1,842
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 17
Quote:
Originally Posted by rajdude View Post

I have tried backup to cloud using Mozy.

Very slow uploads but the deal breaker was the restore problems.

When the time came to restore [I lost a disk] Mozy gave me a hard time. the client kept on giving me errors. I opened a ticket and the support guys kept going roung and round in circles.

However, after about 2 weeks of troubleshooting, they were finally able to restore my data.

have you tried restoring data with Crashplan yet?

I have restored a few files for testing. They do also have a service where you can send them hard drives and they will send them back with the data you specify for an extra fee.
mlknez is offline  
post #37 of 51 Old 01-11-2011, 07:45 AM - Thread Starter
AVS Special Member
 
rajdude's Avatar
 
Join Date: May 2005
Location: Woodbridge, VA
Posts: 2,102
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Amazing!

Questions:

1. Have you ever tried restoring something off crashplan? [already replied, so ignore]

2. How long did it take to send 19 TB to crashplan?

3. Does your ISP not monitor or limit your bandwidth usage?



Quote:
Originally Posted by mlknez View Post

Terabytes.

I now have 18.2TB as of this morning and climbing. My upload speed is 5Mbps but I throttle crashplan at 3.5.


-Rajiv
rajdude is offline  
post #38 of 51 Old 01-11-2011, 08:18 AM - Thread Starter
AVS Special Member
 
rajdude's Avatar
 
Join Date: May 2005
Location: Woodbridge, VA
Posts: 2,102
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
After going through all my options and viewing countless pages on the internet, I have come to the conclusion that my needs may not be served by a NAS.

At this point I have started looking at Windows Home server [WHS].

For keeping the power consumption low I am looking at using Atom processor based Mobos. I am currently installing a WHS on a VM to see how it works.

I will lose iSCSI capability……..well, no big deal if the network throughput is high enough.

A few advantages:
  1. Able to run enterprise backup software like backupexec
  2. Able to use tape drives, but I will probably stay with external HDD and cloud backup as an off site backup solution
  3. Ability to use cloud backup services like Mozy << I just tried...Mozy home does not work. You have to get their MozyPro service...pretty expensive!
  4. RDP connection to WHS……..means I can run a web browser back home and download torrents etc

-Rajiv
rajdude is offline  
post #39 of 51 Old 01-17-2011, 03:41 PM
Member
 
kappaknight's Avatar
 
Join Date: Sep 2004
Location: Atlanta, GA
Posts: 73
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I just picked up a Synology 1511+ and I'm pretty happy with it. When doing the initial data dump from my Mac mini's hard drive, a older NAS and 2 external USB drives, I was getting write speeds of 70 to 90MB/s to the Synology NAS. Streaming large Blu-ray movies have not been an issue and it was one of the reasons I picked this up cause my external USB drives were too slow. With the 1511+, you can also aggregate the link to get a 2 gigabit connection - but I doubt I'll need that for a few more years.

I still don't know how to use the iSCSI feature but it's available... maybe I'll look into it this week.

As for off site backup, there are a number of services out there that are built off of Amazon's S3 storage system like Jungledisk. You can pay for the software then just pay the monthly storage fees to Amazon... YMVV depending on how much media you have. Of course, transferring TB's over the web will be slow... but options for speed are currently limited for cloud based storage.
kappaknight is offline  
post #40 of 51 Old 01-18-2011, 10:01 AM
Member
 
98_1LE's Avatar
 
Join Date: Mar 2008
Posts: 149
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I wanted to append this thread with write performance numbers from my unRAID box (which is low end cheap hardware through and through).

Write speeds over NFS without parity: 64.9 MB/s (4 TB of BD images)
Write speed over NFS with parity: 22.7 MB/s (2 TB of DVD images)
Write speed over SMB using wifi N from laptop 40' away from AP (with parity but it isn't the limiting factor): 10.8 MB/s

My home theater, 16 years and growing in the making (bought the main 5.5's new in 1995)
98_1LE is offline  
post #41 of 51 Old 01-18-2011, 10:31 AM
AVS Club Gold
 
aaronwt's Avatar
 
Join Date: Aug 2001
Location: Northern VA(Woodbridge)
Posts: 20,801
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 259 Post(s)
Liked: 593
What is the maximum size and number of drives that unRAID can scale to now?

39TB unRAID1--53TB unRAID2--36TB unRAID3
LED DLP
XBL/PSN: WormholeXtreme

aaronwt is offline  
post #42 of 51 Old 01-18-2011, 10:42 AM
Advanced Member
 
mbryanr's Avatar
 
Join Date: Nov 2009
Posts: 539
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by aaronwt View Post

What is the maximum size and number of drives that unRAID can scale to now?

20 data, 1 parity, 1 cache.

Max is currently 2TB size, but 3TB support is closer to reality...
http://lime-technology.com/forum/ind...96322#msg96322

Advanced Format (4K alignment) support was released with 4.7-Beta1
mbryanr is offline  
post #43 of 51 Old 01-18-2011, 12:28 PM
AVS Club Gold
 
mrlittlejeans's Avatar
 
Join Date: Jan 2004
Location: the area formerly known as Black Forest
Posts: 4,713
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 191
Yup. 3TB drive support is scheduled for the 5.0 release at which time 60TB will be the capacity.

Absence of evidence is not evidence of absence

mrlittlejeans is offline  
post #44 of 51 Old 01-18-2011, 02:06 PM
AVS Club Gold
 
aaronwt's Avatar
 
Join Date: Aug 2001
Location: Northern VA(Woodbridge)
Posts: 20,801
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 259 Post(s)
Liked: 593
Quote:
Originally Posted by mbryanr View Post

20 data, 1 parity, 1 cache.

Max is currently 2TB size, but 3TB support is closer to reality...
http://lime-technology.com/forum/ind...96322#msg96322

Advanced Format (4K alignment) support was released with 4.7-Beta1

Thanks
I've been replacing a bunch of my 1.5TB drives in my WHS and need to do something with them. So I was thinking about setting up one of my old PCs with unRAID and dedicating that to storing my MKVs made from my HD DVDs. Leaving me more room in my WHS for my BD ISOs. Since with duplication my 46TB of storage is really only half that.

39TB unRAID1--53TB unRAID2--36TB unRAID3
LED DLP
XBL/PSN: WormholeXtreme

aaronwt is offline  
post #45 of 51 Old 01-18-2011, 05:10 PM
AVS Special Member
 
sean_w_smith's Avatar
 
Join Date: Jan 2005
Location: Los Altos, CA and Huntsville, UT
Posts: 4,292
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by aaronwt View Post

Thanks
I've been replacing a bunch of my 1.5TB drives in my WHS and need to do something with them. So I was thinking about setting up one of my old PCs with unRAID and dedicating that to storing my MKVs made from my HD DVDs. Leaving me more room in my WHS for my BD ISOs. Since with duplication my 46TB of storage is really only half that.

unraid works very well for SMB is farily easy to get up and running.... 46Tb of storage on unraid is 44.5TB which is why I never could understand the appeal of WHS. Its so inefficient with respect to space utilization. Its easy to test out on an old PC. Thats how a lot of people start with it.

Sean
sean_w_smith is offline  
post #46 of 51 Old 08-13-2011, 05:22 AM
Newbie
 
this-is-me's Avatar
 
Join Date: Aug 2011
Posts: 3
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
I know this thread is old by now, but I just wanted to mention that iSCSI support can be added to WHS! I am currently doing just that. I use the StarWind iSCSI SAN - Free Edition on WHS. It doesn't have many limits.

You will need to download the Microsoft iSCSI Initiator from here:

Code:
www.microsoft.com/download/en/details.aspx?id=18986
Then you will need to download the StarWind Software from here:

Code:
www.starwindsoftware.com/starwind-free
(choose Download under Free Edition)

Afterwards, you will setup the software on the WHS server. You can do this by using Remote Desktop or by using Advanced Admin Console. Once everything is set up, use the StarWind Installer on one of your other computers to install the management console. Then connect to your server and start using iSCSI. The default user/password is root/starwind.

For me, iSCSI is invaluable because it allows me to run diskless workstations. Windows 7 has native iSCSI support for installation, and I use a combination of TFTPD32 Service Edition on WHS, DD-WRT on the router, and rom-o-matic.net to do net booting. For more information about setting up network iSCSI boot, go here:

Code:
erwan.l.free.fr/iscsi/body.html
The information is old, but still works just fine.
this-is-me is offline  
post #47 of 51 Old 08-13-2011, 06:46 AM
AVS Club Gold
 
aaronwt's Avatar
 
Join Date: Aug 2001
Location: Northern VA(Woodbridge)
Posts: 20,801
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 259 Post(s)
Liked: 593
WHy iSCSI though? Although in the 90's I used SCSI for everything I could except for Hard drives since they were too expensive but I can get plenty of speed using eSATA or internal SATA drives. But even for storage USB is more than fast enough for my BD ISOs. I have 31 drives in/attached to my WHS with 23 of them over USB. in my unRAID I have 22 drives with two on the internal SATA ports and 20 in eSATA enclosures. They all work great with no issues. Over the last 15 years I've used at least 175 non Enterprise drives at home with no issues from them.

39TB unRAID1--53TB unRAID2--36TB unRAID3
LED DLP
XBL/PSN: WormholeXtreme

aaronwt is offline  
post #48 of 51 Old 08-13-2011, 07:10 AM
AVS Special Member
 
mlknez's Avatar
 
Join Date: Jul 2005
Posts: 1,842
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 4 Post(s)
Liked: 17
Quote:
Originally Posted by rajdude View Post

I have tried backup to cloud using Mozy.

Very slow uploads but the deal breaker was the restore problems.

When the time came to restore [I lost a disk] Mozy gave me a hard time. the client kept on giving me errors. I opened a ticket and the support guys kept going roung and round in circles.

However, after about 2 weeks of troubleshooting, they were finally able to restore my data.

have you tried restoring data with Crashplan yet?

I did a 500GB test restore and it worked just fine.
mlknez is offline  
post #49 of 51 Old 08-13-2011, 07:10 AM
Newbie
 
this-is-me's Avatar
 
Join Date: Aug 2011
Posts: 3
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
iSCSI is (for me) just plain fun. Imagine this scenario:

You have a Windows Home Server
You have 3 home comptuers (Most have more. I have 10+, but this is an example)
Each of the computers have > 250GB Hard drives locally installed (Ususally larger. Mine vary in the 250-750GB range, but this is an example).

For this scenario, it would be feasible to convert the individual computers to diskless using iSCSI. You can take each of the hard drives and convert them to iSCSI using cloning software. Then, you can take the physical drives and add them to your WHS Server to expand its storage. Suddenly (in this example) your WHS has maybe another 500GB of usable storage space. Add to this the fact that each computer's boot drive is now stored in a file on a replicated server share, and that means that computer backup is no longer necessary. Also, since the physical drives are in the server, the likelihood of loss of data is almost nil. If one of your desktop's drives (which is now in the server) fails, WHS will take up the slack of being ready for a replacement drive without the need of recovery software.


Now imagine this: You buy a new computer. It comes with a 1TB hard drive (this is an average on most standard desktops). This is more than you need for an individual computer. What is the best use of this space? WHS. With iSCSI, you can take that 1TB hard drive and put it in the server. Any space you would normally use on the physical computer is now used on the network drives.

Let's say that with all programs installed you would normally use 60GB of space on your local computers' hard drives. Well now that 1TB you have is being wasted if it remains in the desktop PC. You could use that extra 940 gigabytes in your server to much better advantage. With iSCSI you have your cake and eat it too. You have extra storage in the server you wouldn't have had, and you have nearly the same functionality on the local computers.

The one caveat to iSCSI boot is that you do not have hibernation support when booting Windows from an iSCSI source. This is one thing that you may find lacking without a local disk. Other than that, iSCSI benefits may outweigh the cons.
this-is-me is offline  
post #50 of 51 Old 08-13-2011, 12:01 PM
AVS Club Gold
 
aaronwt's Avatar
 
Join Date: Aug 2001
Location: Northern VA(Woodbridge)
Posts: 20,801
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 259 Post(s)
Liked: 593
Quote:
Originally Posted by this-is-me View Post

iSCSI is (for me) just plain fun. Imagine this scenario:

You have a Windows Home Server
You have 3 home comptuers (Most have more. I have 10+, but this is an example)
Each of the computers have > 250GB Hard drives locally installed (Ususally larger. Mine vary in the 250-750GB range, but this is an example).

For this scenario, it would be feasible to convert the individual computers to diskless using iSCSI. You can take each of the hard drives and convert them to iSCSI using cloning software. Then, you can take the physical drives and add them to your WHS Server to expand its storage. Suddenly (in this example) your WHS has maybe another 500GB of usable storage space. Add to this the fact that each computer's boot drive is now stored in a file on a replicated server share, and that means that computer backup is no longer necessary. Also, since the physical drives are in the server, the likelihood of loss of data is almost nil. If one of your desktop's drives (which is now in the server) fails, WHS will take up the slack of being ready for a replacement drive without the need of recovery software.


Now imagine this: You buy a new computer. It comes with a 1TB hard drive (this is an average on most standard desktops). This is more than you need for an individual computer. What is the best use of this space? WHS. With iSCSI, you can take that 1TB hard drive and put it in the server. Any space you would normally use on the physical computer is now used on the network drives.

Let's say that with all programs installed you would normally use 60GB of space on your local computers' hard drives. Well now that 1TB you have is being wasted if it remains in the desktop PC. You could use that extra 940 gigabytes in your server to much better advantage. With iSCSI you have your cake and eat it too. You have extra storage in the server you wouldn't have had, and you have nearly the same functionality on the local computers.

The one caveat to iSCSI boot is that you do not have hibernation support when booting Windows from an iSCSI source. This is one thing that you may find lacking without a local disk. Other than that, iSCSI benefits may outweigh the cons.

So is this considerred Desktop virtualization or is that done a different way with specific software?

It sounds like the network would benefit from an upgrade to 10G fiber for a big increase in speed. I have a bunch of scrap fiber from work ready to install at home once the fiber equipment comes down in cost for 10G speeds(I just need to buy the fiber connectors and terminate them). But I also need to wait until they replace the siding on our building since we can't add any more cabling outside until then.

39TB unRAID1--53TB unRAID2--36TB unRAID3
LED DLP
XBL/PSN: WormholeXtreme

aaronwt is offline  
post #51 of 51 Old 08-13-2011, 01:01 PM
Newbie
 
this-is-me's Avatar
 
Join Date: Aug 2011
Posts: 3
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 10
Quote:
Originally Posted by aaronwt View Post

So is this considerred Desktop virtualization or is that done a different way with specific software?

No, Actually that is a "whole 'nother kettle of fish". Desktop Virtualization is where you buy a really powerful (and usually really expensive) server, then use thin clients (or inexpensive PCs) to connect to virtual machines running on the server. You would need a Hypervisor or VDI software on your server to do so. This offloads all processing power to the server. No real work is done on the CPU of the local machine in this case.

This is more "Drive Virtualization" to coin a phrase. The system still performs all functions relating to CPU or GPU locally (thereby allowing for local Blu Ray playback). The only thing different is that the Desktop PC has no hard drive. (Sounds strange to say that)
this-is-me is offline  
Reply Networking, Media Servers & Content Streaming

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off