How many movies can be streamed from one HDD simultaneously? - AVS Forum
Forum Jump: 
Reply
 
Thread Tools
post #1 of 44 Old 05-15-2013, 01:21 AM - Thread Starter
Member
 
PobjoySpecial's Avatar
 
Join Date: Jan 2013
Posts: 172
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 16
...
PobjoySpecial is offline  
Sponsored Links
Advertisement
 
post #2 of 44 Old 05-15-2013, 02:45 AM
AVS Special Member
 
Nevcairiel's Avatar
 
Join Date: Mar 2010
Location: Germany
Posts: 1,003
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 106
The problem with streaming high-bitrate content like Blu-rays from a HDD is not the overall sequential speed, but the random access speed.
If you read from different files at the same time, it needs to seek around on the disc, move the read heads constantly - its just going to be slow.

Personally, i can't say i tested this extensively on actual BD movies, but from experience in similar situations, i would not expect more than 2 or 3 BD movies to work (if even that).

I've had issues myself when trying to watch a single movie while copying data onto the same HDD at full speed, was already enough to trip up playback.
I had to adjust my ripping workflow to directly rip onto the media storage network drive, instead of my local drive first, because writing at BD read speeds is OK, writing at full network copy speed is not. smile.gif
Nevcairiel is offline  
post #3 of 44 Old 05-15-2013, 06:48 AM - Thread Starter
Member
 
PobjoySpecial's Avatar
 
Join Date: Jan 2013
Posts: 172
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 16
...
PobjoySpecial is offline  
post #4 of 44 Old 05-15-2013, 06:55 AM
AVS Special Member
 
bryansj's Avatar
 
Join Date: Feb 2004
Location: Atlanta, GA
Posts: 6,229
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 21 Post(s)
Liked: 161
Because the writing software has a RAM buffer that allows ripping to continue while waiting for head write time. You would need some sort of "reading software" that knows to put the >3 BR files into a buffer and read from that. Or move to SSD media storage drives...

bryansj is online now  
post #5 of 44 Old 05-15-2013, 06:56 AM
AVS Special Member
 
Nevcairiel's Avatar
 
Join Date: Mar 2010
Location: Germany
Posts: 1,003
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 6 Post(s)
Liked: 106
Ripping isn't all that sensitive. In playback, if data comes a bit late, your video already lags a bit, and you notice it.
If the writing is stalled and is delayed a bit, the wrting speed goes down, and ripping may take slightly longer, but it doesn't disrupt the process.

Just some guesses, may also be because of write buffers that Windows uses, which allow some kind of "burst" writing, quickly writing larger blocks, instead of always small blocks.
Nevcairiel is offline  
post #6 of 44 Old 05-15-2013, 06:59 AM - Thread Starter
Member
 
PobjoySpecial's Avatar
 
Join Date: Jan 2013
Posts: 172
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 16
...
PobjoySpecial is offline  
post #7 of 44 Old 05-15-2013, 10:29 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Nev is overly pessimistic, I think.

Any decent player should read in fairly large chunks, hopefully at least 128KiB and preferably 512+KiB.

Most recent HDDs are capable of 40 MB/s throughput with 512KiB random read tests (look at Crystal Disk Mark screenshots if you want to see measurements).

So as long as the player is reading in large enough chunks, you should be able to stream a lot of blu-rays from a single HDD.

Here is a link to CDM results for a couple recent 4TB HDDs:

http://www.legitreviews.com/article/2182/5/

And here for a 3TB WD Red:

http://www.legitreviews.com/article/2092/4/
jim2100 is offline  
post #8 of 44 Old 05-15-2013, 10:56 AM - Thread Starter
Member
 
PobjoySpecial's Avatar
 
Join Date: Jan 2013
Posts: 172
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 16
...
PobjoySpecial is offline  
post #9 of 44 Old 05-15-2013, 10:59 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Probably the reason it is not discussed more is because it is rarely a problem. You can stream a lot of 2-4MB/s videos to or from a single HDD. There is certainly plenty of evidence that a DVR can record multiple streams while simultaneously playing back one stream -- they have been doing it for years now. Of course, that is not exactly what you are asking here, since you are interested in multiple simultaneous reads. But simultaneous writes plus one simultaneous read is a similar workload, I think.

The only time it becomes a problem is when you are simultaneously doing a very high throughput read or write to the drive (such as Nev's example of streaming a movie while simultaneously trying to do a full-speed, probably 100MB/s, write to the HDD)
jim2100 is offline  
post #10 of 44 Old 05-15-2013, 11:42 AM - Thread Starter
Member
 
PobjoySpecial's Avatar
 
Join Date: Jan 2013
Posts: 172
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 16
...
PobjoySpecial is offline  
post #11 of 44 Old 05-16-2013, 07:04 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 20,983
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 51 Post(s)
Liked: 593
I've streamed 4 blurays (30GB MKVs) in my home from my server to HTPCs, desktops and laptops and there wasn't any issue as long as I plugged laptop into LAN cable.

At first only the laptop stuttered but I realized that was wireless limits. I can stream lower bit rate stuff wireless but full blown 1080p I can't.

My desktop (Asus deluxe mobo with Intel LAN) to my server (Intel NIC card) with cat6 cable (short run) and a cheap Asus gigabit switch consistently give me 110MB/sec transfers to or from.

I'm nearly certain in my personal set up my network speed is the limiting factor or bottleneck. My HTPC , desktop and server should perform beyond the network limit in other areas.

My server is almost exclusively 3TB Seagate drives that easily outperform my network speed. I've tried to maximize my performance with an Intel NIC but honestly I got pretty great performance with the crappy LAN built into my AsRock z77 motherboards (both HTPC and server). I'm not sure the intel NIC helps much.

I don't own enough machines to stream more than 4 at a time but I can say with absolute certainty that I can do 4 at once without issue.

If your running a NAS box or Unraid set up or your running slower 2TB WD green drives then its likely you'll have a bottle neck much lower than my Flexraid server. I could see possibly four streams being an issue. I can't imagine two or three at once wouldn't work though.

-


"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #12 of 44 Old 05-16-2013, 07:37 AM - Thread Starter
Member
 
PobjoySpecial's Avatar
 
Join Date: Jan 2013
Posts: 172
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 16
...
PobjoySpecial is offline  
post #13 of 44 Old 05-16-2013, 08:30 AM
Member
 
groove93's Avatar
 
Join Date: Sep 2007
Location: Statesboro, GA
Posts: 193
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by Mfusick View Post

I've streamed 4 blurays (30GB MKVs) in my home from my server to HTPCs, desktops and laptops and there wasn't any issue as long as I plugged laptop into LAN cable.

At first only the laptop stuttered but I realized that was wireless limits. I can stream lower bit rate stuff wireless but full blown 1080p I can't.

My desktop (Asus deluxe mobo with Intel LAN) to my server (Intel NIC card) with cat6 cable (short run) and a cheap Asus gigabit switch consistently give me 110MB/sec transfers to or from.

I'm nearly certain in my personal set up my network speed is the limiting factor or bottleneck. My HTPC , desktop and server should perform beyond the network limit in other areas.

My server is almost exclusively 3TB Seagate drives that easily outperform my network speed. I've tried to maximize my performance with an Intel NIC but honestly I got pretty great performance with the crappy LAN built into my AsRock z77 motherboards (both HTPC and server). I'm not sure the intel NIC helps much.

I don't own enough machines to stream more than 4 at a time but I can say with absolute certainty that I can do 4 at once without issue.

If your running a NAS box or Unraid set up or your running slower 2TB WD green drives then its likely you'll have a bottle neck much lower than my Flexraid server. I could see possibly four streams being an issue. I can't imagine two or three at once wouldn't work though.

Are you running a Dual Band Wireless Network in your system?

This has been an ongoing issue for me as well. Using XBMC I transitioned to NFS from SMB and there was a boost in performance for 1080p playback via wifi to my Raspberry Pi and my Laptop. The Laptop is running on a 5ghz Band and plays movies without a hitch. My Raspberry Pi for now is working on the 2.4ghz and it will play lower bit movies, even the Warner Bros. Discs that have Lossless audio but encoded at 1080p using M2TS as the container. Only movies with HD Audio are a problem streaming to my Raspberry Pi for now. I plan to get an Airport Express with Dual Bands to fix this issue. I have an older APE and that alone improved 1080p playback to the RP with minimal pauses during playback, compared to a wireless N USB Adapter physically connected to the RP.
groove93 is offline  
post #14 of 44 Old 05-16-2013, 11:34 AM
AVS Addicted Member
 
Mfusick's Avatar
 
Join Date: Aug 2002
Location: Western MA
Posts: 20,983
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 51 Post(s)
Liked: 593
No I have a crappy old linksys wireless n router.

It's junk.

-


"Too much is almost enough. Anything in life worth doing is worth overdoing. Moderation is for cowards."
Mfusick is online now  
post #15 of 44 Old 05-16-2013, 11:58 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Just for fun, I started playing a bunch of my MKV blu-ray rips with VLC. They were all playing from the same HDD (a 4TB Hitachi 5400rpm) over a Gbit ethernet connection from my linux server (samba) to my Windows computer. I went up to nine and did not notice any obvious playback issues, but it is somewhat difficult to tell. The sound was a horrendous din, but I did spend a few seconds watching the video of each and I did not see any obvious jumps or freezes. So probably I could have kept going, but nine was enough fun for me.

http://i.imgur.com/QapiGZf.jpg
jim2100 is offline  
post #16 of 44 Old 05-16-2013, 06:53 PM
AVS Special Member
 
olyteddy's Avatar
 
Join Date: Oct 2005
Posts: 3,160
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 26 Post(s)
Liked: 185
I used to torture test my DVD drives by loading a DVD and opening multiple instances of VLC each playing a different chapter. I had an old Samsung that could stream 4 chapters at once. Boy was the drive noisy though! biggrin.gif
olyteddy is online now  
post #17 of 44 Old 05-17-2013, 06:47 AM
Senior Member
 
brianjb's Avatar
 
Join Date: Oct 2007
Posts: 344
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
I see no reason why you couldn't get at least 4 streams going if not more. If you think you will need a lot though, your best bet is to use RAID to increase your IO.
brianjb is offline  
post #18 of 44 Old 05-17-2013, 07:18 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
You can use a cheap SSD drive for caching to increase the number of streams.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #19 of 44 Old 05-17-2013, 07:21 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Quote:
Originally Posted by itznfb View Post

You can use a cheap SSD drive for caching to increase the number of streams.

No, you cannot. At least, not unless you play the same 6 or 7 movies over and over again, and no others.
jim2100 is offline  
post #20 of 44 Old 05-17-2013, 07:31 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by jim2100 View Post

No, you cannot. At least, not unless you play the same 6 or 7 movies over and over again, and no others.

Why can't you? Stream a video file is pretty much the most predictive IO possible. Plus there are plenty of software options to further optimize that caching. None free that I've found though. SRT is perfectly capable though.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #21 of 44 Old 05-17-2013, 07:33 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Quote:
Originally Posted by itznfb View Post

Why can't you?

Because caches are not magic. A cache still has to be filled. And if you can fill the cache in realtime, then you do not need the cache, do you?
jim2100 is offline  
post #22 of 44 Old 05-17-2013, 08:53 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by jim2100 View Post

Because caches are not magic. A cache still has to be filled. And if you can fill the cache in realtime, then you do not need the cache, do you?

So all those people seeing performance gains from using cache must be dreaming huh? Adjust the buffer length on the media server and configure the drive for cache and yes it works. You can build a longer wait queue on the HDD if you are filling the SSD that is serving the real-time data. I have a 64GB mSATA drive in my server acting as cache with 2 drives in RAID1. Prior to adding the mSATA card when I got up to 7~8 people streaming it would start to get pretty choppy. With the mSATA card I've seen 12 people streaming without issue. Not sure if it's gone higher than that as I don't monitor the connections unless someone reports issues.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #23 of 44 Old 05-17-2013, 10:56 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
You might want to get a clue about how a cache works before making more absurd claims.

Caches are great for speeding access to information that is accessed repeatedly and frequently. However, for one-time or very infrequent access to information, a cache cannot help.
jim2100 is offline  
post #24 of 44 Old 05-17-2013, 11:08 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by jim2100 View Post

You might want to get a clue about how a cache works before making more absurd claims.

Caches are great for speeding access to information that is accessed repeatedly and frequently. However, for one-time or very infrequent access to information, a cache cannot help.

I take you're completely unaware of buffering and prefetching? Apparently you are. As I said video streams are among the most predictable reads possible. The entire video can be prefetched and cached if you wanted.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #25 of 44 Old 05-17-2013, 11:12 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
This is going beyond absurd. How can you not realize that "buffering and pre-fetching" involves reading from the slow memory (in this case, the HDD) in order to load it into the cache? And if it is being read from the HDD, then that means the HDD is capable of reading fast enough, and the cache is of no benefit.
jim2100 is offline  
post #26 of 44 Old 05-17-2013, 11:24 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Quote:
Originally Posted by jim2100 View Post

This is going beyond absurd. How can you not realize that "buffering and pre-fetching" involves reading from the slow memory (in this case, the HDD) in order to load it into the cache? And if it is being read from the HDD, then that means the HDD is capable of reading fast enough, and the cache is of no benefit.

That statement isn't even close to being accurate. You're talking about a single stream or multiple streams that would be the exact same bitrate and started at the exact same time. Sure if you have 3 streams that all start at the same and are the exact same bitrate and that is the only activity that is going to occur then prefetching to a cache drive is useless but that is so far from a typical usage scenario. I mean you're completely ignoring work prioritization and scheduling that is going on in the background and you're talking about the server as if it's a completely stupid device that needs to be told exactly what to do and when. I mean there are white papers from the 1990's about this that are still being used today. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.119.7464 This was latest and greatest technology then but now you throw in a SSD and turn on SRT and increase the buffers and everything else is handled by the software.

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #27 of 44 Old 05-17-2013, 11:48 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Quote:
Originally Posted by itznfb View Post

Sure if you have 3 streams that all start at the same and are the exact same bitrate and that is the only activity that is going to occur then prefetching to a cache drive is useless but that is so far from a typical usage scenario.

In this particular case, where we are discussing how many video streams can be played simultaneously from a single HDD, an SSD cache is not helpful. It is not necessary that all the streams start at the same time nor are "the exact same bitrate". If you start the streams a few seconds apart, and they have bitrates varying over a large range, an SSD cache is still not helpful.

If you want to change the subject from what this thread is about to something else, like perhaps a media server that is serving lots of video streams, then it is conceivable that an SSD cache could be of some benefit, but only if the streaming program is extremely poorly written. Because any reasonable streamer is going to do its own buffering, and since buffering is ultimately limited by the HDD performance, the only way that the SSD cache could be of benefit is if it is better at figuring out how much and what to read-ahead than is the streaming program, and that is unlikely.

A disk cache is most helpful for randomly-accessed, frequently re-read data, or for programs that don't do their own buffering. Which is almost the opposite of what is being discussed in this thread. Also, don't forget that most operating systems have their own IO scheduler that re-orders IO requests and does read-ahead in order to optimize HDD access.
Nethawk likes this.
jim2100 is offline  
post #28 of 44 Old 05-17-2013, 10:03 PM
Senior Member
 
brianjb's Avatar
 
Join Date: Oct 2007
Posts: 344
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
I agree with the above. An SSD cache in this specific scenario is of no benefit.
brianjb is offline  
post #29 of 44 Old 05-18-2013, 06:38 AM
Advanced Member
 
itznfb's Avatar
 
Join Date: Jan 2009
Location: Pittsburgh
Posts: 840
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 58
Yet there are tons of people who benefit in the exact scenario.... including myself rolleyes.gif

Running Windows Home Server 2011 Evil Abandoned Edition
itznfb is offline  
post #30 of 44 Old 05-18-2013, 07:34 AM
Advanced Member
 
jim2100's Avatar
 
Join Date: Aug 2011
Posts: 762
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 33
Quote:
Originally Posted by itznfb View Post

Yet there are tons of people who benefit in the exact scenario.... including myself rolleyes.gif

A dubious claim. Feel free to present some data to back it up.
jim2100 is offline  
Reply Home Theater Computers

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off