We're gonna need a faster array... - AVS Forum
Forum Jump: 
 5Likes
  • 1 Post By balky
  • 1 Post By cfineman
  • 1 Post By Aryn Ravenlocke
  • 1 Post By JimP
  • 1 Post By ajhieb
 
Thread Tools
post #1 of 11 Old 08-13-2014, 10:22 PM - Thread Starter
AVS Special Member
 
ajhieb's Avatar
 
Join Date: Jul 2009
Posts: 1,392
Mentioned: 7 Post(s)
Tagged: 0 Thread(s)
Quoted: 305 Post(s)
Liked: 386
Cool We're gonna need a faster array...

Not quite HTPC related, but I thought I'd post it for the storage junkies here.

World's Fastest Camera Takes 4.4 TRILLION Frames Per Second

Quote:
Researchers in Japan have developed a motion picture camera that can take 4.4 trillion frames per second -- making it the fastest camera in the world. They call their technique “sequentially timed all-optical mapping photography” (or STAMP), and the resolution is an impressive 450 x 450 pixels. The work was published in Nature Photonics this week.
Think about that for a second... (while that camera takes 4.4 [Carl Sagan voice] trillion images)

I can't find any info on the color depth, but if we go with 8bpp (not particularly high) that comes out to nearly 800 Petabytes (or Pebibyte I guess? I used 2^10 not 10^3, but I don't know what the silly new prefix they invented is for the base 2 version of Peta) for one second of filming. (or 3.2Exabytes if you go with true-color 32bpp)

For starters, holy crap that's a lot of storage. But the real thing that amazes me is that they're somehow storing that at 800PB/s I doubt it's going straight to traditional hard drives but if it was it would take more than a billion hard drives to be able to do that. And that is if you just take 1 second of footage, and assume that you're going to write to every one of the drives at a full 6Gb/s (which you probably won't because most drives don't have that much cache... it would take around 5 billion or more to sustain that rate.)

Kinda puts your multi-terabyte storage array in perspective doesn't it?

RAID protection is only for failed drives. That's it. It's no replacement for a proper backup.
ajhieb is online now  
Sponsored Links
Advertisement
 
post #2 of 11 Old 08-13-2014, 10:52 PM
Advanced Member
 
balky's Avatar
 
Join Date: Oct 2008
Posts: 843
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 48 Post(s)
Liked: 50
Well, yes, but I'm guessing the image data is probably not going to be written directly to a storage...

There will most likely be (an equally impressive) compression technique that will reduce the data size to something more realistic...

But anyway, you're right, this is really something to think about... I mean like how do they manage to achieve all of this stuff...

And I giggled a bit at the thought of how the "storage junkies" might start to prepare in advance for when this type of technology will go mainstream... if ever...
ajhieb likes this.
balky is offline  
post #3 of 11 Old 08-13-2014, 11:01 PM
Member
 
cfineman's Avatar
 
Join Date: Jan 2008
Posts: 90
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 7 Post(s)
Liked: 16
Yeah I was gonna say... don't presume it's getting written to storage verbatim. You could use something like that to record vectors of, say particles... that takes a little less space.

There was an article in National Geographic in the past few years where they showed cameras that were able to take pictures of lightning bolts in progress. Take THAT Wachowski Brothers!
ajhieb likes this.
cfineman is online now  
post #4 of 11 Old 08-13-2014, 11:27 PM - Thread Starter
AVS Special Member
 
ajhieb's Avatar
 
Join Date: Jul 2009
Posts: 1,392
Mentioned: 7 Post(s)
Tagged: 0 Thread(s)
Quoted: 305 Post(s)
Liked: 386
If they're using it to analyze particle vectors I would imagine that it would need substantially less in the way of storage, but if it is actually images they are after, the numbers are going to be mind boggling no matter what it's final destination is.

If they are compressing the images, I highly doubt they would use a lossy compression scheme. Seems like a waste to go to all that trouble, only to lose some of the detail. And regardless of whether you're compressing it image by image, or as a video stream using intraframe compression, you can only compress it so much. Even if they shrink it by a factor of 1000, (totally not realistic) they'd still need a million hard drives in the scenario above.

Not to mention, what kind of processing you'd need if you were going to compress that kind of data. You're i7 isn't going to cut it.

And while I realize they probably aren't just shoving this stuff onto a bunch of spinning rust, no matter how you slice it you're going to need a lot of ram to be able to keep up with that thing, even for really short durations.

I suspect that it is only being used for analyzing things with a duration of thousandths of a second or even less than that. Even if using it for just a single second, I really don't think we have the technology right now to handle that sort of data rate.

RAID protection is only for failed drives. That's it. It's no replacement for a proper backup.
ajhieb is online now  
post #5 of 11 Old 08-13-2014, 11:36 PM
Senior Member
 
Aryn Ravenlocke's Avatar
 
Join Date: Aug 2006
Location: Tempe, AZ, USA
Posts: 294
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 121 Post(s)
Liked: 59
And here I thought I was mostly future-proofing myself with plans for an eventual 180 TB home server. I don't think swapping out all those 4 TB drives for 6 TB ones is going to cut it.
ajhieb likes this.
Aryn Ravenlocke is online now  
post #6 of 11 Old 08-13-2014, 11:38 PM
AVS Addicted Member
 
Join Date: Sep 2002
Location: Wetumpka, AL
Posts: 15,358
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 112 Post(s)
Liked: 191
Does the article actually say that they store the data or only that they've got a camera that can take that many pictures?
ajhieb likes this.

Samsung 64F8500, Panasonic 65VT50, Oppo 95, Tivo Roamio for OTA, Dish VIP722, Marantz AV8801 preamp, Rotel Amps, Atlantic Tech 8200 speakers, Seaton Submersive HP, Calman 5, Chromapure, Accupel DVG-5000, i1Display3pro, i1pro2, eecolor colorbox.
JimP is offline  
post #7 of 11 Old 08-13-2014, 11:43 PM - Thread Starter
AVS Special Member
 
ajhieb's Avatar
 
Join Date: Jul 2009
Posts: 1,392
Mentioned: 7 Post(s)
Tagged: 0 Thread(s)
Quoted: 305 Post(s)
Liked: 386
Quote:
Originally Posted by JimP View Post
Does the article actually say that they store the data or only that they've got a camera that can take that many pictures?
It doesn't say they stored the data per se, but it does say they've successfully used the camera, which does imply that they've done something with all that data, be it stored it or analyzed it on the fly. Regardless, I suspect it was used for only tiny fractions of a second,. But regardless of the duration, the data rate isn't going to change if you're doing it for a millionth of a second, or an hour and a half. No matter how you slice it, that thing has to move a lot of data in a short amount of time.

RAID protection is only for failed drives. That's it. It's no replacement for a proper backup.
ajhieb is online now  
post #8 of 11 Old 08-13-2014, 11:44 PM - Thread Starter
AVS Special Member
 
ajhieb's Avatar
 
Join Date: Jul 2009
Posts: 1,392
Mentioned: 7 Post(s)
Tagged: 0 Thread(s)
Quoted: 305 Post(s)
Liked: 386
Quote:
Originally Posted by Aryn Ravenlocke View Post
And here I thought I was mostly future-proofing myself with plans for an eventual 180 TB home server. I don't think swapping out all those 4 TB drives for 6 TB ones is going to cut it.
No such thing as future proof

But I have a feeling your server (especially if you go the kapone route) will keep you happy for years to come.
balky likes this.

RAID protection is only for failed drives. That's it. It's no replacement for a proper backup.
ajhieb is online now  
post #9 of 11 Old 08-14-2014, 12:46 PM
Member
 
cfineman's Avatar
 
Join Date: Jan 2008
Posts: 90
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 7 Post(s)
Liked: 16
Quote:
Originally Posted by ajhieb View Post
It doesn't say they stored the data per se, but it does say they've successfully used the camera, which does imply that they've done something with all that data, be it stored it or analyzed it on the fly. Regardless, I suspect it was used for only tiny fractions of a second,. But regardless of the duration, the data rate isn't going to change if you're doing it for a millionth of a second, or an hour and a half. No matter how you slice it, that thing has to move a lot of data in a short amount of time.
Agreed.... pretty bad-assed. As you are probably aware, there's been a lot of work leveraging GPUs for heavy mathematical computation... saw one study that said for FFT a GPU was ~300 faster than and FPGA and ~3000 times faster than a CPU... (two Intel Xeon 5650 CPUs, the Virtex-5 FPGA and NVIDIA’s
GeForce GTX460 and 9800 GTX+ GPUs). Still the time to complete an FFT was still ~8us... still not fast enough (holy cow! fighting the urge to channel Dr Brown from back to the future ;-) )

Cool stuff... gotta love hard core engineering!
cfineman is online now  
post #10 of 11 Old 08-14-2014, 03:07 PM
Senior Member
 
Aryn Ravenlocke's Avatar
 
Join Date: Aug 2006
Location: Tempe, AZ, USA
Posts: 294
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 121 Post(s)
Liked: 59
Quote:
Originally Posted by ajhieb View Post
No such thing as future proof

But I have a feeling your server (especially if you go the kapone route) will keep you happy for years to come.
Yes, I am aware that future-proofing isn't really feasible in the world of technology. I do hope that my system (having indeed gone the kapone route) will serve me sufficiently for many years to come though, and I suspect it will. I may have to eventually re-think a processor in a few more years if 4K becomes an issue, but even then I probably have another good 5 years before I would have enough content to worry about it. So I'm happy with where I am at.

Still, that is some massive data transfer and storage.
Aryn Ravenlocke is online now  
post #11 of 11 Old 08-16-2014, 01:28 AM
Member
 
dew42's Avatar
 
Join Date: Jul 2002
Location: Saskatoon, Canada
Posts: 183
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 14
"But to construct an image, it requires repetitive measurements."


Sounds similar to what MIT did several years back. They are not actually capturing trillions of frames per second in one pass.


Laser pulse shooting through a bottle and visualized at a trillion frames per second


"We have built an imaging solution that allows us to visualize propagation of light at an effective rate of one trillion frames per second. Direct recording of light at such a frame rate with sufficient brightness is nearly impossible. We use an indirect 'stroboscopic' method that combines millions of repeated measurements by careful scanning in time and viewpoints."




"...the video is captured over several minutes by repeated and periodic sampling."
dew42 is online now  
Reply Home Theater Computers

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off