Why we need 1000fps@1000Hz this century -- Valve Software (Michael Abrash) comments - AVS Forum
Forum Jump: 
 
Thread Tools
post #1 of 6 Old 08-01-2013, 01:49 PM - Thread Starter
AVS Special Member
 
Mark Rejhon's Avatar
 
Join Date: Feb 1999
Location: North America
Posts: 8,124
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 102
Michael Abrash of Valve Software has a great article about the problems of trying to simultaneously solve motion blur, judder and strobing when using virtual reality headsets. (In Comments section, Michael also complimented the work Blur Busters is doing). He has several great explanations about these problems, and also touches upon the benefits of 1000Hz-refresh displays.

"Down the VR rabbit hole: Fixing judder"
http://blogs.valvesoftware.com/abrash/down-the-vr-rabbit-hole-fixing-judder

Essentially, the Cliff Notes version of his article:
- Low framerates -- problematic for both motion blur and/or judder
- High framerates -- better, but still have motion blur even at 120fps@120Hz
- Strobing -- solves motion blur but adds problems (flicker problem/stroboscopic problem)
- 1000fps@1000Hz -- Simultaneously fix motion blur, fix flicker, fix stroboscopic/stepping effects

Problem: Motion blur of eye-tracking motion blur on discrete-refresh displays:
.
Using Michael Abrash's diagrams, and corroborated by his vision researcher friends, I've arranged the diagrams in way that helps people understand why true 1000fps@1000Hz will eventually become necessary sometime later this century.
.

Simplified illustration for eye-tracking at 60fps @ 60Hz sample-and-hold (e.g. LCD)
.


Simplified illustration for eye-tracking at 120fps @ 120Hz sample-and-hold (e.g. LCD).
Less motion blur.
.


Simplified illustration for eye-tracking at 60fps @ 60Hz flicker driven (e.g. CRT).
Less motion blur too. But problem if you stare stationary (see next section).
.


Simplified illustration for eye-tracking in real life.
No motion blur bottleneck. No stroboscopic problem.


Problem: Persistence, the stroboscopic stepping effect while staring stationary while object moves past

This creates stroboscopic stepping effects (at low framerates, this is the common sensation of judder). The stepping effect can still be visible even at 120Hz and 240Hz. Visualize a fast moving object which creates a non-continuous trail, e.g. object moving 240 inches per second leaves a 1-inch dotted trail if you're just staring straight ahead. This is fixed by adding motion blur, but that is undesirable in many situations. For VR, you want perfect clarity without motion blur.
.

Continuous illumination (e.g. most LCD)
.


Half persistence (the frame is illuminated half the time)
.


Zero persistence (e.g. laser displays and short-persistence CRT's resembles this)


Problem: Persistence, the stroboscopic stepping effect while staring stationary while object moves past

.
Blur Busters UFO Motion Tests

I let Michael Abrash know about the UFO motion tests, of which Michael likes very much. This is very relevant into the "pick-your-poison" problems of finite-framerate displays. Before proceeding, make sure you understand how these animations work. View these links on a recent fast computer using Google Chrome or another browser supporting perfect framerate=Hz animation.

An excellent animation of eye-tracking-based motion blur:
www.testufo.com/#test=eyetracking (use a supported browser)

An excellent animation of how strobing reduces motion blur:
www.testufo.com/#test=blackframes (use a supported browser)
This also brings some strobing disadvantages

The problem when using virtual reality headsets, you've got head turning that creates very fast horizontal motion. This creates fast panning which can be quite motion blurred. Fast head-turning can create lots of motion blur.

First, look at the stationary image of the Eiffel tower.
http://www.testufo.com/#test=photo&photo=photo-eiffel.jpg&pps=0
This is very clear on almost all displays.
- You can count the number of cars under the Eiffel Tower
- You can count the lattices in the Eiffel Tower

Let's simulate even just a slow head turning speed that's approximately 30 degrees per second (VR headsets):
www.testufo.com/#test=photo&pps=1920&photo=photo-eiffel.jpg (use a supported browser)
Maximize your web browser window on your display (computer or HDTV). Stare fairly close, at a view distance approximately equal to screen width. Now you're viewing motion that's moving approximately 30 degrees per second. That's only a slow head turning speed.
Look at how motion blurry it is on your LCD.
- LCD will show a lot of motion blur
- CRT will show stroboscopic stepping effects
- Even at 120Hz.
- Even at 240Hz.
- Yes, even at perfectly-done 480Hz.

Animation Self-Test Challenges

Challenge for motion blur.
www.testufo.com/#test=photo&pps=1920&photo=photo-eiffel.jpg (use a supported browser)
Track your eyes on the details of the moving photo above
- Try to count the number of cars under the Eiffel Tower
- Try to count the lattices in the Eiffel Tower
You can do typically easily do this on a CRT, but not on most LCD's (except LightBoost)
On a CRT with short-persistence phosphor, the fast moving image remains crystal-sharp.
Also crystal sharp on a LightBoost 120Hz monitor running ToastyX Strobelight programmed to 1.4ms strobes (Control+Alt+1).

Challenge for stroboscopic effect:
www.testufo.com/#test=photo&pps=1920&photo=photo-eiffel.jpg (use a supported browser)
Stare stationary ahead.
- Put your finger along the top edge of the moving photo, at the vertical level where the Eiffel Tower antenna passes underneath.
- Now stare at the finger.
- Notice the strobing effect of the Eiffel Tower antenna, as it passes underneath. You see multiple antennas near the finger.
- Even with 240fps frame interpolation, the strobing effect still remains if you're viewing at this fixed point during 30-degree-per-second motion (the speed of a slow head turn). So 240fps is not enough to fix the stroboscopic effect. There is no consumer display/monitor/HDTV that exists in the world, that fixes this stroboscopic effect problem. It's an artifact that exists on all finite-framerate displays, and diminishes as you go higher in framerate/Hz.

And this is Why 1000fps@1000Hz is Useful Progress

From the perspective of fast-action computer gamers, the persistence of having 1ms frame samples is benefical (the CRT effect) in completely eliminating perceptible motion blur, but even at high-refresh-rate CRT gaming when staring at the videogame crosshairs while doing strafes or turns in video games, you can see stroboscopic effects of finite-framerates (even on 21" high bandwidth professional CRT's being driven at 240Hz). Adding motion blur fixes this, but then you've got the motion blur problem. How do you fix motion blur AND the stroboscopic effect? Simple. 1000fps@1000Hz (or beyond).

So, this century, we will have a use for 1000fps@1000Hz -- in some applications such as FPS gaming, simulators, racing, military training, VR headsets, and everything else that requires all human eye effects to be done naturally (e.g. display doesn't enforce stroboscopic and motion blur limitations upon you). Otherwise, we will never reach Holodeck-league imagery.

Thanks,
Mark Rejhon

www.BlurBusters.com

BlurBusters Blog -- Eliminating Motion Blur by 90%+ on LCD for games and computers

Rooting for upcoming low-persistence rolling-scan OLEDs too!

Mark Rejhon is offline  
Sponsored Links
Advertisement
 
post #2 of 6 Old 08-02-2013, 11:25 AM - Thread Starter
AVS Special Member
 
Mark Rejhon's Avatar
 
Join Date: Feb 1999
Location: North America
Posts: 8,124
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 102
Crossposting my public communication with by Michael Abrash of Valve software who complimented on my work; it appears that he seems to understand and agrees with the work that Blur Busters is doing.
Quote:
Originally Posted by Mark Rejhon 
I am the owner of Blur Busters, and the author of the Blur Busters Motion Tests.
I want to compliment you on your excellent write-up! I created some HTML5 animations that sync to VSYNC in some browsers, view these below links in Google Chrome:

An excellent animation of eye-tracking-based motion blur:
http://www.testufo.com/#test=eyetracking

An excellent animation of how strobing reduces motion blur:
http://www.testufo.com/#test=blackframes

Strobing is a great solution to eliminating motion blur. An example is viewing http://www.testufo.com/#test=photo when LigthBoost is enabled. (ToastyX Strobelight now makes it easy to turn ON/OFF LightBoost via a hotkey). One big problem of strobing is you have to refresh the LCD in the dark, before strobing the backlight on clearly refreshed frames. This adds an average half a frame of input lag, which can hurt VR. However, one could also use a high refresh rate (e.g. 120Hz or 240Hz) to fix this type of lag. 120Hz strobing is far more comfortable on the eyes than 60Hz strobing. Shorter strobe flashes reduce motion blur, so a 90%:10% dark:bright duty cycle reduces motion blur by 90%, but can darken the image by 90%.
Quote:
Originally Posted by Michael Abrash @ Valve Software 
July 30, 2013 at 3:34 pm
Hi Mark – great to hear from you! We really appreciate your work, and in fact Aaron already forwarded around your animations.

We have done black-frame insertion at 120Hz, and it works pretty well, but it’s still hard to get the LCD to settle quickly enough.

–Michael
Quote:
Originally Posted by Mark Rejhon 
July 30, 2013 at 4:25 pm
Thanks for the compliment. This probably means you’re familiar with my LightBoost research so far. Have you tried ToastyX’s new Strobelight Utility, which programs a LightBoost 2D motion blur eliminating strobe backlight without needing 3D Vision drivers, and without registry hacks? It also can reprogram the strobe length, via keyboard shortcuts as well:

Control+Alt+Plus = turn on LightBoost strobe flashing
Control+Alt+Minus = turn off LightBoost strobe flashing
Control+Alt+0 = Lightboost 100%, using 2.4ms flash
Control+Alt+5 = Lightboost 50%, using 1.9ms flash
Control+Alt+1 = Lightboost 10%, using 1.4ms flash

It creates a sort of an equivalent of a programmable-persistence display (to a certain extent). Both Adam Simmons (of TFTCentral.co.uk) and I (of Blur Busters) have used an oscilloscope and found that LightBoost strobe lengths are programmable in approximately 0.1ms increments. Our measurements varies a bit, but both of our oscilloscope measurements match within +/-0.1ms and we’ve established that motion blur of a LightBoost strobe backlight is linearly proportional to its strobe length. It does not eliminate stroboscopic/wagonwheel effects, so 1000fps@1000Hz is still a useful Holy Grail.

The strobe length differences actually show up visually in motion tests too. We were impressed that we could tell the 1 millisecond difference, in a fast-moving motion test:
http://www.testufo.com/#test=photo&pps=1440
(while running ToastyX Strobelight with LightBoost enabled)
Running this animation while hitting Control+Alt+0 versus Control+Alt+1. The windows in the castle at the top obviously became clearer during 1.4ms strobes, than with 2.4ms strobes.

So this is further proof you are correct, that we need a 1000fps@1000Hz display someday this century (one can hope!) — because 1.4ms is equivalent to 1/700sec, and having 1/1000sec display samples is quite close; at 10%, LightBoost is doing roughly equivalent to a 5:1 black frame insertion via backlight means (1.4ms:6.9ms bright:dark during an 8.3ms refresh cycle).

LightBoost displays actually partially framebuffer the refresh and does an accelerated scanout (~1/200sec or ~1/240sec) to artificially create a longer vertical blanking interval for a longer pixel-settling time before strobing the backlight. Then it also uses a Y-axis-compensated response-time-acceleration algorithm, to compensate for the difference in the freshness of the pixels at the top edge versus bottom edge (since the LCD scanouts in the total darkness). Marc Repnow (of did some reverse engineering of the LightBoost behavior, and some TestUFO patterns (e.g. blur trail using blue/yellow) revealed evidence of “RTC zones” along the vertical axis. We even discovered that LightBoost strobe backlight is strobed very late into the VSYNC, and the backlight is still ON when the next refresh begins (turning off about 0.5ms into the next refresh), this is because LCD pixels of the next refresh haven’t yet noticeably begun transitioning. Strobe backlight experiments need a phasing control (to control the timing of strobes relative to VSYNC), as it apparently is beneficial to make the flash as late as possible, and last slightly into the next refresh when transitions are still invisible. (this creates more time for the previous refresh’ pixels to settle).

One needs a fast scanout, followed by a very long VSYNC, for a strobe backlight. And the design of a Y axis variable into the RTC math calculations, will benefit reduction of vertical-axis-asymmetric ghosting artifacts (e.g. increased ghosting along the top/bottom edge).
Quote:
Originally Posted by Michael Abrash @ Valve Software 
July 30, 2013 at 9:11 pm
Interesting stuff! One thought: You can strobe as fast as you want, but current LCDs can’t transition at anywhere near 1000 Hz, and that’s going to be a limiting factor with LCDs for a long time, even if a video channel capable of anywhere near that bandwidth becomes available.

–Michael
Quote:
Originally Posted by Mark Rejhon 
July 31, 2013 at 9:35 am
For current LCD’s, yes, this will limit your strobe Hz to be longer than the pixel’s ability to transition.

Currently, reducing the contrast of the LCD (e.g. reducing VG278H to about 50%-60%) also increases overdrive headroom so that ghosting and 3D crosstalk falls completely below human-perceptible levels (even along bright vertical edges). You can strobe quicker, but you start to get more and more inter-refresh crosstalk the more you “eat” into the next LCD refresh.

The active-matrix OLED’s won’t be fast enough for 1000fps@1000Hz, but passive-matrix OLED are definitely fast enough. Problem is passive-matrix isn’t practical for good brightness on a large-size high-def OLED. This may change (e.g. science lab stuff, such as parallel multiscanning an AMOLED to allow the bandwidth needed for 1000Hz)

We can all hope for Plan B. Blue-phase LCD’s: 10 to 100 microseconds! Enough for 1000fps@1000Hz eventually. This was published in the journal of Society for Information Display (google “blue-phase LCD”. Those can transition in microseconds and they’ve already demonstrated color-sequential LCD’s using this this tech. Color sequential; imagine that — just like DLP.

Interesting Blur Busters history note: Blur Busters Blog started all because of John Carmack's tweet reply to me.
RLBURNSIDE likes this.

Thanks,
Mark Rejhon

www.BlurBusters.com

BlurBusters Blog -- Eliminating Motion Blur by 90%+ on LCD for games and computers

Rooting for upcoming low-persistence rolling-scan OLEDs too!

Mark Rejhon is offline  
post #3 of 6 Old 08-03-2013, 06:46 AM
AVS Special Member
 
Joe Bloggs's Avatar
 
Join Date: Apr 2007
Location: UK
Posts: 2,581
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 151 Post(s)
Liked: 59
Isn't that just for 1080p?
Quote:
although higher frame rates would be required to hit the sweet spot at higher resolutions
What about the optimal fps for 4K and 8K?
Joe Bloggs is offline  
post #4 of 6 Old 08-03-2013, 11:12 AM
AVS Special Member
 
Chronoptimist's Avatar
 
Join Date: Sep 2009
Posts: 2,627
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 1 Post(s)
Liked: 253
I don't know if you've seen it yet, but there was some discussion of this and Valve's research in John Carmack's keynote at QuakeCon. (and your original project with the LED strips got a mention as well)
There was talk of using time warping and frame interpolation inside the game engine to help achieve these kinds of framerates.

If you haven't seen it, here's a demo from a few years back, using interpolation to go from 30fps to 60fps in the game engine: http://and.intercon.ru/releases/talks/rtfrucvg/
Chronoptimist is offline  
post #5 of 6 Old 08-03-2013, 08:18 PM - Thread Starter
AVS Special Member
 
Mark Rejhon's Avatar
 
Join Date: Feb 1999
Location: North America
Posts: 8,124
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 102
Quote:
Originally Posted by Joe Bloggs View Post

Isn't that just for 1080p?
What about the optimal fps for 4K and 8K?
There are a lot of variables, so there's no single answer.

The more resolution you have, the more pixels of motion blur you get at the same angular motion. The angular motion blur is the same, but since the angular motion covers more pixels (more density), you're not hitting the necessary sweet spot of less than 1 pixel of motion blur for fast panning to look exactly as sharp as stationary images to the human eye. The visual acuity of the human eye also plays a role, so beyond a certain point, it doesn't matter, but as long as individual pixels are big enough to be resolved, Mike's correct that you need higher Hz for higher resolution, to keep "pixels-of-motion-blur" the same for the same angular motion speed, and if you're trying to hit the sweet spot of less than 1 pixel of motion blur (imperceptible), there you go... (e.g. strafing and turns in games, as well as when turning your head using VR goggles as Michael Abrash was talking about) This is talked about by John Carmack at about 12mins into his YouTube.

For moderately fast motion of 960 pixels per second, CRT's (and LightBoost 10%) manages to hit the motion clarity sweet spot where http://www.testufo.com/#test=photo *AND* http://www.testufo.com/#test=photo&pps=0 have exactly the same sharpness either moving or stationary (no judder, no stutters, no streaking, no blur, no ghosting, no overdrive).

Even at 1000fps@1000Hz, you can mathematically still have human-perceptible motion blur during high-def computer graphics moving at 2000 pixels/second if your eyes are able to track that fast *AND* the pixels themselves are resolvable. Since 1ms equals 1 pixel of motion blur for every 1000 pixels per second, since your eyes have tracked 1/1000th that distance, causing that pixel to smear. If one pixel is big enough to be resolvable, then blur will be noticed in that pixel if the eye tracking accuracy is good enough and the motion is fast enough. At 2000 pixels/sec, there is 2 pixels of blur at 1000fps@1000Hz sample-and-hold, so even that might still not be the final frontier; it's still a finite-refresh display with diminishing returns, but not disappeared returns required for HoloDeck quality.

Some people may be familiar with the Blur Busters pursuit camera tests at PHOTOS: 60Hz vs 120Hz vs LightBoost, which actually surprisingly corresponds well with what the human eye sees when directly viewing the ghosting pattern at www.testufo.com/#test=ghosting

But head turning at only 30 degrees per second on a 1080p VR googles, the motion blur becomes immediately noticeable in fast pans (much like the motion blur at www.testufo.com/#test=photo which on modern LCD's the motion blur is caused by eye tracking, not by the display response speed.) because it envelopes your vision and your head turns forces panning (a motion blur torture test case).

Not everyone does eye-tracking during fast motion on screen. But the problems of discrete-framerate (even at 240, 480, 960) displays conflicting with the non-framerate nature of human eyes -- this problem becomes noticeable during head-turning in VR -- or when people use computer monitors at close-distances and want fast FPS gaming (strafing/turning) to be as perfectly sharp as stationary images.

There are scientific extreme cases where, humans would notice non-Holodeck-perfect limitations at beyond 1000fps@1000Hz, due to the stroboscopic effect / wagon-wheel effect, but generally speaking, 1000fps@1000Hz woiuld probably eliminate the vast majority of problems for Holodeck-like simulations. You could intentionally add 1 millisecond of motion blur (which is tiny) to fix any remaining stroboscopic/wagonwheel effect problem of discrete-refresh displays, and the motion blur increment would be small enough to not be objectionable except during the fastest head turning.

Mathematically, 1ms equals 1 pixel of eye tracking motion blur with display motion of 1000 pixels/second (1 pixel per millisecond)
(Assuming pure strobes -- on then off, no decay trails, no aftereffects, no sample-and-hold beyond this time)

-- How big are the pixels? 2K vs 4K vs 8K
-- How sharp is the source material? Unfiltered computer graphics is easier to see blur in, than during video.
-- How fast is the panning? Faster panning creates more eye-tracking motion blur on displays.
-- How fast can you track your eyes accurately? That defines your upper limit when pans are too fast for eyes to track. (That's the moment of sweet spot)
-- How accurate is your eye tracking? (saccade factor) Different people are much more accurate at eye tracking, the shiftiness/jerkiness of the eyeball motion.
-- How much angular vision is covered? It's easier to notice motion blur in faster pans in wider displays that span your vision, because you have more time to track eyes accurately.
-- Etc. Other variables exist.

All the study of Vision Research stuff (e.g. people who use www.vpixx.com products and similar)
Joe Bloggs likes this.

Thanks,
Mark Rejhon

www.BlurBusters.com

BlurBusters Blog -- Eliminating Motion Blur by 90%+ on LCD for games and computers

Rooting for upcoming low-persistence rolling-scan OLEDs too!

Mark Rejhon is offline  
post #6 of 6 Old 01-14-2014, 11:53 AM - Thread Starter
AVS Special Member
 
Mark Rejhon's Avatar
 
Join Date: Feb 1999
Location: North America
Posts: 8,124
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 2 Post(s)
Liked: 102
More 1000fps talk, in interview with Oculus:

http://venturebeat.com/2014/01/14/oculus-vrs-brendan-iribe-on-the-virtual-reality-prototype-interview/
Quote:
You could avoid low persistence if you could run the screen at a few thousand hertz and only have, say, one millisecond of time between frames. But that’s not practical to tell game developers, “Hey, if you want to make VR games, you have to run at 1,000 FPS.” We want to say, “You can make great virtual reality, and you only need to run at 40, 50, 60 on your game engine.” The rendering engine will need to run a little bit faster, in sync with the refresh rate of the screen. But it’s very practical. People shouldn’t have too hard a time with where they are today.

Thanks,
Mark Rejhon

www.BlurBusters.com

BlurBusters Blog -- Eliminating Motion Blur by 90%+ on LCD for games and computers

Rooting for upcoming low-persistence rolling-scan OLEDs too!

Mark Rejhon is offline  
Reply OLED Technology and Flat Panels General

User Tag List

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off