Audio Processing in Vista Explained

[amirm]

Some of you know about me and our activities in audio/video compression and HD optical formats such as HD DVD. But probably don’t know about other types of signal processing that my team does at Microsoft. To remedy that , here is high level description of audio processing which is included in Windows Vista.

Some of you know about expensive room correction hardware which in some cases, costs as much as $10,000. Well, in Vista, we have a perceptually tuned version of room correction. Because we don't expect everyone to own an instrumentation microphone, we have designed it so that you can use a cheap cardioid or omnidirectional microphone costing a few dollars to do an excellent job. IF you have a more expensive microphone, you can calibrate your room very accurately. All of this is implemented in in-box Vista software, using less than 5% on 3GHz CPU. Our room correction will equalize frequency response, time delay, and gain between all of your main channels, as well as build a first-reflection-cancellation filter if one or more of your speakers are near a reflective surface. If you do have a high-quality microphone, the room correction system will also flatten the overall frequency response and adjust the subwoofer delay accordingly. All of our adjustments are done as to capture the most obvious problems from the point of view of human hearing, allowing us to do a very effective job with a few machine cycles.

Beyond advanced processing like room correction, we have also added fully configurable Bass Management, Loudness Equalization, Headphone Virtualization for private headphone listening, and channel control tools to fill speakers (for example 5.1) from stereo material, as well as build phantom channels when, for example, your listening setup has no center speaker. In addition, both Headphone Virtualization as well as bass boost are available for laptops, for headphone and laptop speaker use respectively.

For Bass Management, we have provided the tools to set up your system with all large (bass capable), Left and Right large, or no large speakers, with or without a subwoofer. For instance, if you have small speakers and a subwoofer, the bass from all channels will be routed to the subwoofer, and if you have some large speakers and no subwoofer, the subwoofer signal will be routed to the large speakers. We also handle the case where you have some large, some small speakers, and will route the bass accordingly. You can set the crossover point of your system to whatever the loudspeakers require.

In order to explain what Loudness Equalization does, we must first explain some terminology. In the Psychoacoustic Discipline, the term "Loudness" refers to the listener's evaluation of how loud a signal is, and the terms "intensity", "sound pressure" and the like refer to the measured, external to the listener, mechanically determined level. Loudness and intensity are related, but not in a direct fashion, because the response of the outer, middle, and inner ear must be modeled in order to relate the two. Our Loudness Equalization does exactly that, and then equalizes not the intensity, but rather the loudness, of presented signals in order to avoid blasting the listener when switching, for instance, from an analog to a digital TV channel, or from an older, dynamic recording to a modern, highly compressed recording. While this does reduce the dynamic range of the sound, this is often desirable, both in a loud environment where the quiet parts of the music are hard to hear, and in a quiet environment where you want to avoid blasting others with the loud parts of the sound.

Many people (most, if some of my observations on airplanes these days are typical) watch movies on laptops on airplane flights, in airports, at home, work, and so on. In order to provide the full movie experience, we have added Headphone Virtualization which uses Head Related Transfer Functions (which are basically the frequency responses of your head for sound arriving from different directions) to simulate sound arriving not only from the left and right headphone, but rather external to the head, and from front, center, side, and back. In addition we provide artificial reverberation, so that you can chose your desired listening environment, from small to large room. Of course, we also provide this processing on desktop machines so that you can enjoy 5.1, 7.1, and HDTV audio in your headphones on your desktop as well. I suppose I should mention that we also provide a mid-bass boost option for laptop audio, for when you are using those small laptop speakers.

We have added the ability to control the speaker configuration in a very flexible fashion. In addition to the standard configurations of 2.0, 5.1 (with either surround or rear speakers), and 7.1 (with both surround and rear speakers), we offer the ability to separately indicate the presence or absence of the subwoofer (.1) channel, and for the user to indicate that a specific set of speaker/speakers (say side, or rear, or center) is not actually connected. When we detect that a speaker(s) is missing, we will virtualize that channel(s) so that the information in the missing speakers is still presented to the listener in an engaging fashion. Likewise, when we detect that the source material has fewer channels than the listening setup, we provide the ability to "fill speakers" with signal that creates an experience that is more enveloping, and that has a stronger central image and wider listening area.

Finally, for those listeners who do not have multichannel output from their computer, or multichannel inputs into their A/V Receivers, we provide Virtual Surround, a matrixing technology also called LtRt, to provide input to A/V Receiver features that convert 2 Channel sound into multichannel sound inside the A/V Receiver.

All these audio system effects (also referred to as SysFx DSPs) are available on any HD-Audio and USB Audio-equipped computer that uses in-box class drivers. Third party audio drivers that do not use the in-box class drivers will either have similar effects of their own or will re-use the inbox SysFx audio DSPs and expose them through the Control Panel. In short, we provide the functionality offered by a high-end A/V Receiver in the basic Vista in-box software (installed automatically for USB and HD Audio), with performance and functionality that is comparable to the best A/V equipment on the market. If you want to read more on the new Vista™ Home Theatre functionality, please go to page 167, section 4.07 of the Windows Vista Product Guide on http://www.microsoft.com/downloads/d...DisplayLang=en for more information.

Anyway, please let me know what you think of these of features. This is the first time we have put all of this signal processing in an operating system and would be good to know if this is a direction we should continue.

[Ben_Tech]

Thank you very much, Amir, for taking the time to share that with us.

This is very exciting for us high-end HTPC preamp processor veterans! (To say the least.)

[diogen]

Amir, do I understand it right that ASIO/Kernel Streaming won't be needed on Vista to play back DTS wav files?
Will all this functionality be available on today's mobos? Or only on those with HD audio support?

Diogen.

[RobertR1]

Amir,

I'm not techincal with the audio stuff and just getting into it so I'm hoping you can bear with me as I try to learn. One of the reason I stayed away from advanced audio was due to all the techincal knowledge you need to acquire to get your system performing at an optimal level.

Would I able to plug into a receiver into the PC, do the calibration via the PC, then disconnect and the updated information is retained by the receiver? Will current receivers work or will I need a upcoming receiver with either a USB or RJ-45 connector?

If you can provide an "easy" way of people to put together a home audio system, do all the wiring and use Vista to get it 95% optimized, the masses will be quite happy.

Will these features make it into RC-1 of Vista? because I'm just putting my system in tomorrow. The interim Panny XR57 does not have any auto calibration and I can wait a few weeks for RC-1 to become available for calibration

Thanks,
Robert.

[amirm]

Originally Posted by diogen

Amir, do I understand it right that ASIO/Kernel Streaming won't be needed on Vista to play back DTS wav files?
Will all this functionality be available on today's mobos? Or only on those with HD audio support?

Diogen.

I don't know Diogen. I have to find out. Just in case everyone doesn't know, the old kmixer is gone. Replaced by a high performance mixer/resampler in user mode.

[amirm]

Originally Posted by RobertR1

Would I able to plug into a receiver into the PC, do the calibration via the PC, then disconnect and the updated information is retained by the receiver? Will current receivers work or will I need a upcoming receiver with either a USB or RJ-45 connector?

Unfortunatly not. All the processing is done inside the PC and only works if you use the PC as the source, driving your receiver. Take away the PC and there is nothing there anymore. For your reciever to have similar capabilities, it would have to have a ton more hardware in it. The TacT processors do what you say but the one I have cost me a cool $10K and it does less in some respects than Vista!

If you can provide an "easy" way of people to put together a home audio system, do all the wiring and use Vista to get it 95% optimized, the masses will be quite happy.

We have some ways to go on that but Vista takes a pretty big step toward that. Basically, you would just add an amplifier to the PC and you would be good to go. Indeed, there are companies building PCs with built-in amps which should put this technology to good use.

Will these features make it into RC-1 of Vista? because I'm just putting my system in tomorrow. The interim Panny XR57 does not have any auto calibration and I can wait a few weeks for RC-1 to become available for calibration

Thanks,
Robert.

Yes, it is all in RC-1 so you should be able to play with it. There is a nice control panel for every one of these features.

[RobertR1]

Much appreciated Amir.

How would I connect my Panny XR57 receiver to the PC to get this working in Vista RC-1? If Creative ever get around to getting Audigy 2ZS drivers working in Vista, I'll make RC-1 my primary platform. Since my PC is on 24/7, needing it be on and connected isn't an issue but I'm completely lost on how Vista will talk to the receiver, hardware wise.

[lymzy]

Originally Posted by amirm

Replaced by a high performance mixer/resampler in user mode.

Will I be able to use room correction when bypass the new mixer/resampler?

[amirm]

Originally Posted by RobertR1

Much appreciated Amir.

How would I connect my Panny XR57 receiver to the PC to get this working in Vista RC-1? If Creative ever get around to getting Audigy 2ZS drivers working in Vista, I'll make RC-1 my primary platform. Since my PC is on 24/7, needing it be on and connected isn't an issue but I'm completely lost on how Vista will talk to the receiver, hardware wise.

If your reciever has multi-channel analog input so that you can drive it that way from the PC, yes. Basically, you would play the DVD, music, etc. in the PC. The PC would decode the sound, process per my notes above, and then send the output to the receiver.

The receiver should be set up as to not duplicate the same processing in Vista. So if you use the base management in the PC, you don't want to set it again in the reciever (you would configure all the speakers as "large" in the receiver). Most receivers do this for their analogy inputs anyway but you should doublecheck.

[amirm]

Originally Posted by lymzy

Will I be able to use room correction when bypass the new mixer/resampler?

I don't think so as the effects plug-into the new mixer. Are you using an application that would do this?

[RobertR1]

Ah! That makes much more sense now. I forgot about having the PC doing the playback. I sat there wondering how the receiver was going to send the data to the PC, get it processed, receive it and than push it out to the speakers.

Simply put, my HD-A1 and Comcast DVR would not work but I'll still try to play around with it. Naturally, it'll evolve over time.

I'd think the ideal situation would be to have a media center hub that would act as your PC, gaming console, HDMI switch, do all the processing and pass it onto a receiver via HDMI. That would be pretty cool!

[KenLand]

Amir,

Will Vista have native support for DVD-Audio? SACD?
I assume it will have TrueHD through support for HD-DVD. (?)
And will the new mixer have the ability to *not* resample the audio?
And lastly will there be support for multi-zone audio?

My questions sort of focus on Vista replacing my current hardware/media, but does MS see new delivery options that make some of this obsolete? Say, online Hi-Rez versions of iTunes?

Thanks!
Ken

[Ben_Tech]

One suggestion.... would it be possible for Microsoft to offer a complete audio codec (for an additional fee, naturally, due to licensing costs) that is tightly integrated with these features and can decode DD, DTS, DD+, TrueHD et al, instead of relying on the often spotty support offered by third party codecs.

[jimwhite]

does all of this wonderful stuff work with existing sound cards, such as the X-Fi Elite Pro ??

[Ben_Tech]

It should work with any sound card that has Vista compatable drivers, Jim.

[diogen]

By the sound of it, these audio improvements in Vista might be worth by itself the price of admission. Really exciting.
It looks like just an amp is needed in a Home Theatre run by your Vista PC.
Receiver (processor) manufacturers won't be too happy .

Diogen.

[monsteraudio]

Originally Posted by diogen

By the sound of it, these audio improvements in Vista might be worth by itself the price of admission. Really exciting.
It looks like just an amp is needed in a Home Theatre run by your Vista PC.
Receiver (processor) manufacturers won't be too happy .

Diogen.

and what will a ticket cost me, and of course 3 months later the pro edition, I hope they just release 1 good version with all the features

[John Nelson]

A few questions -
1 Sub Eqing - If have a BFD to eq my sub(s), will Vista have have parametric, shelf, LR filters so I can replace my BFD??

2. 7.1 - Will Vista be able to synthesize the 2 rear channels from 5.1 sources like "Logic 7" on my Lexicon??

3. Any hope we'll get better sound cards that don't send clicks and pops to my amps when the PC boots?

4. Any chance that the horribly loud wave that plays when the OS boots will go away?

[Eiffel]

Thanks for sharing this interesting information.

As the D/A converters on my pre-processor are much better than than the ones on my M-audio 410 'audiophile' sound card (and because I can send an S/SPDIF signal over quite a long distance without attenuation/loss), I would like to know if it will be possible to output a bit-perfect 44.1kHz CD signal over S/PDIF without resampling?

I'm currently using ASIO drivers for this functionality, but would be happy to switch to another approach.

Thanks

[RobertR1]

Originally Posted by monsteraudio

and what will a ticket cost me, and of course 3 months later the pro edition, I hope they just release 1 good version with all the features

The "ultimate" edition will have every feature possible in Vista.

[monsteraudio]

Originally Posted by RobertR1

The "ultimate" edition will have every feature possible in Vista.

then thats the one I want any idea ballpark? on price

[summit3907]

Originally Posted by Eiffel

As the D/A converters on my pre-processor are much better than than the ones on my M-audio 410 'audiophile' sound card (and because I can send an S/SPDIF signal over quite a long distance without attenuation/loss)...

expanding on this, it would be great if all the features you described could be rolled up into something akin to a DD/PCM signal to allow the D/A conversion in the preamp/receiver. currently, ac3filter allows real time 5.1 DD encoding for wave, wma, pcm, etc, after any effects are added.

i have a cheap soundcard in my htpc outputting spdif as i do all the D/A conversion in my much nicer preamp. my guess is that many people have a similar situation. while the features you describe probably outclass most preamps, the experience will ultimately be limited by the D/A converters in the soundcards.

[Sherbona]

Originally Posted by John Nelson

A few questions -

---snip---

4. Any chance that the horribly loud wave that plays when the OS boots will go away?

Turn off your windows sounds:

- Start > Control Panel (classic view) > Sounds and Audio Devices
- Front the Sounds tab
- Set the "Sound scheme" dropdown to "No Sounds"

Or if you prefer to have windows sounds except for the startup, then in Program Events select "Start Windows" and for Sounds select "(None)" - but for me I prefer no window event sounds coming over my system so I turn them all off by setting the scheme to No Sounds as shown above.

[John Nelson]

Originally Posted by Sherbona

Or if you prefer to have windows sounds except for the startup, then in Program Events select "Start Windows" and for Sounds select "(None)"

Thanks... Forgot or never paid attention that was there!!

[Sherbona]

Hi Amir

Thank you very much for remembering us HTPC audio folks! Very interesting... If I understand correctly your saying these features (i.e., bass management) are built into Vista and used by the drivers for folks using Intel HD-Audio solutions, and can be used by other soundcard drivers, which may or may not implement these features, right?


Also, I've come across the following from Sept 2005:

=================
Larry Osterman's WebLog

---snip---

Over the years, we've realized that there three major problem areas with the existing audio infrastructure:

1. The amount of code that runs in the kernel (coupled with buggy device drivers) causes the audio stack to be one of the leading causes of Windows reliability problems.

2. It's also become clear that while the audio quality in Windows is just fine for normal users, pro-audio enthusiasts are less than happy with the native audio infrastructure. We've made a bunch of changes to the infrastructure to support pro-audio apps, but those were mostly focused around providing mechanisms for those apps to bypass the audio infrastructure.

3. We've also come to realize that the tools for troubleshootingaudio problems aren't the greatest - it's just too hard to figure out what's going on, and the UI (much of which comes from Windows 3.1) is flat-out too old to be useful.

Back in 2002, we decided to make a big bet on Audio for Vista and we committed to fixing all three of the problems listed above.

The first (and biggest) change we made was to move the entire audio stack out of the kernel and into user mode. Pre-Vista, the audio stack lived in a bunch of different kernel mode device drivers, including sysaudio.sys, kmixer.sys, wdmaud.sys, redbook.sys, etc. In Vista and beyond, the only kernel mode drivers for audio are the actual audio drivers (and portcls.sys, the high level audio port driver).

The second major change we made was a totally revamped UI for audio. Sndvol32 and mmsys.cpl were completely rewritten (from scratch) to include new, higher quality visuals, and to focus on the common tasks that users actually need to do. All the old functionality is still there, but for the most part, it's been buried deep below the UI.

The infrastructure items I mentioned above are present in Vista Beta1, unfortunately the UI improvements won't be seen by non Microsoft people until Vista Beta2.

http://blogs.msdn.com/larryosterman/...19/471346.aspx
=================

Since the above was from awhile ago and things change, I was wondering if you happen to know if the info in that weblog is still true - especially the part about the audio stack moved out of the kernel into user mode?

[diogen]

Originally Posted by Sherbona

...I was wondering if you happen to know if the info in that weblog is still true - especially the part about the audio stack moved out of the kernel into user mode?

See Amir's post #5 above.

Originally Posted by amirm

... Just in case everyone doesn't know, the old kmixer is gone. Replaced by a high performance mixer/resampler in user mode.

Diogen.

[Esben]

Originally Posted by John Nelson

3. Any hope we'll get better sound cards that don't send clicks and pops to my amps when the PC boots?

X-Fi Elite Pro doesn't send any loud clicks, or pops, when the PC boots, or shuts down. But all my previous sound cards have.

[thomaspf]

I addition to all the DSP functionality Vista also comes with a new sound api that provides an application with exclusive access to a card.

In that mode 16/44.1 pass through of DTS/HDCD encoded material should work if the sound card leaves the bits alone.

[RobertR1]

Originally Posted by monsteraudio

then thats the one I want any idea ballpark? on price

No clue. I'm sure you won't find out until close to the RTM release. Unless Amir wants to be extra nice?

[amirm]

Originally Posted by lymzy

Will I be able to use room correction when bypass the new mixer/resampler?

No you can not. If you bypass the mixer, you bypass the effects system where these things hook in.

Hopefully, you don't have to bypass the new mixer in Vista . It works far, far better than kmixer in XP.