Best method/preferred setup for blind comparison - AVS Forum | Home Theater Discussions And Reviews
Baselworld is only a few weeks away. Getting the latest news is easy, Click Here for info on how to join the Watchuseek.com newsletter list. Follow our team for updates featuring event coverage, new product unveilings, watch industry news & more!


View Poll Results: How should blind subwoofer comparisons be matched?
Price 0 0%
Displacement (The limiting factor in output) 0 0%
Enclosure Size (The size of all enclosures combined) 0 0%
Number of Enclosures 0 0%
Other (Please Specify) 0 0%
Voters: 0. You may not vote on this poll

Forum Jump: 
 
Thread Tools
post #1 of 20 Old 01-18-2012, 01:32 PM - Thread Starter
Senior Member
 
kwarny's Avatar
 
Join Date: Dec 2009
Location: Lincoln, NE
Posts: 291
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
I'm interested in what people think or prefer should be the basis for subwoofer comparisons.

The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge - Stephen Hawking

 

Kyle Bosso - SOB

kwarny is offline  
Sponsored Links
Advertisement
 
post #2 of 20 Old 01-18-2012, 02:06 PM
AVS Special Member
 
drewTT's Avatar
 
Join Date: Nov 2007
Location: Anaheim Hills, CA
Posts: 2,021
Mentioned: 1 Post(s)
Tagged: 0 Thread(s)
Quoted: 102 Post(s)
Liked: 133
I would personally only compare the powered versions. Seems like with an outboard amp, many variables come into play...
drewTT is offline  
post #3 of 20 Old 01-18-2012, 02:33 PM
AVS Special Member
 
gtpsuper24's Avatar
 
Join Date: Aug 2008
Location: Ohio
Posts: 2,088
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 5 Post(s)
Liked: 95
Should compare by price IMO. Especially ID brands. Could do multiple price point range, like $800-1200 ect. I think just about all the ID brands in each price point compete well with each other except for Axiom which is half the performance for twice the price. But I don't think we will ever see them offer products for a shootout, unless someone bought one and offered to join in.
gtpsuper24 is offline  
post #4 of 20 Old 01-18-2012, 02:40 PM
AVS Addicted Member
 
cschang's Avatar
 
Join Date: Mar 2003
Location: Manhattan Beach, CA
Posts: 14,845
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 58 Post(s)
Liked: 110
Quote:
Originally Posted by drewTT View Post

I would personally only compare the powered versions. Seems like with an outboard amp, many variables come into play...

Agreed.

If an outboard amp is needed, then what is recommended should be used, and explicit setup guidelines for the amp should be given by the sub manufacturer. Of course, the price of the amp should be factored in.

-curtis

Owner of Wave Crest Audio
Volunteer Mod at the Ascend Acoustics Forum
Like all things on the Internet, do your research, as forums have a good amount of misinformation.
Help beat breast cancer!

cschang is offline  
post #5 of 20 Old 01-18-2012, 02:46 PM
AVS Special Member
 
desertdome's Avatar
 
Join Date: Aug 2004
Location: Elkhorn, NE
Posts: 2,310
Mentioned: 50 Post(s)
Tagged: 0 Thread(s)
Quoted: 439 Post(s)
Liked: 488
I voted "other."

I think the point of a GTG isn't so much to compare subs, but to learn about subs. It is to fun to learn about the trade-offs in size, alignment, DSP, and amp power. You wouldn't learn near as much if the displacement was kept equal or if the size was kept equal. I wanted to know what a 15" sealed servo sub sounded like and what advantages/disadvantages it had over an 18" ported sub.

At the Omaha GTG I liked hearing the variety of single subs/dual subs and the various alignments of ported, sealed, front loaded horn, folded horn, & tapped horn. It was also helpful to hear DIY vs retail and onboard DSP/amp vs outboard amp.

At the KC GTG I enjoyed hearing what incredible output sounded like and being able to listen to the various alignments at a high output level. It was also nice to have all duals to help even out the room response. I wonder if Archaea took any measurements of just one sub? I guess he could still do that with his Cap Pros. By evening out the room response you are able to focus more on the quality of the sub without the room's interference.
desertdome is offline  
post #6 of 20 Old 01-18-2012, 02:55 PM
AVS Special Member
 
monomer's Avatar
 
Join Date: Apr 2002
Location: up north
Posts: 1,767
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
If you were comparing automobiles instead of subwoofers would you disregard price?
In a blind audition all subs should be EQed properly to the room and level matched hopefully in a room that will not put it near the edge of compression, clipping, bottoming, etc. otherwise what's the point? to see which one goes the loudest? you don't need a blind panel of testers to determine that. If the point of the blind panel is to determine which has the best sound qualities (and feel) given a particular set of source material then no sub should be forced to its limits during the test audition otherwise the comparison is flawed rendered useless.

How many people purchase subs by the cubic inch? and if its SPLs that's most important again you don't need a blind panel to tell you who's got the most. Most of us when looking for a sub are seeking one within a price range and so I think "value" is most important to most prospective sub purchasers and therefore something like "specific SQ" point system would be quite useful. Perhaps something like divide the price into the final SQ score or using a price percentage to equalize the field... I don't know at this point, the details would have to be discussed and agreed upon by all parties BEFORE the actual audition happens so you don't have a lot of whining by the losers after the results are posted.

"For deep bass, the listener is not really listening to the speaker, but rather, is listening to the room as it is being played by the speaker."
Juicy
Doggie Jabber
monomer is offline  
post #7 of 20 Old 01-18-2012, 02:57 PM
AVS Special Member
 
desertdome's Avatar
 
Join Date: Aug 2004
Location: Elkhorn, NE
Posts: 2,310
Mentioned: 50 Post(s)
Tagged: 0 Thread(s)
Quoted: 439 Post(s)
Liked: 488
Quote:
Originally Posted by cschang View Post

Agreed.

If an outboard amp is needed, then what is recommended should be used, and explicit setup guidelines for the amp should be given by the sub manufacturer. Of course, the price of the amp should be factored in.

Jeff with JTR had no input into how Archaea connected the amp to his Cap Pros. When you buy passive, the sub mfg shouldn't have any input since everyone is going to have a different setup. It was nice to be able to compare a passive JTR solution with DIY amp setup to the JRT powered solutions. Without this kind of comparison, the end user would think it easy to just add an amp to a sub and start playing loud bass.

The passive JTR solution shut down 3 times during playback while the powered solutions never had this problem. This showed that even with all the work and testing Archaea did ahead of time, it still didn't match the playback capability of the mfg provided DSP amps. This let us as listeners/attendees know the trade-offs involved with passive vs powered and that passive isn't as easy as it might seem. We also saw Archaea's screen shots of the iNuke amps's DSP settings and showed all the filters he needed to smooth the frequency response of the sub.
desertdome is offline  
post #8 of 20 Old 01-18-2012, 03:03 PM
AVS Special Member
 
Snowmanick's Avatar
 
Join Date: Nov 2007
Posts: 1,649
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 74 Post(s)
Liked: 148
It's hard to say just one metric. Take price for example, are you using out-the-door price (sales price, tax, and/or shipping costs factored in) or just MSRP. Even if it is MSRP several ID vendors have MSRP and then a lower ID price. Additionally, brick and mortar shops frequently discount MSRP on the brands they carry, and some ID brands factor shipping into the price (Aperion and now SVS for example). Further, ID products are currently sitting at the top of the pre-assembled heap, but many avid DIY's will tell you that they can construct competitive to world beating subs at a much lower cost than ID's and/or brick-and-mortars, but how do you account for the cost of the time they put into the finished product as well as the diminished resale value and/or lack of warranty that DIY's carry?

Maybe a better way would be brackets based off of price and type of product, but then you are unnecessarily limiting comparisons unless you do a total rank and a rank within a bracket, but that is a lot of work.

I like seeing lots of different methodologies being used, as it allows for a lot of cross platform comparisons. Most of this is going to be subjective anyhow, so why try to overly limit it based off of just one metric.

If you are really determined to choose some limiting factors, then I like the number of enclosures being used as it limits differences due to the room (modal smoothing) that additional properly setup enclosures may provide.

~Nick

Augustine's Law: "The last 10% of performance generates one-third of the cost and two-thirds of the problems."

"If it works the first time, it's not high end." ~ Harry Pierson
Snowmanick is offline  
post #9 of 20 Old 01-18-2012, 03:09 PM
AVS Addicted Member
 
cschang's Avatar
 
Join Date: Mar 2003
Location: Manhattan Beach, CA
Posts: 14,845
Mentioned: 2 Post(s)
Tagged: 0 Thread(s)
Quoted: 58 Post(s)
Liked: 110
Quote:
Originally Posted by desertdome View Post

Jeff with JTR had no input into how Archaea connected the amp to his Cap Pros. When you buy passive, the sub mfg shouldn't have any input since everyone is going to have a different setup. It was nice to be able to compare a passive JTR solution with DIY amp setup to the JRT powered solutions. Without this kind of comparison, the end user would think it easy to just add an amp to a sub and start playing loud bass.

The passive JTR solution shut down 3 times during playback while the powered solutions never had this problem. This showed that even with all the work and testing Archaea did ahead of time, it still didn't match the playback capability of the mfg provided DSP amps. This let us as listeners/attendees know the trade-offs involved with passive vs powered and that passive isn't as easy as it might seem. We also saw Archaea's screen shots of the iNuke amps's DSP settings and showed all the filters he needed to smooth the frequency response of the sub.

Completely understood, but does that mean that Jeff doesn't have recommendations?

-curtis

Owner of Wave Crest Audio
Volunteer Mod at the Ascend Acoustics Forum
Like all things on the Internet, do your research, as forums have a good amount of misinformation.
Help beat breast cancer!

cschang is offline  
post #10 of 20 Old 01-18-2012, 03:17 PM
AVS Special Member
 
monomer's Avatar
 
Join Date: Apr 2002
Location: up north
Posts: 1,767
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by Snowmanick View Post

...Take price for example, are you using out-the-door price (sales price, tax, and/or shipping costs factored in) or just MSRP...

There are just too many marketing schemes being employed by the different manufacturers to use MSRP so its got to be "street" price... meaning whatever someone actually paid for it.

"For deep bass, the listener is not really listening to the speaker, but rather, is listening to the room as it is being played by the speaker."
Juicy
Doggie Jabber
monomer is offline  
post #11 of 20 Old 01-18-2012, 03:23 PM
AVS Special Member
 
Archaea's Avatar
 
Join Date: Apr 2004
Location: Kansas City, MO
Posts: 7,757
Mentioned: 137 Post(s)
Tagged: 2 Thread(s)
Quoted: 1454 Post(s)
Liked: 1298
Quote:
Originally Posted by desertdome View Post

I voted "other."

I think the point of a GTG isn't so much to compare subs, but to learn about subs. It is to fun to learn about the trade-offs in size, alignment, DSP, and amp power. You wouldn't learn near as much if the displacement was kept equal or if the size was kept equal. I wanted to know what a 15" sealed servo sub sounded like and what advantages/disadvantages it had over an 18" ported sub.

At the Omaha GTG I liked hearing the variety of single subs/dual subs and the various alignments of ported, sealed, front loaded horn, folded horn, & tapped horn. It was also helpful to hear DIY vs retail and onboard DSP/amp vs outboard amp.

At the KC GTG I enjoyed hearing what incredible output sounded like and being able to listen to the various alignments at a high output level. It was also nice to have all duals to help even out the room response. I wonder if Archaea took any measurements of just one sub? I guess he could still do that with his Cap Pros. By evening out the room response you are able to focus more on the quality of the sub without the room's interference.

Some of the subs were at my 2011 May meet in single form including the HSU VTF-15H, the Seaton Submersive, and the JTR Captivator. You can see their response there, compared to the frequency response we got in 2012 with duals. In 2011 we didn't use Audyssey and we put the single sub up front in the middle under the projector screen facing the listening area. In 2012 we used two subs, faced them out for best room response, and then used Audyssey to try to provide a more equitable listening experience between the sub pairs.

We have graphs for both meets. Take a look at the threads in my signature for comparison of frequency response. Dual subs definately helps even out the response.

"Without subs it's just background music - with subs it's the main event!"
Archaea's Theatre Room
2011 KC Sub Shootout
2012 KC Blind Sub Shootout
My Subwoofer Recommendations by Pricepoint
Archaea is offline  
post #12 of 20 Old 01-18-2012, 03:47 PM - Thread Starter
Senior Member
 
kwarny's Avatar
 
Join Date: Dec 2009
Location: Lincoln, NE
Posts: 291
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
I just wanted to hear what others thought. I don't really care at a GTG cause I'll probably always be DIY since I enjoy the learning process. One of these days a DIY will be a favorite at one these midwest ones .

However with companies involved, I think it is somewhat smart to level the playing field. If we go by price, do we use the price of their cheapest finish (usually basic black)? Some enclosures also are built better which includes cost. Some include reliable plate amps with dsp tweaked by the engineer which will only increase cost.

Say if we have a 4-5k session, we could have about 2 of each kind of powered captivator, 2 submersives, 4 CHT 18.2, 3 rythmik f25s, 26 Bic F12, and so on (just using what I have an idea of price on without looking). If the CHT or Bic are preferred, then we have to consider that is greater than 4 refrigerators in the room compared to the tiny submersives or sealed caps. Then people may wonder (at least I), I don't have enough room for huge or many ugly boxes in my room, how would half that many still perform to less of the smaller more expensive options ?

If we would of kept the Omaha GTG so each driver was not stressing, we could of never had the volume for anyone to think the larger drivers had any value with their extra displacement.

I would prefer to match by displacement so each driver is performing within their linear range. Displacement is generally the limiting factor and there should be tons of headroom above 30-40 Hz for every setup then. It should easier to play all of these within their linear range. There will be huge price differences but I think the performance is easier to scale. If someone doesn't have the room, they can cut the displacement in half and still have a good idea of the advantages of one versus the other and see if the price difference is worth it to them.

Example - 4 cht 18.2s, 3 rythmik f25, 2 jtr caps compared to 2 submersives now becomes 2 chit 18.2s, 1.5 rythmik f25 , 1 jtr cap, and 1 submersive.

The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge - Stephen Hawking

 

Kyle Bosso - SOB

kwarny is offline  
post #13 of 20 Old 01-18-2012, 06:10 PM
Advanced Member
 
Jeff Permanian's Avatar
 
Join Date: Jun 2007
Location: Wisconsin
Posts: 576
Mentioned: 6 Post(s)
Tagged: 0 Thread(s)
Quoted: 110 Post(s)
Liked: 194
For the most fair comparison you must remove all variables.

1) have the comparison outdoors
2) not equalized (adds variables)
3) level matched to tight tolerance (+/- .5db)
Jeff Permanian is offline  
post #14 of 20 Old 01-18-2012, 06:42 PM - Thread Starter
Senior Member
 
kwarny's Avatar
 
Join Date: Dec 2009
Location: Lincoln, NE
Posts: 291
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 12
Quote:
Originally Posted by Jeff Permanian View Post

For the most fair comparison you must remove all variables.

1) have the comparison outdoors
2) not equalized (adds variables)
3) level matched to tight tolerance (+/- .5db)

I will remember to buy an extra few acres of land in the middle of nowhere for this when I build some decade. This would of almost been possible where I lived last year but my duplex would of reflected the back wave but all the neighbors were pretty far away on the sides (300 ft). Everything would of fired into a park. The garage would of been very simple to exchange setups though. It would have to be done after the birds stopped chirping and before the locusts came out in the evening.

Maybe we could rent the state fair grounds. This would be the perfect place for you showoff that bass rig you have been wanting to build for bass guitar .

The greatest enemy of knowledge is not ignorance, it is the illusion of knowledge - Stephen Hawking

 

Kyle Bosso - SOB

kwarny is offline  
post #15 of 20 Old 01-18-2012, 06:55 PM
AVS Special Member
 
mojomike's Avatar
 
Join Date: Dec 2005
Location: South Florida
Posts: 6,966
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 28
Quote:
Originally Posted by Jeff Permanian View Post

For the most fair comparison you must remove all variables.

1) have the comparison outdoors
2) not equalized (adds variables)
3) level matched to tight tolerance (+/- .5db)

If you are not equalized, how exactly do you level match? At what frequency do you level match and what would make that particular frequency more appropriate than some other frequency?
mojomike is offline  
post #16 of 20 Old 01-18-2012, 07:16 PM
Advanced Member
 
Jeff Permanian's Avatar
 
Join Date: Jun 2007
Location: Wisconsin
Posts: 576
Mentioned: 6 Post(s)
Tagged: 0 Thread(s)
Quoted: 110 Post(s)
Liked: 194
Quote:
Originally Posted by mojomike View Post

If you are not equalized, how exactly do you level match? At what frequency do you level match and what would make that particular frequency more appropriate than some other frequency?

I think I'll say out of the fine details.

Funny that no one picked enclosure size yet since this is one of the most important elements to a subwoofers design. The larger the cabinet the more efficient it becomes or the lower tuned.

The smaller the subwoofer the more expensive it comes to keep up with a larger subwoofer.

If cost is going to be a main factor then there should be a handicap for size.
Jeff Permanian is offline  
post #17 of 20 Old 01-18-2012, 09:06 PM
 
floridapoolboy's Avatar
 
Join Date: Dec 2004
Location: florida
Posts: 3,482
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Cost should definitely be a prime factor in any comparison. Of course, IMO GTGs are for fun, while outdoor ground plane measurements are what is needed for meaningful subwoofer comparisons. I would rather see Ricci get to measure the Submersive, Captivator, etc. than rely on the well meaning but flawed methods used to conduct most GTG shootouts.
floridapoolboy is offline  
post #18 of 20 Old 01-19-2012, 09:00 AM
Advanced Member
 
Jeff Permanian's Avatar
 
Join Date: Jun 2007
Location: Wisconsin
Posts: 576
Mentioned: 6 Post(s)
Tagged: 0 Thread(s)
Quoted: 110 Post(s)
Liked: 194
Quote:
Originally Posted by floridapoolboy View Post

Cost should definitely be a prime factor in any comparison. Of course, IMO GTGs are for fun, while outdoor ground plane measurements are what is needed for meaningful subwoofer comparisons. I would rather see Ricci get to measure the Submersive, Captivator, etc. than rely on the well meaning but flawed methods used to conduct most GTG shootouts.

Unfornately, test data will not reveal the subwoofer's sound quality which is exactly what a blind test does.
Jeff Permanian is offline  
post #19 of 20 Old 01-19-2012, 03:33 PM
AVS Special Member
 
nube's Avatar
 
Join Date: Nov 2005
Location: Albuquerque, NM
Posts: 1,383
Mentioned: 6 Post(s)
Tagged: 0 Thread(s)
Quoted: 75 Post(s)
Liked: 445
Quote:
Originally Posted by monomer View Post

There are just too many marketing schemes being employed by the different manufacturers to use MSRP so its got to be "street" price... meaning whatever someone actually paid for it.

I couldn't agree more that, if you predicate a GTG testing/evaluation methodology on price, it has to be price paid, minus extras for finish. For instance, when I was shopping for subs two autumns ago, I called a bunch of manufacturers. At that time, a single 2011 JTR Captivator + EP4K cost me $1285 delivered, which was more than a single 18.1 ($986 including shipping is what I was quoted for the plate-amp version), but significantly less than, for instance, the Dual CHT 18.1 + Dayton SA1000 setup which was $1450 + shipping.

The problem is that, now, the price of the passive Captivator has gone up to $1499 + shipping, and the price of the CHT 18.1 has gone down to $545 delivered + amp. Definitely market pressures pushing each of those opposite directions, but I'll not go into a big discussion of econometrics here. Suffice it to say that, if you're comparing on price, it should definitely be price paid, minus finish extras. However, I don't think price is the best measure.

Quote:
Originally Posted by floridapoolboy View Post

Cost should definitely be a prime factor in any comparison. Of course, IMO GTGs are for fun, while outdoor ground plane measurements are what is needed for meaningful subwoofer comparisons. I would rather see Ricci get to measure the Submersive, Captivator, etc. than rely on the well meaning but flawed methods used to conduct most GTG shootouts.

GTGs are about having fun, but they can also be about creating repeatable, real-world data. I think that you're too narrow in your consideration of what makes for meaningful comparisons of subs. I think it is equally important to have the specs of the subs in question established by a 3rd party (including ground plane measurements), a blind audition of them against each other, and a subjective "SPL drag race" to get them to show their true colors.

Quote:
Originally Posted by Jeff Permanian View Post

Unfornately, test data will not reveal the subwoofer's sound quality which is exactly what a blind test does.

Definitely agree, and this is an important subjective point. The problem is that the room dominates the frequency response, so it's hard for anyone besides the audition's venue owner to really make purchasing decisions based on what they hear at a given audition. Of course, there's more than FR in subjective like/dislike of a sub's sonic signature, as Mark has fleshed out in the discussions of the SSD in the DIY section in the past couple of days.
nube is offline  
post #20 of 20 Old 01-19-2012, 04:32 PM
 
floridapoolboy's Avatar
 
Join Date: Dec 2004
Location: florida
Posts: 3,482
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Liked: 11
Quote:
Originally Posted by Jeff Permanian View Post

Unfornately, test data will not reveal the subwoofer's sound quality which is exactly what a blind test does.

Really? I believe it actually shows the limitations of the room, compounded by set up variables such as eq, amp choice, etc, and then becomes clouded even further by seating location, aural memory, and so on. I prefer the test data, providing it is done in a consistent and reproducible manner. I bet that when Ricci ever gets these subs on his Data-base the top performers in measurements will also sound the best in whatever room they end up in.
floridapoolboy is offline  
Sponsored Links
Advertisement
 
Reply Subwoofers, Bass, and Transducers

Thread Tools
Show Printable Version Show Printable Version
Email this Page Email this Page


Forum Jump: 

Posting Rules  
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off