How do I verify - or debunk - the claims of The Upgrade Company? - Page 48

From my cold dead hands.

You're completely right. Nobody should kid himself about this--DS and the rest of the subjectivist believers will grab onto anything they can think of to try to "invalidate" a DBT. If the offer had been made to do a manual cable swap, they would have thought up an objection to that -"too much time between selections", etc (of course, that objection is NEVER EVER brought up with sighted comparisons--ONLY with a DBT). It's all part of their game of avoiding objective comparisons, because they KNOW such comparisons won't support their claims.

Just to be clear, the current test was NOT a "DBT" or Double Blind Test. It was an "SBT" or Single Blind Test. The difference is *potentially* significant.

Craig

I'm a little confused about what you mean by "the current test". The test I read about involving Jeff and Rich used an ABX protocol (as I understand it), which would make it double blind. Besides, it makes no sense to say that people are better at identifying which is which in a DBT than a SBT.

Right.


It is single-blind because the operator of the test knew the identity of devices in each trial. It would have been possible to make it double blind but, considering the resources available, that would have been cumbersome.

Ah, ok. I misunderstood the conditions. It stands to reason, though, that Jeff or Rich wouldn't have done better if it was double blind.

A "Double blind test" means that both the tester and the testee are blinded to the DUT. In a "Single blind test", only the testee is blinded. In a DBT, the person operating the switch would not know which DUT was in use for any given setting of the switch. The "significant difference" is that there is the potential for the person operating the switch to provide some subliminal or greater indication as to which device is under test. In the test performed by Jeff, Dennis and Rich, the person operating the switch was not blinded to the DUT, only the listener was blinded; IOW, it was an SBT. Dennis operated the switch for Jeff and Rich. I believe Jeff operated the switch for Dennis. I don't know if the preliminary test results were shared with other listeners before the subsequent tests. I believe there was a screen between the tester and the testee. Still, verbal and non-verbal communication between the tester and testee could give some feedback that could *potentially* influence he results. I'm not saying any of this happened in the current test. I'm just saying it *could* have happened.

Here is an article about the potential pitfalls of performing blind listening tests:
http://www.audioholics.com/buying-gu...listening-test

The "ABX" part of the test is not part of the definition of the "blinding" portion of the test. The ABX definition refers to the identification of X as being the same as A or B. This obviously needs to be done blind for the listener. However, there is no requirement for the switch operator to also be blind. To make the test most rigorous, the test would be a DBT, ABX test. However, that is not how this current listening test was set up.

Craig

I have noticed that many times (including from audio manufacturers who really should know better) DBT is used in place of SBT far too often, and many do not know the difference.

Craig and Kal explained it.

Craig gave an excellent explanation. I was already aware of what he explained, I just didn't realize that the person operating the switch knew which was which. I had assumed that the ABX device was similar to this one.

I agree - a great job on your explanation Craig. I also respect that you've shared your opinion on the switch (you don't think it caused any degregation), and at the same time, recognizing that it's only an opinion until it a comparison can be made with and without the switch. A very fair point of view.

Dave

This is your opinion, an assumption, and not true in my case.

Dave

The switch used is a passive device. To all intents and purposes it is the same as swapping the cables by hand.

Robert,

I don't believe this is true at all. I may not speak for David Schulte, but I do speak for myself. There's nothing wrong with an SBT or DBT.

(I don't believe there's anything wrong with the Zektor switch, either. DS & Zaks think differently, so I'll leave them to argue that one. I'm not 100%, but I'm pretty sure that's the red herring.)

The problem I have is with ABX testing, nothing else. Before this thread started, I never had a strong opinion about it, but having read a lot more now, I'm forming a strong opinion that its difficult for an ABX test to distinguish between any two different audio units. Its not impossible, but its very difficult, and I doubt that I could do it.

However, blind tests, no problem. I've done it before with stock and TUC modified units and passed the test.

Nick

Dave, what is it that you have from TUC?

[quote=thezaks;21947206]
Dave,

I would start with testing both a stock 5508 and a TUC modded 5508 to see how well both handles RFI/EMI issues. Then I would like to see the results of how each compare when in room measurements are done. Of course in room measurements would be taken in the same room as when Jeff did his comparison (with or without the switch). What tests or measurements did you conduct that indicate the TUC modded gear you have lives up to the claims that DS makes?


What tests would you suggest be done on the switch to show that it effected or did not effect the comparisons made?

Bill

^^^

i'd start by measuring/analyzing the unmodded unit to see if there is really any issue to be solved in the first place...

the first leap of faith that is being made by those who purchase the mod is that the "problem" even exists...

Jeff,

Dave and I aren't here because we like David Schulte and we want to support his business nomatter what. We just want the best gear we can get for the money, and we happen to have had a lot of success with with TUC. If we found something better, we'd change our minds.

For me, that WOULD happen if you could distinguish between the 886 & 5508 in an ABX test. As it happens, I'm also working on three other little projects to try get something better.

For the record, I believe the TUC equipment that Dave has owned is:
Pioneer BDP-05
Marantz AV7005
Onkyo SC5508

While I've had:
Onkyo SC885
Denon 3800BD
Pioneer BDP-09
Nuforce BDP-83SE
Oppo 95

Regards, Nick

Nick, in your quest for the "best you can get for your money" have you exhausted all the conventional upgrade paths then and TUC is a sort of 'last fling' for that tiny potential improvement?

What I mean is have you already extensively treated your room for example - knowing that the room is by far the most important 'component' in the system and the one that has the most impact on the SQ?

Have you also splashed out on the very best speakers you can afford - again knowing that it is in speakers where the really big differences in SQ can occur? Ditto for subwoofers if you use them?

If so, would you say that $1400 spent on room treatments and better speakers would bring you even better results than the same money spent on a TUC upgrade of an AVR?

Have you used any of the sophisticated, and proven, electronic EQ methods, such as Audyssey XT32 or Audyssey Pro in order to squeeze the last drop from your system and room?

I am not trying to be confrontational at all - just wondering where a TUC-style upgrade fits into your scheme of things. For me, for example, it would be the very last thing I spent my money on. I'd have done everything possible to treat the room, to use the very best speakers, to buy the very best stock units with the best electronic EQ etc, before I even considered an aftermarket upgrade, especially one which comes with no manufacturer-provided evidence that it actually works.

Anything is "detectable," if one strips away all possible controls and makes the evaluation as leading as possible, of course. One can easily even hear the difference between one box and itself - by twirling the volume control a bit! That's every bit as valid a "test" as any other sighted listening session.


First of all, you're just wrong on fact. There have been plenty of ABX tests that have shown two given DUT's to be different. But that doesn't matter, really. Just one would be sufficient to make you simply wrong on fact.

Second, what if most audio kit does in fact "sound the same?"

That's what the most sensitive controlled subjective same/different listening evaluations tell us, after all!

To deny that possibility is to show a troubling willingness to look at simple reality and be willfully blind to it.


There's a difference between actual controversy and manufactured controversy. To claim otherwise is to say we live in a pomo dystopia where any fact contrary to the interests of any actor instantly becomes "controversial" if that actor whines loudly enough about it.

Here, the simple fact is that a few people with skin in the game are unhappy with observed, and out of some self interest they're trying desperately trying to spin until they can convince others that 2+2 = banana. There is no good reason to give any deference whatsoever to such an obviously manufactured "controversy."

I'm curious about something (and maybe that is why I continue to follow this thread). This is an impressive list of equipment. When you use the various BD players, do you use the digital outputs or the analog outputs for audio to the AVR (or separates)?

But much faster and that, in itself, is a huge advantage.

I would be curious as to if this large list of modded gear had been listened to in its stock form or sent directly to TUC for modification.

Bill

First you say:


Then you say:


Since A and B are being compared to X (an unknown), an ABX test IS a blind test, either single blind or double blind. This means that you're essentially saying:

"There's nothing wrong with an SBT or DBT" AND "The problem I have is with a SBT or a DBT, nothing else"

You're making contradictory statements that cannot be simultaneously true. You're either opposed to blind testing or you aren't. You can't claim to have no problem with it AND have a problem with it.



So, just to be completely clear on this, you would have no problems with the validity of a DBT using manual cable swapping? None at all?

Why would he have a "problem" with those tests? Since there has not been an ABX or DBT done in regard to the TUC components, it's really a moot point. And with no statical sampling of any credible kind to reference, where does that leave all those who are critical of TUC?

I guess it leaves everyone in the same place as TUC who states they've done DBT's.

I currently have an Onkyo 5508 and a Pioneer BDP-05FD Blu Ray Player. I previously had a Marantz AV8003.


Dave

Your opinion, but an unknown until tested with cable swapping vs switch with the TUC upgraded 5508.

Dave

Unfortunate and fairly biased.

Dave

The same place as always...with no critical analysis of the pre or post performance specifications of the "upgraded" components and no reason to believe that some silver tape, copper wires with shielding, and some capacitors do anything at all to improve the performance of the component and that it's a waste of money if one wants to improve sound quality of the system.