Spyware Warrior

eburger68 · Posted: Sat Oct 16, 2004 8:59 am Post subject: ASW Tests - One last time...

Hi All:

I've a completed a third round of tests with 20 different anti-spyware applications and a new collection of spyware and adware. As before, the Guide page explains the tests themselves and lists the "critical" detections for this particular round of tests:

http://spywarewarrior.com/asw-test-guide.htm#detections3

The results of this new round of tests can be found on these two pages:

http://spywarewarrior.com/asw-test-results-5.htm
http://spywarewarrior.com/asw-test-results-6.htm

Questions, comments, and suggestions are always welcome.

Best,

Eric L. Howes

wawadave · Posted: Sat Oct 16, 2004 9:06 am Post subject:

thank you eric for all the work you have done!
_________________
RFID tags! SPYWARE
Tired of proprietary Cor-pirationware?
http://www.openoffice.org/
Installing Vista http://tinyurl.com/2l9qyd

mikey · Posted: Sat Oct 16, 2004 9:13 am Post subject:

Hey Eric,

One last time?

I'm a bit confused. How does a test like this support any conclusions when only less than 1% of known targets are represented?...and many of those are obscure and/or uncommon?
_________________
-
W2K/2K3/XP/2K8/Vista/W7/RHE/DEBIAN/SUSE

Spyware/Adware is NOT freeware, it costs all of us dearly.

Mikey's Stuff

Fiddler and friends...essential web diagnostic, forensic, & development tools.
-

eburger68 · Posted: Sat Oct 16, 2004 10:17 am Post subject:

mikey:

I didn't say that these tests were enough to stand as definitive based on the sample of adware and spyware used. In fact, I specifically disclaimed that here:

http://spywarewarrior.com/asw-test-guide.htm#disclaimers

"One last time" means that this will be the last round of tests for the near future as I'm simply out of time to run these kinds of tests for now.

Eric L. Howes

mikey · Posted: Sat Oct 16, 2004 10:21 am Post subject:

Well, I'm sure you are aware that all over the net, folks are using this test as a comparison and recommendation for the quality of products in spite of the disclaimer. Is there a way you could emphasize it a bit more?

Also, I'm still somewhat curious as to the purpose of the tests if they aren't conclusive. Or is that itself the purpose...to show folks how these tests scattered around(many by rogues) are not a real definitive scale?
_________________
-
W2K/2K3/XP/2K8/Vista/W7/RHE/DEBIAN/SUSE

Spyware/Adware is NOT freeware, it costs all of us dearly.

Mikey's Stuff

Fiddler and friends...essential web diagnostic, forensic, & development tools.
-

TeMerc · Posted: Sat Oct 16, 2004 10:38 am Post subject:

Very good Eric, thanks for all your hard work.

Mikey, I think its a safe bet to figure that as much as Eric tries to distance him slef for any endorsments, people will use this as a reference point no matter what he says.

For his own sake, I would think he needs to remain as distant as possible. After all, as it is he comes under attack by some of these clowns whose products perform poorly. Can you imagine what would happen if he gave any stronger recommendations?

I always point out people just need to make the decisions based on the info at hand. And say this excellent work is done as a matter of education and information.

BTW, Eric, do you have any info on 'StartPage Spyware Removal Tool for IE'.
Its located here:
http://www.nospy.org/1/
_________________

Ultimate Countermeasures Page
Calendar Of Updates
Malware Advisor Blog

eburger68 · Posted: Sat Oct 16, 2004 11:02 am Post subject:

mikey:

Having done these types of tests before, I can assure you that no matter what I do some folks will completely ignore my disclaimers. The disclaimers are prominent and mentioned several times in the text leading up to the disclaimers. The disclaimers themselves are bullet-pointed for easy reference and couched in simple, straightforward language. All the results pages point to the Guide page right at the top. My announcements have urged people to read the Guide page. There's not much more that I can do.

As for the purpose of the tests, that is discussed several times on the Guide page.

Eric L. Howes

eburger68 · Posted: Sat Oct 16, 2004 11:07 am Post subject:

TeMerc:

I've never encountered that tool before to my knowledge, so I can't really comment on it. Looks interesting, though.

Best,

Eric L. Howes

mikey · Posted: Sat Oct 16, 2004 11:24 am Post subject:

eburger68 · Posted: Sat Oct 16, 2004 2:42 pm Post subject:

mikey:

Stop. This is getting silly.

First you were complaining that I was claiming too much for these tests. When it was plainly demonstrated that you, like all the other readers whom you purport to criticize, hadn't read my careful disclaimers and qualifications, you turned right around and complained that I hadn't made the tests definitive enough. Now you make both complaints in the same post.

The only consistent thing here has been an inexplicable compulsion to find something to gripe about.

Still worse, you drag in the personal innuendo once again, implying that the tests that I've run are flawed in the same way as those done by self-interested software companies and that I've somehow been slyly encouraging or tolerating their misinterpretation. Any evidence to the contrary you simply ignore. Even when quoting that short passage from the Guide page, you boldface certain words and ignore, once again, the qualification ("partially") that leads off that sentence.

The tests that I've done are certainly limited, and they may be flawed methodologically (note that you haven't demonstrated any flaws yet, though I'll allow that there may be flaws), but that doesn't come close to being "subjective" and self-interested in the way your post irresponsibly implies.

By turns, that the tests are of limited scope does not mean that they have "little value" at all, and despite any noise to the contrary you haven't offered even the slightest argument why they would carry "little value." Merely asserting that something is so doesn't make it so.

Moreover, it's bit difficult to take you seriously when you purport to dictate to me what my responsibilities are when you hadn't even read (and even continue to ignore) the very disclaimers that you're now griping about others not reading. I am not obligated to run around the Net correcting every misstatement about the tests by people who, like you, can't be bothered to take the five minutes it does to read the material with a careful eye and an open mind.

The bottom line here is that you tried to pin a crime on me without even deigning to look at the evidence. After the relevant passage from the Guide page had been pointed out to you, you continued to make this irresponsible charge even in the face of obvious evidence to the contrary.

If you want to continue with this ridiculous line of argument, that's your choice. I won't be wasting my time responding to it, though.

Eric L. Howes

mikey · Posted: Sat Oct 16, 2004 5:15 pm Post subject:

"mikey:

Stop. This is getting silly.

First you were complaining that I was claiming too much for these tests. When it was plainly demonstrated that you, like all the other readers whom you purport to criticize, hadn't read my careful disclaimers and qualifications, you turned right around and complained that I hadn't made the tests definitive enough. Now you make both complaints in the same post."

M) Actually, I read every word several times.

"The only consistent thing here has been an inexplicable compulsion to find something to gripe about."

M) Well, I guess that's one way to look at it. I prefer to look at it as trying to find truth and understanding. This is what I do and have no intention of stopping now.

"Still worse, you drag in the personal innuendo once again, implying that the tests that I've run are flawed in the same way as those done by self-interested software companies..."

M) This is true. I believe they share the same exact flaws.(as we have discussed before)

"...and that I've somehow been slyly encouraging or tolerating their misinterpretation. Any evidence to the contrary you simply ignore."

M) I didn't say or intentionally imply that.

"Even when quoting that short passage from the Guide page, you boldface certain words and ignore, once again, the qualification ("partially") that leads off that sentence."

M) I emboldened it to highlight what folks are thinking...which is demonstrated all over the net now.

"The tests that I've done are certainly limited, and they may be flawed methodologically (note that you haven't demonstrated any flaws yet, though I'll allow that there may be flaws), but that doesn't come close to being "subjective" and self-interested in the way your post irresponsibly implies."

M) Again, my intention was to point out the flawed interpretations folks are making.

"By turns, that the tests are of limited scope does not mean that they have "little value" at all, and despite any noise to the contrary you haven't offered even the slightest argument why they would carry "little value." Merely asserting that something is so doesn't make it so."

Jester700 · Posted: Sat Oct 16, 2004 7:41 pm Post subject:

Eric, I and many others appreciate your efforts. Yes, people will take the tests for more than they were meant, but morons are everywhere (that was not meant to imply that Mikey was such a one; given our comparative levels of skill here that would be pretty-friggin'-funny). But you can't be held accountable for that. The real issue is simply that there are SO many iterations of spyware and possible solutions that a comprehensive test would be staggering. AND, the usefulness of the results would be short lived anyway.

But at least you give SOME idea of various software's abilities. I'd love to see others take your lead & run with it; maybe in time this would lead to a better overall picture.

3162 · Posted: Sat Oct 16, 2004 8:21 pm Post subject:

mikey · Posted: Sat Oct 16, 2004 8:35 pm Post subject:

3162 · Posted: Sat Oct 16, 2004 8:46 pm Post subject:

mikey · Posted: Sat Oct 16, 2004 8:49 pm Post subject:

3162 · Posted: Sat Oct 16, 2004 8:56 pm Post subject:

That's good. It could have become a giant issue.....*cough*
_________________
Proud member of the Chest Zipper Club!

Nick · Posted: Sat Oct 16, 2004 9:19 pm Post subject:

never mind...

herbalist · Posted: Sat Oct 16, 2004 11:17 pm Post subject:

Eric,
Thanks for taking the time to do all that work. While I realize that these tests aren't intended to be conclusive, they're definitely thorough enough to show some pretty consistent patterns for the different apps. It's obviously unrealistic to even try to do something like that with all possible pests. It would be out of date in a day. A better option would be to invite someone responsible for each of the tested remover programs to view the results, not so much for comparison or discussion, but so they can see what they need to improve on. Some consistently have a false detection problem. Some consistently miss the majority of items. Since there's no realistic way to define a "typical infection" list, a list of common pests like the ones you used, could be used as a type of rough benchmark by them, not for improving test results, but treated as "snapshots" of how their programs did in 3 separate pseudo-typical cases. If they look at it honestly, they'll see what they need to improve on.
Rick

eburger68 · Posted: Sun Oct 17, 2004 1:15 am Post subject:

herbalist:

All excellent suggestions, and I am hoping that anti-spyware vendors are able to learn something from these tests as well.

One thing I haven't done yet is put together a short page that discusses some of the issues that I encountered over the past few weeks that aren't reflected in the raw numbers, which can only say so much. I hope to have that done in the very near future.

Best,

Eric L. Howes