This article was originally published just last November. An email I received the other day reminded me of it so I thought I’d share it again. The bottom line here is scientific method. It’s boring, and expensive and time consuming, but it’s how you test accurately. Now data interpretation is an entirely different ball of wax, but you’ve got to get there first.
We get email all of the time from readers asking why we don’t test this or that. There’s two simple reasons.
First, we do the news and we don’t have enough bandwidth to do proper test & evaluations which leads us to the second point. We don’t have the access to the right facilities and equipment to conduct proper tests.
We see folks all over the internet throwing products on their charcoal grills saying they are conducting burn tests or taking their latest gucciflauge out in the back yard to snap a couple of photos and saying they are testing camouflage or worse yet, shooting a ballistic material at a random range with some random gun. In all cases, they are wrong. All they are doing is making noise and, filling their reader’s heads full of nonsense that in some cases is libelous and in others downright dangerous.
Our editor actually conducted test & evaluation for DoD and later, after he retired from the military for commercial companies. Consequently, he knows how it is supposed to be done and, that is why you won’t see us doing it. If we aren’t going to do it right, we aren’t going to do it at all.
So, next time you see an internet “test” conducted by amateurs, ask them what qualifies them to conduct that test. Ask to see their test plan and their data collection schema. Ask them how many times they conducted the test and how they ensured that the tests were conducted properly and consistently. We could go on and on but quite frankly, question one will stump them.
Not only are there folks posting this nonsense on the internet but they are taking money from unsuspecting companies. Before you pay someone to conduct RDT&E for your company ask them what qualifies them to do this. There’s an old saying in SOCOM, “Don’t confuse enthusiasm with capability.” There are tons of former end-users out there but DoD’s T&E community isn’t very big. Verify credentials.