I've decided to create a list of rules to follow when building a test system. Now I've created an informal checklist over the years of what I do when I put together a new system, but it's time to codify the list. Furthermore, the items on the list are generic and not specific to any one industry. Maybe I'll create a separate rules list someday for actively running a test system, but not today.
Know what you are testing, or work closely with someone who does.
A test system built without knowledge of what it will test and how it will test will not work.
Eventually a technician will need to know how it works, unless you want to run the station yourself forever.
Anything that can go wrong will. It's really just applied statistics. So plan for that when building the system.
Create PM plans and schedules.
Most people think of PM (preventive maintenance) in terms of manufacturing systems: you should have a schedule for when parts need to be oiled, when accelerometers have to be recalibrated, when to clean off gear assemblies. But test systems, especially high volume ones, need this maintenance as well.
Run it with actual parts.
NEVER proclaim the system is ready to go before you have ran it through its paces thoroughly with actual components. I cannot stress this enough.
There's my list. Please comment or email me if you have suggestions of your own. I'll probably update it in a month.