Test Executives - part 2

A couple of weeks ago I started writing about test executive software.  I've decided that this topic is another three-parter, so in this 2nd post I'll write about off the shelf (OTS) software.  The third post will cover in-house test executives.

I mentioned last time that I have my own definition of a test executive software, but I never wrote that definition.  Well, for me the software is defined by what I expect it to do.  The three things I expect at a minimum are:

1. Configurable testing.  I need the capability to switch up the order in which specific parameters are measured and under what conditions they are measured.  This is typically done with scripting or configurable sequence files.
2. A solid, usable GUI.  A test executive is often used by a test technician instead of an engineer, so you need software that doesn't require a lot of care and feeding.
3. Easy to see what's happening.  This is somewhat related to #2 above but more specific.  I like graphs, reports, and charts.

The OTS platforms I have experience with are TestStand and ATEasy.  I wrote about my experiences with TestStand a year ago (a three parter, starting here and continuing with this and that).  To be honest, I'm a lot more nebulous on ATEasy and my knowledge of it consists of three data points:

• I sat through a long seminar on it once.
• I worked with a guy who had used it before and liked it.
• I evaluated it for about a week at a previous company.

Given that limited knowledge, I think it would serve as a cheap, limited alternative to TestStand.


On a final note, NI published a checklist for evaluating test executives.  While the list is a little self-serving and obviously slanted towards NI software, it's worth consideration as a starting point.