Sustaining engineering

When you ask about the duties of a test engineer, the answers fall into one of two categories: creating test stations and maintaining test stations. Many times these duties will be handled by the same person/group - sort of a "you built it, you own it" theory. I've been in several of those situations before. I have also worked for places where the test system was conceived in the R&D group and then turned over to production. Production modified it and maintained it. In that case the person maintaining the system was a sustaining engineer.

I found a Dept. of Defense definition of sustaining engineering as follows:
The technical effort to support an in-service system in its operational environment. This effort spans those technical tasks (engineering and logistics investigations and analyses) to ensure continued operation and maintenance of a system with managed (i.e. known) risk.


I would consider this to be a fairly accurate summary. So, the point of my post is "how much of this is test engineering and how much is it manufacturing engineering?" The sustaining engineer is not creating a test system, but he is responsible for a system's upkeep, making improvements if needed, and monitoring/analyzing the data it creates. These duties are directly related to the test system. But in theory the test system should have been engineering well enough so that it can be turned over to manufacturing without a lot of care and feeding. I mean, Agilent may build a $200k test system that they sell, but they do not sell an engineer with that system.

Personally, I think the answer is that the sustaining engineer is part of the test department (if the dept is large enough to handle that subdivision of labor). He may have had a hand in the creation of the system, and now he helps run it. Ideally, he still has time to work on other projects because the test station has few problems.

Of course, if your test group primarily does R&D work, then all bets are off. Any test system you build will probably be used a few dozen times and then scrapped in favor of something new...

Test group interns, part 3 of 3

In the past two posts I've discussed two different interns (Bill and John) I've worked with in test departments. One performed well and the other...not so much. Some of the lessons drawn from those experiences might apply to any intern, but I maintain that interning within a test group is a very specific experience that has specific requirements. Proceeding from this supposition, here are three things I think you need to do for an intern to succeed in a test environment.

--------------------------------

Hire an intern with broad technical interests. Building and running test stations requires the engineer to be something of a jack of all trades. If the station is automated, you need to know software. You may be called upon to build some fixturing, so you may need to know a bit of mechanical engineering. If you need to debug a circuit, then an electronics background is handy. This can be fun stuff, but only if you _like_ doing different things. All Bill liked to do was program. He did that well, but not much else.


Scope the assignments. There are no courses that teach how to test - it is a very specific skill that is learned from experience. Bill often had little direction from the manager (who was busy looking for a new job at that time) and he floundered. I gave John detailed instructions, and he flourished.


Play to his strengths. This is a corollary to the first point. The intern may have interests in different fields, but he is still getting his degree in a specific discipline. Bill did a great job with the project that involved a lot of software. Since John is a EE major, I've given him tasks that minimize his programming. Choose projects that play to what the intern can reasonably do.

Test group interns, part 2 of 3

I've had some experience with using interns in test groups over the past eight years. What I want to do is present two cases - one that worked poorly and one that is working well - and then draw a few test-group-specific conclusions.

--------------------------------

This summer we've had two interns in my test group. There are several good engineering schools here in the New England area, and they run programs that send students out for an entire semester. The firms get good yet inexpensive help and the students get college credit. We had an intern in our group last year and he worked out well.

One of our interns (call him John) was tasked to work with me on a couple of different projects I have. John is an EE major who has interned with two other companies in the area (with a good track record) and has excellent grades in school. He's a nice, easy-going guy, fairly responsible for his age, and very eager to work with some serious technology.

I have two major projects I am managing. Both projects involve wafer-level testing, although the second is more software-oriented than the first. The first thing I did was make a general plan of what I'd like him to do. First he needed to learn the basics of the test stations and then get some practice time in. Since most of the first project can be viewed as separate mini-projects, I decided that I could parcel out some of that work to John.

I forwarded a collection of documents - work instructions and a couple of presentations I had made for the stations - for him to read. I spent time showing him how the testing worked. Before I went on vacation for a week I asked a production engineer to help him do some testing by himself, and I forwarded detailed instructions of some work he could do.

When I got back from vacation I was pleasantly surprised to see that he had spent most of his spare time in the clean room working on the tasks I had assigned. He had not completed them, but he showed me how far he had gotten and had clear plans for what remained. Since that time I have assigned him other small projects with clear instructions, and he has completed them successfully.


Part 1 of 3 talked about a unsuccessful intern.
Part 3 of 3 will draw a few conclusions.

Test group interns, part 1 of 3

I've had some experience with using interns in test groups over the past eight years. What I want to do is present two cases - one that worked poorly and one that is working well - and then draw a few test-group-specific conclusions.

--------------------------------

In the late 1990s Hewlett Packard had a program (they probably still do) called SEED: Student Employment and Educational Development. Every summer the company hired standout undergraduate students as summer interns. In my division it was a very organized program, the students hired were bright and articulate, and several of the people I worked with were former SEEDs themselves.

One summer my test group retained a SEED intern - let's call him Bill. Bill was a smart guy pursuing a BS in CS at a great school. He had a good deal of experience with designing web sites and JAVA programming (which had only been released a couple of years before). He even did some volunteer work for Red Hat debugging Linux - they gave him stock options which paid out very well when Red Hat went public in 1999. In other words, he was a very clever computer geek.

Bill did very well with one project he was assigned. The tasks were a) write a JAVA program that interfaced with a spectrum analyzer, b) repeadedly run the analyzer through a series of rigorous tasks, c) track the free memory of the analyzer over a long period of time, and d) find any memory leaks and what triggers them. He wrote the program, ran it, and found some important problems.

Bill liked programming, but he wasn't interested in data analysis, hardware (EE stuff) or miscellaneous paperwork. When he had to do those tasks he performed poorly. For example, when he was asked to analyze some data from a series of tests, I spent a lot of time walking him through using advanced features in Excel. If I wasn't there showing him what to do and how, he would tend to chat online with friends and play games. He needed a lot of supervision and guidance.

To be fair, one of Bill's problems was a lack of planning on the manager's part. In my opinion, he didn't always assign tasks that played to Bill's strengths, and sometimes the goals and steps towards those goals were vague. But at the end of the day it was Bill's responsibility to try and figure out things he didn't understand, and to actively seek out help when he needed it (instead of spending the day playing Diablo).


Part 2 of 3 will talk about a successful intern.
Part 3 of 3 will draw a few conclusions.

LabVIEW programming style

So I have to give props to Peter Blume for his book The LabVIEW Style Book. Yes, it repeats a list of best practices for LV programming from different sources. Yes, it is grayscale, when color print would have been much more useful. Yes, it is self-serving: he runs a software consulting firm that is mentioned frequently in the book.

Aside from those gripes, this book is great resource for three specific reasons:

1. It covers specific aspects of LV programming chapter by chapter. These divisions are clear, logical, and discussed in detail.
2. He uses copious examples.
3. He devotes much of chapter 10 to the VI Analyzer, which I have found to be an incredibly useful tool. For some reason NI never pushed this tool, but the book's popularization of it may go some way towards improving the overall standard of LV code.

In fact, I liked the book so much that I started a study group with the other engineers in my group. We cover one chapter per month - we've covered four so far (10 chapters total).


I should post a disclaimer here as well. Back about 4-5 years ago I interviewed with his company and almost started work there. Then I got wind of how little money they would offer and bailed. I was disappointed with the firm for a while, but I still like this book.

Future posts

So I've been posting to this blog for a couple of months now, and I've actually posted all of the things that I wrote sometime in the past 6 months. Of course I still have more topics I want to write, and a couple of other people have suggested ideas. So here's a list of things I'll blog about sometime in the next month or three:

• Test engineer salaries
• Career path for engineers
• Useful background/experience for test engineering
• Typical issues faced by a test engineer
• Sustaining engineering vs. test engineering
• Falling in love with the test system
• Interns
• Outsourcing test
• Labview style
  

This list is as much for me as for anyone reading this. I have a long text file where I've been writing down ideas - this was a good way for me to sift through that list, filter out what I've already written about, throw away bad ideas, and summarize what I had left. Furthermore, all of these items on the list are of current importance to me because I've dealt with them in the past year, or will be soon.

So, take it for what it is worth.

New toy: LabVIEW 8.5

So I downloaded a new toy a couple of days ago - I just couldn't wait for my CDs to come through the mail in a couple of weeks - but with all the things going on at work, I haven't had a lot of time to play with it. Hopefully that will come this weekend. But there are a few things that I noted about it so far:

• NI finally added conditional for-loop exiting. They've needed this for a while. After all, an endless loop is one of the classic software bugs, and in LV the while loop is the prime contributer to this problem.
• They've changed shared re-entrant VIs some. I haven't looked at this in much detail yet, but I've always been a little ambivalent about the re-entrant status of VIs in general. I know there are developers who rely on that feature all the time to use a subVI as a sort of variable for holding data. To me that seems like a bit of a crutch and asking for complicated bugs. I'll play with the new changes to see if it has changed my mind any.
• NI is anal enough to change the names of VIs because they mispelled "zeros" - previous versions of LV include a VI called Find All Zeroes of f(x). I got a real chuckle out of that.
  

Of course, NI almost always releases a new version of LV right around the beginning of August to coincide with NI Week down in Austin, TX. I went last year but was on vacation this year. So I wasn't really surprised the new version came out. But it's fun to have a new toy nevertheless.

Feature Space for software testing

I haven't written a post about software testing in a while, and this is something I had thought about in the past.

When I did software testing for HP, one group of errors I specifically tried to account for were corner cases. The common thinking was that it was always relatively easy to find many bugs - they were in the middle, so to speak. But product quality was always high in mind at HP, so we wanted to dig out odd little bugs that 'hid in the corners.' Besides that, an expensive piece of test equipment, especially one in a lab, can be used in all sorts of odd ways over the course of its life. And the definition of a corner case is a situation that happens under extreme conditions.

After a while I started thinking about extending this idea. What if you referred to a corner case in terms of how the system is used in general, instead of adjusting specific parameters. If the usage of a program, or software embedded in test equipment, or even a test station, is a three-dimensional space then what are the dimensions? To frame it as a physics question, what are the degrees of freedom?

One dimension would have to be the number of functions - all the distinct things that a user can get the software to do. If you use just a few of the software's features (typically the most common ones), then you are not likely to find a corner case bug. The most common features are usually thoroughly vetted by the design team. But if you use a lot of those features, in different orders, then you may find something bad.

The second would be time. That is, the more a program is actively running (not just idling), the more likely it is to encounter a problem. It may be a memory leak, but it could be some other problem as well.

The third and final degree of freedom is the number of users of the system. I've written programs that were buggy, but that was fine because I was the only one using it. I ran it a certain way, to do certain things, and it worked just fine. But if I gave that program to someone else, sure enough they would crash it. This can be generalized to, "The number of ways in which a program can be used and abused increases proportional to the number of users of that program." And this applies to inanimate users as well. If a program can be accessed and used by some other application, then that program is itself a user.


So, lay these three degrees of freedom on orthogonal axes and you have a Feature Space Cube (FSC). Down by the origin you won't find many interesting bugs. The farther out you go, the more interesting things can get. Think about the ramifications for a while, and the concept can be a useful way to picture a system, not to mention entertaining in a geeky sort of way.