Misleading specifications

I've been learning more about solar cell testing over the past year.  I have a couple of test systems I'm developing, I've taken data, and it's been interesting.  It's also been frustrating.

As an example let's consider I-V sweeping, the bread and butter of semiconductor testing.  Most of the basic values of a solar cell can be determined from a quick IV check.  One of those values, the short circuit current (I_sc), helps determine the maximum power a cell could theoretically produce.  A high I_sc is nice to have.

When I first looked at measuring IVs for solar cells last year, I tried to re-purpose my National Instruments PXI chassis to this task.  I knew that the source measure unit (SMU) in the chassis (PXI-4130) could only go up to about 1A, but there was an auxiliary supply that could plug right in to the SMU: the PXI-4130 Power SMU.  On the NI page for the aux the output specs are listed at "12 VDC, up to 5 A, up to 60 W, 0 to 55 °C" (note the "5A" limit).  So I bought the aux supply and got started testing.

But as I tested larger and better cells, with more total current output, I ran into limitations with the NI approach.  I couldn't measure an I_sc above 2 amps.  This confused me somewhat, since I had seen the the five amp limit on the NI page.  Adding insult to injury, I didn't even need the SMU to output that current - the solar cell was generating the current.  

I then considered connecting multiple SMUs.  There is a "knowledge base" article on the NI website that supposedly answers the question, "What is the Maximum Voltage and Current that the NI DC Power Supplies can Source when Cascading Outputs?"  But nowhere on that page does it list a maximum current.  I even found a Photovoltaic Solar Cell I-V Characterization Bundle page that says you can "add more SMUs to get three times the voltage or current."  I struggled with this problem for a month.  

Well, I finally got an answer from NI last week.  The SMU cannot measure above 2A current because the PXI chassis has to dissipate the power, regardless of where the current source is.  The chassis has "a hard limit on the amount of power consumed by each of the SMUs."  In other words, National Instruments has some cleaning to do on their website.

------------------------

So what's the moral of this rant?  First, I'm going to get a different power supply (maybe Agilent).  Second, you cannot always trust the hardware verbage you read online.

LabVIEW patent digging

I started down this path when I read an article about DASYLab in the April issue of Evaluation Engineering.  That triggered something in my memory about Measurement Computing, the company that published DASYLab.  After doing a little digging, I thought I confirmed what I remembered: National Instruments bought them out many years ago (1998).  But then I found a different article from 2003 stating that SoftWIRE, a subsidiary of MC, had patents it had acquired from Fluke that predate the NI patents for LV.  Hmm.  Finally, I found an article on Bloomberg that states, "as of April 29, 2005, Measurement Computing Corporation operates as a subsidiary of National Instruments Corporation."

But doing all that digging started me thinking about NI's patents.  Dataflow programming was first proposed way back in 1966 by Bert Sutherland, so NI couldn't patent that.  Although, if you do a patent search on, say, NI and programming you'll find hundreds of patents (or look here).  To my knowledge, patents can expire in as soon as twenty years.  LabVIEW was first introduced back in 1986.  Doing more digging, I found yet more nuggets:

• According to this news release, one of NI's key LabVIEW patents expires in April of next year.
• NI spent significant time and money defending a patent against The Mathworks (the MatLAB) people in 2002.
• There was a very interesting conversation on LAVA back in 2004 regarding the SoftWIRE v. NI lawsuit.
• Right around the turn of the century NI was in a LOT of activity: licensing patents and buying companies to resolve patent disputes (ex: here, here, over there, and out thatta way).

So where does this trip down the rabbit hole lead?  Here's what I saw:

1. Many of the original LabVIEW patents are getting long in the tooth and will start to expire as soon as next year, if they haven't already.
2. NI has no problems litigating patent infringement.
3. NI will buy companies to protect patents.

Based on that, here's my prediction: history will repeat itself.  In the next few years there will be at least one software company that develops a software package that competes on the cheap with LabVIEW (which currently runs several thousand dollars per license).  They'll be able to do this because of those expiring patents.  Heck, they may even write their compiler so that it can use subVIs written for LabVIEW.  After this happens, NI will sue them, force them out of business, or buy them.  I imagine they'll buy them out - I doubt they'd want the price point on LabVIEW to drop at all.

It'll be interesting to see how it plays out.

New test magazine

Well, I said in my last post that I was going on vacation. But this post was already mostly written, so what the heck.

======================

Keithley is sponsoring a new magazine called Project Test. According to the press release, it is a "unique 32-page electronic magazine edited for test and measurement engineers." In my opinion, the jury is still out. It's obviously meant as a marketing tool - most of the articles in the first issue are written by Keithley employees and spotlight Keithley equipment. I'm not sure if that fits the press release though.

There's nothing inherently wrong with this sort of self-marking document, if it's done right. Hewlett Packard did a good job at this with their Journal this back before the split. The articles were usually written by HP engineers (sometimes in conjunction with marketing), and they often conveyed a great deal of solid technical detail. That is the standard I would hold Keithley to.

The Power of Complaining

Complain (Verb) - To express feelings of pain, dissatisfaction, or resentment


About three years ago I went to the yearly National Instruments Technical Symposium. Held every fall in Massachusettes, it is a mix of companies selling things (roughly 20 booths), programmers getting in touch with each other, and NI showcasing the newest updates to LabVIEW that they promoted at NI Week the previous August.

Well, three years ago I was disgusted with the state of the symposium's presentations. Without fail, all the presentations had a high percentage of marketing and a low percentage of actual technical content. And the technical content seemed to be pitched at either a) a beginner's level or b) extolling the great new things that had been added to LabVIEW (in other words, more marketing).

Most of the seminars and conventions I've attended over the years have a comment/rating sheet where you can grade your experience. Usually I check off a few things, write one or two sentences on what I liked, and that's it. This time, I roasted them. I wrote what I really thought of the day's events, and it wasn't pretty. I went into graphical detail of each talk I attended and why I felt it sucked. I also wrote that other people I had talked with had a similar opinion.

It must've hit a nerve. I received a call from a NI marketing guy in Austin a couple of weeks later. He wanted to talk in more detail about what I disliked (his wording - mine was stronger) and felt should've been done differently. The next time I talked with the local NI rep he mentioned that he had heard about my comments.

Well, over the last couple of years the LabVIEW symposiums I attended definitely had more technical content. A couple of weeks ago I attended the LabVIEW Developer Education Seminar. It's similar in spirit to the technical symposium but without the booths. And I have to say that this time NI did a great job of presenting good technical content. Every seminar I attended had solid information that I can use. Even better, there are actual notes with the presentation materials. I may have to keep that booklet.

So sometimes it pays to complain.

Linux on test systems, pt 5

In July of 2007 I started reviewing app notes that Agilent published about using Linux on test systems. They've put out 5 papers on the subject (full list is here). This blog reviews #5, the last paper in the series.



Tips for Optimizing Test System Performance in Linux Soft Real-Time Applications


One of the first things the paper does is discuss the difference between soft real time applications and hard real time applications. To be honest, I didn't realize there was a noticeable distinction. But I found it referenced in Wikipedia, so it must be real... My experience with real time systems has been of both varieties, but I never quantized the difference. So, now I know something new.

One of the things I liked is the list of tips for optimizing response times. These tips aren't really specifically linked to Linux, and they seem obvious but sometimes it's good to see those "obvious" ideas listed.

■ Avoid it if you can
■ Put the burden of real-time control on your instruments
■ Use a fast PC with plenty of memory
■ Shut down unused services
■ Isolate the real-time part of your application

The author then goes on to discuss specific Linux techniques like time slices, the Linux scheduler, preemptive multitasking, and virtual memory & paging. Each of these discussions are paired with code, diagrams and graphs that dive into some technical details. Most of those details were admittedly beyond my skill level - I'm not a Linux guy - but the paper was written well enough for me to understand the subject matter.

----------------------

This is the last paper in the series, so I don't expect to blog much more on the subject, at least not until I install Linux on that computer at home. I'm still working heavy at my new job, so it may be another couple of weeks before my next post.

Linux on test systems, pt 4

In July of 2007 I started reviewing app notes that Agilent published about using Linux on test systems. They've put out 5 papers on the subject (full list is here). This blog reviews #4.


Using Linux to Control USB Instruments
As I have written before, I sometimes take a dim view of these white papers. It seems that their actual value is proportional to how much input comes from marketing - I just haven't determined if that proportion is direct, quadratic, or exponential. Maybe it depends on the company itself.

Having said that, when I started reading this latest paper I learned about the "USB Test and Measurement Class (USBTMC) specification," which I didn't know about prior to this. Any white paper that actually teaches me something must have something going for it. Plus the author provides sample code for create a generic USB driver that works with current Linux distributions - even better.

I should point out I haven't had a chance to test out this code. I have an older computer at home I am converting over to a Linux box, so I plan to do it at that time. But aside from that caveat, I think this paper was pretty well written.

----------------------

The remaining paper in the series is "Tips for Optimizing Test System Performance in Linux Soft Real-Time Applications." I'll review this one next week.

Linux on test systems, note

In July of 2007 I started reviewing app notes that Agilent published about using Linux on test systems. I reviewed the third of five back in November. In the past couple of months, while I was busy ending one job and starting another, Agilent went and published numbers four and five. So I'll review #4 this week and #5 in another week or two.

The full list of papers is here.

Vendor books about testing - National Instruments

Back in October I posted about recent test system manuals Keithley and Agilent had written. In a post last month I mentioned that NI had also put out a manual that I would eventually review as well. Here it is.


First Thoughts
NI is very good at marketing. They interact well with customers, get knowledgeable sales people embedded with key industries, and support their hardware and software. So when I say they excel at marketing it is truly meant as a compliment. Yet this proficiency also hurts them. Read on and you'll see.


Sections
There are four sections and 14 chapters divided amongst the sections. The first is just an introduction, the second discusses test system guidelines, the third goes over improving a system, and the last one consists of case studies.

Section 1
This single chapter reads more like a position paper for NI being the best ever than an introduction to a test system guide. Pity. For example, on just a single page (1-5) the author referenced three different marketing white papers. My hopes for the manual diminished.

Section 2
There were two saving graces to this section. Chapter five has a good overview of different buses, and chapter seven reviews the PXI standard. Otherwise it is more marketing than substance.

Section 3
These three chapters were somewhat of a revelation. The marketing was minimized in favor of looking at 1) ways to speed up a test, 2) measurement accuracy, and 3) system longevity. Cool.

Section 4
I liked the first chapter in the manual. Describing software-defined radio testing, it was short & too the point. But the other three case studies were all but useless. Okay, so Microsoft used LV and a PXI chassis to test the XBox - why not spend a few pages and describe the test architecture or obstacles that were overcome in the design. Each case study reads like an extended press release.


Summary
Unfortunately, this testing manual is more like the Agilent manual (bad) than the Keithley manual (good). It pushes a theme of "NI products are the best thing since sliced bread." The only time it mentions Agilent is to take them to task for the lack of support of IEEE1394 (VEE isn't mentioned at all). The manual could have used a good editor - the exact same graph, bandwidth as a function of latency, shows up an improbable FIVE times under different titles.

In other words, if it wasn't for section 3 I would write off the whole manual as a waste of space on my hard drive.

Linux on test systems, pt 3

The third paper in a new series of Agilent white papers on using Linux in test systems has just been released: "Using Linux to Control LXI Instruments Through TCP." As has become custom, here is my review.

The previous paper in this series discussed using Linux to control LXI via VXI-11. While that paper gave me the impression that this was the best way to control instruments, the new paper says that TCP (via direct socket connection) is better for short time measurements.

The author gives a very brief overview of the seven layers of TCP/IP and then dives right in to gritty details (including a quick discussion of Nagle's Algorithm). The paper provides several extensive code examples. The examples are in C, but that could be ported easily to LabWindows, or you could wrap it up as a separate object to use in LabVIEW.

I liked this paper better than the last one. To use a Thanksgiving metaphor, there was less marketing feathering and more engineering meat on the bones of the paper. I would really recommend this as a useful paper to read if you were looking at using Linux and LXI, and now I'm feeling more optimistic about the remaining papers.

----------------------

The remaining papers in the series are "Using Linux to Control USB Instruments" and "Using Linux in Soft Real-Time Applications". I'll be reviewing those as they are released. Since these papers have been released just about once per month, I expect to see the next one sometime around the end of the year.

CMMI for testing

There was an article in the September issue of Evaluation Engineering about CMMI ("Capability Maturity Model Integration"). I flagged it for future reading and just had a chance to finish it today.

I flagged this article because I have experience with the CMMI. The division I worked in at HP/Agilent years ago was classed at CMM level 2, and I worked in a couple of projects aimed at moving the department to level 3. I called it "CMM" instead of "CMMI" because back then the older nomenclature was in use. Working in a project group that adhered to those standards, which was very enjoyable and a great learning experience (we used Rational Rose for the heavy lifting, before it was bought by IBM). Testing, and specifically software testing, has a very specific role to fill within such models, and it's significance is not underrated.

In general the article is a cogent overview of the CMMI and how it is applied. It also makes a good point that test engineers involved in creating software - especially for more complicated projects involving multiple people - should learn how to apply the model and use tools associated with it. Many test engineers for hardware testing do NOT have a software background, and don't necessarily have exposure to best practices for programming. But believe me, the CMMI is worth using.

Of course, the author is from NI so I expected some marketing and was not disappointed. The author discussed how NI Requirements Gateway can be used to implement the CMMI, and he also referenced NI programs like LabVIEW and TestStand extensively. But this didn't really bother me - he works for NI and that's his job. Evaluation Engineering has free access, so I expect a modest amount of bias.

No, what really bugged me is that right at the beginning of the article he called the CMMI "Component maturity model integration" instead of "Capability maturity model integration." If you're going to write about something, please get the acronym right. In the engineering world there are way too many acronyms and abbreviations, and doing something like this confuses the issue further.

Vendor books about testing - marketing

Yesterday I posted my review of an Agilent guide to test systems. Eric, who works at National Instruments and runs The Automated Test Blog, added a comment about a test systems book that NI has here. So, I downloaded it and skimmed it quickly. I'll probably review that one as well for completeness sake (thanks for the heads up, Eric).

Of course, originally I wanted to compare the books from Agilent and Keithley to see if they reflected a difference between the two companies themselves: Agilent is much more of a marketing behemoth than it was as HP many years ago. To be honest, I have a bias. I worked in the Test & Measurement group at HP for a few years before and after the switch to Agilent, and I saw firsthand the large amount of resources that went into marketing. But that is a post for another day.


I must tread lightly with this sort of thing. I've had a few marketing/salespeople contact me about products they make. Maybe they want to sell me their products, look for free advertisement on my blog, or just honestly offer information. It could be a blend of those reasons. But I'm an end user of test equipment nowadays, and no one pays me to do this blog. From an ethical point of view I should treat all requests equally. That is, only talk about things I experience, not show unwarranted bias towards one vendor or another, and not lambast someone or something without reason.

Or at least I'll try.

Vendor books about testing - Agilent

About a month ago I talked about testing handbooks recently published by Agilent and Keithley. On October 15th I reviewed the Keithley book. Now I'm going to review the Agilent book, "Test-System Development Guide: A Comprehensive Handbook for Test Engineers," which is available here.


First Thoughts
This book, released in May 2007, is partly a repackaging of other white papers, many of which you can find here. I suspect that it was repackaged like this to compile what had been written separately as well as to heavily promote the LXI interface. I previously posted about Agilent's big stake in LXI, so I won't get into that again.

Other than the marketing-oriented aspects, I found the guide to be somewhat useful.


Sections
There are 4 main sections of this handbook - each section has numerous subsections. The first discusses test system design. The second section covers LAN networking issues. The third is devoted to LXI. The fourth and final section lists some details of RF/Microwave testing.

Section 1 - Test System Design
This section is devoted to going over the various aspects and theory of a test system. Parts of it I found insulting (it appeared aimed at a pure beginner), some of the things they talk about I have posted about in my rules for building systems (here and here), and some of it was actually pretty good.

For about 15 pages the guide discusses software architecture: defining the requirements, controlling instruments, storing data. It's all very general, but I found it extremely funny that whenever they mentioned LabVIEW, their competing product Agilent VEE was written first.

Section 2 - Networking Choices
Here the guide covers networking considerations for a test system. This might be a bit of overkill for some people, since it is aimed for the test engineer who knows very little about networking basics.

Section 3 - LXI: The Future of Test
Yes, that was the actual title of this section. Somewhat presumptuous, and very much market-speak, but that is what the section is called.

Section 4 - RF/Microwave Test Systems
I have no real experience with this kind of testing, so I cannot speak to it's accuracy or whether it was worthwhile or not. To be honest, I skimmed this section.

Summary
When you compare this book to the Keithley book, you can see that that they have two completely different intents. The Agilent guide is polished, views testing from a general point of view, and serves as a vehicle for pushing LXI. The Keithley guide is not so polished, goes over the guts of testing (i.e. - the many pages devoted to discussing passive and active components), and includes numerous examples.

In short, the Agilent book is written for a manager, VP, or someone looking for more information about testing. The Keithley book is written for the engineer. If you're a test engineer, I would recommend reading both of them, file away the Agilent book, and put the Keithley book on the shelf for frequent referencing.

There is a book about test engineering that is supposedly a college-level intro coursebook. Maybe I'll take a look at it for comparison. There is also a free handbook on LXI interfaces available. I haven't looked at it yet myself, but it may be worthwhile.

Linux on test systems, pt 2

Back in mid-July I talked about a new series of Agilent white papers on using Linux in test systems. Well, the second paper in the series, "Using Linux to Control LXI Instruments through VXI-11," has just come out.

The paper begins by defining VXI-11: the GPIB equivalent for controlling instruments via Ethernet. It was added by the VXI Alliance in 2000. The other method the VXI group added, direct TCP socket communications, is a lower-level protocol. This paper maintains that VXI-11 is better for most cases.

It then proceeds to talk about Remote Procedure Calls (RPC). VXI-11 is based on RPC, so Linux will directly support VXI-11 (no $500 GPIB cards or expensive cabling required). To use RPC, Agilent promotes using the rpcgen code generator. They supply several different code examples using generated code.


In general, the white paper was organized, the author knew the subject, and the narrative flowed well from beginning to end. But this paper was not a Linux paper. Other than stating that you can use RPC in Linux (obvious), the paper is really just about VXI-11. To cap it off, the name "Linux" is only mentioned FOUR times in the text of the paper.

The paper is really a veiled push for communicating via VXI-11 regardless of the operating system. But as I stated in my original post for this series: white papers tend to be self serving. They are usually generated by the marketing department. And Agilent certainly has a big axe to grind for using Ethernet to access test equipment. All the big names in test equipment are members of the LXI consortium (LAN eXtensions for Instrumentation), but Agilent was an early proponent of the standard. Also, they were the first company to have LXI-certified equipment.

----------------------

The remaining papers in the series are "Using Linux to Control LXI Instruments through TCP Communication", "Using Linux to Control USB Instruments", and "Using Linux in Soft Real-Time Applications". I'll be reviewing those as they are released. Hopefully the remaining papers will have more substance relative to Linux. Unfortunately, I now harbor doubts.

Vendor books about testing - Keithley

A couple of weeks ago I talked about testing handbooks recently published by Agilent and Keithley. The following is a brief summary of the Keithley book, "Understanding New Developments in Data Acquistion, Measurement, and Control." (http://www.keithley.com/news/prod031407)

First Thoughts
This book is called a "first edition," although portions felt like they were written some time ago and pasted into this new book. Also, I couldn't find a PDF copy of this available online, which is very retro. Finally, there is no summary at the beginning of book. There isn't even a page listing when & where it was published. Clearly the book did NOT come from the marketing department.

Sections
The book has 9 sections and three appendices. Only the ninth section is listed as "examples," but sections 6,7, and 8 are all about different applications as well: temperature, strain, and current measurements.

Hardware
Sections 1 & 2 discuss hardware concerns when building or upgrading a test station. They include mentions of Keithley hardware, but they also cover processors, bus architectures, and networking.

Software
Section 3 discussed software. There was some marketing influence here - several pages were devoted to talking about the Keihtley script programming tool - but they also devoted time covering open source issues, which I think is commendable. They also talked about IVI software. They are a sponsor member, but so are Agilent, NI, Tektronix, and Rohde & Schwarz.

More Hardware
In sections 4 & 5 they examined electronic components (from resistors to op-amps) and how they might relate to test engineering concerns. Very basic, EE stuff, but good to go over as a refresher. Section 6 covered DAQ in some detail, including ground loops (which have bitten me on at least one occasion). As previously mentioned, the remaining sections go over some details in measuring temperature (which I've had to do), strain (which I don't do), and a few other applications.

Summary
This book felt like more of an introductory survey than anything else. It didn't delve deeply into any single topic, yet it presented an overview of a variety of topics and mentioned things that warrant further research. They also had a large variety of application examples. I liked it. It was fairly straightforward, with a minimum of marketing fluff, and was aimed at test engineers.


My next post on this topic will be about the Agilent book.

Vendor books about testing

In the past month I've come across two different manuals about building test systems They are both from big companies in the T&M industry, Keithley and Agilent. They both came out within the past few months. The page count is north of 200 for each book. They both appear to have useful content, in and around the marketing stuff.

That last item is important to me. Sometimes test companies can give you plenty of useful information in books like these - their engineers built the test equipment, and they've done plenty of research on how to use it for testing. But sometimes there's so much marketing fluff that it's hard to separate the useful from the dubious, so I end up ignoring the whole thing.


So, I've decided that I'm going to read through these books and review them. Hopefully I'll learn something, and I might as well pass my opinions of the books to anyone else interested in them. I'll post the first review (probably the Keithley book) next week.

The names of the books (and links to them) are:
Keithley
"Understanding New Developments in Data Acquistion, Measurement, and Control"
http://www.keithley.com/news/prod031407

Agilent
"Test-System Development Guide: A Comprehensive Handbook for Test Engineers"
PDF File
http://cp.literature.agilent.com/litweb/pdf/5989-5367EN.pdf
Hard copy http://www.home.agilent.com/agilent/editorial.jspx?action=download&cc=US&lc=eng&ckey=1244104&nid=-536900530.0.00&id=1244104&cmpid=20580

Autotestcon 2007

I worked for a company for about 3.5 years that build subsystems for aerospace applications. It was about half military and half civilian. Autotestcon claims to be "the United States’ largest conference focused on automatic test systems for US military systems." I never knew anyone who went to it, but that job was over a decade ago, so I'm dated.

Anyway, this convention is happening right now & lasts until tomorrow. It's at the convention center in Baltimore's Inner Harbor (a pretty nice location). I scrolled through the list of technical sessions and was impressed by the list. I have been to trade shows where there was a lot more marketing than actual learning, but the signal to noise ratio appears to be higher for this event.

So, if there's anyone reading this that went to this show this year, or has gone in past years, let me know what you thought of the show. I'll probably write an entry sometime down the road about trade shows for testing, and I'd appreciate the input.

From Test To Sales

The career of Field Applications Engineer has its own entry on Wikipedia. Also known as an applications engineer or sales engineer, they are usually a liaison between the customer and engineering. They must have a technical background and a good understanding of the product, but they must also be able to communicate well with the customer. After all, they are part of the sales department.

I was a field apps engineer for two years, and I did it part time for a year with a different company. Furthermore, I've met several other apps engineers who transitioned from test engineering. Granted this is purely anecdotal evidence, but is there a good career path from one to the other?

Let's look at why a test engineer might do well in this position

• He has a solid technical background.
• If the company has multiple product lines, he probably has written tests or helped to test those products, so he has a breadth of knowledge.
• A test engineer who has seen various product failures can help customers who may have similar problems.
  
• A test engineer has seen the negatives of the product (i.e. - failures) but is still focused on the product (making it work right by correcting failures, or at least weeding out bad products). With this attitude a field applications engineer can build a layer of trust with the customer while at the same time help to sell him on the product.
  

But in the end it all still depends on the person.

Test it until it works

After graduate school, I worked for an aerospace subcontractor firm as my first real job. The company’s products were split 50/50 between the military and commercial fields. On my first manufacturing project I started to run behind on shipments because the yield was slipping. When that happened, I was told by more seasoned engineers (who had initially started this project) to go through the "marginally failed" units and "test them until they work."

The rationale behind this statement was that:

1. The spec was extraordinarily tight for the product (blame was placed on sales & marketing).
2. We were up against the accuracy limits of the system.
3. It didn't really matter if the positioning of the cannon was off by a couple arc-seconds. They had redundant systems in place.

Thinking that this is how it must be done in the "real world," I did what he told me to do and got back on schedule. Granted, I figured out some other things to do to correct the yield, but going through the marginal failures was a contributing factor. But it always bothered me.

Stepping aside from the questionable moral grounds of this situation, let's look at that rationale list from a test engineer's perspective.

1. Spec too tight. Marketing should certainly know to what tolerance the product can be tested. If they don't, then it is the job of test engineering to inform them. If marketing plays the word game of "it is guaranteed by design" then it should not need to be tested, now should it?
   Of course, if marketing knows the limits and chooses to ignore them, then you have much bigger problems...
   
2. Limited test accuracy. If you are trying to test to a spec that is at the limit of what you can measure, then you have serious problems. Buy a more accurate tester, build one if you can't buy it, or do sufficient test system qualification to verify your accuracy. You have no business being anywhere near those limits. Test equipment manufacturers themselves can play "specsmanship" games, so you cannot always trust their numbers.
   
3. The customer doesn't really need that accuracy. I'm sure it's possible that the customer has over-specified what they need. They may have other backup systems in place if the accuracy is not there, they may have an over-tight spec because they don't entirely trust the product (or the company). Or they may just be clueless. But you can't get into the game of second-guessing the customer. That'll get you in deep trouble, somewhere down the line.
   

So, did I really screw up as a test engineer (although I wasn't called a test engineer back then), or was I just doing what I was ordered to do? I think I will just plead 'youthful transgression' and try not to let that happen again.