Testing, Evaluation, and Certification - and why the difference should matter to you. (2024)

If you make decisions based on test reports, evaluations or certifications, and you don't understand the difference, you're at risk.

And you're not alone.

I've spent my career in the conformity assessment industry; I've worked for some of North America's largest testing laboratories, certification bodies, and evaluation centres. I eat, sleep and breathe conformity assessment. I think in quality system terms, and can quote you sections of ISO/IEC 17025, 17020 and 17065 from memory.

But not everyone lives in this world, and I know this because sometimes I have conversations with folks who, through language and context, I can tell don't understand the important differences between a test report, a certification, or an evaluation. These terms have very specific meanings in conformity assessment, and the distinctions have significant implications for anyone who uses these types of assessments to make decisions.

The difference can sometimes mean life or death, it's that important.

I'm looking at you, regulators and authorities.

So what are these 3 types of conformity assessment (as they relate to product approvals)?

1. Test reports - a statement of conformance of a sample ofproduct to a specific property, or set of properties, according to an established method of test.
2. Evaluations - an opinion, usually including some form of judgement, based on some set of evidence, that a product has been determined to meet a specific requirement.
3. Certifications - an attestation that a product, usually bearing a certification mark, has demonstrated conformance to a requirement, and often includes the process by which the product has been produced.

For your consideration, some important differences that may not immediately be apparent:

• Test reports apply only to the specific sample of product which was under test. The results do not necessarily apply to other samples of the same product. If 5,000 objects are produced, and 8 were tested, the other 4,992 objects do not necessarily have the same characteristics.
• Test reports typically include a disclaimer. Many laboratories limit liability for use of their results up to the value of the testing service provided (which may be only a few hundred dollars, depending on the scope.)
• Test reports are only as 'good' as the competence of the issuing lab. Accreditation is a standardized means of having a 3rd party confirm the competence of a lab for specific types of tests (scope of accreditation). Tests results should only be accepted from labs that are accredited for the methods in question.
• Evaluations are opinions. They should be taken as such, and relied on only when the competence of the evaluating organization or person is known. It is important to know and trust the source of an evaluation, as it is their opinion, and based on the set of evidence, they judge to be sufficient and sound.
• Evaluations do not follow a standardized process. They are often issued by professionals who have a license to provide such opinions. By accepting an evaluation, you are accepting the competence of the authoring individual or organization, and whatever recognition of their competence has been established, if any. Depending on the situation there may be a limit to the liability protection afforded by an evaluation.
• Certifications follow an internationally standardized process (ISO/IEC 17065), that includes rules for process, review, and impartiality. Certifications typically include a review of production processes to ensure that all products bearing the certification mark meet the requirements of certification (unlike a test report, which applies only to the specific sample of product tested).
• Certifications can be granted on a batch/lot basis (a set of products from the batch is tested and certification granted to the batch), or on an ongoing process (where the production process is subject to monitoring by the certification body), but in both cases certified products must bear the certification mark.

So how do you know what to use? It's all about the level of risk you can live with.

When making product approval decisions on the basis of these kinds of assessments, it is important to understand the basis of the information you are reviewing. Test reports, evaluations and certifications can all be valuable, sufficient, and acceptable, depending on the situation.

Here are some examples:

• If you're approving the single use of a product that provides a limited function, and life safety risk is low, maybe a test report is sufficient evidence.
• If you're providing a blanket approval for products produced over time, you may want to consider a certification, so that the production process is being monitored.
• If you're reviewing a product which is very innovative, and there is no standardized means of determining its compliance, an evaluation by a professional (whose competence you feel confident in) might be justified.
• If you're determining the life safety risk of a product based on a specific performance characteristic, like fire resistance rating, for example, a certification typically provides the lowest level of exposure.

Like most things in life, you have to decide what is appropriate in each circ*mstance. Hopefully understanding these differences will help you make good choices in the future, and minimize your risk and liability.

Reach out to me anytime if you have questions.

Obligatory government-employee disclaimer:

As a employee of the Canadian government, it is important to note that as with everything I publish, the information presented in this article is the opinion of the author as a subject matter expert, and does not necessarily represent the opinion of the government of Canada, the National Research Council Canada, or the Canadian Construction Materials Centre.