Trial Pass Fail Criteria: A Decision Chart for Product Trials

The chart covers trials of industrial products, such as lubricants, coolants, cleaners, water treatment and coatings, judged against criteria written down before the run. It does not replace a regulated batch release, a statistical dispute procedure such as ASTM D3244-24, or a laboratory's own decision rule under ISO/IEC 17025:2017.

When do you use a trial verdict decision chart?

A trial verdict decision chart is used when an industrial product trial has finished, the results are in, and someone has to state whether the product passed, failed or cannot be judged yet. Typical cases are a plant trial of a new coolant, lubricant, cleaner or coating, a supplier change on an existing line, and a bench round run before a field trial. The person who signs the verdict reads the chart, and the person who has to live with the verdict checks it: a customer's process or quality engineer, a supplier's technical service engineer, or both.

The chart rests on one rule from conformity assessment. ILAC-G8:09/2019, a guidance document for laboratories, states: "It is therefore expected that the decision rule is agreed before the measurements are taken." (ILAC-G8:09/2019, section 4.2). ASTM D3244-24, a practice written for contracts on petroleum products and lubricants, requires its acceptance limit to be determined before testing begins (ASTM D3244-24, section 4.3). A plant trial is not a laboratory conformity statement, but the order carries over: the yardstick comes first and the reading second.

Use the full chart at the end of the trial. Use its first two steps at the planning stage too, to check that the trial will be judgeable at all once results arrive. A plan that cannot pass step 1 on paper will not pass it after the run either.

What are the steps of the trial pass fail decision chart?

The trial pass fail decision chart has 11 numbered steps, and each step's answer either sends you to the next step or stops the chart with a verdict of not judged, invalid run, unassessed or a judgment on one criterion. The table below decides whether an industrial product trial reads as pass, conditional pass, fail or not judged, and what scope the verdict covers.

The key rows are steps 3 and 4: a trial run that lost sample custody or method integrity for the samples a decisive criterion depends on is an invalid run, which is a failed run rather than a product result to interpret. Steps 1 to 6 judge the run; steps 7 to 9 judge each criterion; steps 10 and 11 set the verdict and its reach. A verdict reached by skipping steps 1 to 6 describes what happened in the run but does not show that the product qualifies. Two outcomes in the chart are easy to confuse: not judged means the run cannot be read yet and may become judgeable once a record is found, while invalid run means the run is spent and only a new run can produce a verdict. Write the reason beside every no answer, because the reason, not the tick, is what a second reader checks when the verdict is questioned months after the trial.

What do you need before judging a trial verdict?

Before judging a trial verdict you need six inputs: the trial plan with its frozen criteria, the decision rule, the precision of each test method, the run records, the baseline or control data, and the customer's operating window. If the trial plan is missing, the honest verdict is not judged, and what unblocks it is a new run with criteria frozen before its results.

The frozen criteria list each threshold with its unit, the test method, the quantitation limit, the comparison basis (absolute value, change against baseline, or difference from a control) and the date the criteria were frozen. The decision rule says how a result near the threshold is read. ISO 4259-2:2026, valid from 9 June 2026 and replacing the 2017 edition, specifies how test method precision is used to set specification limits and to settle conflicting supplier and receiver results, and it does not apply to non-homogeneous products (ISO 4259-2:2026, scope). For other product families, use the precision statement of the method itself.

Precision needs its conditions. Reproducibility conditions mean different laboratories, operators and equipment (NIST OSAC Lexicon, 2025), so a customer's result and a supplier's result on the same lot are compared against reproducibility, not repeatability. The run records give the product, lot, line, sample points and dates. The operating window gives the low and high of every condition the product will see in regular use, which step 11 needs.

How do you check run validity before reading trial results?

You check run validity before reading trial results by judging five elements against the record: sample identity and custody, method, baseline, control, and result completeness. Each element is marked valid, deviated or unconfirmed, and a deviation is written as the fact plus its effect on the reading, before any criterion is compared with its threshold.

Sample identity and custody come first. ASTM D4840-99(2018)e1, a guide written for water analysis, describes chain-of-custody procedures whose purpose is "to provide accountability for and documentation of sample integrity from the time samples are collected until sample disposal" (ASTM D4840-99(2018)e1, section 1.1). The same record shows that a trial sample is the sample it claims to be.

Method means the planned test method, unit and detection limit were used. Baseline and control mean there is something to compare against: ASTM D5894-21, a practice for cyclic salt fog and UV exposure of painted metal, says it should not be used for a pass or fail approval unless results are compared with a control exposed at the same time or the test variability is rigorously quantified (ASTM D5894-21, section 5.5). Completeness means each criterion has its planned number of independent results.

Do not invalidate a result because it is unwelcome. FDA guidance on out-of-specification results, Revision 1 of May 2022, says results should be invalidated "when clear evidence of laboratory error exists" (FDA, 2022), and the same logic protects a trial from being tested into a pass.

How do you apply the fixed pass and fail criteria to each trial result?

You apply the fixed pass and fail criteria by comparing each criterion's result, from independent samples, with its frozen threshold, and reading the margin against the method precision or the agreed guard band. Each criterion then gets one of four judgments: met, borderline, not met or unassessed.

ILAC-G8:09/2019 gives the vocabulary for results near a limit. A binary rule allows pass or fail only; a non-binary rule with a guard band reports pass, conditional pass, conditional fail or fail, where the conditional outcomes fall inside the guard band either side of the tolerance limit (ILAC-G8:09/2019, section 4.2). JCGM 106:2012 explains why: an acceptance interval balances the risk that a conforming item is rejected against the risk that a non-conforming item is accepted (JCGM 106:2012, introduction). In a trial, borderline plays the role of the conditional outcomes and is reported as borderline, not as met.

Use the right numbers. Record mean, standard deviation and number of independent results for each criterion, and keep analytical replicates of one sample from counting as separate results. FDA guidance notes that "Reliance on averaging has the disadvantage of hiding variability among individual test results." (FDA, 2022), so report the spread beside the mean.

Do not move a criterion after seeing a result. FDA guidance states that "The number of retests should not be adjusted depending on the results obtained." (FDA, 2022). A trial whose threshold changed after the data arrived is a new run with new criteria, judged on new results.

How do you set the scope a passed trial covers?

You set the scope a passed trial covers by writing, for each operating condition, the low and high the run actually covered, setting that range beside the customer's operating window, and leaving every untested condition outside the qualified scope. A pass on one line, one substrate or one season qualifies the product for those conditions and no further.

Write the tested range per condition in its own unit: for example temperature, concentration, load, speed, water hardness or substrate. Then mark the part of the customer's window the run did not reach. An extrapolated range, where engineering judgment suggests the product will also work, goes in a separate column and is labelled as extrapolated rather than qualified.

The reporting clause of the laboratory standard gives the model. ISO/IEC 17025:2017 clause 7.8.6.2 requires a statement of conformity to identify "to which results the statement of conformity applies" and which specifications are met or not met (ISO/IEC 17025:2017, clause 7.8.6.2, quoted in ILAC-G8:09/2019). A trial verdict should name the same things: the run, the criteria and the conditions.

Fix the identity of the qualified product too, so that a later delivery can be matched to it: the SDS revision, the specification revision, the manufacturing site, the packaging and the Certificate of Analysis (CoA) of the lot used in the trial. A later lot made at another site or under a new specification revision is a change to review, not a product already covered by the trial.

How do you fill in the trial verdict chart and form?

You fill in the trial verdict chart and form in section order: fix the criteria before the run, record conditions and validity as the run goes, enter results, then work the 11 steps and sign the verdict with its scope. The editable Word file at the end of this page has the one-page chart, a four-section form and a separate illustrative example. The table below previews each part of that form, with units and an example entry that is illustrative only.

The key row is section 1: a form whose criteria carry no fixed date earlier than the first result cannot reach a verdict under step 1. Fill section 1 before the trial starts and have both parties sign it, so the date is on record. Fill section 2 while the run is in progress, from the sample log, the laboratory reports and the line records, rather than from memory at the end. Write unconfirmed rather than leaving a cell blank when a record is missing, because a blank cell cannot be told apart from a check nobody made. Enter results in section 3 only from independent samples, and keep a criterion without a valid result in its row, judged unassessed. Work the chart only after sections 1 to 3 are complete, then write section 4 from its answers, including what must still arrive before a conditional verdict becomes final.

Worked trial verdict example (illustrative)

The worked trial verdict example below is illustrative: the product, values, thresholds and dates are invented to show how the chart reads, and none comes from a real trial. A plant trials a new metalworking coolant on one machining line for six weeks, with criteria frozen before the first sample and an unchanged line run on the old coolant as the control (illustrative). The table below shows how each criterion is judged once run validity has passed steps 1 to 6 (illustrative).

Step 7 gives met for tool life because the 4 percentage point margin exceeds the agreed 2 point guard band (illustrative). Step 8 gives borderline for foam because 48 mL is within the 10 mL method precision of the 50 mL limit (illustrative). Step 6 gives unassessed for corrosion on cast iron because no result exists.

Step 10 therefore reads conditional pass, not pass: the decisive criterion is met, but one other criterion is borderline and one is unassessed, and each is named as an approval condition. Step 11 limits the scope to the steel grades and the concentration range the line actually ran (illustrative), and cast iron stays outside. The form names the two measurements that would make the verdict final, a repeat foam test and a cast iron corrosion test, and an owner for each.

Where does a trial verdict chart not apply?

A trial verdict chart does not apply when no criteria were frozen before the run, when the decision belongs to a regulated release process, or when the dispute is a statistical one between two laboratories on a specification property. In each case the chart either stops at step 1 or gives way to a procedure written for that situation.

Without frozen criteria the chart has nothing to apply, so the work is to plan a new run, not to judge the old one. A regulated release, such as a drug batch under US CGMP, follows its own out-of-specification investigation rules, which the FDA guidance of May 2022 sets out (FDA, 2022). A supplier and receiver who disagree on whether a petroleum product meets a numeric specification can use ASTM D3244-24, which combines results from up to three laboratories into an assigned test value (ASTM D3244-24, section 4.6.1), or ISO 4259-2:2026 for homogeneous products. A laboratory issuing its own statement of conformity follows the decision rule it documented under ISO/IEC 17025:2017 clause 7.8.6.1.

The chart also does not judge whether the product is commercially worth adopting; price, supply and change cost sit outside it. Lubinpla's trial result evaluation follows the same order, checking run validity first, applying the criteria frozen before the results, then confirming the qualified scope, and returns that judgment as one cited document for review.