Measurement system analysis (MSA) is one of the core tool in manufacturing industry. Variation present in your measurement system will affect the quality of product and can damage to your company brand. So here in this post we will look into details of attribute measurement system analysis.

You can learn about the Measurement system analysis (MSA) basics by click here.

Now here is the major topic for how MSA should be perform for attribute data type.

After reading of this complete guide you will learn,

What is need of attribute MSA. Types of attribute MSA and when to use. Perform the attribute MSA study.

Let’s see

what are the Measurement systems we have?

Variable MSA

– Caliper length, diameter measurement
– Cycle time measurement
– Torque measurement
– Flatness measurement
– Speed measurement

Attribute MSA

– Customer satisfaction
– Grades
– Defects (Ok, NOK)
– Gages check (G0, No-go)
– Visual defect check,

Now, we have pick here the Attribute MSA.

Purpose of Attribute measurement system analysis

There are below criteria for what purpose the MSA to be perform,

Accuracy check:

To access the customer standard, need to fulfill the customer requirement.

MSA perform to identify how good is our measurement system with our masters.

Precision check:

To determine that, the inspector can measure correctly. Is he check with same criteria across all shifts, machines etc… to measure and evaluate parts. Also called as Repeatability.

To quantify the inspector or gauges. It means inspector or gages can accurately repeat their inspection decision. Called as Reproducibility.

To determine:

To determine that is there any training needed to the inspectors. The gages need any correction or adjustment necessary.

It is to determine where the standards are not clearly define.

Lets see next point of Attribute MSA categories

3 categories of Attribute Data in MSA

As we seen the attribute data types before, therefore it is categorize on base of data.

-For the attribute data of 2 categories – Ok-Nok / Go-NoGO

-Attribute data of 3 or more categories which is not in order Yellow-Red-Blue etc.

-And finally 3 or more categories which is in order – 1-2-3-4, Low-medium-High, etc.

At the base of this categories the Attribute measurement system analysis study is conducted.

These steps should be done in attribute MSA study

-> Plan study
-> Conduct study
-> Analyze and interpret results
-> Improve Measurement System if necessary
-> Ongoing evaluation

Step-1 Plan Study

1. First in the plan study, We need to define the sample size.

Base on sample size selection our study is being analyze.

And the sample size should be 30-50 samples. Or you can take base on customer requirement.

Then the next step in plan is selection of samples from lot / production.

– Selected samples should cover the full range of variation in the process.

– Samples should contain at least 50/50 or 30/70 Good and Bad part combination for better result.

– Single part should have only one defect to avoid misunderstanding.

2. We have to think the main contributing factors in the measurement variability,

Such as

– Operator should be train for inspection
– Defect should be clearly understood
– Environment should be consider while study, ex. Sufficient Lux level at measurement area.

3. Inspection work instruction

In this stage all the study procedure to be well written and documented. This will be more effective inspection and study.

Also need to consider the blind study. Operator do not know which part they are measuring.

Step-2 Conduct study

Now in this step we need to take care of the time and place to conduct the study.

We need to define how we check parts in blind study. Also how much time require to inspect the part.

Finally in conduct study stage we need to observed the following conditions,

– Observed any process deviation in process
– Observed any environment changes while conducting the study.
– Errors in writing the inspection result
– Errors in the measurement methods.
– Operators differences / don’t disturb the operators

Now its time to measure and record the result on paper. The results should not be visible to the operator.

Let’s have a look on practical example,

Practical Example: Attribute study (Minitab)

Your company is producing the Bearing. So you need to check the bearings with the Go-NoGo gauge. Therefore your mission is to identify the defective items.

Then you will choose samples as describe in plan stage.

You must know the exact result of the samples. Means you should know which samples is OK and NOK.

So that the numbering should provide on bearing to identify the sample number. Total 30 number of samples is being tested here.

2 appraisals ( 2 Operator) will check each bearings.

Each trails will be finish in 60 seconds (3 seconds / part)

Each inspector will perform 2 trails
1st Trail
2nd Trail

The operator write the results on the paper ( As OK and NOK).

Be careful to the order and the OK and NOK names have to be the same than the standard used in Minitab® worksheet.

Set-up the data in Minitab®

When the response is binary (ok – nok)
Use the Minitab: Assistant MSA > attribute agreement

When the response is ordinal or nominal (good, very good, bad, very bad etc.)
Use Minitab feature > stat > quality tools > attribute agreement analysis

After click on OK, you get a new worksheet to fill in the result.

Below are the results sheet from the observation result by operators.

Step-3 Analyze and Interpret Results

To analyze the data, go to the process shown below.

Assistant –> Measurement System Analysis –>Attribute Agreement Analysis

You will need to select the below window from the cells

After click on “OK” you will get the analysis windows as below to interpret the results,

1 Report Card:

The following indicators are calculated:

1.% Rated both Way per appraiser
2.% Pass rated Fail per appraiser
3.% Fail rated Pass per appraiser
4.% Accuracy per appraiser
5.Overall Accuracy Rate

Please refer below screenshots of analysis of our attribute study,

1. % Rated Both Way Per Appraiser

What is it?

It shows the repeatability by operator, i.e. what is the ability of each operator to ALWAYS assess a single part the same way ?

How is it calculated?

% Rated both way per appraiser = number of parts not assessed consistently / number of parts assessed in %.

Example : on 20 parts, operator assessed 18parts the same way  and 2 parts sometimes OK, sometimes NOK => % Rated both way = 2/20 = 10 %.

What is expected?

% Rated both way  ≤ 10%.

2. % Pass Rated Fail Per Appraiser

What is it?

It studies the frequency for each operator of assessing a part as NOK when it is OK in reality.

It represents producer’s risk that is risk to scrap or rework OK parts.

How is it calculated?

% Pass rated Fail  = number of time parts assessed NOK when it is OK / number of assessment  with standard OK in %.

Example : on 8 parts with standard 0K, operator assessed 1 part 1 time NOK for a OK standard   => % Pass rated Fail = 1 / 8 * 2 = 6.25%.

What is expected?

% closest to 0 (green <5%, orange <10%, red >10%)

3. % Fail Rated Pass Per Appraiser

What is it?

It gives the frequency for each operator of assessing a part as OK when it is NOK in reality.

It represents customer’s risk that is risk to send NOK parts to customer. n

How is it calculated?

% Fail rated Pass  = number of time parts assessed OK when it is NOK / number of assessment  with standard NOK in %.

Example : on 12 parts with standard N0K, operator assessed 1 part 2 times OK for a NOK standard   => % Pass rated Fail = 2 / 12 * 2 = 8.33%.

What is expected?

% closest to 0 (green <2%, orange <5%, red >5%)

In case of S&R characteristics, the only target is 0%

4. Accuracy Per Appraiser

What is it?

It gives the accuracy for one appraiser that is the % of his appraisals which are matching the standard.

How is it calculated?

Accuracy per appraiser = number of appraisals which are matching the standard / number of appraisals in %.

Example : on 40 appraisals (20 parts 2 trials), the operator assessed 1 NOK part 2 times OK and 1 OK part one time NOK => Accuracy per appraiser = (40-3)/40 = 92.5 %.

What is expected?

Accuracy > 90 %.

5. Overall Accuracy Rate

What is it?

It measures the overall efficiency of the test.

It gives the accuracy for all appraisers that is the % of all appraisals which are matching the standard.

How is it calculated?

Overall Accuracy = number of all appraisals which are matching the standard / number of appraisals in %.

Example : on 80 appraisals (20 parts 2 trials 2 appraisers), one operator assessed 1 NOK part 2 times OK and 1 OK part one time NOK  and the other operator assessed 1 NOK part 2 times OK, 1 NOK part one time OK and 1 OK part one time NOK=> Accuracy per appraiser = (80-7)/80= 91.3 %.

What is expected?

Decision rules:

> 90% :  

Excellent inspection process.

70 to 90% :

An action plan must be set up depending on the how critical the inspection. Inspection method, training process, boundary samples, environment… must be checked and improved.

< 70% :

The inspection process is unacceptable. Reconsider it.

Analysis :

Review the repeatability portion first (% Rated both Way per appraiser), if an Appraiser cannot agree with himself, ignore comparisons to Standard and to other Appraisers and go understand why.

For Appraisers that have acceptable repeatability, review the agreement with standard (% Pass rated Fail per appraiser and % Fail rated Pass per appraiser). We will know if inspectors are well calibrated.

For appraisers that have acceptable calibration, review their accuracy.

Finally check overall accuracy.

Interpret Other Graphs

Check if there is any part assessed mixed way by all appraisers or assessed consistently by all operators but not in accordance with the standard.

Check any accuracy differences (between appraisers, between standard, between trials, …) to look for way to improve.

Step 4 – Improve Measurement System

Once we have established we have a problem or several problems with a Measurement System, we need to figure out how to correct it.

1 – If the % Rated both Way for one appraiser is high, that Appraiser may need training. Do they understand characteristic they are looking for?  Are the instructions clear to them? Do they have vision issue?

2 – If the Accuracy per appraiser is low, the Appraiser may have a differing definition of the categories than the standard– A standardized definition can improve this situation (borderline catalog).

3 – If disagreement occurs always on same part, clarify boundary.

4 – If improvements are made, the study should be repeated to confirm improvements have worked.

How could we improve the Measurement System for our table?

Step 5 – Ongoing Evaluation and Future Actions

– All inspectors making this assessment in production need to be validated with Attribute Gage R&R –> right assessment = validation of skill.

– Any new operator inspecting this part has to be validated with the Gage R&R.

– Frequency to revalidate inspectors has to be defined.

– If borderline catalog is changing (new defect, new boundary, …), Gage R&R has to be updated (new parts to evaluate the defect, …) and inspectors have to be re-assessed.

Conclusion:

The participants will now be able to :

Discuss the need for Attribute Measurement System Analysis.

Describe the types of Attribute Measurement System Analyses and when to use them.

Perform an Attribute Measurement System Analysis.