What share of VA errors are caught by the weekly quality sample?

Overview

Research question and scope

This is a proposed research protocol, not a report of completed findings. It asks what share of errors in a week's work are caught by the weekly quality sample before the work reaches a client. The result would help a team judge whether its sample size and selection rule are adequate. Ask whether a new reviewer could act on the record without a conversation.

The protocol does not measure the total error rate in all work. It measures the errors the sample found and compares them with errors found later through other channels. The sponsor should name the quality decision the result could inform. Use the same format for every handover so the next reader can scan it.

Population and unit of analysis

Define the population as the work items produced during a stated week and the errors recorded against them. The unit is one work item, with the sample status, the error status, and the discovery channel recorded. Items that were never reviewed should remain visible. Test the rule on one real task before applying it to the queue.

Decide how to treat an item with more than one error. Count the item once for the catch rate and count the errors separately for the severity analysis. The two views answer different questions. Write the deadline as a date and a local time.

Philippines evidence beside global context

The table keeps national indicators separate from the checks a buyer must run on one candidate. Values come from the direct sources listed below, and each year stays visible so unlike periods are not presented as the same measurement.

Definitions and measures

Define an error as a factual, procedural, or boundary failure that requires a correction before the work is acceptable. Define a catch as an error identified by the sample before the client reports it. Define a miss as an error found later through a client report, a downstream review, or an audit. Ask whether a new reviewer could act on the record without a conversation.

Report the catch rate, the miss rate, and the severity of the errors in each group. A sample that catches many small errors but misses a serious one is not necessarily working. The severity distribution is as important as the count. Test the rule on one real task before applying it to the queue.

Evidence standard

Link each error to the work item, the sample record, and the correction. A client complaint should be linked to the original work and the date. A correction without a record of what was wrong is hard to classify. Use the same format for every handover so the next reader can scan it.

Keep the original sample selection and the reviewer's note. If the sample was chosen after an error was found, it cannot be used to estimate the catch rate. The selection rule is part of the evidence. Test the rule on one real task before applying it to the queue.

Sampling and collection

Use the sample records the team actually created rather than a new retrospective sample. Pair them with the error log and the client complaint log for the same week. Match on work item and date to avoid double counting. Use the same format for every handover so the next reader can scan it.

Pilot the matching on a few items and compare the reviewers' classifications. Clarify whether a correction made during the sample counts as a catch or as a normal edit. Record the rule and apply it consistently. Keep the boundary visible in the place where the work is assigned.

Analysis and reporting

Report the catch rate by error type and severity, with the sample size and the selection rule. Show the errors that were found later and the time between the work and the discovery. A late catch is different from an early catch. Write the deadline as a date and a local time.

Describe the limits of the sample rather than declaring the process effective. A small sample may have caught all the errors in a quiet week and missed the one that mattered in a busy week. The report should show the uncertainty. A short note with four fields is easier to use than a long narrative.

Limits and sources of error

The error log may be incomplete because some errors are corrected quietly and never recorded. Client complaints may capture only the errors the client noticed. The study should state which channels were used and which were unavailable. Name the owner in the same line as the action.

The reviewer's classification can change after seeing the client complaint. Keep the initial classification and record any later change. The first judgment is part of the evidence about how hard the error was to see. If the source is unclear, route the question instead of guessing.

Review cadence

Repeat the review after a change to the sample size or the checklist. Compare the same error types and the same discovery channels. A higher catch rate is useful only if the errors caught are the ones that matter. Keep the boundary visible in the place where the work is assigned.

Assign the follow-up to the quality owner and set a date. The protocol should produce a specific decision about the sample size, the selection rule, or the checklist. Update the template when the same question appears twice. If the source is unclear, route the question instead of guessing.

For the operating context behind this protocol, review the services page and the boundaries it describes. Use the contact page when the study design needs a written approval or a scoped review. The references below support the definitions and the recordkeeping approach rather than a performance claim. Keep this protocol with the data so a later reviewer can repeat the method.