Skip to content

Missing data in performance evaluation: a policy written before the period

Six kinds of gap and the rule for each, five rules compared on the same score, the case in which redistributing the weight rewards the gap and an eight-step model policy.

By the Pulsika team Performance Published updated 12 minutes read

Two rows set side by side. The figure 0 means the result was measured. The dash means the value is missing and must not be treated as zero.

Missing data in performance evaluation is handled through a policy written before the period, not through decisions taken at the close, when the result is already visible. The policy says which kinds of gap you recognise, which rule applies to each (leaving the part out and redistributing the weight, zero, an extended period, a minimum data threshold), who fills in the data and by when, and how people find out what happened.

The article gives you a classification into six kinds of gap, compares five rules on the same fully calculated score, shows the case in which leaving a part out rewards the gap and ends with a model policy you can copy.

Quick answer. Write a rule for each kind of gap in advance. Leave the part out and redistribute its weight when the gap is not the person’s doing. Count 0 only when the person was supposed to provide the data. Use an extended period for indicators with few observations. Do not publish a score or decide a bonus when the parts with data cover less than a set threshold, for example 70% of the weight.

Definitions: what “missing” means in an evaluation

A missing value is a value that should exist for a person and a period, but does not. It is different from zero: zero sales is a result, a missing sales report is a data problem. The guide on scores people trust explains why the two must not be mixed; here we build the process rules.

Leaving a part out and redistributing its weight (renormalisation) means calculating the weighted average from the parts with data only, dividing by the sum of their weights. The extended period replaces the month’s value with the average of the last months that have data. The minimum coverage threshold is the percentage of the weight that must have data for the score to be published. Imputation means replacing the missing value with an estimated one, for example the team average.

Why the reason for the gap matters

The statistician Donald B. Rubin showed in 1976, in “Inference and missing data”, that a gap can be ignored without spoiling the conclusions only under certain conditions. The literature distinguishes three situations: data missing completely at random, a gap explained by other known data, and a gap linked to the very value that is missing.

The third situation is the dangerous one in evaluation. If precisely the people with poor results do not hand in their report, leaving the part out means hiding exactly the poor results. The right rule therefore depends on the question “why is it missing?”, not just on the fact that it is missing.

Six kinds of gap and the right rule for each

The classification below is a working tool, not a standard. Adapt it to your indicators, but keep the idea: each kind has a rule written in advance.

Kind of gapExampleRecommended ruleWho acts
Does not applyA unit with no display area; a role with no external customersThe part leaves the scorecard for that person or unit; the weight is redistributedWhoever builds the scorecard, before the period
Justified absenceLong annual leave, sick leave, hired mid-monthA target adjusted to the days worked, or “no data” below a minimum number of daysHR, from the attendance records
The source runs lateThe sales report not imported, the coordinator’s evaluation not filled inThe period is not closed before the deadline; completion is followed upThe data owner, with a deadline
The person’s responsibilityThe field agent’s visit report not handed inThe gap counts as 0, a rule announced in advanceThe person being evaluated
Invalid dataAn import with swapped columns, an obviously wrong value“No data” until corrected, then a documented recalculationThe data owner
Too few observationsA single service evaluation visit in the monthExtended period (the average of the last 3 months) or a minimum number of observationsWhoever builds the scorecard

Remember. “Does not apply” is decided before the period, in the scorecard’s set-up. “Missing” is found after the period. If you mix the two, you end up deciding at the close that an indicator “does not apply” precisely for someone with a poor result.

A worked example: five rules, the same person, the same month

Hypothetical example. Fictitious data. A sales consultant’s scorecard has three parts: Sales (weight 50%, score 104), Service quality from evaluation visits (30%) and the Coordinator’s evaluation (20%, score 92). In October no evaluation visit took place, so the second part is missing. In August and September the visits gave 88 and 94; in July there was none. The team average for service quality is 85.

The ruleCalculationScoreWhat it conveys
Left out, weight redistributed(104 × 0.5 + 92 × 0.2) ÷ 0.7 = 70.4 ÷ 0.7100.57The score says what we know, on 70% of the scorecard
The gap counts as 052 + 0 + 18.470.40It penalises the person for a visit that did not depend on them
Extended period (the months with data out of the last three)average (88; 94) = 91 → 52 + 27.3 + 18.497.70It uses real but older information
Imputation with the team average52 + 85 × 0.3 + 18.4 = 52 + 25.5 + 18.495.90It gives the person other people’s result
Minimum coverage threshold of 70%coverage = 50% + 20% = 70% ≥ 70% → the redistributed version is published100.57The score appears only with enough data

Between the mildest and the harshest rule there are 30.17 points (100.57 − 70.40), which in most scorecards is at least one level. The same month, the same facts. Hence the article’s basic rule: the choice is made beforehand, not after you see the figure.

When two parts are missing

If the coordinator’s evaluation is also missing in the same month, only the sales part remains. Leaving the parts out would give (104 × 0.5) ÷ 0.5 = 104: a “complete” score built from half a scorecard. With the 70% threshold, coverage is 50%, so the month’s result is “no score” until the data are filled in. The threshold does not change the figure; it changes the moment you are allowed to use it.

The case in which leaving a part out rewards the gap

Redistributing the weight is a good rule when the gap is not the person’s doing. It becomes a bad rule when the person controls whether the data exist.

Hypothetical example. Fictitious data. Two field agents have the same scorecard: Sales 40%, Visits made 30%, Complaints 30%. Both have 110 on sales and 100 on complaints. Agent X handed in their visit report: 60 against plan. Agent Y did not hand theirs in.

  • Agent X: 110 × 0.4 + 60 × 0.3 + 100 × 0.3 = 44 + 18 + 30 = 92.
  • Agent Y, with the part left out: (44 + 30) ÷ 0.7 = 105.71.
  • Agent Y, with the gap counted as 0: 44 + 0 + 30 = 74.

With redistribution, not handing in the report brings 13.71 more points than the colleague who honestly reported a poor result. For data the person has to provide, the right rule is 0, announced in advance. The guide on choosing indicators mentions this option; here you can see why it is not optional.

The process rules: deadlines, completion, closing

Most gaps are not special cases but delays. A good policy reduces them before they reach the score.

The completion deadline

Set a deadline after the end of the period, for example 5 working days, by which the data owners load the values and the evaluators complete the evaluations. The period is not closed before the deadline, even if someone urgently needs the figures.

Checking completeness before closing

Before closing, someone in HR checks, for each indicator, how many values have been filled in out of how many are expected. For each gap, they note its kind from the table above. Without that note, when a result is challenged nobody knows any more whether the gap was leave, a delay or an error.

The bonus and the coverage threshold

The bonus decision is not taken on a score below the coverage threshold. It is postponed until the data are complete or, if the data will not appear, the rule written for that case is applied (for example, the month’s bonus is calculated on the quarter’s average). Rules tied to money are written into the company’s documents, not only into the scorecard.

Recalculating after completion

When a late value arrives after the close, the period is recalculated once, and the person is told what changed. Do not allow repeated recalculations without notice: every unexplained change lowers trust in all the scores.

How to communicate missing data

  • On the score: the part appears as “no data”, not 0 and not blank, with the rule applied stated in words.
  • In the conversation: the manager says how many parts the score was calculated from before discussing the level.
  • In comparisons: a score calculated on 70% of the scorecard is not put in the same ranking as a complete one without a note.
  • To the person evaluated: tell them what they can do if they know a value is missing by mistake, and whom to write to.

Caution. The General Data Protection Regulation requires personal data to be accurate (Art. 5(1)(d)), and Art. 16 gives the person the right to have inaccurate data rectified and incomplete data completed, “including by means of providing a supplementary statement”.

Romania’s Labour Code (Codul muncii, Law no. 53/2003) requires the internal regulations to contain the criteria and procedures for professional evaluation (art. 242(i)), and dismissal for professional unsuitability requires a prior evaluation under the established procedure (art. 63(2)). Check with the company’s lawyer where the missing-data policy belongs and how it applies to decisions with legal effects.

A model missing-data policy

The steps below produce a one-page document. Complete it for each scorecard before the first period that counts.

  1. List the parts of the scorecard and, for each, who provides the data: the person, the manager, a system or a third party.
  2. Mark what “does not apply” for certain units or roles and remove those parts from the set-up, not from the calculation at the close.
  3. Choose the rule for a gap on each part: left out when the data do not depend on the person, 0 when they do, an extended period when observations are rare.
  4. Set the coverage threshold (for example 70% of the weight) and what happens below it: “no score” until the data are complete.
  5. Settle absences: below how many days worked the month becomes “no data”, and how the target is adjusted above that minimum.
  6. Write the deadlines: by when the data are loaded, by when the evaluations are completed, when the period is closed.
  7. Settle corrections: who may correct a value after the close, how it is recalculated and how the person finds out.
  8. Test the policy on three past months and count how many scores would have come out as “no score” or changed level.

For testing on historical data, see the article on testing a KPI scorecard.

How to apply the policy in Pulsika

In Performance and KPIs, a missing value stays “no data” and never turns into 0 by itself. In manual entry, a field left blank means “no data”, and deleting a value removes it. The value entry screen shows how many values have been filled in out of how many are expected, which is useful for the check before closing.

  • The weighted total, the usual one, leaves out the part without data, and the others take over its weight: the first rule in the table above. If no part has data, the total stays “no data”.
  • The totals “The sum of the parts”, “The weakest part” and “The best part” stay “no data” if a part is missing, except for the parts where you tick “No data counts as 0 in the total” — the rule for data the person provides. For the weighted total, the same rule is written in your own formula, with the piece “If missing, use 0”.
  • The extended period is written in scorecards with the pieces “Average of the last periods” (months without data are not included in the average) and “If missing, use something else” (the first value that has data).
  • Managers’ evaluations give as their value the average of the evaluations submitted in the period; with no evaluation submitted, the value is “no data”. After the grid’s deadline, only someone with the right to manage evaluations can still correct one, and the correction stays on record.
  • At the close, the result shows how many components had no data, and on the person’s score the part appears as “no data — left out”.

The minimum coverage threshold is not a separate setting: you apply it as a process rule, looking at the number of parts without data before deciding a bonus.

Common mistakes

  • The rule chosen after you see the result. Correct it: the policy is written before the period and applied the same way to everyone.
  • The same rule for all parts. Correct it: data provided by the person have a different rule from data provided by others.
  • A zero entered “so the month can be closed”. Correct it: a 0 is a value; a gap stays a gap until it is filled.
  • Imputation with the team average for the bonus. Correct it: do not pay or penalise someone for other people’s results.
  • A score published from half a scorecard without a note. Correct it: a coverage threshold and the note “calculated on X% of the scorecard”.
  • Gaps not recorded. Correct it: every gap gets its kind and reason before the close.

Frequently asked questions

Should a missing value be counted as zero?

Only when the person evaluated was supposed to provide the data and did not, and the rule was announced in advance. In the other cases, a gap counted as zero penalises the person for someone else’s delay or absence. Leave the part out, with the weight redistributed, or use an extended period, depending on the kind of gap and how rare the observations are.

What is weight redistribution?

Weight redistribution means calculating the weighted average from the parts that have data only, dividing by the sum of their weights. If a part worth 30% is missing, the other two, worth 50% and 20%, count as if they were 71.4% and 28.6%. The rule is fair when the gap does not depend on the person and misleading when it does.

How high should the minimum coverage threshold be?

There is no universal threshold or standard: the threshold is an internal convention. A reasonable starting point is 70% of the weight, that is, at least two parts out of three. Test it on past months: if too many scores would come out as “no score”, the problem is data collection, not the threshold. The threshold must be written in advance and applied the same way to everyone with the same scorecard.

How do I handle a month with sick leave?

Sick leave is a justified absence, not poor performance. Set a minimum number of days worked below which the month becomes “no data”, and above the minimum adjust the target in proportion to the days worked. Do not enter zero on volume indicators, and do not ask the person to make up the whole month’s target.

What do I do if the manager’s evaluation is missing at the deadline?

Treat it as a source running late: the period is not closed before the deadline, and the manager gets a reminder. If the deadline passes, the part stays “no data” and the coverage rule applies. Do not count zero: the person evaluated could not complete the evaluation in the manager’s place. Record the delay; if it repeats, it says something about the manager.

Can I use the team average instead of the missing value?

For overall analyses, sometimes yes; for an individual score tied to money, no. The team average gives the person their colleagues’ result, upwards or downwards. When an observation is missing, prefer an extended period from the same person’s data, and when there are none of those either, leave the part out and apply the coverage threshold.

Should the employee be told when their score has missing data?

Yes. The person must see which part is missing, which rule was applied and how they can flag a value that is missing by mistake. The General Data Protection Regulation gives them the right to have incomplete data completed, including by means of a supplementary statement. An explained gap is accepted; one they discover on their own lowers trust.

What to do next

Take the scorecard with the most missing data over the last three months and complete the eight-step model policy. Then calculate the same months with the new rules and compare the levels.

Was this article useful?

See how Pulsika works for your team

Try every module free or book a demo with our team. No card needed.

or let us set up a trial company with your structure