Guard against sub-optimisation
As performance measures, by which employees are evaluated, greatly impact their
behaviour, an improper set of measures can lead to dysfunctional or unanticipated
behaviour (Fry, 1995). In other words, employees seeking to improve the measure of their
performance often make decisions that are contrary to the desires of management. It is,
for example, not rare that an improvement in one area leads to a deterioration in another
and in worst case also resulting in a decline in the performance from an overall point of
view. Skinner (1986) termed this phenomenon as the ‘‘productivity paradox’’ where
dysfunctional behaviour results from poor performance measures. In fact, there are many
examples where productivity improvement programs have failed mostly because they
actually hurt more than they help. For instance, to focus on direct labour and efficiency of
factory workers where direct labour costs is about 10 per cent of sales may not have a
significant effect on the total productivity. Unfortunately, organisations that have
behavioural ramifications in mind when designing performance measures are few and
far between. Often the problem is not that measures have designs that give inaccurate
information, but that the behaviours they are likely to induce in a particular setting are not
considered (Neely et al., 1997). A PMS must therefore guard against sub-optimisation,
possibly by establishing a clear link from the top of the company all the way to the
bottom, to ensure that employee’s behaviour is consistent with corporate goals.
Include a limited number of performance measures
An important aspect is that in order to take appropriate actions it is necessary to use a limited
amount of performance measures (Jackson, 2000). To produce unnecessary data is
expensive, and can lead to more harm than good. More measurement will require a greater
deal of time for analysis by managers, or, alternatively, it is a waste to collect data if they are
ignored. It is therefore important to pay attention to limiting the data requirements to both the
necessary detail and frequency, to consider whether the data are needed for a specific
useful purpose, and whether the cost of producing it is not higher than expected benefit
(Bernalak, 1997). A large number of performance measures also increase the risk of
information overload, meaning that it is practically impossible to distinguish information with
high importance from information with less value. Information overload can lead to their all
information being ignored instead of used. In other words, it is vital that ‘‘old’’ performance
measures, that are no longer of interest, are removed from the PMS.
Class-dependent system requirements
Bourne et al. (2003) describe five different aspects in performance measurement that have
changed significantly during the last 20 years: focus, dimensions, drivers, targets and
desired benefits. A similar classification will be used here when discussing what
requirements increase both in importance and in complexity depending on each system
class:
B requirements regarding used performance criteria;
B requirements regarding stakeholders;
B requirements regarding hierarchical levels;
B requirements regarding time-horizon; and
B requirements regarding information architecture.
A third class PMS requires that:
B Traditional performance criteria are used. The performance measures within this class will
to a majority be influenced by the traditional way of measuring performance (e.g. return
on investment (ROI), cash flow, efficiency). The performance measures control that
different costs are kept at a reasonable level.
B Internal needs are fulfilled. The performance measures are mainly focused on fulfilling the
internal needs of the organisation.
B Top hierarchical levels are included in the PMS. The performance measures should cover
at least the most important hierarchical levels of the organisation.
B The short-term time-horizon is considered. Important measures within the PMS should
include updated targets, which at least cover the short-term time-horizon.
B Some form of information architecture is achieved. The PMS should be designed in a way
so that information is easily retrieved as well as easily understood by those whose
performance is being evaluated.
A second class PMS requires that:
B Both financial and non-financial performance criteria are used. Performance should not
solely be seen from a traditional point of view. A second class PMS consist of various
types of performance measures (i.e. multi-dimensional) covering all the important
aspects that represent the performance of a company (e.g. cost, quality, delivery,
flexibility and dependability). There must in turn be a balance between the various
performance measures in the PMS.
B Internal and external needs are fulfilled. The performance measures are focused on
fulfilling both the intern needs of the organisation and external needs (such as from
suppliers and customers).
Most hierarchical levels are included in the PMS. Performance measures exist on most
hierarchical levels in the organisation and are linked to each other. There should be a
balance between global and local performance.
B Short- and long-term time-horizons are considered. Important measures within the PMS
should include updated targets, which cover both the short- and the long-term
time-horizon.
B A practical information architecture is achieved. Vital information goes to the right
persons with little delay.
A first class PMS requires that:
B The used performance criteria explain important causal relationships across the
organisation. The performance measures within this class must support sustainable
performance improvement and must therefore stand up to the highest standards. The
measures must not only have a multi-dimensional view on performance, they should also
be able to explain what factors that have an influence on the performance of the company
and indicate what to do in order to improve.
B The needs from all stakeholders are fulfilled. The PMS includes performance measures
considering the needs from all possible stakeholders needs (e.g. customers,
shareholders, competitors, suppliers, employees, the society).
B All hierarchical levels are included in the PMS. Performance measures exist on all
hierarchical levels in the organisation and clear links have been established between the
measures on the different levels.
B Processes have been created for natural evolution of the PMS. Secures that the PMS is
constantly updated when needed.
B An advanced information architecture is achieved. Performance measurement
databases and other reporting systems should be fully integrated to each other. The
information in the PMS is updated continuously and directly presented to the persons who
need it.
Improving a PMS
In order to deal with the above-presented requirements in practice when improving a current
PMS, the following simple three-step procedure is proposed. This is a similar approach to
identify weaknesses of PMS as the Performance Measurement Questionnaire proposed by
Dixon et al. (1990). However, that Performance Measurement Questionnaire does not
support the idea of system classes and does not consider all of the requirements presented
here (such as stakeholders):
1. Requirement evaluation. The first thing to do is a subjective evaluation of to what degree
the PMS fulfils different requirements. This is done with the system evaluation form
described in Table II. The idea with the form is to subjectively consider to what degree
each specified requirement is fulfilled by the PMS, on a scale between (1) to (7), where (1)
means that the requirement is not fulfilled at all and (7) that it is fulfilled to the highest
degree possible. It should be noted that there is no particular reason for choosing a
seven-graded scale here other than that this scale is easy to use and gives proper details.
2. System class determination. After the requirements have been analysed it is time to
determine the system class of the PMS (third class, second class or first class). This is
done by comparing the system evaluation form to the descriptions of the system classes
in Table I.
3. Revision of the PMS. Depending on the result of the previous steps, there are two different
strategies to follow when revising the PMS:
B Class completion strategy. This strategy is used if some defined requirements within
the system class have not yet been fulfilled to a satisfactory degree. The class
completion strategy simply means that the requirements within the class that can be improved, receive considerable attention until the PMS has completed all class
requirements.
B Class progress strategy. This strategy is used when all requirements within the system
class have been achieved to a satisfactory degree and the organisation is ready to
move up to a higher class. The class progress strategy will require more efforts for a
shorter period of time than the previous described strategy, since many new
requirements must be focused simultaneously and the PMS will be affected by major
modifications.
Case studies
The three-step procedure has been applied and tested empirically in three manufacturing
companies so far during this research (Table III). In short, the suggested requirements and
the three-step procedure showed several strengths when performing their tasks during the
case studies. First, the three-step procedure was rather simple to understand and to use,
which makes it suitable for assisting measurement practitioners. Second, the classification
of requirements made it easy to pinpoint the limitations and weaknesses of a current PMS as
well as show what to improve. For example, several main weaknesses were identified at one
of the company, even though their PMS had recently been severely modified. Third, the
procedure could also be used independently from the origin of PMS (such as if it had been
designed according to the balanced scorecard or other frameworks). Some limitations were
also identified. The system classes and the system evaluation form force its user to rely on
subjective analysis. Significant work is also required by the user, who for example must
identify how to make improvements.
Conclusions
There are numerous performance measures to select from when designing a PMS. This
gives the measurement practitioner many different options for designing a unique PMS.
However, from a practical point of view, there is often no obvious way to choose between
these performance measures and their abundance has caused much uncertainty (Tangen, 2003). It is also very difficult to fulfil all requirements suggested in literature when designing a
PMS. In this article, it is proposed that the question ‘‘What to measure?’’ and ‘‘How to
measure it?’’ should be handled separately in order to facilitate the design process. Further,
it is suggested that the former question should also be linked to different system
requirements depending on the complexity of the PMS (i.e. system class). Finally, a simple
procedure has been described that can be used when evaluating and improving an existing
PMS.
As performance measures, by which employees are evaluated, greatly impact their
behaviour, an improper set of measures can lead to dysfunctional or unanticipated
behaviour (Fry, 1995). In other words, employees seeking to improve the measure of their
performance often make decisions that are contrary to the desires of management. It is,
for example, not rare that an improvement in one area leads to a deterioration in another
and in worst case also resulting in a decline in the performance from an overall point of
view. Skinner (1986) termed this phenomenon as the ‘‘productivity paradox’’ where
dysfunctional behaviour results from poor performance measures. In fact, there are many
examples where productivity improvement programs have failed mostly because they
actually hurt more than they help. For instance, to focus on direct labour and efficiency of
factory workers where direct labour costs is about 10 per cent of sales may not have a
significant effect on the total productivity. Unfortunately, organisations that have
behavioural ramifications in mind when designing performance measures are few and
far between. Often the problem is not that measures have designs that give inaccurate
information, but that the behaviours they are likely to induce in a particular setting are not
considered (Neely et al., 1997). A PMS must therefore guard against sub-optimisation,
possibly by establishing a clear link from the top of the company all the way to the
bottom, to ensure that employee’s behaviour is consistent with corporate goals.
Include a limited number of performance measures
An important aspect is that in order to take appropriate actions it is necessary to use a limited
amount of performance measures (Jackson, 2000). To produce unnecessary data is
expensive, and can lead to more harm than good. More measurement will require a greater
deal of time for analysis by managers, or, alternatively, it is a waste to collect data if they are
ignored. It is therefore important to pay attention to limiting the data requirements to both the
necessary detail and frequency, to consider whether the data are needed for a specific
useful purpose, and whether the cost of producing it is not higher than expected benefit
(Bernalak, 1997). A large number of performance measures also increase the risk of
information overload, meaning that it is practically impossible to distinguish information with
high importance from information with less value. Information overload can lead to their all
information being ignored instead of used. In other words, it is vital that ‘‘old’’ performance
measures, that are no longer of interest, are removed from the PMS.
Class-dependent system requirements
Bourne et al. (2003) describe five different aspects in performance measurement that have
changed significantly during the last 20 years: focus, dimensions, drivers, targets and
desired benefits. A similar classification will be used here when discussing what
requirements increase both in importance and in complexity depending on each system
class:
B requirements regarding used performance criteria;
B requirements regarding stakeholders;
B requirements regarding hierarchical levels;
B requirements regarding time-horizon; and
B requirements regarding information architecture.
A third class PMS requires that:
B Traditional performance criteria are used. The performance measures within this class will
to a majority be influenced by the traditional way of measuring performance (e.g. return
on investment (ROI), cash flow, efficiency). The performance measures control that
different costs are kept at a reasonable level.
B Internal needs are fulfilled. The performance measures are mainly focused on fulfilling the
internal needs of the organisation.
B Top hierarchical levels are included in the PMS. The performance measures should cover
at least the most important hierarchical levels of the organisation.
B The short-term time-horizon is considered. Important measures within the PMS should
include updated targets, which at least cover the short-term time-horizon.
B Some form of information architecture is achieved. The PMS should be designed in a way
so that information is easily retrieved as well as easily understood by those whose
performance is being evaluated.
A second class PMS requires that:
B Both financial and non-financial performance criteria are used. Performance should not
solely be seen from a traditional point of view. A second class PMS consist of various
types of performance measures (i.e. multi-dimensional) covering all the important
aspects that represent the performance of a company (e.g. cost, quality, delivery,
flexibility and dependability). There must in turn be a balance between the various
performance measures in the PMS.
B Internal and external needs are fulfilled. The performance measures are focused on
fulfilling both the intern needs of the organisation and external needs (such as from
suppliers and customers).
Most hierarchical levels are included in the PMS. Performance measures exist on most
hierarchical levels in the organisation and are linked to each other. There should be a
balance between global and local performance.
B Short- and long-term time-horizons are considered. Important measures within the PMS
should include updated targets, which cover both the short- and the long-term
time-horizon.
B A practical information architecture is achieved. Vital information goes to the right
persons with little delay.
A first class PMS requires that:
B The used performance criteria explain important causal relationships across the
organisation. The performance measures within this class must support sustainable
performance improvement and must therefore stand up to the highest standards. The
measures must not only have a multi-dimensional view on performance, they should also
be able to explain what factors that have an influence on the performance of the company
and indicate what to do in order to improve.
B The needs from all stakeholders are fulfilled. The PMS includes performance measures
considering the needs from all possible stakeholders needs (e.g. customers,
shareholders, competitors, suppliers, employees, the society).
B All hierarchical levels are included in the PMS. Performance measures exist on all
hierarchical levels in the organisation and clear links have been established between the
measures on the different levels.
B Processes have been created for natural evolution of the PMS. Secures that the PMS is
constantly updated when needed.
B An advanced information architecture is achieved. Performance measurement
databases and other reporting systems should be fully integrated to each other. The
information in the PMS is updated continuously and directly presented to the persons who
need it.
Improving a PMS
In order to deal with the above-presented requirements in practice when improving a current
PMS, the following simple three-step procedure is proposed. This is a similar approach to
identify weaknesses of PMS as the Performance Measurement Questionnaire proposed by
Dixon et al. (1990). However, that Performance Measurement Questionnaire does not
support the idea of system classes and does not consider all of the requirements presented
here (such as stakeholders):
1. Requirement evaluation. The first thing to do is a subjective evaluation of to what degree
the PMS fulfils different requirements. This is done with the system evaluation form
described in Table II. The idea with the form is to subjectively consider to what degree
each specified requirement is fulfilled by the PMS, on a scale between (1) to (7), where (1)
means that the requirement is not fulfilled at all and (7) that it is fulfilled to the highest
degree possible. It should be noted that there is no particular reason for choosing a
seven-graded scale here other than that this scale is easy to use and gives proper details.
2. System class determination. After the requirements have been analysed it is time to
determine the system class of the PMS (third class, second class or first class). This is
done by comparing the system evaluation form to the descriptions of the system classes
in Table I.
3. Revision of the PMS. Depending on the result of the previous steps, there are two different
strategies to follow when revising the PMS:
B Class completion strategy. This strategy is used if some defined requirements within
the system class have not yet been fulfilled to a satisfactory degree. The class
completion strategy simply means that the requirements within the class that can be improved, receive considerable attention until the PMS has completed all class
requirements.
B Class progress strategy. This strategy is used when all requirements within the system
class have been achieved to a satisfactory degree and the organisation is ready to
move up to a higher class. The class progress strategy will require more efforts for a
shorter period of time than the previous described strategy, since many new
requirements must be focused simultaneously and the PMS will be affected by major
modifications.
Case studies
The three-step procedure has been applied and tested empirically in three manufacturing
companies so far during this research (Table III). In short, the suggested requirements and
the three-step procedure showed several strengths when performing their tasks during the
case studies. First, the three-step procedure was rather simple to understand and to use,
which makes it suitable for assisting measurement practitioners. Second, the classification
of requirements made it easy to pinpoint the limitations and weaknesses of a current PMS as
well as show what to improve. For example, several main weaknesses were identified at one
of the company, even though their PMS had recently been severely modified. Third, the
procedure could also be used independently from the origin of PMS (such as if it had been
designed according to the balanced scorecard or other frameworks). Some limitations were
also identified. The system classes and the system evaluation form force its user to rely on
subjective analysis. Significant work is also required by the user, who for example must
identify how to make improvements.
Conclusions
There are numerous performance measures to select from when designing a PMS. This
gives the measurement practitioner many different options for designing a unique PMS.
However, from a practical point of view, there is often no obvious way to choose between
these performance measures and their abundance has caused much uncertainty (Tangen, 2003). It is also very difficult to fulfil all requirements suggested in literature when designing a
PMS. In this article, it is proposed that the question ‘‘What to measure?’’ and ‘‘How to
measure it?’’ should be handled separately in order to facilitate the design process. Further,
it is suggested that the former question should also be linked to different system
requirements depending on the complexity of the PMS (i.e. system class). Finally, a simple
procedure has been described that can be used when evaluating and improving an existing
PMS.
Comments