Measuring Knowledge Management Processes
- Dr. Moria Levy
- May 1, 2003
- 4 min read
Many organizations measure knowledge for various purposes. Some use it as a means to drive activity, while others use it for compensation, and there are many other reasons.
The primary purpose of knowledge management activity in organizations is to achieve and leverage organizational goals. Therefore, knowledge management measurement should, first and foremost, examine whether organizational goals are improved as a result of knowledge management activities.
To examine improvement in goal achievement, business metrics must be defined that are content-dependent and not directly related to knowledge management, but to its outcomes. Metrics of this type are called output metrics.
It's not always easy or possible to examine output. This is because output is always influenced by multiple parameters that change simultaneously, making it difficult to isolate the impact of knowledge management on outputs. Additionally, extended time is needed to examine consistent changes in outputs, while measurement is required much earlier. To overcome this problem, managers use, in addition to output metrics, performance metrics that examine the utilization of knowledge management, not just its products. The related metrics are referred to as performance metrics (and/or input metrics).
The metrics are defined, as mentioned, primarily to examine the change resulting from knowledge management, but also to assess the level of knowledge management implementation across various topics and populations, thereby improving and fine-tuning solutions and the implementation plan in different units.
Below are several principles for a possible measurement process
It's recommended to establish several assumptions on which metrics will be defined:
Don't turn the means (measurement) into an end in itself, in terms of investment level in the subject.
Activity measurement is not a substitute for managing the subject, but rather one tool among several that managers use to manage knowledge within their unit.
There's a balance between performance metrics and output metrics. As time passes, the balance will shift in favor of output metrics.
There's a balance of quantitative and qualitative metrics for learning trends on one hand, but also for obtaining an in-depth picture on the other.
The trend of measurements over time is important. Knowledge management is cultural change, and there are populations where the pace is slower. This is legitimate, but progress must be ensured at all times.
Both the level of use and the level of knowledge contribution in different subjects must be examined.
The measurement is not intended to create a hacker-antivirus element. When knowledge management becomes too central a tool, employees are found who devise methods to record positive measurements (usually to receive compensation) based on superficial or non-genuine knowledge management practices, etc. In this case, knowledge managers invest efforts in filtering the above; employees also exert more effort, and beyond the battle of minds, everyone loses.
It's recommended to maintain model simplicity (few metrics - simple and understood).
It's recommended to integrate measurement results into existing reports and regular summaries within the organization, making them part of the organizational culture.
Quantitative Metrics (will serve mainly as performance metrics):
Based on the above assumptions, it's recommended to measure for each mechanism (tool) and each group the performance metrics:
Use/addition-update: breakdown by items.
Use/addition-update: breakdown by users.
Grouping by groups enables tracking where additional or improved implementation is needed, where enforcement and encouragement from management are required, and where recognition should be given for usage achievements.
Qualitative Metrics (will serve mainly as output metrics):
Time savings resulting from knowledge management.
Cost reduction (effectiveness) in performing routine work tasks.
Error reduction resulting from knowledge management.
Quality improvement resulting from knowledge management.
Qualitative metrics will be collected through:
Success/failure stories within implementation (by knowledge management team) and in routine activity (by key role holders).
Semi-annual satisfaction survey (questionnaire) that will be conducted for each type of tool and filled separately according to content worlds where the tool type is implemented.
Quarterly sample interviews will be conducted each quarter for some of the tools and content worlds.
Where possible, a financial assessment will be conducted to evaluate the benefits and savings resulting from the use of knowledge management tools.
Organization-wide Metrics:
In addition to metrics for each of the activities (tools and mechanisms), the level of use, quality, and cultural state of overall knowledge management in the organization will be measured. These will be measured through summarizing the results of various existing metrics via an annual questionnaire. It will include a reference to:
Usage:
Number of users.
Usage profile.
Level:
Content quality.
Information currency.
Outputs:
Time savings.
Task effectiveness.
Number of recurring errors.
Quality improvements.
Culture:
Knowledge management initiatives that didn't result from knowledge management group activity.
Success stories.
Recurring feedback.
Organizational learning.
Summary:
Measurement is not an easy process. Beyond that, it's not simple because it's difficult to quantify the real advantages of knowledge management, especially in the short term and on a regular periodic basis. Despite this, due to both organizational pressure for ROI and the right thing to do, every organization should strive, with the transition to systematic knowledge management, to find the golden path for measurement that's both meaningful and not overly dominant.
And finding this golden path is what every knowledge manager in their organization should work diligently to achieve.
Comments