Quantity vs Quality
- Dudi Rozental
- Oct 1, 2007
- 4 min read
Let's start from the end...
A young man named Albert Einstein once claimed:
"Everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted..."
Almost everyone in knowledge management needs to measure and quantify their activity outputs. In this world, measuring what needs to be measured is important, not what can be measured.
As knowledge managers, we are sometimes required to present success metrics to our supervisors for the technological platforms we requested, established, and maintained with determination.
Usually, as soon as the requirement arrives, we head to the report generator and start counting visits to specific pages, clicks, virtual discussions, and any other type of action that hints at using the tool we represent. The easiest metrics to show are statistical data about visit quantities, where the metric is clear: the more users visit, the more popular the page.
Statistical tools are supportive infrastructure for intelligent virtual environment management. However, one should be careful when concluding high statistical rankings of sites, pages, etc. Statistical popularity doesn't necessarily indicate value-based significance. Visit quantity is not an ultimate measure of organizational learning and knowledge management because it only indicates how many times a user reaches a certain place—and that's it.
First of all - quantity doesn't indicate how the found information was used, the value created from it, its contribution to subsequent actions, meaning its importance to the user, and the creation of new business value for the organization.
In our world of knowledge management, what creates value is a qualitative measure (What did I find? What is the meaning of what I found? What action am I taking as a result? What new value was created?) rather than quantitative measures (How many touches/hits/visits/return visits?).
Additionally - often, the most meaningful metric isn't a large number of 'hits' but rather a single hit that brings great value to the user and, consequently - to the organization.
Usually, the most popular page is the organizational phone directory, cafeteria menu, or similar organizational information. This is very important, but it's trivial. Examining low visit quantities to sites or pages dealing with the organization's core subjects is more important. This can give a more accurate indication of whether the knowledge items we worked so hard to input are relevant to anyone.
Are people consuming knowledge or just shouting that it isn't there? When consuming knowledge, is it knowledge relevant to the main subjects the organization deals with?
Managing a virtual environment without supporting tools that analyze behaviors, frequencies, movement, flow, and other variables in knowledge infrastructure is very difficult. However, as I mentioned at the start, it's more important to measure what needs to be measured than what can be measured. As such, knowledge management measurement should first and foremost examine whether organizational goals are improved following knowledge management activity.
To examine improvement in goal achievement, business metrics need to be defined. These metrics are content-dependent and not directly related to knowledge management but to its results. They are called output metrics.
The metrics are defined, first and foremost, to examine the change following knowledge management. During the measurement process, it's recommended to implement several assumptions on which the metrics will be built:
Don't turn the means (measurement) into an end in terms of investment in the subject.
Activity measurement isn't a substitute for managing the subject but rather one of several means among managers seeking to manage knowledge in their unit.
There's a balance between performance metrics and output metrics. As time passes, the balance will shift in favor of output metrics. The reason is that performance metrics are very important for supporting the implementation process, so their importance decreases over time.
There's a balance of quantitative and qualitative metrics for learning trends on the one hand and for getting an in-depth picture.
The trend of measurements over time is important. Knowledge management is a cultural change, and some populations move slower. This is legitimate, but constant progress must be ensured.
Both the level of use and the level of knowledge contribution in various subjects should be examined.
So, what are those qualitative metrics through which knowledge management in an organization can be measured?
These metrics, as I argued, should measure the organization's business output:
Time savings resulting from knowledge management.
Cost reduction (effectiveness) in performing routine work tasks.
Reduction in errors that resulted from lack of knowledge management.
Quality improvement resulting from knowledge management.
It's recommended that the metrics be continued refining according to the business needs that knowledge management deals with.
The qualitative metrics can be collected in several ways:
Success/failure stories within the implementation (by the knowledge management team) and ongoing activity (by key position holders).
A (Semi-Annual)annual satisfaction questionnaire (survey) will be conducted for each type of tool and filled out separately according to the content worlds where the tool type is implemented (qualitative!).
Quarterly sample interviews will be conducted for some tools/content worlds each quarter.
When possible, a financial assessment will be made regarding the benefits/savings resulting from knowledge management tools.
Measurement isn't an easy process. Beyond that, it isn't simple since it's difficult to quantify the true advantages of knowledge management, certainly in the short term and on a regular periodic basis. Despite this, both due to organizations' pressure for ROI and because it's the right thing to do, every organization should try, with the transition to systematic knowledge management, to find the golden path for measurement that will be meaningful on the one hand but won't become the central path on the other.
Comments