Needs Assessment
- Amit Starikovsky
- May 1, 2006
- 8 min read
"Oh how I wanted a dog - all I asked for was a dog. Every night I prayed for a dog, and dad brought me a little cat" (Yonatan Geffen, "Oh how I wanted a dog").
Think for a moment about that child who so wanted a dog, but dad brought him a little cat: The child wanted a pet, so dad bought a cat. What could be more natural than a father bringing his child what he wants or needs? But the father, in this case, failed. A cat is indeed a pet, but not what the child wanted...
Solutions are built and adapted for small and large organizational units as part of daily organizational routines. After characterizing the solution and building it, we sometimes encounter user problems. Usually, these problems emerge during the implementation phase and manifest as low usage of the solution built for them and/or low satisfaction, contrary to expectations, just like that child who wanted a dog but received a cat.
To properly adapt the solution we are building, it is extremely important to conduct a needs assessment. This is a process of approaching the target audience for whom the solution is intended, examining their knowledge needs, and reflecting on the solution to be built. Needs assessment is the foundation of the characterization process, resulting in a detailed definition of user needs and the nature of the appropriate solution.
It's important to note that this process is integral to any solution built within the framework of a knowledge management project (and much beyond) – from organizational and/or professional portals to insight repositories and lessons-learned processes. The process sounds trivial, but it isn't: if we expand our resource investment in it too much, we will create expectations among users who will "fantasize" but ultimately receive nothing from us beyond sympathy (as we cannot provide solutions to everyone simultaneously); if we expand the resources invested in this stage too much, we will waste time in the not-so-long window of opportunity we received from the organization to prove that knowledge management can indeed help them; the same applies to money. Therefore, as with other organizational knowledge management processes, we must be very focused but ensure we are on the way to the desired dog, not a cat.
The needs assessment process involves both content providers and potential users. This overview will focus on potential users—the knowledge consumers—as this is a more complex and sensitive process.
The needs assessment process has two stages:
The first stage is defining the target audience. Who is the solution we are building for? What are the characteristics of the target audience? Are they computer literate? Accurate answers to these questions will prevent extreme mismatches, such as creating a portal for production workers who lack computer access and are a priori computer illiterate.
In the second stage, choosing the method by which we will sample potential knowledge consumers is important. In our experience, there are three effective methods for this purpose: questionnaires, interviews, and focus groups. Of course, there are other legitimate methods, but due to limited scope, we will focus on these methods.
Questionnaire: The questionnaire is a useful tool for sampling a large and broad group of users. It allows for the collection of extensive data in a short time with relatively little effort, with most of the energy directed toward its proper construction. The collected data can be easily analyzed using appropriate statistical software such as SPSS and SAS, or even basic analyses can be performed using Excel.
The notable disadvantage of using a questionnaire is that usually, to facilitate completion, we use only a limited number of closed questions, revealing a little while concealing much more. The result in these cases is that the information collected is superficial and not deep enough. Additionally, many biases related to respondents can give an incorrect direction.
However, the main disadvantage is qualitative. People who do not know what knowledge management is tend to answer questionnaires, which are inherently more superficial, by presenting their "perception of attitudes" toward the subject. This perception often differs from the actual situation.
Nevertheless, when dealing with a large and dispersed population, the questionnaire can be suitable for defined purposes, especially as an opening salvo that will help provide a general direction of needs (when backed by in-depth interviews). We recommend not relying on a questionnaire as the sole tool for conducting needs assessment but as a complementary tool.
Several important tips:
In the introduction, be sure to state the purpose of the questionnaire, the purpose of the process, and its background.
Ask questions simply and unambiguously.
Recommended number of questions – no more than 10.
Generally, closed questions are recommended, but this leaves room for users to add free text.
Be sure to thank those who responded to the interview for their time and contribution. The best way is to send that person a thank-you letter/email, with their direct manager copied for information.
Interview
The interview method, especially in-depth interviews, is the most suitable for conducting needs assessments.
It is recommended that the interview be planned: define the desired outcomes from the interview and build a questionnaire to guide the interview toward achieving them. It is important to use the questionnaire as an outline for the interview, not as the interview itself, because sometimes it turns out that the direction we thought of is untrue. We need to use it wisely so that it doesn't fix us and prevent us from identifying our real needs.
As part of preparing for the interview, think about places where objections are likely to arise from the interviewees. Proper planning of objections will allow dealing with them in real-time and addressing them already at this activity stage. Remember: objections that we don't deal with may resurface at later stages, so it's very important to address them at this stage.
It is especially important to set expectations before the interview. We should send interviewees a short letter describing the process and explaining why we want to meet with them. In addition, it's advisable to attach up to 5 guiding questions as points for thought, allowing interviewees to prepare in advance. It is also recommended that the activity leader or a managerial figure familiar to the interviewee should sign the letter to create a certain degree of commitment.
The significant disadvantage of using this method is the resources it requires, mainly regarding the physical time of an interview conversation. An interview of less than an hour may not be effective, while an interview of over an hour and a half may be tiring, especially for the interviewee. As part of preparing for the interview, it's important to converge on a reasonable timeframe that will allow efficient use of time without exhausting the interviewee.
Several important tips:
It is recommended that more than one interviewer be present. However, avoid a situation in which five interviewers face one interviewee.
It is possible and even desirable to conduct interviews with more than one interviewee, especially when dealing with interviewees from the same content world; however, it is not recommended to interview more than four interviewees simultaneously.
Be sure to set expectations at the beginning of the interview, even if a preliminary letter was sent, as not everyone bothers to read and prepare.
Some interviewees are stressed by the fact that we are documenting their words. Remember that what is said in the interview will be used only within the characterization framework.
Be attentive to the words’ tone and "music,” especially to hidden objections. Cynicism is the first sign of this. However, avoid asking direct and aggressive questions to examine these issues.
Be sure to thank those who participated in the interview for their time and contribution. The best way is to send that person a thank-you letter/email, with their direct manager copied for information.
Focus Group
The focus group method is particularly suitable when the target population is large, heterogeneous, and dispersed. Of all the methods, it seems to be the most complex, but it helps achieve a more accurate picture of the needs of that population, albeit in a slightly less focused manner.
This method's complexity stems from the need for meticulous preparation, so it is recommended that we use it only after we have conducted interviews with knowledge producers/providers and have a good and focused picture of the organization and the content infrastructure of the solution we wish to build.
Key advantages to using this method: As mentioned, it is possible to simultaneously sample a large group of potential users, where the existing group dynamics naturally create a joint brainstorming and allow reaching more needs, some of which we did not anticipate existed.
Another advantage is that the focus group creates organizational visibility and serves as a marketing activity, helping to establish the desired solution.
Several important tips:
Ensure that the group is heterogeneous and covers all the different units so that the needs encompass as many users as possible.
The recommended size is no more than 10 people per group. Above that, multiple meetings are recommended.
If key people are known for their opposition, it is recommended that they be invited. Their participation in the group may contribute to engaging them in the process, mainly because group dynamics typically moderate. However, in the case of a general atmosphere of opposition, it could intensify. In such cases, it is recommended that this method be avoided.
Let people express their opinions, but ensure the discussion does not deviate from its purpose.
Divide the meeting into at least three parts: group work, discussion, and summary.
It is recommended that the working groups include a facilitator who will ensure that the discussion proceeds in the desired direction.
Be sure to conduct the focus group as close as possible to the final stages of characterization. Remember that participation in a focus group creates expectations among users, so it's important to maintain continuity. Too long a period will damage the momentum created.
Be sure to thank those participating in the focus group for their time and contribution. The best way is to send each person a thank-you letter or email, with their direct manager copied for information.
So, how do we know which method to choose?
Here are several parameters to consider when choosing the method:
Population size - A focus group is recommended for a large population, but a questionnaire can be complementary.
Cooperation - When the level of cooperation from potential users is high, a focus group is recommended. When it is low, it is recommended to avoid questionnaires, as they are likely to receive laconic and non-advancing answers.
Availability - If dealing with people who are unavailable, it is recommended to avoid interviews and try to use a focus group, which is the most efficient method.
Openness - If there is an atmosphere of openness in the organization, it is recommended that a focus group be conducted. In organizations where the level of openness is low, it is recommended to use questionnaires, where the element of anonymity will help identify the desired information.
In conclusion, the needs assessment process is critical and essential in building a new solution in an organization. We strongly recommend not avoiding and communicating it as much as possible within the organization.
We all want to avoid building "white elephants." Conducting a focused, systematic, and structured needs assessment process will help create an effective solution that addresses real needs.
Needs assessment does not end with the establishment of the solution. It is an ongoing process designed to ensure user utilization and satisfaction. More to come on that.
Comments