LAUNCH 281 shown in the model in Exhibit 10.1, there are four major steps in the assessment process. While this chapter is designed to provide an overview of the assessment process, there are many good sources that provide the details. Three excellent sources are David Nadler’s (1977) classic, Feedback and Organization Develop- ment: Using Data-Based Methods, Church and Waclawski’s (1998) Designing and Using Organizational Surveys, which provides a comprehensive analysis of the positives and negatives of methods using technology such as emails or websites, and Watkins and Mohr (2001), Appreciative Inquiry, which addresses assessment from an appreciative inquiry perspective. • Efforts are made to attract, retain, develop, and fully utilize committed and tal- ented people who are a good fit with the organization. • Efforts are made to make working conditions and the work environment a plus rather than a minus. 7. Take Care of Your Customers • Being customer driven to both internal and external customers is a high priority. • Employees from top to bottom are encouraged to know their internal and external customers and their needs. • Building good relationships with present and potential customers is valued. • The organization has a reputation for treating customers well. 8. Build Teamwork • Teamwork is encouraged and developed at the top, within teams, between teams, and outside the organization with groups key to the success of the organization. • There is a one-team mentality with minimal barriers between groups. • Involvement and collaboration are a way of life. 9. Never Stop Learning, Improving, and Building a Great Organization • A strong emphasis is placed on continuous learning, improvement, and devel- opment at the individual, group, and organization levels. • Many opportunities are provided for people to share ideas and make improvements. • Complacency and maintaining the status quo are not options. 10. Keep Score and Get Results • Measures of excellence are simple and clear and allow the organization to know where it stands regarding performance, human resource indicators, culture, customers, and other important measures. • Decisive decisions are made to make needed adjustments to get the best results without damaging the culture or compromising the core values. Practicing Organization Development, 2nd Ed Copyright © 2005 by John Wiley & Sons, Inc. Reproduced by permission of Pfeiffer, an Imprint of Wiley. www.pfeiffer.com 16_962384 ch10.qxd 2/3/05 12:17 AM Page 281 Planning the Assessment A well-planned assessment can provide valuable information that can be used in making organizations, groups, and individuals successful. It can confirm what is being done right and may also uncover blind spots or potential opportunities that, when addressed, result in significant improvements or even a turnaround situation. A poorly planned assessment, however, can demoralize people and cause division. Therefore, it is very important that someone with experience and expertise lead the planning process. This is not the place to try to find the cheapest consultant! The stakes are too high. The size and scope of the assessment, level of expertise of the person or per- sons leading the assessment process, and the commitment needed to make the assessment successful will determine who and how many should be involved in the planning process. A known, trusted, and experienced internal or exter- nal OD practitioner may be able to plan an assessment with minimal involve- ment. However, many efforts require considerable involvement in establishing goals, agreeing on what and who will be assessed, choosing methods, deciding on how to collect the data, and fulfilling the other steps in the planning process. Planning Tips • It is particularly important in planning the assessment to agree with the appropriate leaders on how the results will be utilized and to coach them on how they can help or harm the assessment process. • Assessments can become colossal failures if leaders are not committed to doing anything with the results or if they misuse the results. Some lead- ers have been known to simply file the reports. Others have been known to chastise employees in general or some in particular for the poor results or to make hasty decisions that greatly damage the credibility of the assessment process and may discourage any future assessments. • A lack of milestones can result in missed deadlines and commitments and undermine the credibility of the assessment. Particularly damaging is the failure to feed the results back and involve people in action plan- ning in a timely manner, which dilutes the value of the data and confi- dence in the entire OD effort. Fresh data that are delivered and utilized as quickly as possible are far more motivating than stale data that are neither delivered nor utilized in a timely fashion. Data Collection Technology has made it possible to collect and analyze data very quickly. How- ever, every situation must be analyzed to determine the most effective and real- istic way to collect data. For example, while a wealth of information can be collected and quickly analyzed through questionnaires, people may be more 282 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION 16_962384 ch10.qxd 2/3/05 12:17 AM Page 282 open in face-to-face interviews and communicate things that cannot be picked up in questionnaires. The most frequently used methods of data collection are available informa- tion (an organization’s vision, mission, values, strategic goals, organization charts, turnover rates, and so forth), using questionnaires and interviews (indi- vidual and group), and observations. Ideally, both quantitative and qualitative data are preferable. Quantifiable (most quantifiable data are still subjective) data, which typically are collected through questionnaires, make it possible to know the magnitude of an issue. An average of 2.5 on a 7-point scale, with 7 being the highest favorable score, has a far different meaning from a 6.5. However, quantitative data will not always tell you what is behind the numbers. Qualita- tive data are generally collected through one-on-one or group interviews that make it possible to pick up information and impressions that explore a wide range of issues, including what is behind the issues. Data Collection Tips • Making data collection as user friendly and non-time-consuming as pos- sible is important. • Whenever possible, it is usually important to have a top-level leader or respected professional notify participants in an assessment ahead of time regarding the purpose and importance of the assessment. Some credibility is lost when participants are caught off-guard when assess- ments suddenly appear. • If both questionnaires and interviews are used, ideally it is better to col- lect the data from the questionnaires before doing the interviews so that the interviewers will know ahead of time what to ask and where to probe. In practical terms, this is rarely possible because of the time it takes to collect and analyze data. • Interviews should consist of core questions that are asked of all so there is consistency in the process and the flexibility to go with the flow. • With rare exception, anonymous questionnaires will produce far more valid results than those that are not. • When interviewing is not feasible, well-designed open-ended question- naires can capture some of the valuable information normally gained from interviewing. However, open-ended questionnaires also make data collection and analysis far more time-consuming for the OD practitioner. • If open-ended comments are to be used in the reported findings, which is often the case, it is important to note this in the questionnaire, as some people may feel that their confidence has been betrayed. It is also important to decide ahead of time whether to summarize key thoughts from open-ended comments if a large sample size makes the comments LAUNCH 283 16_962384 ch10.qxd 2/3/05 12:17 AM Page 283 too numerous for people to read it and whether to report comments ver- batim or to provide modest editing to eliminate inflammatory comments and capture thoughts in a more constructive way. (I prefer the latter, as I have seen unedited comments cause considerable division and conflict.) The appreciative inquiry approach does have an advantage in this regard, as the information gathered is positive and uplifting. • Interviewing in a conference room is to be preferred over interviewing in offices, as it takes time to go from office to office, and there are many distractions in offices. • Never use a tape recorder in interviewing! Tape recorders can be intimi- dating, and they double the amount of time required of the OD practi- tioner, as everything must be listened to twice. Most professionals take their own version of abbreviated notes in interviews, although another alternative is to use a laptop computer. However, some people being interviewed may also find this distracting. • Online questionnaires are less likely to experience contamination from participants sharing ideas about the questions than are printed question- naires, but they are also likely to get a smaller response rate for a variety of reasons, such as some people not being proficient at computers and others being tired of spending so much time with computers. A good source for online questionnaires is www.zoomerang.com. • The highest response rates come from the collective approach, where opportunities are offered multiple times during the day for groups of people to complete questionnaires electronically or on paper. However, the individual approach, where respondents can complete question- naires at their own discretion by a prescribed due date, is less complex and labor intensive to organize and manage and provides greater privacy and more time for respondents to think through their responses. • Training may be necessary to provide consistency in the collection of data. This is particularly true of the appreciative inquiry approaches because of the uniqueness of how data are collected and analyzed. Data Analysis The OD practitioner or practitioners leading an OD process usually compile, ana- lyze, and prepare a report of the assessment results. Technology has made it possible to automate the compilation and analysis part of an assessment, and with large numbers of people being assessed, this is almost a necessity for effi- cient assessment. The data go in and a report comes out, complete with attrac- tive charts and graphs. However, no matter how dazzling the technology and 284 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION 16_962384 ch10.qxd 2/3/05 12:17 AM Page 284 resulting report may be, of much greater importance—regardless of how the data are analyzed and presented—is that the analysis should provide valid and useful information and that the information should be presented in a way that is brief and easy to follow and utilize. Some question whether this can be done solely by machine or without an experienced expert or experts doing the analy- sis. Nevertheless, the point is that this part of the assessment and action plan- ning process is extremely important, and a poor analysis or presentation of the results will lead to a low use and/or misuse of the assessment. Data Analysis Tips • A good analysis should involve both deductive and inductive reason- ing. Deductive reasoning from an OD perspective comes from evaluat- ing and making conclusions about data based on general theories and models. This requires OD practitioners to be knowledgeable about OD and what makes organizations successful. Inductive reasoning comes from evaluating data based on what emerges from the data itself and using one’s experience in working with organizations to interpret the data. • Multiple sources of data, such as data from questionnaires and inter- views, make it possible to cross-validate data. In other words, look for data that support one another. • Experienced OD practitioners analyzing data not only look for trends and patterns rather than isolated data that show no consistency with other data but also have the experience to know when someone sees something important that others do not see. • A seasoned person analyzing data will know when some information is best shared personally or left out of formal reports where the informa- tion could be inflammatory rather than helpful. • If sophisticated statistical methods are used in the analysis, good judg- ment determines how much the statistics are used by the analyst and how much are used in reporting the results that may confuse the recipients. • The main body of reports should be kept brief and arranged so that the major findings are summarized, easy to understand, and can be used for action planning. For example, a report could begin with a summary of important assessment demographics, the major numerical results, the highest and lowest rated items, and a summary of major strengths and opportunities for improvement based on an analysis of all sources of data. This could then be followed by the actual results and open-ended comments for each question in appendices. LAUNCH 285 16_962384 ch10.qxd 2/3/05 12:17 AM Page 285 Data Feedback If and how data are fed back can have a significant impact on the success or failure of OD efforts. For example, leaders may not like what they see or have little interest in feeding the data back and therefore forego the feedback process altogether, or may delay the feedback so long that the data lose their relevance or motivating potential. When any of these events happens, there will be a loss of credibility and trust in the OD process and the leaders, which may discour- age people regarding future change efforts. On the other hand, feedback prop- erly handled can energize people, create momentum for change, and ensure that organization members trust and own the data. The feedback process must be designed for each unique situation, but typi- cally includes some version of the following steps: 1. A strategy is developed by the OD practitioner in collaboration with the appropriate people about who gets what information, how, and when. 2. If appropriate, training is provided for those leading feedback sessions so that there will be consistency in philosophy, methods, and out- comes in each session. 3. The top-level leader involved will usually be briefed on the findings and coached on behaviors that tend to help or hinder the feedback process and what the feedback process consists of. 4. The feedback then is usually presented to the primary group it is intended for or cascaded down the organization, starting with top management. Each group receives the information that is appropriate for it to see and respond to. There are exceptions where a “bottom-up” approach is used, with recommendations eventually formulated to pre- sent to top management. 5. While some feedback sessions at this point are designed simply to brief people on the results, most follow a presentation of the results with some phase of the action-planning process. For example, a group may be used to evaluate, validate, and prioritize data and brainstorm alter- natives and then pass this on to an action planning group, or they may serve as the action planning group and develop specific actions. Data Feedback Tips • Data feedback should be a positive and helpful experience and not a negative and harmful experience. Early OD efforts often focused pre- dominantly on what was wrong and how to fix it, took somewhat of a confrontational, in-your-face approach, and took great pride in doing so. Being confrontational, getting people to “spill their guts,” and involving 286 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION 16_962384 ch10.qxd 2/3/05 12:17 AM Page 286 people in all kinds of experiential learning became the badge of courage for some OD practitioners. These approaches gave OD a reputation in many circles for being a “touchy-feely,” confrontational approach led by emperors without clothes, who were so engaged in their own approaches that they did not realize how they, their approaches, and the field of OD were being perceived by many. However, contemporary OD focuses more on understanding present realities and future ideals and how to reach the ideal step by step. It is a much more positive and ener- gizing approach and recognizes that there are helpful, not just poten- tially harmful and undiplomatic, ways to deal with reality and tell the truth. There is also, of course, the appreciative inquiry approach, which focuses entirely on the positive. • Prior experience or training is, of course, important for all involved in the feedback process. However, it is also important to provide at least a brief amount of training at the beginning of feedback sessions to relieve anxiety about the findings and how they will be used, to assure people that the process is intended to be used for helpful and not harmful pur- poses, and to understand how to interpret and best utilize the informa- tion. For example, it may be helpful to point out that it is important to look for patterns and trends rather than some isolated finding. • Those who want to undermine OD efforts can do so quite skillfully by trying to discredit some aspect of the assessment process. They know someone who did not understand how to complete the questionnaire and others who were too intimidated to be truthful, and they can name many variables that influenced the results or may challenge the statisti- cal or sample methods used. This can be overcome by telling partici- pants in feedback sessions that the data are a starting point for them to decide what is important. This minimizes criticisms and the need to jus- tify every detail of the assessment process. • It is important for people to have feedback on the whole organization and to have opportunities to contribute to increasing the success of an organization. However, feedback becomes much more meaningful when the data can be broken down to pertain directly to groups involved so that they can contribute and take ownership for things that directly affect them and that they can directly influence. • While quantitative and qualitative data are both important and each has its advantages and disadvantages, quantitative data have an advantage in motivating change. Executives know how to relate to numbers and are less likely to dismiss such information. For example, when they see that morale received a 2.3 rating on a 7-point scale, it is more likely to get their attention and motivate change than learning that interviews LAUNCH 287 16_962384 ch10.qxd 2/3/05 12:17 AM Page 287 revealed a morale problem. The most powerful motivator is quantitative data that is supported by qualitative data. • Whenever possible, it is helpful to have a knowledgeable person in the organization review feedback reports before distributing them or sharing the results. Minor errors, such as wrong names for groups or misspelled words or numerical errors, can raise doubts about the accuracy of the results. Assessment Methods One of the many interesting aspects of assessing organizations is that a variety of methods can be used to find out what is going on and what is possible. Each method has advantages and disadvantages. For example, interviews and ques- tionnaires are the most frequently used methods, but both have advantages and disadvantages. Interviews make it possible to collect a wide variety of data, find out what is behind issues, and see and dialogue with respondents, but may suf- fer from interviewer bias. However, they also can be time-consuming and expen- sive if a consultant is used and large numbers of people are interviewed. Questionnaires make data quantifiable, but may suffer from respondent bias and are not likely to reveal what is behind the numbers. A summary of the major advantages and disadvantages of various assessment methods is shown in Exhibit 10.3, and a brief discussion of the methods follows. 288 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION Exhibit 10.3. Advantages and Disadvantages of Organization Assessment Methods Method Advantages Disadvantages Examining Existing Data • Past and present • Information already exists • Reality between what is published information • Generally easy to access stated and what is may • Brochures • Quickly familiarizes the differ • Vision, mission, values, person doing the assess- • Some existing goals statements ment with the organization information may be • Organization charts • Efficient and inexpensive difficult to access • Available data on profits, • Relatively objective • Gives only a surface sales, turnover, etc. view of the organization 16_962384 ch10.qxd 2/3/05 12:17 AM Page 288 LAUNCH 289 Method Advantages Disadvantages Interviews • Structured (specific • Makes it possible to • Can be time-consuming questions designed to collect rich, valuable, and expensive if large elicit specific responses) in-depth information number of respondents • Non-structured • Can be used for a wide involved (open-ended questions) range of assessment • Subject to interviewer • Individual purposes bias and influence • Group • Makes it possible to • Interpreting and • Face-to-face probe and elaborate on summarizing information • Phone or electronic information and pursue from interviews can be new lines of questioning difficult • Can build empathy, trust, • May not give a sense support for OD efforts of magnitude or importance of information collected • Risks inconsistencies in interviewing style and interpretation of the results if more than one interviewer involved Questionnaires • Used to gather data on • Make it possible to • Misses qualitative data whole organizations, quantify and objectively especially if open-ended groups, or individuals, analyze results questions not included or a specific focus • Can survey and involve • May not reveal what is • Quantifiable large numbers of people behind the numbers • Open-ended questions • Numbers and a sense of • Lacks the flexibility of magnitude can be strong interviews motivators for change • Subject to respondent • Relatively inexpensive bias, especially if • Can seed the organization respondents lack for change knowledge pertaining to • Can compare before and some questions after results • Interpreting and summarizing data may require a high level of expertise 16_962384 ch10.qxd 2/3/05 12:17 AM Page 289 Examining Existing Information. This part of the assessment begins when an OD practitioner first gets involved and starts learning about an organization. It serves as a first step in the assessment process, even though it often precedes a formal assessment. It could include, for example, studying past and present information available on the organization, including information published by or about an organization, such as an organization’s history; reputation; past and present success; and brochures or publications that may describe the organiza- tion’s vision, mission, core values, strategic goals, organization chart, and prod- ucts and services. It could include information on turnover rates, absenteeism, or even prior assessments. The goal of this search for information is to become a knowledgeable student of the organization. 290 PRACTICING ORGANIZATION DEVELOPMENT, 2ND EDITION Exhibit 10.3. Continued Method Advantages Disadvantages Observations • Formal (specific things to • Organization behavior, • Not always easy to observe and information processes, and systems arrange and can be to collect) can be observed first-hand distracting to those • Informal (observing, • Can obtain a better feel being observed talking, attending for the culture of the • Possible observer bias in meetings, etc.) organization or group interpreting what is • Real-time data observed • Flexibility in terms of what • Can be expensive and is observed time-consuming for the value received • May be difficult to analyze what is observed Live Assessments • Data collected and • Interesting, engaging, • May miss valuable analyzed live at meetings real time, and provides information that comes and workshops fast turnaround and use from a variety of sources • Data collected and of information over time analyzed in real time • With skilled facilitator can • Dependent on having electronically be used with large groups key people present or of people commitment may be lost • Quickly involves people • Somewhat risky in the and builds commitment event that unforeseen to change things can happen that undermine the process 16_962384 ch10.qxd 2/3/05 12:17 AM Page 290 . available informa- tion (an organization s vision, mission, values, strategic goals, organization charts, turnover rates, and so forth), using questionnaires and interviews (indi- vidual and. used for helpful and not harmful pur- poses, and to understand how to interpret and best utilize the informa- tion. For example, it may be helpful to point out that it is important to look for. often precedes a formal assessment. It could include, for example, studying past and present information available on the organization, including information published by or about an organization,