Validation on the DecisionWise Leadership Intelligence 360 Degree Feedback Surveys
The Leadership Intelligence 360 degree surveys are designed to collect relevant and actionable multi-rater feedback for development and appraisal. Versions are specifically designed for Executives, Business Leaders, Team Leaders, and Individual Contributors. These surveys contain the most important leadership competencies and behaviors based on over 15 years of DecisionWise research. They also include a special Derailer section that identifies behaviors that are detrimental to leadership success. Three open-ended questions are used to identify key deficiencies, leadership strengths, and specific things a leader can do to become more effective. Combined with coaching, the unique feedback produced from the Leadership Intelligence 360 increases self-awareness, acts as a catalyst for change, and creates a road map for improvement.
The following provides a brief overview of the extensive research conducted by DecisionWise between 1997 and 2014 to develop the Leadership Intelligence 360 survey tools.
Competencies and Behaviors Development: Peak Performance Study
This is the original research that resulted in the core competencies and behavior statements for the Leadership Intelligence 360 instruments. The study consisted of a comprehensive literature review of academic articles and management books and publications in the subject of leadership competencies between 1960 and 1997. In addition, the researchers conducted extensive interviews with 70 leaders that were identified as excellent leaders across a variety of industries and levels of management. The study included a survey of 250 leaders ranking the importance and performance level of over 100 competencies. The outcome of this study produced a set of core leadership competencies and behaviors to be used in the Leadership Intelligence 360 Feedback surveys.
Leadership Derailer Development
A thorough assessment of what effective leaders do (the competencies) did not capture the impact that a negative tendency may have on a leader’s effectiveness. In other words, the multiple virtues of a leader could not compensate for the presence of a derailer- a negative behavior that stands in the way of achieving the leader’s full potential. The initial development of derailers is based on existing research by the Center for Creative Leadership (Executive Derailment Study), Hogan & Hogan (Why CEOs Fail), and the Extraordinary Leader study of Fatal Flaws. From that foundation, we conducted a survey among corporate executives at three different companies to validate the DecisionWise set of derailers.
Validation through Field Application and Observation
Between 1997 and 2009 DecisionWise conducted 360-degree feedback assessments with thousands of leaders at all levels of the organization, representing different industries, company sizes, professional backgrounds, world regions and age groups. For each project, we worked closely with line managers and HR professionals to refine the survey instruments. The researchers were involved in coaching each leader on the results of the 360-degree feedback and observing through direct experience the validity and reliability of the instrument as a predictor of leadership effectiveness. These studies provide the basis for the construct validity of the survey.
As part of refining the survey instruments, we conducted statistical analysis of the 360 feedback data available at three different stages. The data was analyzed in four areas:
- Factor Analysis: Factors contributing to the relationships between the items
- Variance: Descriptive statistics of central tendency and variance among the items
- Reliability: Reliability of the survey scales and the overall survey
- Validity: Concurrent validity in measuring leadership qualities
The results from these studies and analyses were used to refine and improve the Leadership Intelligence 360 surveys over the years to culminate in the current versions being used today. These surveys have proven to be superior instruments for collecting actionable and relevant feedback to build effective leadership.
360 Survey Customization
DecisionWise specializes in creating customized 360-degree feedback surveys that are relevant to the client’s needs. Customized surveys are best when the client already has a leadership competency model in place or would like to align with a specific strategy or training program. Customized surveys also promote greater buy-in among participants because they feel the items are relevant to their responsibilities. Most DecisionWise clients use a customized survey for these reasons.
When creating a 360 survey, a DecisionWise senior consultant works with the client to define the competencies and behaviors that are unique to leadership success in the organization. Existing items from our extensive database are used to create the survey or new items are developed as needed. The final survey is reviewed with the client for accuracy and reliability. DecisionWise can also conduct a validation study on the survey by conducting a small pilot administration and analyzing the results.
Based on the survey results from thousands of managers over the years, DecisionWise has created an extensive database of comparison data that we use for benchmarking purposes. Clients can choose to show benchmarking comparisons on individual reports using up to two of the following:
- All participants in the DecisionWise database
- All participants in a specific industry
- All participants of a particular management level
- All participants in the client’s company
- All participants in a department or team
- The top 10% (or 25%) of participants in the database
Primary comparison scores (norms) are shown using a red vertical line on the report. A secondary norm, if used, is shown using a green vertical line. These indicators help participants to understand how their scores compare and where to focus their development efforts.