
This question is likely on the minds of many Talent Management leaders and Chief Learning Officers who are charged with bringing the right development solutions that narrow the competency gaps between what leaders can do today and what they must do tomorrow.
Surprisingly, many of them are still uncertain and reticent about deploying mechanisms to confidently uncover the answer with compelling and objective evidence. Instead, they have largely relied on attendance figures or “smile sheets” to determine whether their investments in the development of their executives – which for large Fortune 500 organizations can easily run into millions of dollars – translates into greater performance by leaders and strategic achievements for the organization.
The rationale they offer is often the same: how would one demonstrate with confidence that an executive’s participation in a development program solely accounts for changes occurring in their performance down the road?
Their hesitation is understandable but can easily be challenged with the consideration of simple solutions that, when applied with the proper level of discipline and focus, yield straighforward conclusions.
So how can it be done? Four simple questions provide the foundation of a methodology to evaluate any executive development program:
1.What is the experience of executives going through the program?
Start by asking the program participants how they are experiencing the program and determine how consistent it is with the way it was intended to be. For example:
•Did they attend and fully engage during the entire program?
•Was the program executed according to plan?
•Have participants acquired the intended skills and knowledge?
•Are participants using the newly-acquired skills and knowledge back on their jobs?
This may appear to be quite basic, but even this fundamental level of assessment often gets overlooked. There are a number of ways to accomplish this assessment can be done, including having checkpoints in the course of the program to monitor attendance and engagement in the program activities, knowledge tests, and behavioral changes resulting from program attendance.
2.How are participants using their learning and program’s takeaways to change the way they carry out their work and achieve results?
Several techniques can be used to answer this question including self-report surveys, focus groups, and observational methods. Some organizations have found great success using Behavioral Event Interviewing (BEI), a powerful interviewing technique which solicits evidence or examples of a specific competency or skill from interviewees. These data can then be organized in a dashboard that gets shared with senior management.
3.What business results are achieved by the program?
This is where the rubber meets the road and most organizations will want to know what is the final return on investment of their executive development program. Putting a dollar value on the results of a development program can be a difficult task but not an impossible one. A good place to start is to lay out a simple model showing hypothesized relationships between the executive skills and knowledge targeted by the program and “outcome” measures – those indicators of operational and financial performance that matter to your organization. For example, a Private Equity firm may be most interested in looking at the link between deal partners’ competencies in Risk Management or Analytical Reasoning and their investment performance.
4.What can be done to fix or maximize the impact of the program?
Good program assessment should logically translate into robust recommendations for program enhancement priorities. For example, when we administer a survey to program participants, we look at respondents who provided very high scores or very low scores for the program evaluation (along with their post-program 360-degree competency scores if these are available). We then conduct follow up in-depth interviews about what worked and what did not work for them in the program and focus on these results to determine which program enhancements should be considered.
If you wish to know more about our program assessment methodologies and tools, please contact us as info@fisher-rock.com.