Reporting Results: Managing Measurement

The ability to provide valid measures of success for any learning program is key to achieving support for future initiatives. Knowledge training and assessment programs can provide data in multiple formats to transform learning business operations.

Return on investment (ROI) and risk mitigation are important drivers of many corporate learning initiatives. But without proper assessments, organizations lack the ability to measure either of these attributes. The need for valid measurement is at the very core of every learning program. Knowledge training and assessment programs provide an abundant source of rich data, in multiple formats, that could transform the way your business operates.

If not carefully supervised, the often vast amount of data generated by learning and assessments can make information management difficult. This information might include training durations, time spent on different subject areas and assessment exam questions, integrated survey responses, employee demographic information, question sequencing, correct response and test-taker response ratios, exam results, biometrics and other security data. To grasp the true potential and ROI of any corporate learning program, there must be a valid and reliable system for capturing, storing, evaluating and communicating the sheer volume of this data. Digital data management services make this possible.

Data management lessens the administrative burden associated with managing and accessing the data from learning programs. Organizations that take advantage of these services have actionable measures of job knowledge and skill requirements, and the proof needed to validate learning and development investments. Utilizing this knowledge is essential to the creation and maintenance of any corporate-sponsored learning program, and the key to success is high-quality, trusted data that allows you to learn more about your employees’ abilities and properly allocate your learning budget. As a result, you are able to make better, more informed decisions regarding the future of your corporate learning programs and ultimately, the future of the business.

The introduction of digital data management services is a forward-looking practice. Its ability to meet every business’s objective of increasing customer satisfaction is revolutionary and is making its way around the business world. It is actively used throughout the certification and licensure industry. “Data management services provide the ability to more efficiently manage and grow our certification programs,” said Bill McCulloch, training and certification programs manager for Adobe Systems. This kind of self-sufficiency permitted by data management services puts the power of the learning program in your hands.

According to research outlined in the American Society for Training and Development’s (ASTD) 2004 State of the Industry Report, the percentage of payroll invested in employee learning in 2004 increased from 2.2 percent to 2.5 percent among all companies. The percentage of learning expenditures outsourced, as well as the number of hours spent learning, has increased steadily from 2002. It’s worth noting that the use of technology for delivery, specifically Web-based delivery, continues to gain market share. Given the increase in money spent on educating employees, the need to prove ROI is more important than ever. This is further evidence of the increasingly important role knowledge assessment plays within the corporate market.

In addition to Web-based delivery, consideration also must be given when delivering more sophisticated assessments into secure testing centers. Advanced items, such as those involving workplace scenarios, complex sets of activities and simulations, must be accounted for in your data management planning. The need for robust data management becomes even more important if you offer assessments internationally, particularly in multiple languages.

Managing this information through the right data management service allows you to isolate learning performance differences and determine potential risks. This simplifies the recovery, maintenance, analysis and communication of learning results. Also, it can be essential to the ongoing advancement of the learning process and the mitigation of risks that come from poor hiring decisions and employee placement.

How Does It Work?
With the right learning measurement service in place, you are meeting the three key functional areas of any successful data management program:

  • Deployment of a robust framework to support the collection, access and analysis of data.
  • Development and ongoing maintenance of strict program privacy and security measures.
  • Providing the highest level of customer satisfaction possible.

Through this access to comprehensive and simplified data and support, you are in control of your learning program. You can turn this data into knowledgeable and opportune action—action that will positively impact the success of your learning program and the business. (See Figure 1.)

To start, you have the option of hosting the data management technology or outsourcing the hosting responsibilities. While the technology could be designed, implemented and managed by you and your staff, outsourcing can be more efficient and cost-effective. These outsourced programs are normally configured and customized for each client. Choose a vendor with solutions that are scalable, flexible and secure, that acclimate to your current business and that have the flexibility to evolve and grow with your business. The best are seamless despite your operational environment and, used appropriately, promise to become a transparent part of your organizational culture.

What’s more, fully hosted solutions eliminate the need to implement and sustain data management hardware, as well as the need to maintain the expertise necessary to manage the technology. Data can be sent directly to the outsourced vendor for instant uploading. In turn, this information is instantly available for your review and use. Standardized data also can be made available through a Web interface. No matter what size your learning program is, you won’t need programming skills to create or update the system’s content. Quality vendors will collaborate closely with you and your staff to determine the full requirements and map the right data management system and content for your business.

Executive-level management needs to focus on the results of your corporate learning programs, not the technical infrastructure that supports them. Once implemented, those in your organization who support the learning programs will be relieved of the administrative time and headaches associated with customer support, allowing them to more fully and efficiently address customer satisfaction issues. Instead of mediating employee concerns, you will be able to focus these resources to positively influence business outcomes. You will help your staff work more effectively and provide your customers—your employees—with greater self-sufficiency to manage their personal learning performance.

Digital data management services often include real-time access to customizable, comprehensive reports. These reports configure the data to identify learning result patterns and trends. Ideally, this information serves three main purposes for the business, all designed to mitigate risk and prove ROI:

  • Triggers program maintenance and development.
  • Manages the administration of the program.
  • Directs strategic and operational business planning.

For example, you will be able to identify areas that employees have mastered or, equally important, areas that are common pitfalls. This allows you to effectively allocate your learning budget to the areas that need improvement. Acting on that information virtually guarantees an effective learning program.

Reports can easily include statistical information and synopses of employee, training and assessment exam performance. Assessment results are analyzed for item difficulty variations and overall performance. With this knowledge, your assessment questions or objective parameters can be altered to match the evolution of the corporate strategy.

Report information also can include training history, assessment scores and learning patterns or trends. All reports can be customized to the business and localized into different languages. Each is exportable in a variety of formats, such as PDF, spreadsheet and word processor documents.

Format options are useful when creating employee performance and objective planning reviews or reports. As a result, you will have a single, shared view of your employees’ capabilities with a complete audit trail of changes and updates. With this access, you can promptly check and confirm employee history and resolve any learning gaps.

For example, let’s say you want your HR employees to achieve HR certification status. Using your reports, you will be able to determine all of the employees who started the path toward certification but never achieved this status. Using this data, you now are able to target that audience and promote the benefits of completing the HR certification track.

Often, corporations use the results of pre-employment knowledge assessments to measure personality traits against predetermined job requirements. By using the data derived from these knowledge assessments, you can enhance employee recruitment and hiring practices, ensuring the right applicants are hired and placed in the right roles within the company. As a result, hiring costs are reduced through lower employee turnover. As these employees’ performance is monitored on an ongoing basis using the data-management reporting function, the business has the assurance and proof it needs to align human capital with business objectives.

The best data management services maintain strict privacy and security levels. The system should protect your data using multiple levels of sophisticated security protocols. These security measures include user sign-on requirements and user verification. During implementation of the data management technology, you will be given the opportunity to define who from your business has access, and their user roles and rights, including such controls as who can view and create data, and who has permission to edit and delete. You will be given the confidence you need to know that only the individuals you assigned are accessing the data appropriately, as outlined by their user rights.

Your Learners
Organizations that provide employees with access to formal learning programs face a variety of fascinating and trying opportunities. If used correctly, the data derived from these opportunities can not only improve risk mitigation and the way learning programs are administered, but also provide the knowledge needed to get closer to the workforce and their needs.

The improved synergy between employees and the business comes from timely and accurate feedback of their training results, benchmarked against their learning objectives and a snapshot of their personal learning progress.

Data management simplifies the process, while improving productivity, reducing costs and increasing customer satisfaction. Theoretically, employee performance could be monitored 24 hours a day, seven days a week. This anytime, anywhere access to such critical information is a powerful method for achieving business results. Correctly managed, this knowledge can enhance learning and development, allowing you to truly manage the program, rather than just the program’s data.

Use this knowledge to engage and develop the value of your human assets. Assess the quality of learning and development efforts, and institute ongoing improvements and updates to further mitigate risks. Analyzing and communicating the data associated with learning programs is made less expensive through the sheer volume of easily accessible business information. Having this kind of visibility into the learning program will allow you to make decisions that positively impact employee performance, thereby enhancing profitability.

Do something proactive for your learners: Track your human capital and help improve the quality of their learning program. Make real-time business decisions and grow the value delivered by your learning program. This is true business intelligence.

Michael Brannick is president of Thomson Prometric. He can be reached at

June 2005 Table of Contents