Frequently Asked Questions
Need help? Check below to see if we have posted an answer to your question!
We typically analyze the comments using self-coded themes. That is, we ask the respondent to select the theme from a list of common themes we have aggregated over the past 15 years.
Generally, we advise that specific comments should not be used as the basis for specific action at the corporate level, or even action at smaller units of the business. However, they are very valuable to better understand respondent nuances.
We also can provide more in-depth comment analysis. We apply sophisticated software to analysis trends, themes and sentiment.
TalentMap recommends as much transparency as possible. The ideal is to share all comments “as is” but very few management teams have the fortitude. Full buy-in is needed from your CEO for this approach. For most organizations, the following seems to work:
- All members of the senior team see all comments
- Directors see all comments that fall in their respective portfolios
The challenge is how or whether to disclose comments to managers.
Is the manager’s group large enough to protect anonymity?
It’s human nature to try to guess who said what. Typically comments are NOT to be shared when a manager has less than 20 direct reports.
Is it possible to share comments without causing too much upset?
Some managers are sophisticated and can put “hurtful” comments in proper context, but there are many less able to do so, and they’ll struggle with negative comments.
A way to bridge these issues is to provide managers with a random or representative sample of comments from the larger department or division. In other words, all managers in a given department get the same set of comments. This approach gives managers exposure to the tone and nature of verbatim comments, which can be very direct and personal.
These seven factors are key to improving engagement:
1. Focus on the key drivers and root causes
2. Maintain the dialogue with employees
3. Involve employees "appropriately" in the change process (i.e. to provide input and test solutions, not to lead the design)
4. Select only a handful of change initiatives and focus on them
5. Integrate the chosen change initiatives with strategic and operational planning
6. Ensure accountability and ownership for the change initiatives
7. ensure strong advocacy from the CEO
To measure engagement drivers using multiple regression analysis, there need to be approximately 30 individuals for every dimension measured. In a typical TalentMap survey, we measure 12 dimensions of engagement.
Accurately measuring all of the drivers requires 360 respondents.
That said, for smaller sample sizes, a driver analysis could be done using a linear regression model instead of a multiple regression models, but it would not be an apples to apples analysis, and it would be inconsistent with how the drivers are measured for the company as a whole.
TalentMap uses linear regression analysis for smaller organizations and we find that a minimum sample size of approximately 30 people is needed to effectively do a driver analysis. However, multiple regression analysis is the preferred method when possible as it takes into consideration all the variables in the relationship and not just one variable at a time, painting a more accurate picture.
The neutral category provides an option for the respondent who truly does not have an opinion on a question so that they are not forced to respondent positively or negatively.
If you do not include a neutral option, you are forcing employees to make a choice, when in reality there are times when people have no opinion. Without a neutral rating, the tendency is for people to choose the more positive rating, and accordingly, it will falsely skew results towards favorable.
Research confirms that you should not use an even-numbered rating scale (e.g. 4-point or 6-point). TalentMap utilizes the five-point Likert scale, the most common scale in attitudinal research because a five-point scale is widely considered to represent the “natural” number of opinion plateaus or levels.
Key Driver Analysis is a powerful way to derive business value from your employee survey data. It helps you target improvement efforts to get the 'biggest bang for your buck’ by identifying the key leverage points for improving the engagement at your organization.
How does it work?: A Key Driver Analysis uses a statistical technique known as multiple regression to identify patterns in the responses of employees. By using multiple regression, it is possible to identify which questions have the strongest and most meaningful association with your employee engagement score. When you prioritize your intervention efforts based on the results of statistical analysis, the results are far superior to simply basing intervention efforts on the lowest scoring items in a survey. Typically the lowest scoring items (like compensation) have a very weak connection with employee engagement, and thus even large investments to improve these low scoring items do not result in large improvements in engagement.
"Key driver analysis is a powerful way to derive business values from your employee survey data."
— Sean Fitzpatrick, President TalentMap
How does TalentMap determine key drivers for my company?
TalentMap has a scientific method for identifying key drivers of engagement for your specific organization. Our methodology takes into consideration the size of your organization, regression analysis, comparison data and overall performance scores.
After the survey is completed and an employee clicks the “submit” button, there is no option to go back and change responses. Once responses are submitted, they are saved and locked.
We use a methodology called Linear Regression. In statistics, linear regression is a technique used to model the relationship between a scalar dependent variable y (engagement) and one or more explanatory variables denoted X (the other dimensions measured in our survey).
When the model is employed with just one explanatory variable the technique is called simple linear regression. Modeling with more than one explanatory variable is called multiple linear regression.
In linear regression, data are modeled using linear predictor functions, and unknown model parameters are estimated from the data.
It is important to remain focused on what drives engagement rather than low scores.
Engagement is what motivates your employees to be their best. It could be, for instance, that your organization scores low on one area, such as Work/Life Balance. However, if that area does not have a strong statistical relationship to Engagement, an investment of time and resources to improve that area will not have a large impact on Engagement.
It is human nature to want to invest in the areas that score the lowest; however, if the focus is on improving engagement, the focus should remain on those areas that affect engagement the most.
Investing in areas where the scores are low but have shown statistically do not have a strong relationship with engagement, will likely result in improved scores in that area and perhaps some internal complaining will subside, but little or no improvement in the organization's overall engagement scores.
Yes, TalentMap prevents respondents from submitting duplicate surveys.
TalentMap recommends using a unique URL for each respondent. Each respondent obtains it's own unique URL and once completed the link is closed and cannot be accessed again.
A key challenge with survey deployment is to ensure we get a sufficient response rate, however, we do ensure respondents don't complete the survey more than once.
The ideal deployment method is using a unique URL which is specific to the individual respondent. Once that survey is completed the link is closed and can't be used again or re-accessed.
If deploying by unique URL is not an option and the survey is launched using an open link, TalentMap has means for checking duplicates including looking at response rate patterns, reviewing open-ended comments etc.
75%+, and our goal is to help you reach an 80% response rate.
TalentMap works closely with its clients to achieve a high response rate. We provide you with best practices on how to obtain high response rates, and we monitor response rates closely to ensure that there is continuous improvement in our systems and the best practices we share with you.
One of the keys to our high response rates is our online reporting tool. With our online reporting tool, the survey champion can view response rates in real time and filter down by demographics to see what groups require a little more encouragement and support to complete the survey.
Individuals who we survey are the lifeblood of TalentMap. Consequently, the protection of individual survey responses is paramount. And, despite wide-spread misinformation, standard Internet technologies provide a solid foundation to ensure the anonymity of individual survey respondents.
You, your computer and all computers at your office sit within your company’s network (LAN). Your company’s firewall protects your computer and its identity, from all other computers on the Internet. Your company’s firewall manages the translation of its ONE INTERNET ADDRESS (the one it displays to other computers on the Internet) into the multiple computer addresses that are found within your company’s network and protects your computer from other unwanted forms of access.
TalentMap’s computers (servers) capture your individual survey responses from your company BUT because your company’s firewall translates all computers addresses inside your company’s network into one single computer address we are unable to trace your survey responses back to you or to your computer.
Understanding Internet Technology and How it Impacts Your Privacy
Every computer that accesses or delivers data on the Internet requires a unique address. This address is called an IP address.
Protects a grouping of computers connected to the Internet. Effectively makes several computers look like one computer to other computers on the public Internet.
Like your computer at your office, our database - where your survey response data is kept - is protected by a firewall. In fact, our database is even more secure than your computer. Our database has no direct connection to the Internet. The standard internet port on which it runs is closed to the Internet making it virtually impossible to access from the outside.
Different organizations have different ways to measure engagement – all are effective, but they have different twists and unique features.
TalentMap’s research methodologies are sound and based on best-practices and research. The argument of as to whether response scales should run "negative to positive" or "positive to negative" has support both ways, but the tendency is to go negative to positive. If the positive scale is first, a slight positive bias arises in the scores. however, if the negative scale is first, people are more likely to look through the answers and contemplate their choices, thereby eliminating bias in either direction. Accordingly, we believe this provides a more accurate/nonbiased selection. Below is one researcher’s thought on this – and we, like many in our industry, tend to follow this philosophy…
"Left to right or right to left?" I always advocate displaying scales from the negative to the positive, left to right. It seems more logical to me and some automated survey software mark your answers and calculate the responses for graphs on this basis, e.g. that the first point is the lowest. But I've heard others argue that it should be the opposite way around - put positive to negative, left to right - as people will click on the first point by default in online surveys - which I personally don't believe. I've not yet found any academic reference supporting either way but looking at all examples in academic articles, 95% are written as negative to positive, left to right - some evidence in itself."
Sometimes, in transitioning to a new consulting firm, you will experience some transition in methodologies. It is possible that in comparing year over year scores that there is a slight positive bias to your previous period data if your former provider did their surveys using the reverse scale, but research indicates that the amount of bias and difference between result sets should be very small or negligible.
After years of helping clients implement action plans, we’ve learned a few things along the way:
Getting local-level business unit leaders to share their survey results and develop action plans directly with their staff will increase management’s commitment to take action.
Developing action plans at all levels of an organization is helpful, and ensuring that employees are involved in action planning is most effective at increasing employee engagement at the business-unit level.
Designating the oversight of action-planning programs to a senior manager or executive at the highest levels ensures greater accountability and adherence to action plans.
Ignoring problems or issues that the organization can’t address, without explaining the situation to employees, only leads to greater disengagement. Managers who explain why they can’t use some of the employees’ suggestions help employees accept the organization’s limitations. Also, employees won’t repeat the issue when responding to future surveys.
Tying bonuses, compensation packages, and/or performance metrics to survey results needs careful consideration. Although you often hear “what is measured gets managed,” remember to guard against local-level managers and staff who may be attempting to manipulate survey items.
You will not be asked this information, as this information is provided in advance to TalentMap.
On a related note, you can be confident that only aggregate results will be provided to your organization. We will not report any results with less than 5 responses to ensure confidentiality and privacy, and no manager will be able to determine "who said what".
The Client survey data comprising all TalentMap survey responses received from Client respondents, and any tabulation, reporting or analysis thereof, is the property of the Client, will be kept strictly confidential and will not be divulged by TalentMap to any third party for any purpose whatsoever except with the express written consent of a duly authorized Client representative.
We create an Employee Engagement roadmap designed specifically for your organization.
We clearly outline your priorities for developing an effective range of employee programs and practices — from leadership development to performance management and rewards, to communication and workplace management.
Anonymous and confidential feedback from your workforce, a handful of employee focus groups and our knowledge and experience of your industry’s best practices provides the underpinning for our recommendations that are guaranteed to drive your employee engagement efforts forward.
“The one thing we liked most about TalentMap versus other vendors is flexibility. They listened, understood and adjusted their approach to fit our needs, not theirs.
— VP HR Financial Services
TalentGage is a proven, scientifically designed employee engagement questionnaire designed to measure and benchmark engagement. It measures twelve core key drivers of engagement. It includes about 65 radio button type questions as well as open-ended type questions. It takes about 15-20 minutes to complete.
You, your computer and all computers at your office sit within your network (LAN). Our firewall protects your computer and its identity when you access the Internet. When TalentMap’s servers receive your responses, they only know which organization the data came from not which computer. Therefore, even TalentMap is unable to trace your survey responses back to you or your computer.
TalentMap specializes in employee surveys and we ensure that all of the survey data is collected outside of our company via a secure internet connection (SSL) and remains confidential.
The information that we collect goes directly to TalentMap’s secure servers. To ensure anonymity, TalentMap promises that at no time during or after the survey will individual responses be divulged to anyone at your organization; only aggregate results (total responses) will be provided. Managers will not be able to determine 'who said what'.
Most clients keep their survey open for employees to complete for two weeks. This window allows enough time for the vast majority of employees to respond while providing a clear deadline which can be used to encourage responses. If the survey period is too short, say one week, you may miss some people. If it open for too long, say four weeks, the process can get stale, it is difficult to motivate employees with a pending deadline, and you risk losing momentum for the 'analyze and act' phases for implementation.
Most organizations take 3-4 months from start to finish. However, some can complete the entire process in just a month while others can take up to 6 months.
The survey is a key source of information for engaging employees in a discussion about strengths and areas for improvement at all levels of their organization. It is an important tool for measuring progress in leadership and management practices. By providing information on topics such as performance management and employee engagement, it allows for the continuous improvement of people management practices.
We have a complete library of standard, proven and reliable questionnaires to choose from. Although most of our work is focused on employee engagement we also deploy safety surveys, compensations surveys, customer and/or member satisfaction surveys, board effectiveness surveys and others. Furthermore, we can help you design and deploy any type of survey feedback you need.
The quartile map called “Engagement vs. Benchmark” is a graphical representation of the distribution of all engagement scores within a particular benchmark.
It shows your organization’s results as compared to every other organization included in the benchmark in a more visual representation and gives you a much better sense of how your organization compares to others. The graph shows the variation of scores included in that particular benchmark.
The top quartile represents the best of 25% of scores in the benchmark and shows you the distribution of scores. Similarly, you can observe the spread of the poorest performing organizations making up the 4th quartile.
All scores for each organization within that benchmark are divided evenly among four quartiles. The top scoring organizations (those in the top 25% of scores) are graphed to the top quartile (or quartile 1. This means the top 25% of engagement scores within the benchmark are represented in this category. The top 50% of engagement scores are in Quartile 1 and 2, while Quartile 4 represents the lowest 25% of scores. The decile calculation works in a similar way. The top decile of scores includes the top 10% of favorable engagement scores.
The following set of engagement drivers appear consistently across work published by leading practitioners and academics:
1. Trust and integrity
2. Nature of the job
3. Line of sight
4. Career growth opportunities
5. Pride about the company
6. Coworkers/team members
7. Employee development/training
8. A personal relationship with one’s immediate manager.
TalentMap measures each of the eight common drivers listed above, albeit using slightly different terminology. We also go beyond simply measuring the drivers of engagement by adding four additional drivers that measure business performance.
This survey model includes additional measures of perception as they relate to:
1. Customer/client/patient service
3. Work/life balance
4. Work environment
While these last four dimensions are not always significant drivers of engagement nor do they appear among the eight common dimensions mentioned above, they are significant business performance drivers leading to positive business outcomes. They are also important metrics for any organization that wants to improve both engagement and performance.
By adding these additional four (4) survey drivers, TalentMap’s engagement survey covers areas that are not normally addressed.
There are supplemental engagement drivers that can also be surveyed - for example, health & safety, work diversity, etc. These drivers are not typically key to the success of a corporation but some companies feel that they want to gather facts and data around them. TalentMap can help you survey on a number of these supplemental drivers.
Our survey software allows you to save your responses before you finish. You can always come back and complete your survey at a later time.
We keep or questionnaires short so that most surveys are completed in one sitting. This is a best practice that we encourage. However, when surveys are deployed using unique URLs, responses are saved each time that the respondent clicks the button at the bottom of each page before he/she moves on to the next page. As a result, the respondent can return to a partially completed survey (by clicking on their survey URL) at a later time to pick up where they left off and complete the survey.
You can reach us live at 1-888-641-1113 ext 501 - Monday to Friday (9:00 am to 5:00 pm EST). During off hours and holidays email us at firstname.lastname@example.org - we will get back to you quickly, usually within the hour.
Focus groups can serve many purposes. Here are 5:
1. Gain deeper insights into the responses to the survey and clarify what
the responses meant to participants;
2.Ask the follow-up questions not possible in the survey itself and help determine the relative importance of the issues that surface;
3. Identify and clarify the underlying causes driving low engagement and develop a basis for the most impactful improvement recommendations for future action;
4. Confirm whether or not certain issues are localized in certain employee segments or widespread across the organization;
5. Understand the current negative impact of specific workplace issues and the benefits of addressing them.
Should We Provide Incentive to Complete the Survey? Employee Engagement Survey Response Rates Above Image by Gavin Whitner is licensed under Read More …
Explaining The Beta Score, Regression Analysis and Their Relationship to Engagement Drivers For large organizations, TalentMap uses a Beta score Read More …
Employee Engagement Questionnaire Can we map our old one to TalentMap’s? We can map your old questionnaire to TalentMap’s employee Read More …
Why Is It Important For Everyone in an Organization to be Working on the Same Key Engagement Drivers?
Why Is It Important For Everyone in an Organization to be Working on the Same Key Engagement Drivers? In short, Read More …