Measurement has been one of the main themes of this blog since we started it and this time we’re using it to explain some of the background to ICEmarq™, our brand new internal communication service.

With spending in many organisations under the microscope, demonstrating return on investment has never been more important.  Our goal with ICEmarq™ is to help organisations measure that return by providing them with what they need to run an effective measurement programme. We explain how ICEmarq™ works on the ICEmarq™ site but here I’d like to share some of our observations about measurement and how they have influenced our approach to ICEmarq™.

• No single ‘right’ way. If you ask an organisation what aspects of internal communication it would like to measure you will get many answers. Some will focus on cut-through of key messages, others on the performance of key channels and a few will focus on the wider of benefits of good communication such as advocacy or higher discretionary effort. There is no ‘one way’ of approaching the measurement challenge and what an organisation measures will ultimately come down to what it really wants to achieve with its communication. 
• In-house tools and expertise. We’ve observed an increase in the number of internal surveys in recent years thanks largely to the availability of easy-to-use tools such as Survey Monkey™.  The existence of such tools and the expertise to use them means that organisations no longer require the full-service approach to research and instead prefer the opportunity to buy only those components it actually needs. At the top of that list sits establishing the right questions to ask and interpreting results effectively.
• Placing results into context. Whilst we believe an organisation can always draw conclusions from a set of survey results, we also believe those results are more meaningful if they can be compared to an external benchmark. We tested this point repeatedly with those we’ve researched ICEmarq™ with which is why we’ve made the availability of statistically reliable and representative data a cornerstone of the service. We’ve also sought to extend the principle by offering ICEmarq™ Unique for any organisation wanting a precise benchmark sourced from a panel made up exclusively of people working in its industry sector and/or countries in which it operates.

We’re very excited about ICEmarq™ and over the next few weeks we intend to share some of the most interesting ICEmarqs™ via this blog. But for now, we’d love to hear any feedback you have about internal communication measurement in your organisation!

A CEO recently questioned the results of a pulse survey. The survey was short – 6 quick questions – and was part of an ongoing programme to measure the extent to which colleagues engage with the new corporate brand. And the results were good with double digit percentage increases across most of the 6 indicators. So why the challenge and how best to respond?

The first angle to cover is the statistical one. Can we treat the results as statistically significant and representative of the whole population? This sounds complex but actually isn’t with easy to use online calculators available to determine minimum sample sizes. As a rule of thumb, if the sample size is small and if the overall size of the population is less than 2,000 then you will have to settle for a wider confidence interval (which means less certainty) than if both are larger.

It’s unlikely though that concerns about statistical validity were uppermost in his mind. More likely is that he has a different perception of what people are saying, thinking and doing than respondents say they have.

There might therefore be an issue with the questions being asked. Are they specific enough or do they allow for too much interpretation? An obvious consequence of keeping a survey short is that questions tend to get more general. So rather than ask whether respondents see evidence of the brand being lived every day, maybe we ask whether they see evidence of prompter or more courteous customer service or whatever else may make up the brand promise.

What about the degree of direction respondents were given? A lot of organisations push surveys hard and in some cases hold managers responsible for response rates. This can obviously have an impact on the mindset of the respondent and it can be positive or negative. In this case, the survey was live for 2-3 weeks and other than a couple of emails to make people aware of it, no particular pressure was applied. Importantly, all responses were given anonymously.

And is employee perception really a reliable measure? As an alternative, what about using clear objective measures (that are not usually tracked) such as arriving at meetings on time or responding to email within a particular timeframe and observing behaviour. This moves us away from perception and into cold, hard reality. But it also requires more effort and will almost certainly cost more to do.

Organisations often rely on the perceptions of employees as a way of measuring success or otherwise. It’s easy to do, creates involvement and can be a reliable indicator of performance. Agendas are often based on the results and managers held accountable for future performance in key areas. All of which suggests that the CEO should simply accept the results and take the credit he and his team deserve.

But that allows those of us who run surveys on a regular basis off the hook. A challenge like this is good as it forces us to really assess the validity of what we are doing. Are we really asking the right questions given the wider objective (in this case, stronger brand engagement)? How certain and open are we about the statistical validity of what we have learnt? And is a survey of employee perception really the best way of determining success? All good questions and all ones we should ask next time the brief requires us to run a simple survey.

People rarely use tools which they don’t believe offer personal benefits, much less ones they don’t understand or know about. Raising awareness and educating colleagues – activities communicators are ideally placed to do – are vital to building adoption; but they aren’t enough.

Recently, we’ve been looking at collaboration readiness in organisations. As usual, quantitative research was followed by conversations with employees, many of which reminded us that metrics only ever tell part of the story.

One way we look at collaboration readiness is to consider three dimensions:

• The ability the individual has for using collaboration tools (as determined by their current use outside work via social media sites like Facebook)
• The appetite the individual has for using collaboration tools at work
• The perceived availability of the collaboration tools in their organisation today

In some organisations we’ve seen figures which suggest low levels of perceived availability and appetite – regardless of actual availability. For those which have already deployed collaboration tools, an instinctive reaction is to focus on raising awareness, a challenge which falls to communicators.

In truth, we’re often being asked to create appetite. Viewed from that perspective, awareness and education are only steps in a journey to the moment when a colleague decides to invest their time and faith in migrating to a new way of working (or at least investigating it).

Today, most communicators contend that we need to move from a broadcast model to one of dialogue, and I believe this to be particularly true when driving adoption of Enterprise 2.0.

In conversations with employees who claimed to be sceptics, it was frequently possible to make them concede at least an interest – once the discussion had moved beyond education and become a personalised case for adoption. Many such conversations followed a similar pattern, and eventually I modelled it:

For me, it’s interesting how many different areas one must draw upon to respond to each question or statement convincingly:

Just glancing at this list, I suspect most communicators would agree that whilst we might take a lead on several activities, others will require expertise and action from a diverse range of stakeholders: leaders, managers, HR specialists, business analysts, trainers. Whilst the principle objective of adoption may be better communication (and performance), accomplishing it requires us to employ a broader range of capabilities than communication alone. That shouldn’t be a surprise, since ambitious change is often cross-functional. Neither does it mean communicators cannot – or indeed should not – take the lead in building a coalition of capabilities.

However, for me the paramount principle is that to gain real traction the proposition has to be made personal: and that means starting – and sustaining – a genuine conversation; and that’s where communicators can really make the difference.

We talk a lot about good employee advocacy as a discernible benefit of effective internal communication. Whether its BP employees fielding questions about the oil spill, Google staff explaining Street View or bankers justifying their annual bonus, communicators want to ensure people receive the information they need to back the official line. But how well is the average employee equipped to handle more basic questions about the latest price plan, special offer or new product? Two recent experiences suggest ‘not very’.

First, one organisation I know will shortly ask the vast majority of its employees to leave their desks for the day, head out to the streets, cafes and shopping centres and interact directly with prospective customers. The case for doing so is compelling: the company has recently changed its name and has introduced some exciting, potentially game-changing new products that it wants to shout about. Many of its staff have never had any direct contact with a customer so what better time to bring employees and the customers who pay their wages closer together. The trouble is, a high percentage of those employees are very worried that they don’t understand the products well enough to make the most of the opportunity. Advocates yes, but salespeople, probably not.

Second, a chance conversation with a local teacher suggested that parents at his school know rather more about that schools stance on several key issues than the majority of staff. As a teacher, his role is in the classroom but as a member of the local community he is expected to know the school’s view on a wide range of issues. His honest assessment though is that he does not.   Willing advocate certainly but genuine salesman? Again, probably not.

It’s a fair assumption (though by no means a given) that product information is generally provided to those who most need it. But could and should organisations be doing more to equip all staff with the information they need to become genuine salespeople? After all, every employee comes in to contact with dozens of potential customers every week but I’d be prepared to bet that a majority of these people, however willing, would be more than a little uncomfortable about actively selling to them. Time perhaps to look more closely at what we really mean by advocacy.

Arguably the worst part of the professional communicator’s role is the need to communicate bad news, or more specifically redundancies. It’s a role many of us have had to fulfil in recent months and given the range of approaches taken, begs the question as to whether there is a ‘right way’.

To start with, let’s clarify precisely why the communication component of any announcement is important. First, with redundancy high on any list of stressful experiences, behaving and being seen to behave with common decency is important. After all, brand is what brand does. Second, a majority of staff will ‘survive’ and making sure we support and continue to engage this group is equally important. And third there is still a business to run and a brand identity to uphold so if nothing else the communication process must do all it can minimise the damage done to business as usual.

Moving to the debate, is there a right way to structure the overall process? Employee consultation is a legal requirement in the UK and across the EU which means an announcement stating intention to proceed followed by the consultation period itself is a mandatory component. So little room for manoeuvre here. (Go to http://www.direct.gov.uk/en/Employment/Employeeinformationandconsultation/DG_10028095 for more details). Choices though arise once the consultation is complete and the outcome known and this in my experience is where division of opinion is most likely.

One coherent argument is that once ‘the list’ of affected employees is finalised, we should simply notify those colleagues of their fate and not involve anyone unaffected. Assuming notification happens face to face (and thankfully in my experience no organisation has sought to do it any other way), the issue is efficiently dealt with and broadly speaking achieves each of the objectives set out above. There is though a very obvious counter-argument that such an approach pre-announces the news (face to face meetings need to be scheduled so if you have one, you know what’s coming) and makes the process much harder to implement. Much better therefore to give everyone a slot regardless of their fate and work it through properly. That way everyone has the chance to talk through what it means for them. So plenty of scope for discussion here!

Another area for debate is around the ‘who’. Whose voice should be heard at what stage in the process? There should be little dispute that the most senior voice should be heard at the beginning (any other trivialises the news) but what about the act of notification. The line manager is the obvious choice though s/he may lack the skills. HR is an alternative as it is their specialism though this absolves managers of their responsibilities. There is also the local senior manager/director though the numbers involved may make this logistically challenging.

Then there is the discussion about ownership. Is a redundancy announcement a communication process requiring HR input or the other way around? This is very likely to depend upon the complexity of the announcement and factors such as numbers involved, variation in redundancy terms being offered and union involvement. The more complex the situation, the more significant the role of HR becomes.

If isn’t already obvious, the answer is almost certainly that there is no ‘right way’. I’ve yet to be involved in two processes that were the same or where the post-announcement debrief didn’t identify things we would have done differently. Core principles yes, but rigid process absolutely not. All of which underlines the importance of giving the communication side of any redundancy announcement the attention it deserves so that those all important objectives can be achieved.

One night in my late teens me and my best friend from university plugged a 14.4 modem into a wall socket and uploaded our first website (technically it was his second). 14 years later I found it again, completely intact, in an Internet archive. That should make you stop and think for a moment.

Beneath every social media website or collaborative tool is a database, and truly deleting content from a database is considered to be a bad idea. Asking for it to happen usually makes engineers wince and mutter about ‘referential integrity’. Much better, they’ll argue, to label the content as obsolete and stop displaying it. Storage is now so cheap that it’s usually commercially preferable to hide the data than work to remove it.

All of which assumes one is actively considering the question of what to do with old content. Increasingly, at least on a personal level, we don’t. Social media and collaborative tools are inherently conversational, and the natural tendency of most individuals (once they’re comfortable with online discussion) is to treat their contributions as they might their words in a face-to-face dialogue. Once said, or typed, it’s history. Unless you’re a public figure you don’t expect to be quoted back to yourself later, and the rate of technological change has rendered most of what we typed barely years ago inaccessible (just how would you read documents stored on floppy disks today?). So the persistence of our comments and postings is rarely front of mind when we make them.

That’s not to say we are unthinking contributors. Thanks to some painful lessons learned about indiscretions posted to Facebook, most people are also conscious of the need to exercise common sense when participating in online discussions. That applies even more for internal spaces, where it is usual for employees’ contributions to be attributed. But what if even the good things we write (or perhaps the ‘good at the time of writing’ things) have the capacity to cause us problems in the future?

A decade ago a multi-national I knew decided to create a database of every key promise or commitment made by its senior leaders in the previous 5 years. In the then largely un-digitised world that task was laborious enough to warrant a sizable team of researchers, despite the relatively small amount of data that had to be located and sifted. Digitisation and the evolution of search technologies mean that even with the explosion of content in the 2.0 era, a contemporary exercise would not only be easier, but yield richer data.

What that multi-national was interested in was transparency and accountability, words very much beloved of the social media scene. To be clear, this was a defensive action: the company knew external special interest groups were effectively performing the same exercise and it wanted to understand its exposure. The important aspect for us now is that the level of granularity was at the corporate level: this was about what a company had said (albeit through the mouths of its people), not an individual. At that time the only individuals who could reasonably expect to invite that level of scrutiny (and indeed generate that much easily-accessed content) would be public figures.

A decade later, within organisations the capacity for ordinary employees to publish and navigate the published content of colleagues has changed completely. If your organisation is investing heavily in collaborative platforms today, think ahead to 2015 and imagine the body of searchable, attributable content that will attach to any future member of middle or senior management who has risen through the ranks. It will be non-trivial. It’s tempting to think that because common sense will have been exercised throughout, we have nothing to worry about; but think about three practical aspects of communicating as a leader:

• Often you must speak or write on behalf of your role or organisation, not as a private individual reflecting her/his own views
• It’s not uncommon for a business to change direction, rendering obsolete what was said or promised before
• It’s also not uncommon to be plain wrong

None of these make you a liar or inauthentic, but in many disputes advantage is gained by pointing to opponents’ documented mistakes and changes-of-heart. It’s almost a rule that candidates in any US election will have a charge of ‘flip-flopping’ levelled against them, and the commentariat will mysteriously forget that good decision making often means knowing when to change direction. By comparison, corporate leaders have historically had an easier ride: beyond heavily unionised environments the degree to which leadership cadres could control the internal news agenda and re-position issues has been significant. As organisations move towards giving everyone a voice, as well as the tools to search the past, change is unavoidable.

When the controversy first raged about drunken pictures posted to Facebook and how this might affect an individual’s future career prospects, many advanced the argument that changing social attitudes would quickly negate the risk. I’d like to think that will happen, but I’m mindful that forty years on from the 1960s politicians still break into a sweat when asked about smoking dope at university.

The smart Gen Y individual already knows how to manage her/his profile, and will leverage that advantage over less discreet peers in coming years. Moreover, the emergence of sites like the (quickly suppressed) Web 2.0 Suicide Machine suggests that, at least in the public sphere, some have new misgivings about their digital footprints.

If attitudinal change is required for organisations to live comfortably with the legacy of long term (digital) transparency, it probably begins with the collaboration cultures many of us are trying to build today. It probably requires us to work with IT and KM specialists to look critically at issues of content curation, the longevity of systems, and what we really mean – and want – when we use words like ‘transparency’ (and whether the answers hold true at all levels of the organisation).

It would be a shame if we rushed headlong into the next level of collaboration (which I believe offers us many benefits), only to later recoil at the thought of accidentally building our own Panopticon from the inside.

We recently uploaded case studies featuring internal communication reviews we undertook for two very different organisations. One was for British Airways and was based around an extensive survey questionnaire with a strong channel focus. The other was for a UN agency called UNRWA and was based around a series of focus groups and interviews, undertaken during two fascinating weeks in the Middle East. Two projects with a very similar purpose but with quite different approaches.

The output was also quite different. In the case of BA, our main focus was an extensive analysis of the answers we received from the survey which led to a series of conclusions and recommendations. With UNRWA, our output was focused less on what we’d learnt and more on the channels and supporting infrastructure needed if communication was to improve. A quite different challenge altogether.

But the two projects were similar in one very important way. Both were one-off exercises that need only happen every 2-3 years and both were undertaken on behalf of those directly responsible for communication. And both also included the recommendation to introduce an ongoing measurement programme which brings me to my main point.

Communication reviews and audits are great as diagnostic tools for understanding how communication is currently viewed and where improvement can be made. But they are essentially inward looking and designed to inform future planning. More a means to an end. A measurement programme is an ongoing assessment of performance against pre-agreed criteria. Its purpose is to summarise the impact of communication activity and provide the organisation with evidence that the investment is worthwhile. An end therefore in itself.

In my view, both are equally necessary and valid exercises. But the crucial point is not to confuse the two. Put very simply, review once and measure often!