Are your meeting evaluations reliable?

evaluation 4175299981_7752cbe323_o

Can the way we evaluate meetings change how participants view their experience? Possibly, given the findings of research reported in the June 2013 Personality and Social Psychology Bulletin. The study indicates that when we ask people for their reasons to justify their choices they focus on aspects that are easy to verbalize, and this can distort their overall judgement. Here’s Tom Stafford‘s description of the experiment:

Participants were asked to evaluate five posters of the kind that students might put up in their bedrooms. Two of the posters were of art – one was Monet’s water lilies, the other Van Gogh’s irises. The other three posters were a cartoon of animals in a balloon and two posters of photographs of cats with funny captions.

All the students had to evaluate the posters, but half the participants were asked to provide reasons for liking or disliking them. (The other half were asked why they chose their degree subject as a control condition.) After they had provided their evaluations the participants were allowed to choose a poster to take home.

So what happened? The control group rated the art posters positively (an average score of around 7 out of 9) and they felt pretty neutral about the humorous posters (an average score of around 4 out of 9). When given a choice of one poster to take home, 95% of them chose one of the art posters. No surprises there, the experimenters had already established that in general most students preferred the art posters.

But the group of students who had to give reasons for their feelings acted differently. This “reasons” group liked the art posters less (averaging about 6 out of 9) and the humorous posters more (about 5 to 6 out of 9). Most of them still chose an art poster to take home, but it was a far lower proportion – 64% – than the control group. That means people in this group were about seven times more likely to take a humorous poster home compared with the control group.

Here’s the twist. Some time after the tests, at the end of the semester, the researchers rang each of the participants and asked them questions about the poster they’d chosen: Had they put it up in their room? Did they still have it? How did they feel about it? How much would they be willing to sell it for? The “reasons” group were less likely to have put their poster up, less likely to have kept it up, less satisfied with it on average and were willing to part with it for a smaller average amount than the control group. Over time their reasons and feelings had shifted back in line with those of the control group – they didn’t like the humorous posters they had taken home, and so were less happy about their choice.
—Tom Stafford, When giving reasons leads to worse decisions

What might this imply for event evaluations? If we’re asked to give our reasons why we evaluated an event a certain way, this research indicates that we’re likely to focus on reasons that are easy to express. Ever noticed in your event evaluations that attendees’ opinions about food and accommodations are often far more detailed than what they write about specific sessions or the event as a whole? It’s much easier to express an opinion about the former than the latter, and that’s OK in itself. What should concern us, though, is that evaluations themselves, by focusing on the easily quantifiable, may bias how participants perceive our event’s value.

Perceived value is an important component of event Return On Investment (ROI). I’ve mused about Return On Investment (ROI) for social media (I’m skeptical about measuring it) and participant driven events (I believe they improve ROI). How might this research affect the calculation of meeting ROI?

Conversations => Relationships => Value (Part 2)

In Part 1 of this post I introduced a core component of Conference 2.0.

Conversations => Relationships => Value

Here’s why this sequence is an important consideration for modern meeting design, and how it’s enhanced by Conference 2.0 designs.

Why should customers buy from you?

Sometimes, business value grows out of the barrel of a gun. When you have a monopoly on a product or service, you can charge as much as the market will bear. But when competition exists, you must use different strategies. For example, you can play race-to-the-bottom: squeezing your suppliers for rock bottom costs that, hopefully, are lower than your competitors. Or, you can differentiate what you offer in many other ways: better service, more options, faster delivery, longer warranties, superior customer support, etc. Thousands of books have been written about how to profitably and consistently market and sell. And, except perhaps for the most cutthroat commodity markets, the ability to build and maintain good relationships with your customers is a key component of most techniques.
This ability is even more crucial in today’s markets, because of four factors:

  • The increased complexity of products and services.
  • The increased variety of products and services.
  • The increased speed of product and service development.
  • The increased transparency in many marketplaces caused by online customer reviews and feedback.

The first three factors make it harder for potential customers to evaluate whether a specific product or service is a desirable fit for their needs. The last amplifies any deficiencies (perceived or otherwise) that may exist, any of which could prove fatal to sales.

In this new business environment, creating and maintaining good, trustworthy relationships with your customers becomes crucial.

Relationships are the new impressions

In the good old days, the more people heard about your product through broadcast marketing (impressions), the greater your sales. Today, business value, especially for non-commodity products and services, is becoming increasingly linked to the strength and quality of buyer-seller relationships. Traditional marketing can’t manufacture relationships, which are built through conversations between you and potential customers. Some of your conversations will turn into relationships, and some of those relationships will lead to value for your business.

Not all meetings are alike

Meetings provide wonderful opportunities for conversations. But, for two reasons, some meeting environments provide better opportunities than others.

First, for all but very small meetings, the number of conversations doesn’t scale with event size. For example, at a one-day, two hundred attendee event you can’t have more ten-minute conversations than you can with a hundred in attendance. In fact, at a large conference it’s often harder to find the people you really want to talk to than at a smaller, more focused event.

Second, Conference 1.0 sessions don’t foster conversations. Conversations only take place in breaks and socials. Compare this with Conference 2.0 designs, which excel at providing opportunities for relevant conversations

How Conference 2.0 designs support conversations

I’ve quoted Howard Givner before and I’ll quote him again. (Why? Because he made this highly positive remark about one of my conferences 😀.)

I easily established triple the number of new contacts, and formed stronger relationships with them, than at any other conference I’ve been to.

Why is Howard’s experience a common one at Conference 2.0? Let’s take Conferences That Work as an example. This conference design starts with initial roundtables that not only provide a structured forum for attendees to meet and learn about each other’s affiliations, interests, experience, and expertise but also effectively uncover the topics that people want to discuss and share. Within a couple of hours, every attendee has the initial introductions and information necessary to go out and start the right conversations about the right topics with the right people. Other Conference 2.0 designs encourage fruitful conversations by giving attendees the ability to meet around topics that they choose during the event.

The bottom line: Conference 2.0 formats routinely lead to more meaningful conversations, which in turn lead to more relationships, which in turn lead to more business value.

Does Conversations => Relationships => Value make sense to you?

Conversations => Relationships => Value (Part 1)

I admit it
I do not have a good reaction when someone talks about the return on investment (ROI) from attending an event.

My initial internal response is a rant:

Do we ask for the ROI when we buy tickets to a concert?

How can you evaluate the ROI for learning something new or seeing something in a new way?

And my favorite: So, what is the ROI on a wedding? (Please don’t respond with an analysis of the average value of wedding gifts versus the cost of the wedding. I’d probably argue diminished responsibility at the subsequent trial.)

Conversations Relationships Value part 1In some ways, my reaction is alarmingly similar to the message of the brilliant formula for the MasterCard advertisement:

[List of mundane items with $ assigned]
[Intangible item – Priceless!]

The delicious subtext: Forget the money, whip out the credit card, and go to the event anyway!

The morning after
OK, it’s strong black coffee time. Whether the benefits are intangible or concrete, we all know that there is some kind of calculation that goes on when a potential attendee decides whether to attend an event. I’ve written about how existing event ROI methodologies are a noble attempt to quantify this calculation and give it as much respectability and logic as we can. So, enough on ROI; here’s a core component of Conference 2.0.

Conversations => Relationships => Value

In Part 2 of this post, I explain why this sequence is now an important driver of modern meeting design, and how it’s enhanced by Conference 2.0 designs.

Photo attribution: Flickr user alanchan

How participant-driven events can improve event ROI

skeptical 2402001517_fab6e95cdb_o“Why do you want to go to this conference?” is a question that you’ve probably been asked at one time or another. The real Question being asked (usually by our boss) is, of course, Is it worth the money and time invested in having you attend? It can be a hard question to answer, especially when the event in question has no or few predetermined sessions, like the peer conferences I design and facilitate.

Probably the most exhaustive methodology of planning and evaluating event ROI has been developed by Jack Phillips and Elling Hamso. It’s long and comprehensive, and here’s a summary of it.

According to this methodology, one of the components involved in evaluating Event ROI is the degree of Relationship Learning, which Elling defines as follows:

Relationship learning refers to the building of affinity between people, getting to know others, trust and liking. All forms of peer learning benefit from the strength of personal relationships, it is the foundation for subsequent information, skills and attitude learning in the peer relationship. Relationship learning may be measured in much the same manner as other forms of learning. At the most detailed level, individual relationships of trust and liking, for example, may be scored on a scale from very low to very high, or more general reports of relationship learning may be collected.

Well, this is exactly this kind of learning experience at which peer conference designs like Conferences That Work excel! Here’s how Howard Givner described a recent peer conference:

…one of the most innovative and eye-opening professional experiences I’ve had. Aside from coming back with lots of new tips and ideas, I easily established triple the number of new contacts, and formed stronger relationships with them, than at any other conference I’ve been to.
—Howard Givner article: The Un-Conference: Participant-Driven Agenda + Mashup Networking = Relationship Building on Steroids

Conversations, and the subsequent relationships that are formed, are very important. Doc Searls, co-author of the Cluetrain Manifesto, wrote a great article about their pivotal role: Building a Relationship Economy. Well-designed peer conferences provide an environment that encourages and supports a rich abundance of the initial components of the following sequence (on which I’ll write more soon):

Conversations => Relationships => Value

“Value” here means the kind of business worth your boss is thinking about: more prospects, new sales, increased customer satisfaction, etc. All the things that translate into funding for your paycheck, profit for your company, and a happy boss.

So, when your boss next asks you The Question about the participant-driven conference you want to attend, take a deep breath and tell her you expect to make many more business relationships at this event than you would at a conventional conference—relationships that will turn into solid business value for your organization. Communicate exactly why you want to go, and explain that participant-driven conferences are much better than traditional events at building sessions around the content that attendees really want. Good luck!

Photo attribution: Flickr user andyi