Raise your hand if every conference session you’ve ever attended was accurately described by its program blurb.
Bet your hand didn’t go up.
When we have to sit through a session that bears little resemblance to what was advertised, our time is invariably wasted and we tend to blame the presenter. But, in my experience, it’s often conference producers we should be holding responsible. Last week, Peggy Duncan sent me an example:
The above quote comes from an interesting blog post by two Netflix engineers who explain why the company has never fully implemented the algorithm that won the Netflix $1 Million Challenge to improve Netflix’s customer movie recommendations.
Why didn’t Netflix use the improved movie recommendation algorithm?
Although an earlier part of an earlier version of the algorithm was incorporated into the way Netflix recommends movies, by the time the prize was awarded in 2009, Netflix’s world had drastically changed:
“Netflix launched an instant streaming service in 2007, one year after the Netflix Prize began. Streaming has not only changed the way our members interact with the service, but also the type of data available to use in our algorithms. For DVDs…selection is distant in time from viewing, people select carefully because exchanging a DVD for another takes more than a day, and we get no feedback during viewing. For streaming members are looking for something great to watch right now; they can sample a few videos before settling on one, they can consume several in one session, and we can observe viewing statistics such as whether a video was watched fully or only partially.” —Ibid
Conference program building: DVDs or streaming?
The traditional way we build conference programs is like how we order DVDs on Netflix. We decide in advance the content we want and order it from presenters. Once the conference starts we have no choice except to view the presentations we’ve ordered.
At participant-driven events, building the conference program is much closer to how we pick a streaming movie to watch on Netflix. When it’s time to settle down on the couch with our popcorn, we pick the movie we want at that moment and start to watch. Not to our liking? We can stop the stream and try something else.
Notice there are two things different about the streaming movie choice. First, we get to choose what we want to watch when we’re ready to watch it, rather than having to make a choice several days in advance. And second, we can switch to any other movie right away if what we choose isn’t to our satisfaction—an option not available with DVDs.
Netflix’s move towards streaming delivery of movies is the reason why the carefully tuned, Prize-developed-at-great-expense algorithm has been largely abandoned. It turns out that people choose different movies when they can decide what they want to watch when they’re ready to watch, and when they can change their mind if their choice is unsatisfactory.
Lessons for your event, courtesy of Netflix
Netflix’s observations parallel my experience of building conference programs. Programs that participants create at the event are noticeably different (and, in my experience, better) than those chosen in advance by a program committee. As I’ve observed before, half or more of the session topics that participants choose via the Conferences That Work design are not predicted in advance by a program committee. If program committees do such a poor job of creating conference sessions that registrants actually want to attend, perhaps you should consider replacing at least some of your conference sessions with those chosen via an effective tool like Open Space, World Café, or Conferences That Work‘s peer session sign-up.
I subscribe to both DVDs & streaming from Netflix, and notice that my DVD queue is getting shorter and shorter these days; we watch far more streaming movies than we did just a year ago. We’re not alone; Netflix streaming users now outnumber DVD subscribers 2:1. With the rise of online, providing topical, up-to-the-minute conference sessions is becoming more important than ever. Let’s plan for a streaming rather than a DVD future for our conference programs.
“There is talent everywhere. We just don’t know how to find it.”
Today’s Wired article by Jonah Lehrer describes recent research on the NFL scouting combine that concludes that highly paid sports scouts barely do better than chance at picking great players like Jeremy Lin out of the pool of promising candidates.
Sports scouts, with all the information, statistics, tests, and direct observations at their disposal can’t pick the best players! So why should we believe that “conference curators” can pick the best presenters and presentations?
In my twenty years of organizing conferences, I’ve never found a program committee that predicted more than half of the session topics that conference attendees chose when they were given the choice. During that time I’ve seen no evidence that any one person, whether they are given the title of “curator” or not, can put together a conference program that can match what attendees actually need and want.
Sure, taking a thematic, big picture approach to constructing a conference program and then soliciting appropriate presenters may produce better results than issuing a call for speakers and picking sessions from the offerings of those who choose to respond. If you insist on leaving attendees out of the loop, it’s probably the best you can do. (Sadly, I’ve found that polling attendees before the event doesn’t work.) But it doesn’t, in my experience, create a conference program that truly serves attendees.
The smartest person in the room
It’s elitist and untrue to claim that only “curators” can put together a conference experience that attendees will value. “Attendees don’t know what they don’t know,” says Jeff Hurt. Yes, that’s often true if you’re comparing the knowledge of a single attendee with the knowledge of an expert. But, in my experience, attendees collectively know what they don’t know far better than any outside “expert”. As David Weinberger puts it in his latest book Too Big To Know: “The smartest person in the room is the room.”
Finally, who are these conference curators? Is it possible to be a conference curator for any kind of conference, or do you need to be a subject matter expert on the conference topic? What are the credentials a conference curator needs? None of the articles I’ve read answer these questions.
The conference curator is a myth
I think that the need for a conference curator is a myth created by those who desire to maintain the role of experts in the construction of conference programs. Let it go, guys. The people formerly known as the audience can do a much better job.
I’m sticking my neck out again. It’s a great way to learn. Are you a champion of the conference curator? Chop away in the comments below.
When I was an IT consultant I used to build custom database management systems—complicated, company-specific software that handled the unique way an organization did things. The normal way to do this is the Microsoft Word or kitchen-sink approach. Add every feature and ability you can think of (or that any important customer asks for) into the application. Then, let the user work with the entire glory of what you’ve created.
Over the years I found I could make a good living creating integrated systems that did things a little differently. Instead of company staff facing a complete set of menus, choices, and features, most of which they never used, I built interfaces where users only saw the functionality they required. Once logged on to the system, it appeared to contain only the functions and information needed to do their work. Yet, because the software spanned the entire company, any departmental changes were immediately available elsewhere in the organization.
Employees loved these systems because they gave them just what they wanted and no more. Without unneeded menus, options, and reports, employees worked with minimal distraction, leading to less stress and higher productivity.
Large traditional conferences exemplify the kitchen-sink approach I described above. The thinking goes: “if we have a program that includes sessions on anything that attendees might want, then they’ll come and be happy”. And perhaps this seems like the only answer, given that traditional conferences, at best, do a poor job of predicting and then offering what attendees really want.
Give attendees just what they want
Well, we can do better. When we ask attendees what they want to have happen, it turns out they are remarkably good at telling us. Especially if you’ve just presented them with a smorgasbord of possible topics gleaned from the entire group. That’s what the Conferences That Work roundtable and peer session sign-up sessions do. First, they uncover participants’ needs, experience, and expertise. Next, within a couple of hours, they turn these discoveries into a conference program that optimally matches just what attendees want, and no more.
Attendees love these conference programs, because they contain just what they want and no more. Wouldn’t you?
What do you think about the feasibility of determining your conference program at the start of the event?
Over and over again, attendees report in conference evaluations that the predetermined program was a poor fit to what they would have liked to have happen. In my experience, the average participant gives lukewarm ratings to over half of the sessions available to them at a conventional conference. In other words, predetermined programs don’t work.
How can we do better?
One obvious approach is to poll attendees before the conference. Nearly all conscientious event planners do this.
The problem is, asking your attendees for their input on the upcoming conference program simply doesn’t work very well. Here’s why:
Attendees are busy people
Attendees, like all of us, are busy people. How many of yours are going to fill out a long (or even a short) questionnaire about what they want at an event that’s happening six months from now? Not many. Even if you force them to answer as part of the registration process, how much time are they going to spend to really think about the three most important topics they’d like you to offer? Sure, a minority of attendees will be conscientious and may give you some good ideas. But do you know if they represent an unbiased sample of your attendees? Do you want to base your conference program on their responses?
Six months is an eternity
Most food goes stale. (OK, Twinkies don’t, but how many of us enjoy eating Twinkies?) Similarly, most conference topics have expiration dates. The topic that’s hot now may be cold by the time your conference rolls around. So even if lots of your attendees tell you they’re really into sushi now, it may be Cambodian Cha knyey when it’s time to actually sit down for the conference meal.
What do you want to talk about now?
There’s a world of difference between a response to the question “What do you want” when it’s asked about the distant future and when it’s asked about what you want in the next five minutes. At Conferences That Work, when the roundtable facilitator announces that, in five minutes, people will start to answer three questions out loud to everyone present, minds become wonderfully concentrated. That’s when you find out what attendees really want. Not before.
I’m not saying we should give up asking in advance what attendees want in a conference program. Sometimes you’ll get good suggestions for conference presenters or session topics that you can turn into valuable sessions at the event. But you’ll rarely be able to create the bulk of a conference program that fits as well as one that’s created at the event.
So, don’t sweat about creating the perfect conference program in advance. Remember, predetermined programs don’t work. Instead, relax. Use event crowdsourcing to ask your attendees what they want. Your attendees will build the best program possible themselves—and they’ll thank you for the opportunity!
In 2005, spec homes—homes that builder start, and sometimes finish, before selling them—made up a quarter of the homes being built in the United States. Today, in the aftermath of the bursting of the housing bubble, almost no spec homes are being built. From 25% market share to 1-2% in just four years.
A traditional conference is like a spec home. The program is designed and built for you based on what a program committee thought people like you would want.
I don’t think the traditional conference market is going to implode like the market for spec homes. On the other hand, I’ve found during my eighteen years of experience running Conferences That Work that the best program committees predict only half the topics that participants at attendee-driven conferences actually request.
If conference organizers continue to believe they can predict what their attendees want to share, learn, and do at their conferences they may, at some point, experience the bursting of a bubble of their own.
The three communication modes used among a group of people are one-to-one (individual conversations), one-to-many or broadcast (presentations and panels), and many-to-many or conferring (discussions). Traditional conference sessions are predominantly one-to-many, with perhaps a dash of many-to-many at question time.
One-to-one conversations are infinitely flexible; both participants have power to lead the conversation along desired paths. Many-to-many conversations are powerful in a different way—they expose the participating group to a wide range of experience and opinions.
In contrast, one-to-many communication is mostly pre-planned, and thus relatively inflexible if the presentation involves a passive audience. At best, a presenter may ask questions of her audience and vary her presentation appropriately, but she is unlikely to get accurate representative feedback when her audience is large. Some presenters are skilled at creating interactive sessions with significant audience participation, but they are the exception.
Presentations and panels are appropriate when we are training, and have expert knowledge or information to impart to others. But with the rise of alternative methods for adults to receive training—reading books and articles, watching recordings of presentations, downloading answers on the Web—what can’t be replicated at a face-to-face conference is the conversations and discussions that occur. So why do we still cling to conference sessions that employ the one communication mode for which a variety of alternatives can substitute?
One of the questions I asked when interviewing conference attendees for my book was:
“Most conferences have a conference schedule and program decided in advance. How would you feel about a conference where, at the start, through a careful conference process, the attendees themselves determine what they want to discuss, based on what each person wants to learn and the experience each attendee has to share?”
Forty-five percent of my interviewees were unable to conceive of a conference that did not have a schedule of conference sessions decided on and circulated in advance.
The most common response was that the interviewee wasn’t sure she’d want to go to such a conference without knowing what was going to happen there.
The next most common response was that the idea sounded great/interesting/intriguing, but the interviewee had no idea of how one would create a relevant conference program at the start of the conference.
Suspend disbelief for a moment, and assume that at the start of a conference it is somehow possible to use available resources to create a conference program that reflects actual attendee needs. Imagine attending such a conference yourself, a conference tailored to your needs. (You might want to reflect on how often this has happened for you.) Wouldn’t it be great?
What is the origin of the assumption that a conference program must be pre-planned? Perhaps it arose from our experience of learning as children, from our teachers in school who knew or were told what we were supposed to learn following a pre-planned curriculum. Certainly, if one thinks of conferences as trainings by experts, a pre-planned schedule makes sense. But conferences are for adult learners, and adults with critical thinking skills and relevant experience can learn from each other if they are given the opportunity. We’ll see that there are ways of putting conference attendees in charge of what they wish to learn and discuss. But this cannot be done effectively if a conference’s program is frozen before attendees arrive.
The peer conference model described in Conferences That Work does indeed build a conference program that automatically adjusts to the actual needs of the people present. Read the book to find out how.
Still skeptical that the peer conference can build an optimum program for your conference, a program that’s better than anything your program committee could come up with?
Imagine you’d never seen a bicycle or any other two-wheel conveyance, and someone gives you one and says, “You can ride that thing without falling off.” Wouldn’t you be skeptical of them too?
Sometimes you just have to experience things in this world to find out that life is not always what it seems. So, talk to anyone who’s been to a peer conference and see what they say. Or, best of all, organize or attend a peer conference yourself. I’m confident you’ll discover what thousands have already experienced: that the roundtable and peer session sign-up used at the start of every peer conference offer just the content that attendees really want – every time.