crowdsourcing Archives : Page 2 of 2 : Conferences That Work

Dear Adrian Episode 1—is there a way to crowdsource a conference program before an event?

Monday, January 7th, 2013 by Adrian Segar

Is there a way to crowdsource a conference program before an event?

If Abby and Prudence can do it, so can I! Here’s the inaugural issue of an occasional series—Dear Adrian—in which I answer questions sent to me about event design, elementary particle physics, solar hot water systems, and anything else I might conceivably know something about. If you have a question you’d like me to answer, please write to me (don’t worry, I won’t publish anything without your permission).

Q. Dear Adrian,

I have read with great interest your articles on ‘Conferences that Work’. Whilst most of my clients’ events fall into the many hundreds of delegates there is an opportunity in most cases to run a ½ day or full-day pre-event for one or more particular segments of the audience such as new entrants or master classes.

I would like to find a way to incorporate your concept of peer events into this structure.

Have you ever tried creating the ‘peer session sign-up’ online through a social media site or the client’s internal communication system? This would give the client the confidence that the logistics can be handled cost-effectively, ie, the number of rooms held to accommodate concurrent sessions and ensuring that the appropriate topic experts were available. This could have the dual benefit of ensuring that a significant part of the event is attendee-driven and probably increase delegate registration numbers due to the relevance of the program.

In the past, I’ve run qualitative research to assist in the pre-planning of the program in an attempt to more closely align the event with attendee needs but an online component would permit the addition of a quantitative element.

I would be very interested to hear if you have any experience in this area that you could share.

With thanks and kind regards,

Wendy Hand, Bright Red Fish Pty Limited, Mosman NSW, Australia

A: Dear Wendy,

People frequently ask me your question. Is it possible to build a peer session sign-up system to crowdsource a conference program before an event?

The answer is a “Yes…But”.

Yes, it’s not hard to build such a system. I’ve done some kind of pre-conference “what-do-you-want-to-talk/hear-about” online survey since the start of the world wide web; i.e. for 15+ years. These days it’s pretty simple to create such a system; the SXSW Panel Picker is just one example.

But. For twenty years now, I’ve had the luxury of comparing what these pre-conference surveys conclude attendees want with what they actually choose when they pick sessions on site. And I’ve found that the predictive power of pre-conference surveys (and conference program committees) is extremely poor.

The best surveys/program committees predict only half of the sessions that attendees actually want. In other words, half or more of the pre-chosen sessions at a traditional event are not what attendees want.

This is a dirty secret of traditional conferences that no one ever talks about.

I believe there are a number of reasons for the poor predictive power of attendees and conference program committees:

It’s hard to get busy people to spend significant time thinking about what they would like to see on the program of a conference that’s six months down the road. Only a small number of attendees respond to such pre-conference requests.
The attendees who respond are not representative of the entire body of attendees.
I’ve seen pre-conference crowdsourcing gravitate to “hot” topics that are largely irrelevant by the time the event rolls around. For example, six months before Windows Vista came out, it was clear that everyone at a technical conference I was facilitating thought implementation would be a hot topic. We left it off the traditional program. By the time the event arrived, it was clear no one wanted to hear about it.
When you use a participant-driven design to create sessions at the event, you always uncover unknown attendee expertise and experience that other attendees are eager to tap. Such uncovered resources become sessions that are often some of the most valuable time at the event.
As a participant-driven conference, one attendee can share a great idea for a session. If others concur, we can make the session happen. This is hard to duplicate in a pre-event crowdsourcing program. Why? Because a late submission is unlikely to be seen by earlier respondents.

Those are some of the reasons why it’s important to build a conference program at the event. Crowdsourcing a pre-conference program may improve it slightly, but I’ve found that, on balance, it’s simply not worth the effort. On the other hand, building the conference program at the event, which takes about half a day to do right, improves the event immensely. Thousands of evaluations over the years validate this.

Here’s a 2011 post that reflects on some of these points.

I hope this gives you some background as to why I am still skeptical of the value of trying to crowdsource a conference program before an event rather than during it.

With best wishes,

-Adrian Segar-

Tags: crowdsourcing| Posted in Dear Adrian | 1 Comment »

Tip: How to live blog a public conference session without going crazy

Tuesday, June 21st, 2011 by Adrian Segar

Here’s a simple idea, courtesy of edACCESS colleague Bill Campbell, that can come in real handy when you want to live blog a public conference session without devoting most of your time to keeping up with what the speaker or participants are saying.

Crowdsource your event recording! How? Before the session, create a public Google Doc, shorten the weblink to the document, and publish the shortened URL on the Twitter feed for the event and/or on the projection screen in the room before the session starts, together with a request to help out with session notes. Anyone with the web link will be able to log in and help share the work of documenting the session.

Do you ever live blog a public conference session this way? Share your experience in the comments below!

Photo attribution: Flickr user catspyjamasnz

Tags: blogging, crowdsourcing, event recording, live blog| Posted in Tip | 1 Comment »

Three perspectives on crowdsourcing events

Saturday, September 18th, 2010 by Adrian Segar

This week has seen three articles published on crowdsourcing in the events industry. Crowdsourcing events is hot! Here are comments from someone <shameless brag> who has been crowdsourcing conferences since 1992.

Are We Thinking What I’m Thinking?

The longest and most detailed article is Are We Thinking What I’m Thinking? by Barbara Palmer. It appears in the September issue of PCMA Convene magazine. I, of course, love this article because I feature extensively in it. (And, all the photos are mine!) Massive kudos to Barbara for quoting me accurately and clearly conveying my crowdsourcing philosophy, perhaps the most radical approach described, that of using crowdsourcing to determine sessions at the event.

Barbara also interviewed Sam Smith, who, together with Ray Hansen, organized last week’s Event Camp Twin Cities (at which I ran a couple of sessions). ECTC used crowdsourcing very successfully at the session design level by reaching out to the #eventprofs community, asking for suggestions for novel session formats and content, and then creating a conference program that incorporated many cutting-edge ideas. One of the refreshing strengths of Event Camp Twin Cities was its overt philosophy. “We are trying new stuff here, and probably some of it won’t work.” The result was a truly innovative conference, full of enlightening experiments (with very few failures, as it turned out).

The 2011 GMIC conference

Next, Barbara asked Elizabeth Henderson about her ongoing work on the design of the 2011 Green Meeting Industry Council Sustainable Meetings Conference. Elizabeth is concentrating on creating crowdsourced event design teams with members drawn from a variety of backgrounds and viewpoints. This is another way to use crowdsourcing at the pre-event level that helps to avoid a groupthink mentality.

What meeting professionals think

Finally, Barbara quotes various meeting professionals’ views on crowdsourcing, around the theme of the role of the expert compared to the role of the crowd. In this section, the viewpoints expressed downplay the value of the crowd’s input. This is the crowdsourcing as a minor fad point of view. The possibility of influencing session topics and content is mentioned. But the description of crowdsourcing’s value for “entry and exit music, entertainment options, ways to green an event, reception themes, and party-venue options” implies relegating its importance to the relatively superficial.

And the final two comments—”I suggest that event professionals crowdsource topics, content ideas, and suggestions – and steer clear of crowdsourcing speakers and actual sessions” and “Crowds can tell you what they want…But an expert can say, ‘You are going to need to know this'”—revert to the old worldview of novice attendees being guided by wise non-attendee experts. Such a worldview ignores the reality that conference audiences invariably contain a healthy mixture of novices and veterans.

The Good and the Evil of Crowdsourcing Conference Content

The second article is Michelle Bruno’s The Good and the Evil of Crowdsourcing Conference Content from the Trade Show News Network. Michelle starts by saying “Event organizations that crowdsource conference content are learning there is a right way and a wrong way to solicit community feedback.” This formulation is unfortunate for two reasons. First, as we’ve seen above, there are a number of ways to approach crowdsourcing an event, and it’s simplistic to say that any of them are right or wrong. Second, the article offers no evidence that any particular approach is better or worse, except for the gut instincts or opinions of the people Michelle quotes.

The 2010 PCMA Education Conference

First, Michelle describes Mike McCurry’s work on developing peer networking sessions for the 2010 PCMA Education Conference held in June. Since the event is over, it’s a shame that there are no follow-up comments as to the effectiveness of Mike’s pre-event crowdsourced topic selection. Perhaps this information exists but is unpublished.

South By Southwest

Next up is South By Southwest’s crowdsourcing of 30% of their conference sessions. I tend to agree with Jeff Hurt (quoted in the Convene article) that for a conference of this size, the process SXSW uses becomes a popularity contest rather than participatory crowdsourcing. This is because there’s no significant pre-selection communication between attendees and/or the folks offering the sessions. So we end up with what is basically online voting on thousands of suggestions.

NTEN

According to Michelle’s article, the same kind of process is used by the Nonprofit Technology Network, which points out that it takes about 90 minutes to vote on every one of the 400+ sessions suggested for their 2011 conference. It’s unreasonable to expect more than a small fraction of attendees to expend this kind of effort, so the question then arises: How representative of all attendees are the responses NTEN gets?

The article continues with comments by Chris Bucchere about using (what he calls) crowdsourcing in a very different way from the above examples. He says “Using your community to ‘spec’ work for you is an ‘evil’ way to crowdsource.” This declaration ignores the common thread in all the above uses of the term: namely that crowdsourcing is also a win for participants because they get the sessions & formats they want. Chris’s idea of using crowdsourcing seems to be about building community/brands via contests with incentives for the winners. Perhaps this is an effective strategy for “anyone who wants to leverage the power of the social web to build a brand”, but it’s hardly new; advertising contests go back to at least the 1920s (e.g. Ivory Soap) and it’s a stretch to relabel them crowdsourcing, even if we’re using a social media platform for them now.

In closing

In closing, Michelle says “On the other hand, there is such a thing as giving the community so much power that they begin to make specific demands that may or may not be in keeping with the goals of the conference or the organization.” This brings up the issue of control at events, something that I’ve written about before. In 20 years of using crowdsourcing at events, I’ve never experienced an instance where a majority of participants made unreasonable demands. On the contrary, on every occasion when something unexpected (to me) has emerged from good group process, it’s turned out to be an accurate and useful way to improve the conference.

How web video powers global communication

The final article isn’t, in fact, an article, it’s a just-released video of a 19-minute TED talk by Chris Anderson entitled How web video powers global communication. In it, Chris describes what he terms Crowd Accelerated Innovation: “a self-fueling cycle of learning that could be as significant as the invention of print”. Chris argues that the rapidly increasing availability of web video is drastically speeding up the ability of people all over the world to focus on and improve the best-of-breed sharing they discover on their screens.

In a sense, web video becomes a tool for effectively crowdsourcing experience and experiments. Chris believes that this new medium has the potential to revolutionize learning and the development of new ideas and their implementation. But he cautions “…to tap into its power, organizations will need to embrace radical openness.” I think there’s a certain degree of hype in his presentation, but it’s well worth watching.

So, there you have a variety of ways in which crowdsourcing is entering and affecting the world of events. Which ways, if any, speak to you?

Tags: Convene, crowdsourcing, eventcamp, eventprofs, PCMA, peer session sign-up, TED| Posted in Event design | 7 Comments »

Comparing pre-conference and at-conference crowdsourcing

Tuesday, July 6th, 2010 by Adrian Segar

I’ve written about my skepticism of attempts to crowdsource session topics before a conference. After running edACCESS 2010 last month, I realized I could analyze the success of our pre-conference crowdsourcing. So here is a real-life comparison of pre-conference and at-conference crowdsourcing.

pre-conference crowdsourcing: list of a few of the topic suggestions at edACCESS 2010 — A few of the topic suggestions at edACCESS 2010

Conference details

edACCESS is a four-day conference for information technology staff at small independent schools that has been held every year since 1992. In 2010 there were 47 attendees, 9 of whom had not attended before. The conference uses the Conferences That Work event design.

Before the conference, we solicited suggestions for session topics via messages to the edACCESS listserv and email to registrants. Ideas could be posted on the edACCESS wiki, which requires registration. Some of the new attendees did not register on the wiki before the conference, but all returning attendees were registered (since most of each year’s conference content and discussion is posted there.)

Seven attendees (15%) posted a total of fourteen topic suggestions to the wiki before the conference.

In contrast, during the conference roundtable, 147 topics were suggested, an average of over three per participant.

The schedule for edACCESS 2010 included a vendor exhibit, two predetermined sessions (an attendee-created Demo Showcase and a Web 2.0 Demo Tools Workshop), and nine one-hour time slots for peer sessions with topics crowdsourced through the Conferences That Work peer session sign-up process, for which 78 topics were suggested.

We had space to run four simultaneous sessions in each of the nine time slots and ended up scheduling the 33 most popular (and feasible) of the 78 suggestions.

Findings

Of the 14 topics suggested prior to the conference, only 5 ended up being chosen for sessions by attendees at the event. So only 15% of the 33 actual conference sessions were predicted by attendees in advance of the event.

Of the 147 topics suggested during the conference roundtable, 26 were subsequently chosen for sessions. That’s 80% of the final sessions. Interestingly, seven of the final session topics were not mentioned in the roundtable notes. It’s quite common for participants to think of new topics after the roundtable is over. Once seen by other attendees, some of these ideas turn into desirable sessions.

Conclusions

It’s hard to know how to improve on our process of soliciting session ideas before the conference, short of forcing attendees to make suggestions during registration—something that would not go over well with the typical edACCESS attendee. In my experience, the above analysis is pretty typical for peer conferences I run. Having session topics and formats that fit participants’ needs is vital for the success of any conference. Given the poor showing for pre-conference topic crowdsourcing, (and, by extension, efforts of a conference’s program committee) I feel that having attendees brainstorm and then propose topics at the start of the event, as is done at Conferences That Work, is well worth the work involved.

I hope this comparison of pre-conference and at-conference crowdsourcing is informative. Do you think we should even be trying to crowdsource topics for a conference? Are you still skeptical of the utility of crowdsourcing topics at the beginning of a conference? What would you do to improve the success of crowdsourcing sessions before an event?

Tags: crowdsourcing, edACCESS, Event design, event-planning, peer session sign-up, session topic| Posted in Event design | 4 Comments »

Does asking attendees in advance for program suggestions work?

Monday, February 15th, 2010 by Adrian Segar

Does asking attendees in advance for conference program suggestions work?

Attendees repeatedly report in conference evaluations that the predetermined program was a poor fit for what they would have liked to have happened. In my experience, the average participant gives lukewarm ratings to over half of the sessions available to them at a conventional conference. In other words, predetermined programs don’t work.

How can we do better?

One obvious approach is to poll attendees before the conference. Nearly all conscientious event planners do this.

The problem is that asking your attendees for their input on the upcoming conference program simply doesn’t work very well. Here’s why:

Attendees are busy people

Attendees, like all of us, are busy people. How many will fill out a long (or even a short) questionnaire about what they want at an event that’s happening six months from now? Not many. Even if you force them to answer as part of the registration process, how much time are they going to spend to really think about the three most important topics they’d like you to offer? Sure, a minority of attendees will be conscientious and may give you some good ideas. But do you know if they represent an unbiased sample of your attendees? Do you want to base your conference program on their responses?

Six months is an eternity

Most food goes stale. (OK, Twinkies don’t, but how many of us enjoy eating Twinkies?) Similarly, most conference topics have expiration dates. The topic that’s hot now may be cold by the time your conference rolls around. So even if lots of your attendees tell you they’re really into sushi now, it may be Cambodian Cha knyey when it’s time to actually sit down for the conference meal.

What do you want to talk about now?

There’s a world of difference between a response to the question “What do you want?” when it’s asked about the distant future and when it’s asked about what you want in the next five minutes. At Conferences That Work, when the roundtable facilitator announces that, in five minutes, people will start to answer three questions out loud to everyone present, minds become wonderfully concentrated. That’s when you find out what attendees really want. Not before.

Conclusion

I’m not saying we should give up asking in advance what attendees want in a conference program. Sometimes you’ll get good suggestions for conference presenters or session topics that you can turn into valuable sessions at the event. But you’ll rarely be able to create the bulk of a conference program that fits as well as one that’s created at the event.

So, don’t bother asking attendees for conference program suggestions in advance. Remember, predetermined programs don’t work. Instead, relax. Use event crowdsourcing to ask your attendees what they want. Your attendees will build the best program possible themselves—and they’ll thank you for the opportunity!

Image attribution: http://www.flickr.com/photos/joshuacraig/ / CC BY-ND 2.0

Tags: crowdsourcing, event-planning, program, what attendees want| Posted in Uncategorized | 4 Comments »