The COVID-19 pandemic has devastated the in-person meeting industry. Though it took too long to recognize that COVID-19 spreads via air transmission, we finally have effective procedures (vaccine mandates, masking, air quality standards, and social distancing) to reduce infection risk at in-person meetings. Now, meeting planners can add an affordable air quality tool to their site visits.
How can you determine air quality at a prospective venue?
Look around the room at an in-person event and you’ll see if masking and social distancing are taking place. We can implement vaccination mandates using third-party vendors such as sharemy.health, CLEAR Health Pass, Safe Expo, and others. But how can we determine the air quality at a prospective venue?
Currently, we don’t know how to detect airborne COVID-19 viruses. (This is likely to be true for a long time. We still have no test for airborne tuberculosis bacterium (TB) transmission two centuries after identifying TB as a distinct disease.)
Luckily, under the conditions I’ll outline below, we can obtain useful information about a venue’s air quality by using a device that measures a proxy for air pollution: carbon dioxide (CO₂).
People breathe in air, typically containing about 0.04% CO₂. They breathe out a mixture of gases containing about 4 – 5 % CO₂. People with COVID-19 co-exhale respiratory aerosols containing the SARS-CoV-2 virus.
If an occupied building space has effective ventilation, the occupants’ excess exhaled CO₂ is quickly diluted with fresh air, and the CO₂ level in the air remains close to normal values. Measuring the level of CO₂ in the air can, therefore, tell us whether effective ventilation is present or not.
Here are the generally accepted standards for CO₂ levels:
~400 parts per million (ppm) – Normal outdoor air level. 400 ~ 1000 ppm – Typical value level indoors with good ventilation. 1,000 ppm – the OSHA/ASHRAE recommended maximum level in a closed room. > 1,200 ppm – Poor air quality – requires ventilation in the room. 2,000 ppm – This level of CO2 typically produces a significant increase in drowsiness, tiredness, headaches, lower levels of concentration, and an increased likelihood of spreading respiratory viruses.
Until recently, meters that measure CO₂ levels in the air cost hundreds of dollars. (Some models with especially accurate sensors or the capability to measure other air pollutants still do.) But today we can buy an affordable air quality tool — a hand-held CO₂ meter for under $100. The one I just purchased (illustrated above) cost $80, and there’s a wide variety to choose from (for example, from here or here).
My 3.27″ (diameter) x 1.26″ (depth) meter measures CO₂ levels from 0 – 5,000 ppm. It can run on standby for 18 hours, supports USB charging, and includes a battery level indicator and temperature and humidity readings. While its specifications omit accuracy, inexpensive CO₂ meters are typically reliable within ±100 ppm. This is good enough to provide a decent estimate of the air quality in an enclosed space.
My unit shows a concentration of ~350 ppm CO₂ outside my rural Vermont home, which was built tightly. In my home office, the level increases to about 450 ppm and rises to about 525 ppm if I’m sitting next to the unit for a while. Slightly cracking open a window quickly brings down the reading.
I haven’t had time to explore other buildings yet, but am looking forward to seeing what I find out when I do.
Is a CO₂ a proxy for indoor air quality in occupied spaces?
Can measuring CO₂ levels give us a useful indication of indoor air quality?
The answer is a qualified yes. It depends!
First of all, we need to measure CO₂ levels in occupied spaces. A meeting planner doing a site visit should take CO₂ readings in occupied meeting rooms, restaurants, hotel lobbies, etc. Taking measurements in empty spaces will only show high readings if the building ventilation system is grossly inadequate (with CO₂ infiltrating from other areas.) Also bear in mind that increasing the number of occupants in a space increases the likelihood that an infectious person will be present and the number of people possibly infected. Doubling occupancy can thus cause a four-fold increase in risk of transmitting COVID-19.
Finally, air treatment options, such as MERV 13 or better filtering, or possibly ultraviolet-C radiation, may reduce the prevalence of active COVID-19 aerosols. When venues employ these mitigation strategies, CO₂ levels will not be decreased. Of course, if a venue has deployed these preventative measures, they will surely inform you about them when asked!
Due to these factors, you shouldn’t rely solely on measurements of CO₂ levels to determine whether a space is ventilated enough to mitigate transmission risk.
However, a simple CO₂ meter like the one I now own can be an effective air quality tool, providing valuable information to anyone who wants to investigate the air quality of occupied spaces at venues, hotels and properties, restaurants, and other meeting locales. I’ll be bringing mine when I travel, and I encourage you to do this as well!
More information on the relationship between CO₂ levels and COVID-19 exposure
If you’d like to learn more about the relationship between CO₂ levels and COVID-19 exposure risk, here are some useful references:
Sadly, while I acknowledge and appreciate Freeman’s significant work on the case for recommencing meeting in person, I believe this claim is misleading, and the underlying modeling and research include flawed assumptions.
Make no mistake; I love to design and facilitate in-person meetings. I strongly desire to be able to safely return to facilitating and attending in-person events. But, as meeting professionals, we have a professional duty of care during the COVID-19 pandemic. So, I think it’s important to provide a realistic assessment of risk for meeting stakeholders—especially potential attendees. Articles are already appearing in meeting publications (1, 2) that highlight the one-line summary of the Freeman announcement above. Such opinions, buttressed by what seems to be solid research and modeling, can easily give our industry the impression that in-person meetings can safely recommence.
My concerns about Freeman’s statements
I have two broad concerns about Freeman’s summary of research “Inside LIVE: The data you need to navigate the Delta variant for events” on the safety of in-person events. You can watch Freeman’s 55-minute webinar, posted on August 25, 2021, below.
1—Freeman’s overall conclusion is misleading
My first concern is that Freeman’s big-picture conclusion that “in-person events are actually safer than many daily activities, like trips to the grocery store” is a misleading characterization of the statistics they present.
Here are the statistics (from the next webinar slide).
This slide compares the mid-August, 2021 rate of COVID cases amongst the entire population in the United States with the reported rates from four recent large in-person events. The second column shows the infection rate as a percentage.
The entire U.S. infection rate is indeed higher than the reported rates from the listed recent in-person events. (I’ll add that we know that reported rates are typically significantly lower than actual rates, but let’s assume that both sets of statistics are undercounted to the same degree.)
Because the statement conflates the risk of a masked visit to a grocery store with the overall risk in the United States of getting infected! The latter regrettably includes a significant fraction of the U.S. population who:
Won’t or can’t be vaccinated;
Don’t wear masks to protect against airborne transmission of COVID-19; and
Don’t social distance.
The risk of contracting COVID-19 during a grocery store masked visit is far less than the overall risk for everyone in the U.S.
The headline statement is, therefore, comparing apples to oranges. You’d expect any event that implements precautions against COVID-19 transmission to have a lower infection rate than the entire United States. That doesn’t mean that attending an event is a safe enough choice for attendees and staff.
This brings us to what’s actually important to people trying to make a decision about whether to attend an event. The event modeling, performed for Freeman by Epistemix, and discussed later, suggests that those who are currently likely to attend a large in-person business event that implements mitigation strategies such as vaccination requirements, masking, and social distancing, are significantly more likely to be vaccinated (~80%). That statistic seems credible to me.
Such potential attendees, who are already more careful than the average American about how they live their lives in a pandemic, aren’t interested in whether an event environment provides a risk of getting COVID-19 comparable to the average risk of the entire population of the U.S. Rather, they want to know if attending the event will significantly (defined by them) increase their likelihood of contracting COVID-19. And that brings me to the second concern about the assumptions made by Epistemix’s event risk model.
2—The event risk model used for risk calculations is flawed and incomplete
When I heard about the Freeman webinar (thanks Julius Solaris!), I posted some initial responses. Freeman’s Jessica Fritsche was kind enough to reach out to me and arrange a Zoom call with John Cordier, the CEO of Epistemix, to walk through the data modeling used in the research. And John generously offered an hour of his time for us to talk. Sarah Shewey, Founder/CEO of Happily, also joined us. Sarah was interested in learning more about how infection rates at meetings could be modeled.
During our hour together, John shared an overview of the Epistemix model. This gave me a better understanding of Epistemix’s approach. The model essentially attempts to simulate the entire population of the United States at an impressive level of detail. It includes numerous geographic and social factors that affect infection risk. However, during our conversation, I asked about a number of important factors that I believe Epistemix has not incorporated into its model of calculating meeting risks.
Probably the most important of these is adequately modeling the air quality at the event, given the paucity of information available about the safety of specific venues and properties from an air quality perspective. In addition, the model does not include the additive risks for travel to and from an event, and staying in a hotel during an event. Though it’s likely possible to model the increased risk during (unmasked) eating and drinking social activities during the event, it doesn’t appear that the Epistemix model does this. Finally, though the Epistemix model incorporates information about COVID-19 variants as they become known, I’m skeptical that it can accurately predict in a timely manner the impact of brand-new COVID-19 variants.
In the following sections, I’ll expand on these issues in more detail.
Flaws and omissions in Epistemix’s meeting model
First, a tiny introduction to modeling human systems. All models are an approximation of reality. Consequently,
I learned to program computers in high school, over 50 years ago. Through a series of summer jobs, undergraduate and graduate work, and consulting assignments, I’ve spent years creating computer models of city traffic systems, the interactions of high-energy particle beams bombarding matter, the consequences of obscure physics theories, and the functions of complicated administrative systems.
Two fundamental considerations when building and trusting computer models are:
The assumptions one makes in building a model are key to the model being actually useful rather than wrong. Computer models are very seductive. They seem precise and authoritative, and it’s hard to discover and accept their limitations and/or even their completely wrong predictions. Choosing the right assumptions is an art, not a science. One poor assumption can doom a model’s reliability.
Even if you choose good assumptions, implementing them correctly in computer code is difficult. It’s hard to be sure that an implementation faithfully reflects core assumptions. An incorrect implementation of a potentially useful model typically leads to incorrect predictions. If you’re lucky, it’s obvious that a model’s outputs are wrong. But sometimes, predictions are subtly wrong in ways that are easy to overlook.
I’m going to assume that Epistemix models faithfully implement the assumptions made to create them (#2 above). However, I’ve identified four factors that I feel Epistemix has not incorporated into its model of calculating meeting risks. Some of these factors are interlinked.
1—Adequately modeling airborne COVID-19 transmission at a specific event
While talking to John, it became clear to me that the current Epistemix approach does not adequately model the air quality—and the consequent risk of COVID-19 transmission—at a specific event. The model has some capacity to estimate risks (which are generally minimal) in very large, high-ceiling spaces like convention halls. But, of course, the typical meeting venue contains multiple meeting spaces, some of them small, and, critically, the venues do not in general have a good handle (if any) on the air quality in those spaces. (Or, if they do, they’re not talking publicly about it.)
When I wrote about this issue six months ago, I put out an industry-wide request to learn of venues and properties that had upgraded their HVAC systems to current ASHRAE recommendations (typically ~5 air changes/hour plus MERV 13 or better air filtering). I promised to publicize the venues that had made these upgrades.
I know such upgrades can be costly. But you’d think that venues and properties that have implemented them would love to promote themselves as having air quality that meets current pandemic-based standards.
To date, I have not been told of a single venue that is now compliant with ASHRAE pandemic recommendations. (I hope that by now there are some and that they will share this information.) During the webinar, Freeman said that they have been and are doing such work. Please share this information, folks! Meeting planners want to know!
Frankly, without this information a) being made available and b) being incorporated into the Epistemix model it’s hard to have much confidence in the infection risks Epistemix’s model predicts.
2—Additive risks for travel to and from an event, and staying in a hotel during an event
Epistemix’s model does not include the additive risks for attendees (and staff) traveling to and from an event. The main concern is air travel. The air industry has stressed that air change rates in aircraft are high (over 10 air changes/hour) and, now that masks are mandatory, infection risks should therefore be low. An excellent investigation by the New York Times “How Safe Are You From Covid When You Fly?” has tempered this assessment somewhat. Of particular interest are comments from a couple of readers who monitored the carbon-dioxide level—an excellent proxy for air quality—during their entire travel. They found that boarding and deplaning air quality was drastically reduced, as well as during the last thirty minutes of one person’s flight. Exposure at terminal restaurants, where masks must be removed, is also potentially risky.
Quite apart from the “event” itself, staying in a venue may greatly increase one’s risk of infection. I wrote about venue and property ventilation concerns in detail in April 2021, and later articles by PCMA (1, 2) and the New York Times (1) have echoed this concern.
Again, travel risks are not included in Epistemix’s model. They can be significant. They have to be included to determine the relative risk for an event attendee who is choosing whether to participate or staff an event, or not.
3—Modeling the increased risk when masks are off for socials and group meals
Most in-person meetings include meals and socials, when masking is not possible. Unless you hold such unmasked get-togethers outdoors or in safely ventilated venues, airborne transmission of COVID-19 amongst everyone present (attendees and staff) is a potentially significant and unknown risk. Outdoor locations are only possible for limited periods in much of the U.S. As mentioned above, venues and properties remain silent on whether they’ve upgraded and certified their facilities to current ASHRAE recommendations on air quality.
We have also seen reports of numerous cases of reduced, unmasked social distancing at socials and meals. We can understand this in a world where we’ve masked for so long. But it is still a risky activity, especially in spaces where ventilation is inadequate.
My understanding (which may be incorrect) of Epistemix’s model is that masking is a global parameter for an event. The model does not handle unmasking in specific event spaces for periods of time. Even if the model does have this flexibility, the lack of knowledge of whether such spaces are safely ventilated prevents an accurate risk assessment.
4—Can Epistemix model the appearance of brand-new COVID-19 variants?
I am also still skeptical that Epistemix can build new variants into the model predictions in a timely fashion. The world took about six months after the delta variant was first identified to realize that it was radically changing COVID-19 transmission rates. While Epistemix’s model includes the infection characteristics of multiple variants and new variants can be added once they are identified, I wonder if an event organizer who made a go/no-go decision about a fall meeting early this summer based on the Epistemix model would be happy about the increased predicted risks once the delta variant was added.
But John and I didn’t have time to fully explore this issue, so this concern may be overblown.
Are in-person events COVID safe?
I really appreciate John Cordier’s willingness to share an overview of Epistemix’s infection risk model for events. Obviously, my brief introduction means there’s no way I can authoritatively review the extensive assumptions that are built into the model. Epistemix’s model is impressively detailed and, if correctly implemented (which I have no reason to doubt), seems to comprehensively cover core demographics, the data needed to model infection spread in regional populations, and most major components for predicting infection at a specific event.
When I brought up the concerns I’ve listed in this post, I felt that John largely talked past me, continuing with an explanation of the model without responding directly to what I was asking. This was somewhat frustrating. The two exceptions to this were:
My question about whether the model could accurately predict in a timely manner the impact of brand new variants. This arose at the end of our meeting. John indicated that he believed the model was able to do this, but we didn’t have enough time to explore this issue fully. I’m still skeptical, though he might well have been able to convince me otherwise if we’d had more time.
My primary concern about modeling air quality in detail. John admitted during the meeting that the model does not currently handle specific venue air quality architecture at the detail that’s necessary to simulate, say, what happens when you have a session in a smaller classroom with an HVAC system that is not up to current ASHRAE recommendations. It also omits risks due to event participants (and staff) spending time in properties that may have inadequate air quality. He wrote to me afterward that “he’d be glad to follow up on the air-quality parameters that you think are most important”.
The limitations of modeling
I’ve seen so many pretty models of systems over the decades. To a casual viewer, they look impressive. It’s only because I spent years building and validating such models that I know how misleading they can be. It’s difficult, but important to identify the key factors and approximations that form the basis of the model and limit its scope and/or accuracy.
Leaving out detailed venue-specific air quality modeling, plus the incoming and return travel risks and accommodation risks during an event, plus inadequate modeling of the risk of transmission during socials and food & beverage sessions make the outputs of the Epistemix model suspect. And I’m skeptical that Epistemix can build new variants into the model predictions in a timely fashion.
Finally, I haven’t covered in this article the feasibility of implementing the various mitigation strategies that are available to reduce the risk of COVID-19 infection at meetings. Personally, I’d insist on proof of vaccination (no exceptions) and maximal masking at any event I’m likely to attend in the near future. But I’ll just add here this observation from the Healthcare Information and Management Systems Society‘s HIMSS21 Las Vegas conference for 19,000 attendees. Vaccination was mandatory for all attendees. There were six positive test results (0.03% infarction rate). However, this PCMA article on the event includes the statement: “…you will not be able to service your show if you require every single vendor employee, every single supplier employee, every single temp employee to be vaccinated — there’s just not enough labor out there.” Something to bear in mind.
Are in-person events COVID safe?
Are in-person events COVID safe? Ultimately, each of us needs to decide the answer to this question. But, in my opinion, until the COVID-19 case count drops drastically and/or venues can show that their facility ventilation is safe, it’s a violation of our professional duty of care to mislead attendees and those who work in our industry by telling them “in-person events are actually safer than many daily activities, like trips to the grocery store”.
I’d like to be clear that I don’t hate in-person meetings, despite what some have been posting recently on a Facebook group for meeting professionals:
“Often wondered why so many on this feed hate live events.”
“It is my opinion that this group does not support any in-person meetings or gatherings of any kind…”
” I am sad to see so many industry giants verbally destroying our industry – apparently with glee.”
Let’s explore what’s causing opinions and feelings like this in the meeting industry.
The tension in the meeting industry
As I’ve said before, the pandemic’s impact on lives and businesses has been devastating, especially for the meeting industry. COVID-19 has virtually eliminated in-person meetings: our industry’s bread and butter. Many meeting professionals have lost their jobs, and are understandably desperate for our industry to recover. We are all looking for ways for in-person meetings to return.
Unfortunately, I and many others believe there is a strong case to make against currently holding in-person meetings. Ethically, despite the massive personal and financial consequences, we should not be submitting people to often-unadvertised, dangerous, and life-threatening conditions so we can go back to work.
I’ve been posting bits and pieces of the case against currently holding in-person meetings on various online platforms and decided it was time to bring everything together in one (long for me) post. I hope many meeting industry professionals will read this and respond. As always, all points of view are welcome, especially those that can share how to mitigate any of the following concerns.
The strong case against holding in-person meetings right now
Here are four important reasons why I think we shouldn’t be holding “large” in-person meetings right now. (Obviously, “large” is a moving target. Checking Georgia Tech’s COVID-19 Event Risk Assessment Planning Tool as I write this, a national US event with 500 people is extremely likely (>95%) to have one or more COVID-19-positive individuals present.)
meticulously observed social distancing and masking;
could safely travel to and from events;
be housed safely; move around event venues while safely maintaining social distancing; and
eat and drink safely.
Even if one could meet these difficult conditions, I questioned the value of such in-person meetings. Why? Because meetings are fundamentally about connection around relevant content. And it’s impossible to connect well with people wearing face masks who are six or more feet apart!
In addition, there’s ample evidence that some people won’t follow declared safety protocols. Since I wrote that post, we have heard reports and seen examples of in-person meetings where attendees and staff are not reliably social distancing, and/or aren’t wearing masks properly or at all.
Orlando, Florida, OCCC Together Again Expo, July 2020
This is most likely to happen during socials and meals, where attendees have to temporarily remove masks. It’s understandably hard for people to resist our lifetime habit of moving close to socialize.
2) We perform hygiene theater—but please don’t ask us about our ventilation systems
Many venues trumpet their comprehensive COVID-19 cleaning protocols. Extensive cleaning was prudent during the early pandemic months when we didn’t know much about how the virus spread. But we now know that extensive cleaning is hygiene theater (1, 2); the primary transmission vector for COVID-19 is airborne.
A recent editorial in the leading scientific journal Nature begins: “Catching the virus from surfaces is rare” and goes on to say “efforts to prevent spread should focus on improving ventilation or installing rigorously tested air purifiers”.
I haven’t heard of any venues that have publicly explained how their ventilation systems minimize or eliminate the chance of airborne COVID-19 transmission!
Why? Because it’s a complicated, and potentially incredibly expensive issue to safely mitigate. And venues are reluctant or unable to do the custom engineering and, perhaps, costly upgrades necessary to ensure that the air everyone breaths onsite is HEPA-filtered fast enough to keep any COVID-positive attendee shedding at a safe level.
Adequate ventilation of indoor spaces where people have removed masks for eating or drinking is barely mentioned in governmental gathering requirements (like this one, dated March 3, 2021, from the State of Nevada). These guidelines assume that whatever ventilation existed pre-COVID is adequate under the circumstances, as long as all parties are socially distanced. We know from research that there are locales — e.g. dining rooms with low ceilings or inadequate ventilation — where this is not a safe practice, since it’s possible for COVID-carrying aerosols to travel far further than 6 feet.
In case you are interested, current recommendations are for MERV 13 filtering throughout the venue. Does your venue offer this?
P.S. I expect there are venues that have done this work. Do you know of venues that have done the engineering to certify a measurable level of safe air on their premises? If so, please share in the comments! We should know about these conscientious organizations.
3) Inadequate or no pre-, during-, or post- COVID testing, and contact tracing
Shockingly, many in-person meetings now taking place require no pretesting of staff or attendees. (News flash: Checking someone’s forehead temperature when they enter a venue will not detect anyone who is infectious for the two days before symptoms appear, or who is asymptomatic.)
Even if everyone in the venue is tested daily, the widely used quick tests are simply too unreliable. From Nature again:
“Deeks says that a December trial at the University of Birmingham is an example of how rapid tests can miss infections. More than 7,000 symptom-free students there took an Innova test; only 2 tested positive. But when the university researchers rechecked 10% of the negative samples using PCR, they found another 6 infected students. Scaling that up across all the samples, the test probably missed 60 infected students.” —Nature, February 9, 2021, Rapid coronavirus tests: a guide for the perplexed
Finally, I find it upsetting that venues like the OCCC keep claiming that they are #MeetingSafely when they are doing no post-event follow-up! If an attendee contracts COVID-19 at the event, returns home and infects grandma, how would the OCCC ever know?! Under the circumstances, I think it’s misleading, dangerous, and unethical for such a venue to publicly claim that they are providing an #MeetingSafely environment.
4) We’re meeting safely—but you can’t sue us if we’re not
“I voluntarily assume full responsibility for any risks of loss or personal injury, including serious illness, injury or death, that may be sustained by me or by others who come into contact with me, as a result of my presence in the Facilities, whether caused by the negligence of the AKC or OCCC or otherwise … I UNDERSTAND THIS IS A RELEASE OF LIABILITY AND AGREE THAT IT IS VALID FOREVER. It is my express intent that this Waiver binds; (i) the members of my family and spouse, if I am alive, and (ii) my heirs, assigns and personal representatives, if I am deceased.” —Extract from the Orlando, Florida, OCCC American Kennel Club National Championship Dog Show, December 2020, Waiver
I’m not sure how you can bind people to a contract who may not know they are a party to it. But, hey, I’m not a lawyer…
So, can we safely and ethically hold in-person meetings right now?
For the reasons shared above, I don’t believe we can safely and ethically hold in-person meetings right now. Consequently, it’s alarming that many venues, and some meeting planners, are promoting in-person meetings in the near future.
Do I hate in-person meetings?
By now it should be clear that I stand with meeting professionals like Cathi Lundgren, who posted the following in our Facebook group discussions:
“I’m not going to be silent when someone holds a meeting in a ballroom with a 100+ people and no masking or social distancing…I own a global meetings company—and we haven’t worked since March but no matter how much I want to get back at it I’m not going to condone behaviors that are not positive for the overall health of our industry.”
—Cathi Lundgren, CMP, CAE
And here’s how I replied to the first Facebook commenter quoted at the top of this post:
“For goodness sake. I LOVE in-person events. It’s been heartbreaking for me, like everyone, to have not attended one for a year now. But that doesn’t mean I am going to risk stakeholder, staff, and attendee lives by uncritically supporting in-person meetings that are, sadly, according to current science, still dangerous to attend. When in-person meetings are safe to attend once more — and that day can’t come soon enough — you bet I’ll be designing, facilitating, and attending them.”
I hope it’s clear that I, and those meeting professionals who are pointing out valid safety and ethical concerns, don’t hate in-person meetings. Realistically, the future of in-person meetings remains uncertain, even with the amazing progress in developing and administering effective vaccines. More mutant COVID-19 strains that are resistant to or evade current vaccines, transmit more effectively, or have more deadly effects are possible. Any such developments could delay or fundamentally change our current hopes that maintaining transmission prevention plus mass vaccination will bring the pandemic under control.
I’m cautiously optimistic. But, right now, there are still too many unknowns for me to recommend clients to commit resources to future large 100% in-person events. Hub-and-spoke format hybrid meetings look like a safer bet. Regardless, everyone in the meeting industry hopes that it will be safe to hold in-person meetings real soon.
In the meantime, please don’t attack those of us in the industry who point out safety and ethical issues and the consequences of prematurely scheduling in-person meetings. We want them back too! We all miss them.