Event professionals: it’s time to talk about COVID, duty of care, and the meeting industry. Actually, we should all have been talking about this for the last couple of years, but better late than never.
Air quality has a significant effect on human health. During the COVID-19 pandemic, it has become an especially critical issue. Why? Because COVID-19 spreads via aerosols that can float in the air for minutes to hours. Although there is currently no commercially available way to measure the presence of COVID-19 in air, I’ve written about how measuring carbon dioxide (CO2) concentrations can act as a useful proxy for COVID-19 infection risk. Small, inexpensive CO2 meters are now widely available.
So when I took a deep (masked) breath and decided to accept an invitation to design and lead a two-day meeting industry leadership summit in Puerto Rico, I decided to bring my CO2 meter with me. What would I learn about the air quality in the airports, planes, and ground transportation I used, as well as my hotel and the summit’s convention center? Well, I uncovered significant air quality concerns in places that may surprise you. Read on to find out what I discovered. But first, a brief explanation of what CO2 measurements mean.
How do CO2 levels correlate with the risk for COVID-19 infection?
It’s complicated! Measurements of indoor CO2 concentrations can often be good indicators of airborne infection risk. But clear conclusions on the CO2 level corresponding to a given COVID-19 infection risk are currently lacking. Multiple factors influence the risk. These include exposure duration, the mixing of air in the vicinity, the exhalation volume and rate of infected individuals, and, of course, the use of masks, virus-removing air filtration, and UVC and far-UVC radiation. This article gives some idea of the complexities involved. The American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) has summarized current thinking on indoor CO2. ASHRAE takes the position that “indoor CO2 concentrations do not provide an overall indication of IAQ [indoor air quality], but they can be a useful tool in IAQ assessments if users understand the limitations in these applications.”
More research is required, especially because of “the ubiquity of indoor concentrations of CO2 in excess of 1,000 [parts per million] ppm.” And ASHRAE reports that “indoor concentrations of CO2 greater than 1,000 ppm have been associated with increases in self-reported, nonspecific symptoms commonly referred to as sick building syndrome symptoms.” To summarize, currently, there is insufficient research suggesting CO2 levels that indicate a significantly increased risk for COVID-19 infection. However, many authorities have tentatively proposed maximum levels of around 1,000 ppm CO2 as guidelines.
OK, enough of this; you probably want to know what I found. Here we go!
I flew JetBlue flights 261 and 462 between Boston (BOS) and San Juan (SJU). My outbound flight, on an Airbus A321, lasted 3 hours and 43 minutes. My return flight, on an Airbus A320, took 4 hours and 40 minutes. (Don’t ask.) On both flights, I had an aisle seat in row 15. As you can see from the photo at the top of this post, I perched my little CO2 meter on my knees when tray tables had to be up. The rest of the time, it nestled perfectly into the little tray table drink recess. Here’s an annotated graph of the CO2 readings I took on my outbound flight.
My key flight observations
Boarding the aircraft led to a large spike in CO2 levels. Levels increased sharply in the jetway as I approached the passenger door. Slowly walking down a packed aisle to my seat I saw readings around 2,000 ppm. Once in my seat, the levels dropped somewhat but were still high (1,600 ppm) when they closed the door.
Levels stayed high (above 1,500 ppm) while taxiing until we took off. We had been on the plane for about 50 minutes at this point.
I estimate that about 30% of the passengers were unmasked, as well as most of the flight attendants.
During the cruising portion of the flight, the CO2 level stayed at REHVA’s “upper range of reliable air quality” of 1,000 ppm. The level in the bathroom was 1,200 ppm.
Once we started our descent, levels rose a few hundred ppm. On landing, we were at 1,300 ppm.
During deplaning, levels soared again. I took the photo at the top of this post, showing a reading of 2,074 ppm, at this point.
As soon as they opened the passenger door, levels dropped to around 1,200 ppm.
On my return trip (which took close to five hours) I saw similar readings, except that:
The cruising flight CO2 level was significantly higher (1,200 – 1,400) ppm.
The boarding peak was lower (1,500 ppm).
The deplaning peak was an unsettling 2,400 ppm.
To summarize, these readings are troublesome. Aircraft ventilation systems reportedly filter out aerosols, assuming that the HEPA filters are regularly replaced. However, the close proximity of passengers (both flights were full) still allows people to infect others close to them, as this NY Times article illustrates. The high readings I saw indicate that in-flight ventilation was not fully operative during embarkation and deplaning on either flight. I am glad I wore a high-quality N95 mask during both.
BOS airport levels were around 600 ppm. At SJU I saw readings between 650 – 800 ppm. Both of these are acceptable. Neither airport was especially crowded, however, and I would be cautious about assuming it’s OK to go unmasked there.
This was a shocker to me. In the U.S. during the pandemic, when driving with others I’m used to having the car windows open, at least a little. Puerto Rico was hot and humid, and the vehicles I was in had the A/C on and windows closed. My client had arranged a car and driver to pick me up from the airport and drive me to the convention center for a couple of technical rehearsals and then to my hotel. Just the two of us in a Chevy Suburban quickly raised the CO2 level to around 1,500 ppm for the 30 minutes we were together. Luckily we were both masked.
I saw the same readings during my trip to the airport at the end of the event.
But I saw the highest readings during my travel in a shuttle bus bringing us to the opening reception. There were, perhaps, 20 of us on board. Readings spiked to over 3,000 ppm! And some of the passengers were unmasked.
The conference center
The conference center was far from maximum capacity and I only saw readings well below 1,000 ppm. We held the summit in four meeting rooms with high ceilings. We left the meeting room doors open, and my meter typically showed readings between 500 – 600 ppm. If the venue had been packed or the doors closed it might have been a different story.
I was concerned about the air quality in my (large) hotel room because I expected it to have no openable windows due to San Juan’s climate, and this proved to be the case. Over the three nights I was there I noticed the same pattern. On entering the room during the day, readings were about 600 ppm. As evening approached, the readings slowly climbed to about 900 ppm.
I had reason to be concerned.
The increase in CO2 as evening approached was probably due to increased occupancy of nearby rooms. Building heating, ventilation, and cooling (HVAC) systems typically recirculate interior air, mixing together air from all the rooms in the building. So as guests retire to their rooms in the evening, the overall CO2 concentration in every room increases.
That means that although I was alone in my room I was breathing exhalations from other guests. If any of those guests had COVID-19, it’s possible that their aerosols would travel into the air I was breathing. There was nothing I could do to protect myself other than wearing a mask the whole time I was there (which obviously included sleeping!)
Commercial HVAC systems
Commercial HVAC systems include filters to remove dust and dirt. Typical HVAC filters will not stop COVID-19 aerosols unless they have been upgraded to MERV 13 or better (e.g. HEPA). They also need to be regularly replaced to work correctly.
Whether these mitigation measures have been performed at a hotel is hard to know. My hotel was modern, but that doesn’t mean its HVAC system was well-designed and safe. I have stayed at hundreds of hotels over the years. Some of them, based on the odor of the rooms, had ventilation problems of some kind. Paradoxically, the single-unit heating and cooling systems common in inexpensive lodgings could be safer because air entering the room only comes from outside.
Concerns like these have made me cautious about staying in accommodations that don’t have windows that can be opened. That wasn’t possible in Puerto Rico, and my CO2 monitor gave me at least some reassurance that air quality levels weren’t too bad. However, many commercial lodging offerings don’t offer this option. The inspection and, if necessary, re-engineering of hotel HVAC systems is an important step to protect guest health. Yes, it costs money, but if the owners have done this work they should publicize it as a reason to stay.
As I write this, I’ve been isolating for four days since my return, and just performed my fourth daily rapid antigen test. All have been negative. So it looks like I’ve escaped getting COVID-19 during my first major travel since the pandemic began. I recommend travelers purchase an inexpensive CO2 meter and bring it with them.
I hope the information I’ve shared in this post is helpful in warning other travelers of potentially dangerous environments. COVID-19 is far from over. As the pandemic continues, monitor your air quality while traveling—and mask up.
A rare opportunity! Hosted by CSAE Manitoba, this free one-hour online Participate Lab will introduce you to the design of participation-rich events through the direct experience of participatory meeting techniques and formats. All are welcome to attend this event at no charge (both CSAE members and non-members).
We’ve known for a long time that lectures are terrible ways to learn. Today’s attendees are no longer satisfied sitting and listening to people talking at them. If you want to hold meetings where effective learning, connection, and engagement take place, you need to build in authentic and relevant participation.
Our time together at this Participate Lab will cover:
Why creating participation-rich meetings is so important.
Human spectrograms: a simple tool for learning about other participants.
The Conference Arc:
Building connection while uncovering wants, needs, and resources.
Creating the right program.
Facilitating individual and community growth.
Ask Adrian Anything: using a fishbowl sandwich to facilitate group discussion on meeting design and facilitation.
The old meeting industry normal is long over, and many event professionals are still hoping and waiting for a new normal.
In October 2020 I wrote two posts [1, 2] about what a meeting industry new normal might look like. As I write, two years have passed since the COVID-19 pandemic devastated the world and the event industry. It’s time to take another look. How have my predictions held up? And what does the future hold?
Six months into the pandemic, I wrote that three fundamental things had to happen for everything to go as well as possible in the global fight against the coronavirus.
1. “If we’re really lucky, we’ll have a safe, inexpensive, effective vaccine sometime before the end of 2021.”
Even if no further variants appear, the above factors mean that COVID-19 is here to stay for the indefinite future. As I write, for example, South Korea is experiencing a massive surge, the largest of the entire pandemic. The dominant COVID-19 variants are so contagious that it’s currently impossible to prevent further spread and outbreaks until most of the world population is adequately vaccinated or builds up enough (weaker) immunity through repeated infections.
We may eventually tame the pandemic by developing effective and inexpensive antivirals and making them widely available to those contracting COVID-19. However, the virus is likely to develop resistance to such drugs, which are currently in short supply and expensive, so continued R&D will be needed.
Finally, it’s important to remember that we still do not understand the health impact of long covid. The American Medical Association estimates that “anywhere from 15% to 80% of patients might experience long COVID after recovering—even if they weren’t very sick in the first place”. I have friends and family that are still suffering serious effects of long covid—you probably have too. Now vaccines and better treatments have reduced the risk of dying from COVID-19. But that doesn’t mean we can dismiss its significant long-term health consequences going forward.
Holding in-person meetings: what do we now know?
Here’s a quick overview of what I see as the relative risks involved in attending in-person meetings at this point. Two important caveats are that I’m assuming travelers:
Are fully vaccinated; and
Use good quality masks when in public enclosed spaces.
Risks of serious illness for the unvaccinated are at least an order of magnitude higher. See below for situations when masks cannot be worn.
Airline travel seems reasonably safe these days. Airlines claim “cabin air is refreshed 20-30 times an hour.” If correct, this is more than adequate. The main exposure risks occur during boarding and deplaning when in-flight airplane ventilation systems are not operating. However, I would avoid long plane flights for now if possible, as it’s somewhat risky to unmask to eat or drink on a plane.
Train travel has a similar risk exposure. Amtrak says that its “trains are equipped with onboard filtration systems with a fresh air exchange rate every 4-5 minutes”. Again, if accurate, this is more than adequate.
If attendees and staff follow precautions, traveling to and from meetings is not as high-risk as the following activities.
As described below, very few hotels (and venues) seem to have implemented ASHRAE’S building readiness standards for air quality in their properties. Sleeping in a hotel room when one can’t wear a mask has an unknown and potentially high risk for COVID-19 infection unless you can obtain fresh air by opening windows. Consequently, I currently prefer to stay in self-contained Airbnb properties. There, I can be confident that air from an unknown source won’t contaminate indoor air.
Dining and socializing
Currently, eating and drinking indoors is quite risky unless the location has upgraded its HVAC systems to adequately filter COVID, the space has very high ceilings, or copious fresh outdoor air is available from open windows.
Understandably, people want to connect at in-person meetings. We are drawn to do this during meals and socials where masks cannot be or are not worn. Which can lead to consequences like this:
“…now myself and at least 25% of our participants are sick with COVID. I am hearing from someone else every day…All the precautions in the world don’t really matter if you abandon them when people eat and drink. We all know this yet we are all still doing it for the most part.“ —Quote from a meeting planner’s January 2022 conference report
I’ve heard reports of this natural but hazardous behavior at many conferences held over the last couple of years. Given the ease of transmission of dominant COVID-19 variants, the best way to minimize such risks is to hold meals and socials outdoors. Obviously, this is not always practical.
Currently, hardly any in-person events report post-event attendee and staff COVID cases. In many cases, there is no apparent effort made to perform post-event case tracking.
Consequently, while we all desire in-person meetings, I think it’s incumbent on every event stakeholder to consider the effect of their event on the health of participants and staff, and determine whether, in good conscience, the meeting should best take place in-person or online.
Looking forward: What the meeting industry still needs to do
Two years after COVID-19 started, we know what to do to keep in-person meetings safe. Currently, it’s still critical that vaccination and masking requirements are in place for events to occur safely. Yet the meeting and hospitality industries still have their heads in the sand in one crucial safety area.
Upgrade air quality in venues and accommodations
As we start thinking about returning to in-person events, it’s crucial to check that venues are upgrading their HVAC systems to handle potentially virus-infused air. This does not appear to be happening! Since I wrote at length about this important safety requirement back in April 2021, I have only heard of one additional venue that is providing COVID-safe ventilation — the Javits Center in New York City. [Heard of others? Let me know, either directly or via comments on this post!]
Let me put this in simple terms. COVID-19 is here to stay for the indefinite future. Would you want to stay in a hotel room with ventilation that includes air from the room next door where a COVID-positive person is sleeping? Do you want to mingle, unmasked, during a meeting social with strangers where the ventilation rate is inadequate to clear the air of COVID-19 aerosols? Even if you’re cavalier about such infection risks, we have a duty of care to attendees and staff.
Right now, updating venue ventilation for COVID-19 is a competitive advantage. Being able to say a property is compliant with current ventilation guidance is a great selling point, as the Javits Center illustrates.
Plan for future COVID-19 variants (and new pandemics)
To date we’ve had several COVID-19 variants play havoc with our in-person meeting plans. We now need to assume that another new dominant variant could appear at any time.
Dominance occurs because a new variant is more transmissible than older ones. A dominant variant may or may not cause more severe disease than other variants.
What this means is that we now need backup plans for switching in-person meetings that can’t be postponed to online formats at relatively short notice. Yes, our work just got even more complicated than it already was. Meetings sure aren’t getting any easier to plan!
Conclusions for a new meeting industry normal
Finally, it should be clear that at this point I’m still cautious about returning to in-person meetings. Millions of people—the elderly, the immunocompromised, and young children who cannot yet be vaccinated—are particularly vulnerable to severe consequences if they catch COVID-19. Some may have to wear masks for the rest of their lives. Premature removal of mask and vaccination mandates at meetings will cause additional, possibly fatal illnesses amongst this population. I hope meeting planners do not rush to relax these important mandates in the mistaken belief that we have reached or are about to return to the old meeting normal.
My concluding paragraph from Part 2 of these posts still applies:
“We are living in unprecedented times. Experimenting with new approaches to designing and convening meetings is essential. What may be even harder is discovering what works and adopting it, rather than staying locked in the old comfortable ways of making meetings. Meetings will continue to occur, and the meeting industry will survive. But don’t passively buy into the myth of a new meeting industry normal. That is if you want to remain a player in one of the most important industries the human race has created.”
“It was rare, I was there, I remember it all too well.”
Listening to Taylor Swift’s lament in her beautiful and evocative “All Too Well: The Short Film” I feel my own grief well up. My last in-person engagement was a wonderful two-day workshop with several hundred cardiologists in Texas. January 28 and 29, 2020. As I’m writing this, that was twenty-two months ago.
Since then, I’ve worked with many groups online. But it’s not the same.
I’m sure you can relate. Yes, it’s wonderful to be instantly connected, with video and sound, to likeminded folks, friends, and family scattered around the country or globe. So much better than the only option in my youth — the telephone. Long-distance phone calls then cost so much that speaking to someone far away or, heaven help us, internationally was a rare treat.
But it’s not the same.
I miss doing what I love to do. Facilitating connection between people around what matters to them. Creating meetings that become what the participants want and need. The magic of the unexpected that appears when you least expect it, and, sometimes, changes peoples’ lives.
Online, we meet using group-focused platforms that don’t have the power, nuance, and flexibility of in-person meetings.
We can’t touch, hug, or connect physically.
Even if an individual’s camera is on, the resolution still isn’t good enough to read their micro expressions of emotion and body language that inform our experience of and connection with them.
We can’t move to different environments online like we can in person: from sharing in a circle to learning about other participants via human spectrograms, from sharing with a neighbor to talking while walking.
The platforms themselves impose additional restrictions. In Zoom, for example:
Spontaneous side conversations are restricted to private chat — if it’s enabled.
A facilitator can’t “feel the room” during small group work, because there’s no way to simultaneously monitor breakout rooms. This important task is far easier to do in person, by simply walking around and noticing what’s going on.
Attendee attention is hard to sense. Are they listening intently, ignoring what’s going on, or browsing TikTok? Even when their camera is on, it’s difficult to tell. And if their camera is off…
Online social platforms can provide an experience much closer to that of an in-person social. Participants can see who’s “in the room” and decide whom to talk with, either one-to-one or small group, in public or private. In the last couple of years, I’ve enjoyed holiday parties with folks who could never have practically got together in person, and these platforms are well worth exploring if you haven’t already.
But it’s not the same as hanging out with and making new friends in person.
And we’re back to the grief. “It was rare, I was there, I remember it all too well.” I see a photo of a meeting I attended with so many friends, and I miss them, and wonder if/when I’ll see them again in-person rather than on a screen.
I feel it. It’s good to remember the past, to feel the pain of its absence now, to be in touch with it, to acknowledge its presence. And then I return to working on being in the present, with my grief a part of me.
The COVID-19 pandemic has devastated the in-person meeting industry. Though it took too long to recognize that COVID-19 spreads via air transmission, we finally have effective procedures (vaccine mandates, masking, air quality standards, and social distancing) to reduce infection risk at in-person meetings. Now, meeting planners can add an affordable air quality tool to their site visits.
How can you determine air quality at a prospective venue?
Look around the room at an in-person event and you’ll see if masking and social distancing are taking place. We can implement vaccination mandates using third party vendors such as sharemy.health, CLEAR Health Pass, Safe Expo, and others. But how can we determine the air quality at a prospective venue?
Currently, we don’t know how to detect airborne COVID-19 viruses. (This is likely to be true for a long time. We still have no test for airborne tuberculosis bacterium (TB) transmission two centuries after identifying TB as a distinct disease.)
Luckily, under conditions I’ll outline below, we can obtain useful information about a venue’s air quality by using a device that measures a proxy for air pollution: carbon dioxide (CO₂).
People breathe in air, typically containing about 0.04% CO₂. They breathe out a mixture of gases containing about 4 – 5 % CO₂. People with COVID-19 co-exhale respiratory aerosols containing the SARS-CoV-2 virus.
If an occupied building space has effective ventilation, the occupants’ excess exhaled CO₂ is quickly diluted with fresh air, and the CO₂ level in the air remains close to normal values. Measuring the level of CO₂ in the air can, therefore, tell us whether effective ventilation is present or not.
Here are the generally accepted standards for CO₂ levels:
~400 parts per million (ppm) – Normal outdoor air level. 400 ~ 1000 ppm – Typical value level indoors with good ventilation. 1,000 ppm – the OSHA/ASHRAE recommended maximum level in a closed room. > 1,200 ppm – Poor air quality – requires ventilation to the room. 2,000 ppm – This level of CO2 typically produces a significant increase in drowsiness, tiredness, headaches, lower levels of concentration, and an increased likelihood of spreading respiratory viruses.
Until recently, meters that measure CO₂ levels in the air cost hundreds of dollars. (Some models with especially accurate sensors or the capability to measure other air pollutants still do.) But today we can buy an affordable air quality tool — a hand-held CO₂ meter for under $100. The one I just purchased (illustrated above) cost $80, and there’s a wide variety to choose from (for example, from here or here).
My 3.27″ (diameter) x 1.26″ (depth) meter measures CO₂ levels from 0 – 5,000 ppm. It can run on standby for 18 hours, supports USB charging, and includes a battery level indicator and temperature and humidity readings. While its specifications omit accuracy, inexpensive CO₂ meters are typically reliable within ±100 ppm. This is good enough to provide a decent estimate of the air quality in an enclosed space.
My unit shows a concentration of ~350 ppm CO₂ outside my rural Vermont home, which was built tightly. In my home office, the level increases to about 450 ppm, and rises to about 525 ppm if I’m sitting next to the unit for a while. Slightly cracking open a window quickly brings down the reading.
I haven’t had time to explore other buildings yet, but am looking forward to seeing what I find out when I do.
Is a CO₂ a proxy for indoor air quality in occupied spaces?
Can measuring CO₂ levels give us a useful indication of indoor air quality?
The answer is a qualified yes. It depends!
First of all, we need to measure CO₂ levels in occupied spaces. A meeting planner doing a site visit should take CO₂ readings in occupied meeting rooms, restaurants, hotel lobbies, etc. Taking measurements in empty spaces will only show high readings if the building ventilation system is grossly inadequate (with CO₂ infiltrating from other areas.) Also bear in mind that increasing the number of occupants in a space increases the likelihood that an infectious person will be present and the number of people possibly infected. Doubling occupancy can thus cause a four-fold increase in risk of transmitting COVID-19.
Finally, air treatment options, such as MERV 13 or better filtering, or possibly ultraviolet-C radiation, may reduce the prevalence of active COVID-19 aerosols. When venues employ these mitigation strategies, CO₂ levels will not be decreased. Of course, if a venue has deployed these preventative measures, they will surely inform you about them when asked!
Due to these factors, you shouldn’t rely solely on measurements of CO₂ levels to determine whether a space is ventilated enough to mitigate transmission risk.
However, a simple CO₂ meter like the one I now own can be an effective air quality tool, providing valuable information to anyone who wants to investigate the air quality of occupied spaces at venues, hotels and properties, restaurants, and other meeting locales. I’ll be bringing mine when I travel, and I encourage you to do this as well!
More information on the relationship between CO₂ levels and COVID-19 exposure
If you’d like to learn more about the relationship between CO₂ levels and COVID-19 exposure risk, here are some useful references:
Sadly, while I acknowledge and appreciate Freeman’s significant work on the case for recommencing meeting in person, I believe this claim is misleading, and the underlying modeling and research include flawed assumptions.
I have two broad concerns about Freeman’s summary of research “Inside LIVE: The data you need to navigate the Delta variant for events” on the safety of in-person events. You can watch Freeman’s 55-minute webinar, posted on August 25, 2021, below.
1—Freeman’s overall conclusion is misleading
My first concern is that Freeman’s big picture conclusion that “in-person events are actually safer than many daily activities, like trips to the grocery store” is a misleading characterization of the statistics they present.
Here are the statistics (from the next webinar slide).
This slide compares the mid-August, 2021 rate of COVID cases amongst the entire population in the United States with the reported rates from four recent large in-person events. The second column shows the infection rate as a percentage.
Because the statement conflates the risk of a masked visit to a grocery store with the overall risk in the United States of getting infected! The latter regrettably includes a significant fraction of the U.S. population who won’t or can’t get vaccinated, don’t wear masks to protect against airborne transmission of COVID-19, and don’t social distance. The risk of contracting COVID-19 during a grocery store masked visit is far less than the overall risk for everyone in the U.S.
The headline statement is, therefore, comparing apples to oranges. You’d expect any event that implements precautions against COVID-19 transmission to have a lower infection rate than the entire United States. That doesn’t mean that attending an event is a safe enough choice for attendees and staff.
This brings us to what’s actually important to people trying to make a decision whether to attend an event. The event modeling, performed for Freeman by Epistemix, and discussed later, suggests that those who are currently likely to attend a large in-person business event that implements mitigation strategies such as vaccination requirements, masking, and social distancing, are significantly more likely to be vaccinated (~80%). That statistic seems credible to me.
Such potential attendees, who are already more careful than the average American about how they live their lives in a pandemic, aren’t interested in whether an event environment provides a risk of getting COVID-19 comparable to the average risk of the entire population of the U.S. Rather, they want to know if attending the event will significantly (defined by them) increase their likelihood of contracting COVID-19. And that brings me to the second concern about the assumptions made by Epistemix’s event risk model.
2—The event risk model used for risk calculations is flawed and incomplete
When I heard about the Freeman webinar (thanks Julius Solaris!), I posted some initial responses. Freeman’s Jessica Fritsche was kind enough to reach out to me and arrange a Zoom call with John Cordier, the CEO of Epistemix, to walk through the data modeling used in the research. And John generously offered an hour of his time for us to talk. We were also joined by Sarah Shewey, Founder/CEO of Happily who was also interested in learning more about how infection rates at meetings could be modeled.
During our hour together, John shared an overview of the Epistemix model. This gave me a better understanding of Epistemix’s approach. The model essentially attempts to simulate the entire population of the United States at an impressive level of detail. It includes numerous geographic and social factors that affect infection risk. However, during our conversation I asked about a number of important factors that I believe Epistemix has not incorporated into their model of calculating meeting risks.
Probably the most important of these is adequately modeling the air quality at the event, given the paucity of information available about the safety of specific venues and properties from an air quality perspective. In addition, the model does not include the additive risks for travel to and from an event, and staying in a hotel during an event. Though it’s likely possible to model the increased risk during (unmasked) eating and drinking social activities during the event, it doesn’t appear that the Epistemix model does this. Finally, though the Epistemix model incorporates information about COVID-19 variants as they become known, I’m skeptical that it can accurately predict in a timely manner the impact of brand new COVID-19 variants.
In the following sections, I’ll expand on these issues in more detail.
Flaws and omissions in Epistemix’s meeting model
First, a tiny introduction to modeling human systems. All models are an approximation of reality. Consequently,
I learned to program computers in high school, over 50 years ago. Through a series of summer jobs, undergraduate and graduate work, and consulting assignments, I’ve spent years creating computer models of city traffic systems, the interactions of high energy particle beams bombarding matter, the consequences of obscure physics theories, and the functions of complicated administrative systems.
Two fundamental considerations when building and trusting computer models are:
The assumptions one makes in building a model are key to the model being actually useful rather than wrong. Computer models are very seductive. They seem precise and authoritative, and it’s hard to discover and accept their limitations and/or even their completely wrong predictions. Choosing the right assumptions is an art, not a science. One poor assumption can doom a model’s reliability.
Even if you choose good assumptions, implementing them correctly in computer code is difficult. It’s hard to be sure that an implementation faithfully reflects core assumptions. An incorrect implementation of a potentially useful model typically leads to incorrect predictions. If you’re lucky, it’s obvious that a model’s outputs are wrong. But sometimes, predictions are subtly wrong in ways that are easy to overlook.
I’m going to assume that Epistemix models faithfully implement the assumptions made to create them (#2 above). However, I’ve identified four factors that I feel Epistemix has not incorporated into their model of calculating meeting risks. Some of these factors are interlinked.
1—Adequately modeling airborne COVID-19 transmission at a specific event
While talking to John, it became clear to me that the current Epistemix approach does not adequately model the air quality—and the consequent risk of COVID-19 transmission—at a specific event. The model has some capacity to estimate risks (which are generally minimal) in very large, high-ceiling spaces like convention halls. But, of course, the typical meeting venue contains multiple meeting spaces, some of them small, and, critically, the venues do not in general have a good handle (if any) on the air quality in those spaces. (Or, if they do, they’re not talking publicly about it.)
I know such upgrades can be costly, but you’d think that venues and properties that have implemented them would love to promote themselves as having air quality that meets current pandemic-based standards.
To date, I have not been told of a single venue that is now compliant with ASHRAE pandemic recommendations. (I hope that by now there are some, and that they will let this be known.) During the webinar, Freeman said that such work has and is being done. Please share this information, folks! Meeting planners want to know!
Frankly, without this information a) being made available and b) being incorporated into the Epistemix model it’s hard to have much confidence in the infection risks Epistemix’s model predicts.
2—Additive risks for travel to and from an event, and staying in a hotel during an event
Epistemix’s model does not include the additive risks for attendees (and staff) traveling to and from an event. The main concern is air travel. The air industry has stressed that air change rates in aircraft are high (over 10 air changes/hour) and, now that masks are mandatory, infection risks should therefore be low. An excellent investigation by the New York Times “How Safe Are You From Covid When You Fly?” has tempered this assessment somewhat. Of particular interest are comments from a couple of readers who monitored the carbon-dioxide level—an excellent proxy for air quality—during their entire travel. They found that boarding and deplaning air quality was drastically reduced, as well as during the last thirty minutes of one person’s flight. Exposure at terminal restaurants, where masks must be removed, is also potentially risky.
Quite apart from the “event” itself, staying in a venue may greatly increase one’s risk of infection. I wrote about venue and property ventilation concerns in detail in April, 2021, and later articles by PCMA (1, 2) and the New York Times (1) have echoed this concern.
Again, travel risks are not included in Epistemix’s model. They can be significant, and have to be included to determine the relative risk for an event attendee who is choosing whether to participate or staff an event, or not.
3—Modeling the increased risk when masks are off for socials and group meals
Most in-person meetings include meals and socials, when masking is not possible. Unless you hold such unmasked get-togethers outdoors or in safely ventilated venues, airborne transmission of COVID-19 amongst everyone present (attendees and staff) is a potentially significant and unknown risk. Outdoor locations are only possible for limited periods in much of the U.S. As mentioned above, venues and properties remain silent on whether they’ve upgraded and certified their facilities to current ASHRAE recommendations on air quality.
We have also seen reports of numerous cases of reduced, unmasked social distancing at socials and meals. This is understandable in a world where we’ve been masked and apart for so long. But it is still a risky activity, especially in spaces where ventilation is inadequate.
My understanding (which may be incorrect) of Epistemix’s model is that masking is a global parameter for an event. The model does not handle unmasking in specific event spaces for periods of time. Even if the model does have this flexibility, the lack of knowledge of whether such spaces are safely ventilated prevents an accurate risk assessment.
4—Can Epistemix model the appearance of brand new COVID-19 variants?
But John and I didn’t have time to fully explore this issue, so this concern may be overblown.
Are in-person events COVID safe?
I really appreciate John Cordier’s willingness to share an overview of Epistemix’s infection risk model for events. Obviously, my brief introduction means there’s no way I can authoritatively review the extensive assumptions that are built into the model. Epistemix’s model is impressively detailed and, if correctly implemented (which I have no reason to doubt), seems to comprehensively cover core demographics, the data needed to model infection spread in regional populations, and most major components for predicting infection at a specific event.
When I brought up the concerns I’ve listed in this post, I felt that John largely talked past me, continuing with an explanation of the model without responding directly to what I was asking. This was somewhat frustrating. The two exceptions to this were:
My question about whether the model could accurately predict in a timely manner the impact of brand new variants. This arose at the end of our meeting. John indicated that he believed the model was able to do this, but we didn’t have enough time to explore this issue fully. I’m still skeptical, though he might well have been able to convince me otherwise if we’d had more time.
My primary concern about modeling air quality in detail. John admitted during the meeting that the model does not currently handle specific venue air quality architecture at the detail that’s necessary to simulate, say, what happens when you have a session in a smaller classroom with an HVAC system that is not up to current ASHRAE recommendations. It also omits risks due to event participants (and staff) spending time in properties that may have inadequate air quality. He wrote to me afterwards that “he’d be glad to follow up on the air-quality parameters that you think are most important”.
I’ve seen so many pretty models of systems over the decades. To a casual viewer, they look impressive. It’s only because I spent years building and validating such models that I know how misleading they can be, and the difficulty and importance of identifying the key factors and approximations that form the basis of the model and limit its scope and/or accuracy.
Leaving out detailed venue specific air quality modeling, plus the incoming and return travel risks and accommodation risks during an event, plus inadequate modeling of the risk of transmission during socials and food & beverage sessions make the outputs of the Epistemix model suspect. And I’m skeptical that Epistemix can build new variants into the model predictions in a timely fashion.
Finally, I haven’t covered in this article the feasibility of implementing the various mitigation strategies that are available to reduce the risk of COVID-19 infection at meetings. Personally, I’d insist on proof of vaccination (no exceptions) and maximal masking at any event I’m likely to attend in the near future. But I’ll just add here this observation from the Healthcare Information and Management Systems Society‘s HIMSS21 Las Vegas conference for 19,000 attendees. Vaccination was mandatory for all attendees. There were six positive test results (0.03% infarction rate). However, this PCMA article on the event includes the statement: “…you will not be able to service your show if you require every single vendor employee, every single supplier employee, every single temp employee to be vaccinated — there’s just not enough labor out there.” Something to bear in mind.
Are in-person events COVID safe? Ultimately, each of us needs to decide the answer to this question. But, in my opinion, until the COVID-19 case count drops drastically and/or venues can show that their facility ventilation is safe, it’s a violation of our professional duty of care to mislead attendees and those who work in our industry by telling them “in-person events are actually safer than many daily activities, like trips to the grocery store”.