Google’s Audio Overview is an AI love-bath and a propagandist’s dream
Why are people raving about the conversations produced by Google NotebookLM‘s Audio Overview? Because Google’s Audio Overview is an AI love-bath!
I can’t take credit for characterizing Audio Overview’s appeal like this. Kudos to Johnnie Moore, who responded to my first post on Audio Overview with this insight:
“And you’ve really nailed the trouble with the relentless enthusiasm. When I tried Notebook the first time on my own stuff I loved it. Cos, I now realise, I was flattered by the AI love-bath. But now I see it says this to all the people it’s dating!”
—Johnnie Moore
I’ll return to the dangers of this seductive tool later in this post. But first, let me share what happened when I fed content containing inconsistencies and incorrect information into Audio Overview.
Testing Audio Interview with inaccurate content
In last week’s post, I showed how Google’s Audio Overview tool in NotebookLM generates bogus output from accurate content. This week, I tested its performance when fed inaccurate content.
I asked ChatGPT to write a short article on the solar system, deliberately including a few inconsistencies and errors. Below is the article, along with ChatGPT’s list of factual errors and inconsistencies.
ChatGPT_inaccurate_article_about_the_solar_system
Now here’s the audio “conversation” that Audio Overview generated from the article, along with a transcript:
Google NotebookLM Solar System conversation
Man: All right. Get ready for liftoff. We’re diving deep into the solar system today, and you’ve given us a stellar lineup of readings on each planet, from the Sun to Mars. We’re gonna ditch the boring textbook stuff and blast off into the really mind blowing parts.
Woman: Exactly. Think cosmic secrets, the kind of stuff that’ll have you winning all the trivia nights and impressing your friends.
Man: You know how everyone thinks they’ve got the solar system basics down?
Woman: Yeah.
Man: Sun in the middle, planets go round and round.
Woman: Sure, sounds simple enough.
Man: Well, what if I told you even those basics are about to get seriously shaken up?
Woman: It’s true. Sometimes all it takes is a fresh perspective to completely change how we see even the most familiar things. Like, take our sun, for example.
Man: Big ball of fire. Keeps us warm. Pretty important, right?
Woman: Absolutely. But we tend to picture it as this fixed point in the center of everything. But the reality, it’s way more dynamic than that.
Man: Okay, you’ve got my attention. How is the sun not just some stationary star?
Woman: Well, picture this. The Sun, with all its planets in tow, is actually hurtling through space on this incredible cosmic journey.
Man: Hold on. The Sun’s on the move. It’s not just chilling in the center of the solar system. Where are we headed, and do we need to pack snacks for this cosmic road trip?
Woman: Well, imagine a giant spiral, a cosmic waltz, if you will, where our Sun and its planetary entourage are gracefully swirling around the center of our Milky Way galaxy. That’s our trajectory.
Man: Okay, I’m kind of trying to picture this cosmic waltz, but it’s definitely not your average ballroom.
Woman: You’re not wrong there. And when it comes to measuring this journey, we’re not talking miles, we’re talking galactic years.
Man: Galactic years. Okay, now you’re speaking a language that’s out of this world.
Woman: One galactic year. The time it takes for our solar system to make one loop around the galactic center is roughly 230 million Earth years.
Man: 230 million years. That makes driving across the country on a family vacation seem like, well, nothing. It really puts things in perspective, doesn’t it?
Woman: It really does. It’s a reminder that we’re part of something so much bigger than ourselves.
Man: So, back to the sun for a second. One thing that really surprised me was that it doesn’t just give off energy. It actually absorbs some, too.
Woman: It’s true. While the sun’s main energy source is nuclear fusion happening way down in its core, it does have this interesting habit of, well, snacking on space dust.
Man: Space dust? Like the stuff that collects on my furniture?
Woman: Kind of, but on a cosmic scale. And it’s not just some quirky cosmic habit. It actually plays a big role in those incredible solar flares we see.
Man: Wait. Those solar flares that can mess up our satellites and even cause blackouts here on earth, you’re telling me that space dust is involved.
Woman: You’ve hit the nail on the head. When the sun absorbs energy from space dust, it can trigger these massive bursts of radiation.
Man: So the sun is building up energy from these space dust snacks, and then, boom, solar flare.
Woman: That’s a good way to put it. It’s like the sun releasing all that pent up energy, and sometimes we get caught in the crossfire.
Man: Okay, note to self, the sun is a force to be reckoned with, even if it enjoys a little cosmic snacking. It’s on the move, it’s absorbing energy, and it can definitely pack a punch.
Woman: It’s anything but boring.
Man: Alright, with that in mind, let’s shift gears and zoom in on the planet closest to this mighty star, Mercury. Most people probably think it’s this tiny, insignificant little planet.
Woman: Right, easy to overlook.
Man: But the research you sent paints a completely different picture.
Woman: It’s time to ditch that outdated image of Mercury as a dull, gray rock. This planet is full of surprises. For one thing, it’s much larger than people think. It’s almost the size of Earth.
Man: Really? Wow, I had no idea it was that big. But size isn’t everything, right? What else makes Mercury so interesting?
Woman: Well, let’s just say it’s not the fastest spinner in the solar system. In fact, it’s an incredibly slow one.
Man: Okay, how slow are we talking? Like, if we’re on Mercury, how long are we waiting for the sunrise?
Woman: Well, pack a lunch and maybe a book, because a single day on Mercury, meaning one full rotation on its axis, takes a mind boggling 59 Earth days.
Man: 59 Earth days.
Woman: And since Mercury’s orbit is oval shaped, not a perfect circle, a full day night cycle from sunrise to sunrise actually lasts a whopping 176 Earth days.
Man: Wait, so that means you’d have to wait three Earth months to see the sun peak over the horizon.
Woman: You got it.
Man: I think I’d be bored to death by then. Speaking of extreme conditions, you know what else blew my mind? The fact that there might be ice on Mercury. Ice. On a planet that’s practically touching the sun? How is that even possible?
Woman: It does seem counterintuitive, right? But remember that incredibly slow rotation in those long, dark nights. Well, believe it or not, those are the perfect conditions for ice to survive on Mercury, tucked away in those deep, dark craters at its poles.
Man: So it’s like hiding from the sun in plain sight.
Woman: You could say that. Those craters are so deep the sun’s rays never reach them, leaving those spots in permanent shadow.
Man: So we’ve got this paradoxical planet. It’s surprisingly large, incredibly slow, scorching hot during the day, but with hidden pockets of ice. Who knew? This solar system is full of surprises. Alright, next up, Venus, Earth’s so called twin, but I have a feeling those similarities are only skin deep.
Woman: Venus, huh? It’s always been this bright, beautiful thing in the sky. We always hear it called Earth’s twin.
Man: Right. Easy to assume they’re practically identical.
Woman: But after going through this research, you said, I’m starting to think this twin has a bit of a wild side.
Man: You’re definitely on to something. Venus might share some surface level similarities with Earth, like its size and what it’s made of, but that’s about where the family resemblance ends. This planet is like the rebellious sibling, always going against the grain.
Woman: Rebellious how? Give me the dirt! For one thing, Venus spins in the opposite direction of most planets in our solar system. It’s called retrograde rotation. Imagine, instead of the sun rising in the east and setting in the west, it’s the opposite on Venus.
Man: Wait, seriously, the sun rises in the west on Venus. That’s wild. Any idea why it spins backwards like that?
Woman: It’s a bit of a mystery. The leading theory is that way back when, early in its life, Venus got slammed by something huge, like a massive collision that sent it spinning off in the opposite direction.
Man: A cosmic fender bender, huh? Makes you wonder what else Venus has up its sleeve.
Woman: Well, its atmosphere is nothing like ours. It’s incredibly thick, full of carbon dioxide, and there are these clouds of sulfuric acid just hanging out up there.
Man: Okay, sulfuric acid clouds. Definitely not on my travel bucket list. So much for Earth’s twin, more like a weird distant cousin we don’t talk about much. But speaking of this crazy atmosphere, I read about these super bright flashes of light that have been observed on Venus. What’s the deal with those? Alien disco parties.
Woman: I wish I could say it was aliens, but the reality is probably a bit less exciting, though just as intense. Scientists believe those flashes are caused by these megastorms with lightning way more powerful than anything we see here on Earth. Venus’s thick atmosphere traps heat, creating this runaway greenhouse effect.
Man: So picture this. We’re on Venus, braving the heat and the sulfuric acid rain, the sun setting in the east, and then bam, these megastorms erupt, lighting up the entire sky.
Woman: You’re getting a pretty accurate picture, but it doesn’t stop there. There’s one more thing about Venus that really sets it apart. It’s magnetic field. Now, we know Earth has a magnetic field. It’s like our protective shield. It’s pretty stable.
Man: Yeah, it keeps us safe from all that nasty radiation from space. Essential stuff.
Woman: Exactly. But Venus, its magnetic field is a whole different story. It’s all over the place, shifting dramatically every few years.
Man: Every few years, that’s nuts. I can’t even imagine what would happen here if our magnetic poles went haywire that often. Our compasses would be spinning like tops.
Woman: Right, it would be total chaos. And on Venus, these magnetic field shifts likely contribute to its extreme and unpredictable weather. Like, Venus can’t catch a break, even with its own magnetic field.
Man: Okay, I think it’s time we retire the whole Earth’s twin nickname. Venus is in a league of its own, and frankly, I’m not sure I’d want to take it to that party. Speaking of strange planetary companions, can we talk about something that really threw me for a loop? Earth’s second moon. I always thought we were a one moon planet.
Woman: It’s not exactly a moon in the traditional sense. Think of it more like Earth has a cosmic dance partner.
Man: Okay, I’m intrigued. Tell me more about this dance partner.
Woman: This object, called Cruithne, isn’t orbiting Earth directly. Instead, it’s locked into this gravitational dance with both Earth and the Sun. It traces out this horseshoe shaped path around our star.
Man: So Earth and Cruithne are in this cosmic waltz around the sun, never quite colliding, but always kind of connected.
Woman: Exactly. And while it’s fun to think about Earth having a secret companion, Cruithne is technically classified as a quasi satellite, not a real moon.
Man: Quasi satellite. That sounds even cooler, if I’m being honest. But should we be worried about Cruithne? Is it going to crash our cosmic party anytime soon?
Woman: Not to worry. Kruthne’s orbit is very stable. It’s not a threat to Earth. More like a curious neighbor, always nearby, but keeping a safe distance. Speaking of neighbors, our next stop is a planet that’s been in the news a lot lately. Mars.
Man: Mars, the red planet, right? Yeah. I mean, it’s always been this red, dusty world in our imaginations.
Woman: Yeah, rusty red deserts as far as the eye can see.
Man: Exactly. But these articles you said, they mentioned something about blue areas on Mars. Is that some kind of Martian mirage?
Woman: Not a mirage at all. We’re so used to thinking of Mars as this red planet, and most of it is thanks to all that iron oxide. But it turns out the Martian color palette is more diverse than we thought. Recent observations have actually revealed patches of blue, especially in certain areas around its equator.
Man: Blue on Mars, huh? Ha ha. Alright, Color Me Intrigued, what’s causing these blue zones to appear?
Woman: It’s pretty cool actually. These blue regions are thought to be caused by concentrations of different minerals, not just iron oxide. Picture ancient volcanic eruptions spewing minerals up from deep inside Mars. These eruptions could have literally painted these blue streaks across the surface.
Man: So we’re talking about a planet that’s not just a flat red desert. It’s got this hidden layer of geological activity and even a dash of color. And speaking of hidden wonders, this next one really blew me away. Mars has auroras, just like Earth, but on a way grander scale. Apparently these Martian light shows can stretch across the whole planet. Talk about a sight to see. What makes these auroras so different from what we see here on Earth?
Woman: You’re right. The auroras on Mars are in a league of their own. See, on Earth we have our magnetic field, right? It acts like a shield, funneling those charged particles from the sun towards our poles. That’s how we get those beautiful but localized auroras. Mars, on the other hand, well, its magnetic field is weaker and, let’s just say, a lot more erratic.
Man: Okay, so instead of a shield, it’s more like a sieve.
Woman: Exactly. Those charged particles aren’t being channeled to the poles on Mars. They’re free to roam across the entire planet.
Man: Wow, so they collide with the Martian atmosphere all over the place.
Woman: And create these incredible, planet wide auroral displays. Imagine looking up at night and seeing the entire sky lit up with those shimmering lights.
Man: Now, that’s an Instagram post. I’d love to see. But okay, let’s talk about the discovery that really has everyone talking. That underground water network on Mars. We’re not just talking about a little bit of ice here and there anymore, right?
Woman: You’re right. This is huge. For years, we’ve been finding hints of water on Mars, but now we have solid evidence that there’s liquid water beneath the surface forming this vast interconnected network.
Man: So like a hidden Martian oasis.
Woman: You could say that. And this discovery has really profound implications, especially when it comes to the search for life beyond Earth.
Man: Because where there’s water, there could be.
Woman: Life. Even if it’s just microbial life, finding it on another planet would be revolutionary. It would totally change how we understand our place in the universe. It would mean that life, in some form, might be way more common, much more resilient than we ever imagined.
Man: It’s mind blowing to think that something as seemingly simple as water could unlock one of the biggest mysteries of our existence. Are we alone? It makes you wonder what other secrets are out there, just waiting to be discovered.
Woman: That’s the thing about space exploration, isn’t it? Every time we answer a question, we uncover a dozen more. It’s this incredible journey of discovery that never really ends.
Man: Well said. From the sun’s cosmic dance to those hidden depths of Mars, we’ve journeyed through some truly mind blowing corners of our cosmic neighborhood. It’s a good reminder that there’s always more to learn, more to explore, just beyond the horizon. Keep looking up, everyone. You never know what you might find.
Evaluating Audio Overview’s performance
So, how did Audio Overview respond to the errors in the article? According to ChatGPT, the original article contained thirteen factual errors. Here’s how Audio Overview handled them:
- Errors Omitted: Three factual errors (23%): the Sun’s age, sunspot-related gravitational effects, and Mercury’s temperature were not mentioned in the audio conversation. It’s unclear whether the system identified and omitted these errors or simply failed to include that content.
- Errors Omitted: Three errors (23%)—concerning Venus’s atmosphere composition, the description of Cruithne, and Mars’s mineral colorations—were avoided, possibly suggesting some filtering or avoidance of incorrect data.
- Errors Repeated: The remaining seven errors (54%) were repeated, including the Sun dragging planets in a spiral, exaggerations about Mercury’s size, Venus’s magnetic field shifts, blue regions on Mars, highly visible Martian auroras, space dust triggering solar flares, and exaggerations about Mars’s underground water network.
To summarize, Audio Overview’s conversation left out 23% of the factual errors in the content I fed it, avoided 23% of the factual errors, but repeated the majority (54%) of the factual errors, creating a plausible “conversation” that reinforced them.
The Real Danger of Audio Overview
The real danger of Audio Overview’s “AI love-bath” lies in its ability to produce convincing, natural-sounding, audio conversations between two computer-generated voices. The seductive quality of these conversations is magnified by what Johnnie Moore rightly termed “relentless enthusiasm.”
In today’s digital world, people are increasingly skeptical of what they read. However, an audio recording featuring two people eagerly summarizing content feels far more trustworthy—even when it’s completely AI-generated and contains errors.
Rather than celebrating how authentic Audio Overview’s conversations sound, we should be wary of how easily such tools can become a propagandist’s dream.
Here we go again—another generative AI service churning out plausible-sounding, but misleading content. This time it’s Google’s 
Get rid of Human Resources. Human Resources: what an ugly term!