When Seeing is Hearing

Isolation isn’t conducive to writing about communication difficulties. Since communication difficulties are what this blog is about, I haven’t written much in recent months.

I’m getting along just fine with no one to talk to. I’m hearing well. Or at least I feel like I’m hearing well. But that’s because everything I hear – the television, the telephone, Zoom calls – is captioned. When I can see the words, I can also hear them.

That seeing enhances hearing is a well known phenomenon, which I’ve written about in the past. Researchers call it the McGurk effect, named after one of the British scientists who discovered in the 1970s that people comprehend speech better if they also see it. They called it “hearing lips and seeing voices.”

This is why good communication strategies are important. It’s why we need to make sure we can see a speaker in order to hear them. We all intuitively speech read. The speaker’s facial expressions contribute to our comprehension.  How the words are formed in the mouth and on the lips is also important, which is why we used to call it lipreading. Now we know that lipreading is augmented by facial expression and body language, and we call it speechreading.

I’m a good speechreader, but only if I can also hear what’s being said. If I hit the trifecta — hear the speaker, see the speaker, AND get captions — I hear perfectly! Fortunately for me, that’s often the case with virtual meetings and conversations.

There are times when captions fail me, however, especially with live TV. I like to watch the network news to catch up on the day. One network, my favorite, has terrible captioning. The captions routinely start stuttering, the same few words repeated over and over again while the speakers cluelessly move on. Eventually I give up and change the channel. Readers if you also have this experience, is it worse on a particular network? I’m reluctant to slam mine, but feel free chime in.

This network is presumably using ASR — Automatic Speech Recognition. Unfortunately the network’s system can’t even recognize the names of the network’s star correspondents. Far superior is CART captioning — Communication Access Real Time Translation. It would seem well worth the small investment in a good CART captioner. Networks, listen up! increase your viewership. The same problem exists, by the way, with live sports captioning.

Maybe I AM hearing better. Without the anxiety and stress of trying to hear and understand all day — trying to communicate — I’m more relaxed. These past few months, as bad as they’ve been, have for me included one benefit, a big benefit. I can hear.


Here is a link for filing a complaint to the FCC. You will not be able to complain about a recurring problem, just one specific station/show on one specific date. So if you are trying to watch NBC Nightly News, for instance, you will have to choose one specific date and time to complain about.

14 thoughts on “When Seeing is Hearing

  1. Hi Katherine,
    So true! NBC and NBC, our go-to news channels, are seriously delinquent in captioning. Often it’s two sentences late. How can we pressure the powers that be to fix this?


  2. thank you, Katharine . . . and yes to the importance of captioning (especially when placed in the lower half of the screen so as not to cover a speaker’s eyes, a basketball hoop, etc.) And yes! to the McGurk effect . . . which partly explains the challenges of hearing in a masked world. For my 2 cents on that see https://community.macmillanlearning.com/t5/talk-psych-blog/hearing-in-a-face-masked-world/ba-p/12399

    With continued appreciation for your writing,
    Dave Myers


  3. I agree, Katherine. Captions are a godsend!
    Since my Cochlear Implant I am no longer totally reliant on them (blessedly) but I surely WAS.
    And I was not alone.

    As I raptly listened to President Biden’s Inaugural Speech on 1/20/21, I noticed in horror that the words of the speech shifted for a while to news about Trump leaving for Florida that AM. I was appalled.
    What if those written words were all I had!

    Yes, captioning needs to get better.
    But I am still grateful for it and wish ZOOM and all virtual conferencing had free auto-generated captions
    as Google Meet does. Thank heaven, too, for apps like Otter and Ava and Innocaption.

    We just need to keep speaking up and letting people know what we need.
    Thanks for the reminder 🙂


  4. I use captions, too, but I had a cochlear implant operation this past November 19, as my sister did two years ago. I’m still waiting for the upgraded AB equipment, which is supposed to be shipped now in March. My surgeon, Dr. Thomas Roland, named you as someone for whom the implant worked. It wasn’t working very well when you and I met not long after you got yours. Has it improved? Does it just take time? I continue to be told that everyone is different, but what you’re describing now in your post about reading what is said is what I’ve been doing. But there’s no going back since the residual hearing in the ear with the implant no longer exists. And though I got the “loaner” processor on December 3, I’m still getting feedback from the implant processor. I see my Langone audiologist tomorrow in the city, but I’m getting discouraged. My sister used to hear far worse than I did. Now she hears better. I read Debra Cerruti’s response to what you wrote, and she’s someone for whom captions no longer matter as much, as is the case for my sister. Still, as Debra notes, everything needs to be captioned. I love the theater, and I even write plays, but though none of us can go to live theater now, we will again, and I’d love to be able to hear what the actors are saying. I know you, too, are a writer, so you know words matter. Any advice would be very welcome!


    • I love my cochlear implant. It took me a long time to learn to hear well with it, because I was still working full time and couldn’t afford not to wear my hearing aid in the other ear. But over the years I have done auditory rehab both formally and informally (online programs, listening to recorded books) and my word recognition is much improved.
      I also feel very unbalanced without my cochlear implant. My hearing aid ear is still the better one, because at the time I was implanted (2009) I had been profoundly deaf in the implant ear for at least two decades. That’s a long time to wait. The FDA changed its regulations in 2007 or 2008, so that even though I heard well enough with the hearing aid ear I was then eligible for a cochlear implant.
      I too am waiting for the Marvel. I’m way overdue for an upgrade, at least two years I think.
      My primary complaint about the implant is that it doesn’t fit well. I have small ears and I also wear glasses, so there’s not a lot of room behind my ear for the earpiece. It frequently gets dislodged when I’m taking off a hat or scarf, or even if I just turn my head too quickly. The Marvel is a thinner processor I think, and I’m eager to try it.
      Keep at it. It’s worth it.


      • Thanks, Katharine! That’s very encouraging! I’ll keep at it. And I’m glad to hear the processor is thinner. That will help me, too. I’m practicing, but I know it’s only been a short time. I love hearing that this may all end up being worthwhile!


Please leave a reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s