Blind Gaming

JAMES PARKINSON: Video games are a rich and detailed entertainment medium, with both visual and audio components. I know I probably take my sight and my hearing for granted most of the time, but for people who are visually impaired, playing games is often very difficult.

The good news is that the games industry is changing. Developers and console manufacturers are prioritising accessibility, making their experiences more inclusive and more enjoyable for everyone.

Hey, I’m James Parkinson. From Lawson Media, you’re listening to Gameplay, stories about video games and the virtual worlds that power culture and community. And today on the show, a story from the fantastic podcast Twenty Thousand Hertz. Now, I’ll hand you over to its host, Dallas Taylor.

[music in]

Steve: I realized, my entire life, that I'd been telling myself I sucked at video games, but in reality, it was that video games sucked for me.

Dallas Taylor (narrator): That’s Steve Saylor. He’s loved video games since he was little, but playing them wasn’t always easy.

Steve: I would mostly watch my brother play video games [sfx: Mario NES]… sure, it would've been great to play those games, but I just felt, at the time, that I sucked at them.

Steve: Because I just wasn't as good at it as my brother or even my mom was. Actually, my mom beat Mario before my brother and I did.

Dallas Taylor (narrator): But it turned out that there was a very specific reason why Steve was struggling.

Steve: I have a condition that's actually called nystagmus. It's an onset from albinism. Essentially, the condition that I have is that my eyes move back and forth without me being able to control them.

Dallas Taylor (narrator): This causes Steve’s vision to be extremely blurry.

Steve: My blindness never felt like it was, to me at the time, it didn't clue me in that that was the reason. I just felt like, "Well, I can't really see the screen that well, so that's probably why I suck." It wasn't until much later that my mentality around that had changed.

[music out:]

Dallas Taylor (narrator): For people with visual impairments, gaming can sometimes be a struggle. Steve is a big advocate for accessibility in gaming, and he hosts a Youtube channel called Blind Gamer with Steve Saylor.

[sfx clip: Hi, I’m Steve Saylor. I’m blind, and I play video games.]

Dallas Taylor (narrator): In order for Steve to see the screen, he has to sit really close.

Steve: I have a 50-inch TV that's about a foot and a half away from me. And even then, there's definitely some things that I'm just not able to do in video games, whether that's fast action [sfx] or reading small text [sfx], that kind of thing.

Dallas Taylor (narrator): About seven and a half million Americans have some form of visual impairment, but only about 10% of them have no vision whatsoever. This leads to a lot of confusion around the definition of the word blind.

Steve: Nine out of 10 people who say they are blind, do have some vision.

Dallas Taylor (narrator): Blindness is more like a spectrum, with some vision at one end, and zero vision at the other. Twenty twenty vision is considered average. This means that you can see the normal amount of detail in an object that’s 20 feet away.

Steve: For me, with my glasses on, it's more like 20/200. Something that's 20 feet away looks like it's 200 feet away. And with my glasses off, it's more in like the 20/1700 range.

[music in]

Dallas Taylor (narrator): In general, most video games are very visually-focused. So for anyone with a visual impairment, there are a few things that really help make a game more accessible.

Steve: Yeah, generally what we would like to be able to see, at least from a blind player's perspective, is large text or the ability to be able to control the size of the text within either the menus or the in-game HUD, or heads up display.

Dallas Taylor (narrator): In a game, the HUD is the information you have on screen while you play. It might tell you how much health or ammo you have, or what items you have equipped. Some games allow you to adjust the size of that information, but many of them don’t.

Steve: Also, the ability to separate the important information on the screen as compared to the environment, because there's times where enemies or your own allies are blended in with the game environment themselves. So, it's hard to distinguish which is which. So, having a bit of a high-contrast mode where I could be able to see, “Okay, this is an enemy, this is my ally, or this is a tree,” being able to distinguish between those would be beneficial.

[music out]

Dallas Taylor (narrator): Another thing that’s really helpful is when a game supplements the visual information with audio. For instance, when characters are talking, there might be text captions and recorded dialogue. Here’s an example from Dark Souls.

[sfx clip: Dark Souls: Have a seat and get comfortable. We’ll both be Hollow before you know it. Hahaha.]

Dallas Taylor (narrator): In the game Bastion, there’s a narrator who often comments on what’s happening on screen.

[sfx clip: Bastion: Kid takes a last look around, ‘cause he ain’t coming back.]

Dallas Taylor (narrator): Games can also use audio to give players direction about what they’re supposed to do next. This could be as straightforward as Slippy telling you which button to push in Starfox 64.

[sfx clip: Slippy: Hold the A button to charge your laser.]

Dallas Taylor (narrator): Or it could be more subtle, like the tone that plays in The Legend of Zelda: Link’s Awakening when you’re close to a hidden key.

[sfx clip: Link’s Awakening sfx]

Dallas Taylor (narrator): In most video games, sound adds extra information on top of what you see on screen. But in some cases, sound makes up the entirety of the game.

Connor: So audio games are games which primarily don't have any visuals.

Dallas Taylor (narrator): That’s Connor Scott-Gardner. Connor is a disability rights activist, and a big fan of audio games.

Connor: So some audio games will have, usually fairly rudimentary visuals, but you don't need to see at all to play them. So all the in-world stuff is constructed through sound.

[music in]

Dallas Taylor (narrator): Unlike Steve, Connor isn’t able to play through a game by sitting close to the screen.

Connor: I'm almost totally blind. So I have a condition called Lebers congenital amaurosis, and I've been blind since birth. So for gaming, I really need audio cues and everything to be able to be accessed completely non-visually.

Dallas Taylor (narrator): When Connor was young, he had no idea that audio games even existed.

Connor: I remember being just totally fascinated because my sister who's sighted used to play The Sims all the time [sfx: Simlish sounds].

Connor: So I would follow her around and pester her, to read to me, and to help me play it, because I couldn't play it at all.

Dallas Taylor (narrator): When he tried his first audio game, it was a complete revelation.

Connor: It was totally crazy the first time I was ever able to play a game by myself. I remember just feeling, “Wow this is unreal, how this feels.”

[music out]

Dallas Taylor (narrator): Most sighted gamers probably aren’t familiar with audio games. But the roots of these games actually go all the way back to the early 1980s. At the time, personal computers were becoming more and more common. Many of these early PCs came with text-to-speech engines. For instance, Apple had one called MacinTalk. Here’s Steve Jobs showing off the feature in 1984.

[sfx clip: Steve Jobs: For the first time ever, I’d like to let Macintosh speak for itself.

MacInTalk: Hello, I am Macintosh. It sure is great to get out of that bag [cheers]… ]

[music in]

Dallas Taylor (narrator): One type of game that was popular on these early computers was called a “Text Adventure” game, where you move through the story one paragraph at a time, and make choices along the way. It was basically like a Choose Your Own Adventure book in video game form.

Dallas Taylor (narrator): Early on, these games didn’t include any audio. But over the years, people started to combine them with text-to-speech software so the stories would be read out loud. Here’s an early 80s game called Zork 2, using a more modern text to speech engine.

[sfx clip: Zork 2: A huge red dragon is lying here, blocking the entrance to a tunnel leading north. Your sword has begun to glow very brightly.

Attack dragon with sword.

Dragon hide is tough as steel, but you have succeeded in annoying him a bit. He looks at you as if deciding whether or not to eat you.]

[music out]

Dallas Taylor (narrator): Since the text adventure days, audio games have gotten a lot more advanced. These days, they also include music, sound effects, and dialogue. Many of them also integrate with the text to speech engines that most blind gamers already have set up on their computers. And even though they’re audio only, there’s still a huge variety.

Dallas Taylor (narrator): There are first person shooters like Shades of Doom, which is inspired by the Doom franchise [sfx: Shades of Doom].

Dallas Taylor (narrator): There are also racing games like Top Speed [sfx: Top Speed].

Dallas Taylor (narrator): And role-playing games like Paladin of the Sky [sfx: Paladin of the Sky].

Dallas Taylor (narrator): Since audio games typically don’t have any graphics, building a world from sound is crucial.

Connor: Sounds are used in various different ways. So one way is for example, a character's footsteps [sfx]. So depending on what surface your character is walking on, there may be different sounds. And this is very popular in audio games to have different sounds for like gravel [sfx] or dirt [sfx], and then a very specific sound for a road [sfx]. So your character would know when it's hit a road that it maybe wants to follow.

Connor: Same, for things like water [sfx], your character will either fall in [sfx] or be walking like in a stream or something. Sounds are also used very much panning from left to right [sfx]. So if you need to aim to kill something in the game [sfx] you'll want that sound to move as your character moves so that you then can orient your character in relation to whatever it is you're shooting at.

Dallas Taylor (narrator): Connor even created an original audio game called Beyond the Ice. To make the game, Connor had to make sure every detail was explained through audio.

[music in]

Connor: So I am not a developer. My coding skills are limited. So I built it with an engine called SABLE, which allows you to build audio games without needing to code. But it was really fun. I think for me the best thing was creating the story because I'm a writer, story is something I'm really passionate about anyway.

Connor: And so for me, it was, “How do I tell this story actually through a really immersive experience?” And then I had to think about, “Okay, what sounds right? How do I make this interesting?” And I suddenly realized the overwhelming amount of detail that had to go into it.

Dallas Taylor (narrator): Here’s Connor demonstrating the game on Youtube. You’ll hear audio from the game itself, as well as Connor’s screen reader that’s being fed text by the game.

[music out]

[sfx clip: Beyond the Ice]

[music in]

Dallas Taylor (narrator): Audio games are very accessible, because they’re usually designed specifically for visually-impaired gamers. But in recent years, the accessibility of mainstream games has ramped up a lot, and these games have started to include some really cool options. Xbox even went so far as to recreate the audio experience of Steve Saylor’s dreams… And I mean literally, from his actual dreams.

Dallas Taylor (narrator): That’s coming up, after the break.

[music out]

[AD BREAK]

JAMES PARKINSON: Welcome back to Gameplay, and we return to our story on blind gaming from Twenty Thousand Hertz. Here’s Dallas Taylor.

[music in]

Dallas Taylor (narrator): Whether it’s a giant game company or a tiny indie studio, game developers put a lot of thought into the sound of their games. Not only does audio help tell the story, it also gives players crucial information about what’s happening on screen. In audio games, there’s no visual information at all, meaning that developers have to be even more precise with their sound design.

[music out]

Dallas Taylor (narrator): One of Connor’s favorite audio games is a first-person zombie shooter.

[sfx clip: Swamp]

Connor: There’s a game called Swamp and it is multiplayer, so you play online. And so that's cool because obviously you can talk to your friends on the game, and mission with them and stuff. It’s quite fun, yeah.

[Swamp FX out]

Dallas Taylor (narrator): For Connor, a big part of what makes Swamp so great comes down to its developer, Jeremy Kaldobsky.

Connor: The game developer has spent a lot of time getting to know the blind community. He is a sighted person, but he's really engaged with the community and put a lot of his own time and resources towards it. And I think he takes community feedback very seriously, which is something that perhaps sets the game apart.

[music in]

Dallas Taylor (narrator): In the past, it’s been pretty rare for mainstream game makers to put that kind of effort into the accessibility of their games. But that’s been changing. Steve Saylor does a lot of work consulting with game companies about how to make their games more accessible. He’s worked on games like Watch Dogs Legion, Assassin's Creed Valhalla, and The Last of Us 2.

Steve: I use The Last of Us as a good example. They actually have an option in there that's called audio cues where whenever coming to a specific object that you could interact with, there is an audio sound effect that plays, that tells you which button to push, but also what object you're interacting with.

Dallas Taylor (narrator): In The Last of Us 2, there are dedicated sounds for all of the items you use. There’s a sound for the rope [rope], a sound for the arrows [arrow], a sound for the bottle [bottle], and the list goes on.

Dallas Taylor (narrator): If you turn these settings on, and then walk up to an object you can interact with, you’ll hear two sounds. The first is the generic “Object Interact” sound. [Object Interact] The other sound tells you which item to use. [Rope] So if you hear this, [Object Interact + Rope], you’ll know to press the button for your rope.

Steve: You can do that pretty much throughout the entire game, regardless of what you're interacting with [sfx].

Dallas Taylor (narrator): There are also options for navigational assistance.

Steve: You can have this listen mode, where, if you push a button in that mode, it does this audio ping [sfx] in the environment you're in to tell you which direction there are items or enemies in the room that you're in.

Steve: At least you can know, “Okay, there's an enemy off to my left, [sfx] but there is also a drawer I can open to my right” [sfx].

Dallas Taylor (narrator): There’s even a high contrast-mode. When it’s enabled, it turns your character into a bright color of your choosing, and it highlights key objects and important parts of the environment.

Steve: It would take me hours to be able to go through all 60 accessibility options that are in The Last of Us, and it is something that, across the board, is accessible to at least a majority of people with disabilities.

[music out]

Dallas Taylor (narrator): Playing with these settings enabled can be a fascinating experience, even for sighted players. Here’s a short segment of the Last of Us 2 with these settings off.

[sfx clip: Last of Us 2 - audio settings off]

Dallas Taylor (narrator): And here’s that same section with the audio cues and listen mode enabled.

[sfx clip: Last of Us 2 - audio settings on]

Steve: That's the real magic of accessibility, the real beauty and genius of accessibility, is that when it's integrated like that, it just makes it so that it's still fun to play, and it's still challenging to play, but also is accessible to play.

[music in]

Dallas Taylor (narrator): Recently, Steve worked with Xbox on a project that was unlike anything he’d ever done before.

Dallas Taylor (narrator): To promote their new Series X console, Xbox launched an advertising campaign called “Power Your Dreams.” As part of this campaign, Xbox wanted to try and recreate the dreams of someone who had recently experienced one of their games, by making a kind of audio art piece. Here’s Steve telling the story on his Youtube channel:

Steve on Youtube: Essentially, they wanted to turn my dreams into reality. So they teamed up with a prominent university that was conducting a dream study, where essentially, they were able to have people go into this sort of lucid, dream-like meditative state, but still allow the participant to be fully aware and cognitive of what’s going on and what’s happening.

Dallas Taylor (narrator): The game that Xbox had chosen for Steve was one of his favourites, a sci-fi first person shooter called Destiny 2. Specifically, they wanted to promote an upcoming expansion for the game called Beyond Light, which takes place on Jupiter’s icy moon Europa.

[music out]

Steve on Youtube: So the way they tried to theme my dreams around Destiny, is they actually had me watch what they called a stimulus.]

Dallas Taylor (narrator): The stimulus was unreleased footage of the Beyond Light expansion.

[sfx clip: Beyond Light trailer: We are deploying Guardians to all corners of the system to find answers.

I’m picking up a distress signal. Someone’s in trouble.

My friends, we are all in great danger.]

Steve on Youtube: About a half an hour to 45 minutes before my sleep session, they would have me watch this footage over and over and over again, essentially to stimulate my brain in order to be able to keep thinking about the game. And then when I would go into the sleep session, the person running the dream study would sort of prompt me to keep thinking about the game and thinking about being on Europa, and Destiny, and all of that kind of helps basically grow my imagination, and expand it so that I could be dreaming about Destiny.

[music in]

Dallas Taylor (narrator): During the sessions, Steve wore a specialized headset that could monitor his brainwaves, his heart rate, and his head movements. He also wore headphones that allowed the researcher to talk to him while he drifted off.

Steve on Youtube: It was an interesting sort of state because it wasn't like I was fully asleep, but it wasn't like I was fully awake either. It was dipping in and out of dream state and I had full cognizance of what I was actually doing, so I knew that I had basically sort of dreams happening in my head, but when the guy conducting the study was asking me questions about what I was dreaming about, I could fully say what I was seeing.

Dallas Taylor (narrator): This is audio from one of those sessions. It starts with a question from the researcher.

[sfx clip: Researcher: Can you tell me any more about what was going through your mind?

Steve: We're running through a bunch of different hallways. Trying to, no, we’re being chased by something.

Steve: It was really trippy. It felt really weird, but it was such a cool experience.]

[music out]

Dallas Taylor (narrator): Steve did 2 of these sessions, which lasted 2 hours each. Next, Xbox and their partners spent several months making an audio recreation of Steve’s dreams. They also had Steve record some voice over. Here’s a clip of the final piece.

[sfx clip: Power Your Dreams Clip: I’m on Europa… It’s a vast landscape of snow… Now, a bunch of creatures are chasing us.]

Dallas Taylor (narrator): That video is meant to give people a taste of the 3D audio capabilities of the Xbox Series X. If you’re familiar with Surround Sound, 3D audio is similar, but with a few subtle differences.

Dallas Taylor (narrator): For surround sound, imagine you’re in a movie theatre, watching the movie Thor Ragnarok.

[sfx clip: Thor Ragnarok]

Dallas Taylor (narrator): The audio is played through a set of speakers that surround the audience. Generally, the positioning of the sounds will match the action on screen. So when the Hulk swings his axe from the left to the right, you’ll hear it quickly pan from the left speaker, to the middle left, to the middle.

Dallas Taylor (narrator): Video games have been doing basically the same thing for years. So in Mario 64, if there’s a ghost over on your right, you’ll hear it out of the right channel [sfx: Boos].

Dallas Taylor (narrator): But with the processing power of modern consoles, it’s possible to take this a lot further. In a newer game, there might be hundreds of things making sound all at once.

[sfx clip: 3D audio scene in]

Dallas Taylor (narrator): As you move through the game, the placement and tonality of these sounds will constantly shift around your character. When it’s done well, it can almost trick your mind into thinking you’re in a real physical space. And this is what’s known as 3D audio.

Dallas Taylor (narrator): For example, in Battlefield 5, there might be planes overhead, and soldiers shouting, and bullets flying from every direction [sfx clip: Battlefield 5].

Dallas Taylor (narrator): In the game Returnal, you might hear the alien creatures change positions as you dodge around them [sfx clip: Returnal].

Dallas Taylor (narrator): The great thing about 3D audio is that it doesn’t require a bunch of speakers, like surround sound. It's pretty much all done through advanced processing. This means that all you need to experience 3D audio is a good pair of stereo headphones.

[music in]

Dallas Taylor (narrator): Today, both Xbox and Playstation are making 3D audio a big selling point for their newest games. And for gamers like Steve, it’s exciting.

Steve: For blind players, having the ability to know where an enemy is, if they're behind and to the left [sfx], it makes it either scary but also really cool at the same time.

Steve: It just immerses a blind player into the world, so it feels like that they are a part of it or that they're in the world that they're playing.

Dallas Taylor (narrator): The reason that so many of us love video games is because they let us experience worlds that are so different from our own.

Connor: It’s just something fun and different, and I can lose myself in a different world, I guess? And that's kind of fun. I mean, I think that's what attracts a lot of us to gaming, right?

Dallas Taylor (narrator): Just like immersive audio and stunning graphics, great accessibility options can help erase the borders of the screen, and put you in the world of a game. These settings might seem like they’re just for a specific group of people, but the truth is that anyone can benefit from them. Even if you’re sighted, they might totally change the way you experience a game. And as more and more game companies start making this a priority, who knows what kind of awesome new features we’ll start to see and hear.

Steve: There is a famous quote of, "This isn't the end. This isn't even the beginning of the end. This is the end of the beginning.” We're just getting started, and I'm really excited for the future. We all are all in this together. Sure, I have my own space in it, and I'm very happy and proud of the work that I've been able to do, but it's a lot of us together that have been able to help make it happen. And It's really cool to be able to see.

[music out]

JAMES PARKINSON: This story came from the podcast Twenty Thousand Hertz, the show that brings you the stories behind the world’s most recognisable and interesting sounds. If you enjoyed this one, there are many more to check out. And a good place to start is their episode on the sounds of Minecraft. You can find the show in your favourite podcast app or visit their website - 20k.org - that’s 2-0-k dot org.

CREDITS

JAMES PARKINSON: Gameplay is a production of Lawson Media.

JAMES PARKINSON: Our theme music you're hearing right now was composed by Breakmaster Cylinder, our artwork is by Keegan Sanford, and additional music in this episode from Breakmaster Cylinder.

JAMES PARKINSON: You can follow the show on Twitter, Instagram and Facebook at ‘gameplaypodcast’. We have a Discord as well, so come and join the community. And please tell your friends about Gameplay and help spread the word in your favourite gaming communities. Word-of-mouth is still one of the best ways you can help us grow. You’ll find links, transcripts and more on our website, gameplay.co. And we’ll be back with an all new story on the next episode of Gameplay. Until then, I’m James Parkinson, thanks for listening.