Up, Down, Left, Right

JAMES PARKINSON: If you’ve played video games for long enough, you probably have a favourite game controller. The one that feels the most comfortable to you, and maybe best suited to the types of games you like to play. For example, if you’re serious about Super Smash Bros, you might favour the GameCube controller. It’s a very personal thing, because controllers, and specifically their design, is central to the overall experience.

NEIL MANSFIELD: Controllers are absolutely vital. At the most basic levels, if you have the wrong controller, you're going to find the game impossible to play. Or if you have an inferior controller, you're going to be at a disadvantage.

JAMES PARKINSON: Neil Mansfield is a Professor of Human Factors Engineering, or you might know it as ergonomics.

NEIL MANSFIELD: And it's all about designing for human use. So what we want to do is to think about the real users, the inclusive populations that are going to use any product and design for those people.

NEIL MANSFIELD: But really what we're looking for in a gaming controller is something that actually makes the game as much fun as possible. Something where you can develop skill and you get that feedback, something which makes it so that the controller isn't getting in the way. I think we’d take it on the chin, if we lose a life because we took a bad decision. But when it's because we weren't able to actually communicate that decision to the game itself, that's when the controller starts to fall down, and the enjoyment of the game starts to drop.

JAMES PARKINSON: Controller design is incredibly difficult to get right, because you need to cater to such a wide variety of people. It’s why the original Xbox controller, known as “The Duke”, received so much backlash for it’s oversized proportions - despite Microsoft’s rigorous testing process.

[Archive Audio]

Bill Gates: “The design here was driven by spending time with gamers, actually putting the control in their hands. We tried out over one hundred different form factors…”

NEIL MANSFIELD: Curvy doesn't mean it’s ergonomic!

JAMES PARKINSON: Controllers have evolved in different ways over the years but some things have also remained much the same. We’ve come to expect particular layouts and buttons. And when you mess with a good design that works, it can disrupt how we play. When Nintendo released the Switch in 2017, they actually reintroduced a problem that they’d already solved nearly 40 years ago.

JAMES PARKINSON: I’m James Parkinson. From Lawson Media, this is Gameplay, stories about video games and the virtual worlds that power culture and community.

NORMAN CARUSO: The late 70s, early 80s was basically the beginning of the video game industry.

JAMES PARKINSON: This is Norman Caruso, creator of the YouTube documentary series, Gaming Historian.

NORMAN CARUSO: You had arcades, which were very popular, but you also had the beginnings of the home console generation. So Atari puts out the Atari 2600, Coleco puts out the ColecoVision, Mattel puts out the Intellivision, you have tons of Pong consoles everywhere. So it was really the beginning of the video game industry.

JAMES PARKINSON: It was a time of experimentation, but not a whole lot of innovation. Games at the time were fairly basic, as were the controller interfaces. The arcade industry had set the tone early with a variety of knobs, switches and of course the joystick, built into the cabinets.

NORMAN CARUSO: The joystick was the primary device for directional movement, and that came from the arcade industry. And a lot of home console developers basically just copied whatever the arcade industry was doing. So you had the joystick, you had Intellivision with a bunch of numbered buttons - it almost looked like a phone. But you also had a disc that could move in sixteen different directions. Atari Football, in the arcades, used a massive trackball controller. So you had to spin the trackball as fast as you can to move around, and you know, it really hurt your hand when you played. So, it was a great time for experimenting with controller interfaces.

JAMES PARKINSON: And around this time, Nintendo were also moving into electronics. Founded in 1889, as a company that manufactured Hanafuda, Japanese playing cards, Nintendo also started developing toys in the late 1960s, thanks to one of their factory workers, Gunpei Yokoi.

NORMAN CARUSO: Gunpei Yokoi was probably Nintendo's greatest engineer.

NORMAN CARUSO: He basically did repair work on these assembly lines at Nintendo. But in his spare time he liked to create toys. And he took his ideas to Nintendo's President Hiroshi Yamauchi. And I believe the first idea he brought to him was the Ultra Hand. It’s like an extendable arm toy. And that was one of the first toys that Nintendo manufactured. And it was very, very popular. And basically Gunpei Yokoi went from repairing assembly line machines to making toys full time, because he was making Nintendo a lot of money from the Ultra Hand.

JAMES PARKINSON: This was 1966, just a year after Yokoi was hired by Nintendo. The Ultra Hand sold over one million units, and it set the company on a new path, with toy manufacturing becoming a core part of Nintendo’s business. Gunepei Yokoi’s engineering and mechanics background was also pivotal in the transition to electronic games, and by the early 70s he was appointed General Manager of the newly created Research & Development Department.

NORMAN CARUSO: He also designed kind of an odd device called the Love Tester. Basically, two people grab onto these electrical nodes of this device and it determines if there is compatibility between the two people. Strange but popular.

JAMES PARKINSON: Although several of Nintendo’s toys were popular, they still struggled to compete with more established companies in the market. But with the rise of video games, they saw new opportunities in a growing industry.

NORMAN CARUSO: They had Pong consoles, just like almost every other electronics company at the time. You know, the Color TV-Game 6, the Color TV-Game 15. But they also did arcade games. And in 1980, they created a series of handheld games, called the Game & Watch.

JAMES PARKINSON: And the person behind it? Gunpei Yokoi.  

NORMAN CARUSO: The story is, Gunpei Yokoi was taking a train ride into work, and he noticed a businessman, to pass the time, he was just playing with a pocket calculator. And Gunpei Yokoi thought that was really interesting, and what if Nintendo could come up with some sort of small electronic device, just to pass the time when you're riding the train to work? So fast forward, Nintendo President Hiroshi Yamauchi had a business meeting and his regular driver couldn’t make it. And so oddly enough, Gunpei Yokoi had to drive Hiroshi Yamauchi to his business meeting. And he was kind of upset about it, because you know he's an engineer at Nintendo. Why is he being a chauffeur, driving around the President of the company? But as he's taking Hiroshi Yamauchi to his meeting, he makes small talk and he mentions the businessman he saw on the train playing with a pocket calculator. And Hiroshi Yamauchi is notoriously quiet, kind of a grumpy guy. And he just says, “huh, interesting”. And Gunpei Yokoi thought that was the end of it. And of course a few days later, salespeople from Sharp Electronics show up at Nintendo and they want to speak to Gunpei Yokoi. And Yokoi is confused, as to why Sharp wants to talk with him, and Yamauchi intervenes and says, “Well, you talked about this pocket calculator game idea. So we're going to work with Sharp, and we're going to make it.”

[Archive Audio: Game & Watch Commercial - Japan ]

JAMES PARKINSON: The Game & Watch marked an important step in Gunpei Yokoi’s career at Nintendo, which defined his personal philosophy and approach to hardware design.

NORMAN CARUSO: Lateral Thinking with Withered Technology.

JAMES PARKINSON: Essentially, this means using cheap and established technology in new and unconventional ways.

NORMAN CARUSO: It was much cheaper to use this old technology and create new products than to go for the latest and greatest. And the other reason is availability. It's very easy to get all of these parts.

JAMES PARKINSON: A team of five engineers were recruited from Nintendo’s Creative Section, and assigned to develop the Game & Watch, under Gunpei Yokoi. Each developer was a jack-of-all-trades and the team worked closely together. It included Makoto Kano, Takehiro Izushi, Masao Yamamoto, and Shigeru Miyamoto.

JAMES PARKINSON: The foundation for Game & Watch devices was a 4-bit CPU and small LCD screens. PC’s weren’t common at the time, so instead of programming, games were all developed with hardware. In a 2010 interview with Nintendo, Takehiro Izushi reflected on his memories of the time, saying quote: “If a hardware guy wanted to increase the speed somewhere, he'd bring in a soldering iron and change the wiring.”

JAMES PARKINSON: Gunpei Yokoi and fellow engineer Satoru Okada developed the prototype Game & Watch, before the first official title was released in 1980.

NORMAN CARUSO: I believe the first Game & Watch game was Ball. It was a very simple game, it was a guy juggling and you had to move his arms and make sure he could keep juggling the ball. And Nintendo put out a lot of Game & Watches, and each Game & Watch had a different game on it. It was very successful. So successful that they released it worldwide, which before - Nintendo, a lot of their products stayed within Japan. But Game & Watch was so popular they were able to expand and release it worldwide. So the Game & Watch came out in Europe and North America.

[Archive Audio]

Game & Watch Commercial (USA): “Nintendo Game & Watch, that’s pocket power! Widescreen or multi-screen games you can play indoors or out. They tell you the score, and even the time! They’re pocket power, they’re Game & Watch. 14 and all, only from Nintendo!”

NORMAN CARUSO: And Nintendo was able to take a lot of their arcade games and convert them to Game & Watches - it was a very primitive version of the arcade game, but it worked. And the Game & Watch was, I'd say from 1980 to 1984/85, the Game & Watch was one of the most successful products Nintendo ever made.

JAMES PARKINSON: The Game & Watch sold just over 43 million units globally, throughout its lifespan, and Gunpei Yokoi had produced another hit product for Nintendo. But it’s production also led to an important innovation. In the 1970s, controller interfaces hadn’t evolved beyond the joystick, because there wasn’t really a need to. But that changed in 1982.

NORMAN CARUSO: Nintendo was wanting to take Donkey Kong, which was by far their most successful arcade game of all time, and they wanted to make a Game & Watch version of Donkey Kong. This Game & Watch would be two screens. So normally Game & Watches were on one screen, and you hold it and you play it. But when you have two screens, they had to add a hinge, so the Game & Watch actually closed. So this presented a problem when they wanted to make the controls.

NORMAN CARUSO: It was a problem for two reasons. One is, previous Game & Watch games were very simple, in that you only moved on the X axis, so you move left and right. But Donkey Kong, you had to move on the Y axis as well, which required four directions. So you can't just have a left and right button, you have to have an up and down button as well. And so, Gunpei Yokoi’s presented with this problem, that not only do we need four directions, we also have to have something that works with a Game & Watch that opens and closes.

JAMES PARKINSON: Gunpei Yokoi and his design team experimented with many prototypes. The engineers would bring Yokoi a mock-up try out, he’d test it, and say, "No, it's not right,", and send them back to try again. One of the designers, Makoto Kano, actually went out and bought a bunch of compacts, the small makeup cases, while researching different types of hinges. This was important in getting the clamshell design just right. But for the control interface, Gunpei Yokoi was very specific. Getting the Game & Watch to close was one part, but how the device felt in the hands of the player was crucial too.

NORMAN CARUSO: And so at first, they just use a classic joystick. And it worked, but it was very fragile, and it broke easily, and you couldn't open and close the Game & Watch with the joystick. So, Gunpei Yokoi tries a joystick with, like, a soft plastic shell around it. So it makes the joystick less fragile, but you still can't close the Game & Watch. And then Gunpei Yokoi says “Okay, let's just do four directional buttons up, down, left, right”. And it worked, it was thin, and so you could close the Game & Watch. But it didn't feel intuitive. When you have four different buttons, you may have a tendency to look down instead of looking at the screen and try to figure out what button you are pressing, because all the buttons look the same. And so Gunpei Yokoi was worried about that. He said it didn't feel - it took the player out of the game, because they were worried about what button they were pressing.

JAMES PARKINSON: This need for an instinctive player experience is what drove Yokoi to innovate, landing on a simple but perfect solution.

NORMAN CARUSO: So he comes up with an idea. Instead of four different buttons, "I will put a cross-shaped piece of plastic on top of the buttons.". And therefore it's basically one piece that you rock up, down, left or right, to determine what direction you want to go.

JAMES PARKINSON: Yokoi called it the directional pad, otherwise known as the D-pad.

NORMAN CARUSO: The way the D-pad works is you have four input buttons on the board; up, down, left, right. And there's also a piece of rubber that goes on top of the board. And on top of that piece of rubber, you have the plastic D-pad. And the D-pad itself, it’s a plastic cross, and then it has a flat circular base. And then in the center of the base, underneath, is a ball and that is the pivot. And that is how the D-pad can rock and move in all four directions. You can also press two directions at the same time to allow, possibly for eight directional movements. Of course, they didn't need that for the Donkey Kong Game & Watch, but it was possible to do it.

NORMAN CARUSO: And because it was in the shape of a cross, it was intuitive to feel what direction you were going.

NEIL MANSFIELD: It has that tactile shape to it. So that's a key part to it. So you don't need to keep looking at it to work out where you are. And it works even for a novice user. It's implicit in the design, is how it's supposed to be used.

JAMES PARKINSON: Professor Neil Mansfield again.

NEIL MANSFIELD: So it's that tactile feedback, through the surface and the shape. And also the proprioception from - which is the way you feel - of how the position that your thumb is in, it's constant. The range of movement of the thumb is only a few millimetres, in those, what we call X and Y directions, where we do a lateral and four and half movement.

NEIL MANSFIELD: So the movement is only a few millimetres, but what it means is that you can slide from one position to the other without getting the abrasion on the thumb itself. So it's a comfortable thing to play for long periods of time. And you get that reliability, because instead of pushing the button one way, it pushes down on a button underneath, rather than pushing it sideways, and that means you get a much more reliable response from that D-pad controller, which is where the elegance in the design is so impressive.

NEIL MANSFIELD: You are immersed within the game itself, right within that gameplay itself.

JAMES PARKINSON: When developing the Donkey Kong Game & Watch, Gunpei Yokoi considered positioning the D-pad on the right side of the device, given the majority of people are right-handed. But he instead opted for the D-pad on the left, with a single jump button on the right, because that’s how it worked in the arcades. Not just the Donkey Kong arcade edition, but pretty much all cabinets that used a joystick and button combination. It was a familiar layout, and of course, one that would continue far beyond the Game & Watch itself.

JAMES PARKINSON: Coming up, the simple workarounds that allow Nintendo’s competitors to replicate their patented design.

[ AD BREAK ]

JAMES PARKINSON: While the games industry was still young in the 80s, the D-pad was an innovation like it had never seen before. But in the moment, its importance wasn’t initially realised. Here’s Norman Caruso.

NORMAN CARUSO: It's interesting because even after the directional pad was invented, future Game & Watch releases went back to the directional button layout. For example, Donkey Kong Jr. Game & Watch used four directional buttons instead of the D-pad. So it's very possible the D-pad could have been a one-hit wonder with the Donkey Kong Game & Watch.

JAMES PARKINSON: In fact, Nintendo didn’t even patent the D-pad right away.

NORMAN CARUSO: It's still kind of a mystery, the whole patent experience with the D-pad. From what we know, Gunpei Yokoi was a very busy man. He did a lot of stuff at Nintendo; he did game development, he did engineering, he did hardware development, he did game testing, he did consulting. So he was a very busy guy, and a lot of his creations, he didn't think too much of. He didn't feel like it was that important. And that's how he felt with the D-pad. He thought, “I've solved a problem with the Donkey Kong Game & Watch, and that's that, we're done, I'm moving on”. But a young engineer named Ichiro Sharai reminded Gunpei Yokoi that we need to file patents for new stuff that we create. And Gunpei Yokoi, a busy man, says, “well just put your name on it”, you know, “put all of your information, I'm busy”. And so it took a year, but the patent for the D-pad was finally filed in 1983, a year after the Donkey Kong Game & Watch came out.

JAMES PARKINSON: And it’s lucky that they did, because that same year, the D-pad demonstrated its true value when it was included in Nintendo’s next big product.

NORMAN CARUSO: When Nintendo was making the Famicom, also known as the Nintendo Entertainment System, they had strict orders to keep the system as cheap as possible. And they were also wanting to make the system as powerful as possible. And so they needed a controller interface that could do eight different directions, that felt intuitive, and that could allow for all sorts of gameplay experiences. And of course, they went to the tried and true joystick first, and they tested it and again, it worked. But there was an engineer named Taka Osawano. And he used to work for Research & Development 2 which was the team that was developing the Famicom. But now he worked in Research & Development 1, which was Gunpei Yokoi’s team. But he kept in contact with his old co-workers, and he heard that they were working on the Famicom controller. And so he suggested The D-pad from the Donkey Kong Game & Watch. And of course, the development team thought that was a crazy idea. But they decided to test it anyway. So what they did was they interfaced a Donkey Kong Game & Watch to their prototype Famicom, and used the D-pad. And they were just amazed at how well it worked and how natural it felt to play a game using the D-pad. And of course, they kept that design when they brought it over to North America as the Nintendo Entertainment System.

JAMES PARKINSON: It’s well known that the American video game industry was on the verge of collapse in the early 80s, before the release of the NES breathed life back into the market. The console was a huge commercial success, and part of it was down to the D-pad’s ability to deliver a frictionless experience for players of all ages.

NEIL MANSFIELD: I really loved the NES controller. You know, it's that classic design. I’d still probably say it's my favorite gaming controller and it's just because anyone can pick that up and they know what to do immediately. And mechanically, the design is really elegant. If you take one apart, you find that there's not many parts within it. It's a really beautiful piece of engineering. And the beauty is in its simplicity.

NORMAN CARUSO: I don't think anyone has ever asked me, like people always ask, “now, what does the ‘B’ button do?”, “What does the ‘A’ button do?”, like, “how do I fire?”, “How do I do this?”. But I don't think anyone has ever asked, “how do I move?”, because I think you look at the D-pad and you’re like, “it just clicks”. It just makes perfect sense what that’s supposed to do and how it works.

JAMES PARKINSON: The Nintendo Entertainment System cemented the directional pad as a core input for movement in game controllers, and fuelled the development of 2D platform games. It changed the entire approach to controller design, and inspired other companies to try and replicate the D-pad. One of the most striking examples of this comes from Sega. Their SG1000 console, like many systems before it, used a joystick. And there was nothing particularly wrong with that. The Atari 2600, for example, was a very popular system. But the SG1000 was released in Japan on the exact same day as the Famicom, in 1983, and it just couldn’t compete. Thanks to Nintendo, the home console had moved on from the joystick era.

NORMAN CARUSO: So, Sega goes back to the drawing board and makes the Sega Master System, which had sort of a D-pad. It worked very similar to the D-pad, but it also had, I believe it had threads in the middle of it, and you could screw in a little joystick handle if you wanted to play it that way. Of course, the Master System, it did better than the SG1000. But Sega didn't really take off in the home console market until the Sega Genesis. And because Nintendo has this patent on the D-pad, companies can't just take apart an NES controller and see how the D-pad works, and just copy that. So they have to come up with tiny little differences to make their D-pad unique.

NORMAN CARUSO: As I mentioned earlier, on the Nintendo D-pad, under the plastic piece is the ball pivot. And that's how the controller rocks and moves around. But Sega put the ball pivot on the piece of rubber that goes under the plastic piece. That one tiny difference allowed them to get away with using the D-pad. It works basically exactly the same way as Nintendo's D-pad.

JAMES PARKINSON: At this point, there was basically no turning back. If a console manufacturer wanted to compete in the market, they had to come up with a controller that was as fun and easy to use as the NES, and the Super NES. The Sega Genesis sold much better than previous efforts, but the real challenger to Nintendo emerged in Sony, with their release of the PlayStation in 1994.

NORMAN CARUSO: They did something very similar, but instead of the ball pivot, being on the rubber or the ball pivot being on the plastic D-pad, they have a completely separate piece that just contains that ball pivot. You have the circuit board, you have the piece of rubber, then you have the ball pivot piece, and then you have the plastic. And they mounted the D-pad kind of under the controller shell. It's kind of a cool look, and it works really well. But again, that's just another small, subtle difference that allows these other companies to make replica D-pads.

JAMES PARKINSON: As technology improved, the rise of 3D games saw manufacturers reconsidering the necessity for the D-pad. They needed an input device that allowed for 360 degrees of movement, and the solution was the analogue stick, an evolution of the joystick. Sony released the Dual Analog controller for the PlayStation in 1997, and while it forever changed what we think of as a video game controller, it still included a directional pad. And with Nintendo’s patent expiring in 2005, the D-pad remains as relevant as ever.

NORMAN CARUSO: I think we will always have a D-pad on video game controllers. With a lot of these indie developers creating retro style games, or Nintendo putting out, you know, classic Nintendo games with their Switch Online service, the D-pad is by far the best way to play those games. And the D-pad has, you know, gone outside of video games. On TV remotes, you’ll find D-pads. Nintendo won an Emmy award for the D-pad in 2007 because it’s so influential in, not only the video game industry, but the electronics industry as a whole. So I don’t think we’ll ever see the D-pad go away.

JAMES PARKINSON: When the Nintendo Switch was announced in 2016, the most surprising thing wasn’t it’s incredible versatility, but that the D-pad had been replaced by four individual directional buttons. It was the first time a Nintendo home console had shipped without one since their first generation Color TV-Game series in the late 70s. Although Nintendo omitted the D-pad with good reason - the detachable pair of Joy-Con’s needed to function as two standalone controllers - many players were frustrated when they went to play those classic platform games. The kind of games that were created because of what the D-pad offered.

NORMAN CARUSO: I think because of the D-pad, developers were able to try new games, create new types of games with new types of movement. You know, with the Famicom, it was a pretty powerful home console. And so you were able to try new things like you know, instead of moving in two directions, we can move in four directions. Now, we could even move in eight directions, which was huge.  I think because of how intuitive the D-pad is and how comfortable it is, you're able to create much more fun, immersive gaming experiences.

JAMES PARKINSON: Gunpei Yokoi died in a tragic road accident 1997. He’d left Nintendo the year before to start his own toy company, Koto Laboratory, which still exists today. But his legacy at Nintendo lives on, not only the D-pad, and his other creations that made use of it, like the Game Boy. But his design philosophy remains at the core of the company's approach to making video games fun and accessible for all.

JAMES PARKINSON: Many thanks to Norman Caruso and Neil Mansfield. If you enjoyed this episode, go and check out Norman’s excellent YouTube channel, just search for 'Gaming Historian'. And coming up, Games Archive, the first of many recurring segments on the show, and first up, the unsung hero of Nintendo’s handheld consoles.

JAMES PARKINSON: Gameplay is a production of Lawson Media. This episode was written and produced by me, James Parkinson. The Gameplay theme was composed by Breakmaster Cylinder, our artwork is by Keegan Sanford, and other music in this episode comes from Breakmaster Cylinder, Blue Dot Sessions, and Epidemic Sound. You can follow the show on Twitter, Facebook and Twitch at 'gameplaypodcast', and come and join our community over on Discord. You’ll find all those links, and more, on our website, gameplay.co.

[SEGMENT: GAMES ARCHIVE]

JAMES PARKINSON: So, Gunpei Yokoi often gets much of the praise when talking about Nintendo’s history. And all of that is deserved. But another man is often overlooked, and he was one of Yokoi’s closest colleagues, Satoru Okada.

JAMES PARKINSON: Okada arrived at Nintendo in 1969. Passionate, and stubborn, with a talent for electronics, he was just 22 years old. He’d already worked for several years as an engineer but took the Nintendo job out of desperation, having missed out on another opportunity. Okada found himself designing plastic toys and card games at Nintendo. And he was disappointed by this, not really knowing what he signed up for. But the company’s entry into electronic games quickly saw his skill with a soldering iron put to use. The first product he worked on, under guidance from Gunpei Yokoi, was the Kousenju SP, a light-beam toy gun. The technology was then adapted into Nintendo’s first arcade game, the Laser Clay Shooting System in 1973, and Wild Gunman for the TV-Game 6 console a year later.

JAMES PARKINSON: But it was handheld games where Okada would really shine - not just the Game & Watch series, but the full line of consoles, up to the Nintendo DSi. It started with the Game Boy though, which Gunpei Yokoi approached in a similar way to the Game & Watch - a simple handheld with a short lifespan - more toy than fully fledged console. But it was Okada that had the grander vision, inspired by the success of the Famicom. He and Yokoi fought over this for some time, until Yokoi gave in. Satoru Okada had his superior’s blessing to lead the project, and he made it his own. Yokoi still partly got his way, pushing for a monochrome screen, that was integral to the Game Boy’s great battery life and kept the costs down - there’s his design philosophy again. But the console would not have been as successful as it was, selling 119 million units combined, if it wasn’t for Okada’s ambition. In a 2017 interview with Retro Gamer magazine, Okada said, quote: “My wife often tells me I am the happiest man in the world, because throughout my life, I only did whatever I wanted!”. Satoru Okada left Nintendo in 2010. I’m James Parkinson, thanks for listening.