Preface – Deep Reinforcement Learning in Action


Deep reinforcement learning was launched into the spotlight in 2015, when DeepMind produced an algorithm capable of playing a suite of Atari 2600 games at superhuman performance. Artificial intelligence seemed to be finally making some real progress, and we wanted to be a part of it.

Both of us have software engineering backgrounds and an interest in neuroscience, and we’ve been interested in the broader field of artificial intelligence for a long time (in fact, one of us actually wrote his first neural network before high school in C#). These early experiences did not lead to any sustained interest, since this was before the deep learning revolution circa 2012, when the superlative performance of deep learning was clear. But after seeing the amazing successes of deep learning around this time, we became recommitted to being a part of the exciting and burgeoning fields of deep learning and then deep reinforcement learning, and both of us have incorporated machine learning more broadly into our careers in one way or another. Alex transitioned into a career as a machine learning engineer, making his mark at little-known places like Amazon, and Brandon began using machine learning in academic neuroscience research. As we delved into deep reinforcement learning, we had to struggle through dozens of textbooks and primary research articles, parsing advanced mathematics and machine learning theory. Yet we found that the fundamentals of deep reinforcement learning are actually quite approachable from a software engineering background. All of the math can be easily translated into a language that any programmer would find quite readable.

We began blogging about the things we were learning in the machine learning world and projects that we were using in our work. We ended up getting a fair amount of positive feedback, which led us to the idea of collaborating on this book. We believe that most of the resources out there for learning hard things are either too simple and leave out the most compelling aspects of the topic or are inaccessible to people without sophisticated mathematics backgrounds. This book is our effort at translating a body of work written for experts into a course for those with nothing more than a programming background and some basic knowledge of neural networks. We employ some novel teaching methods that we think set our book apart and lead to much faster understanding. We start from the basics, and by the end you will be implementing cutting-edge algorithms invented by industry-based research groups like DeepMind and OpenAI, as well as from high-powered academic labs like the Berkeley Artificial Intelligence Research (BAIR) Lab and University College London.