While there are a number of books about refactoring, most of them deal with the nitty-gritty of improving small bits of code one line at a time. I believe that the most difficult part of a refactor is usually not finding the precise way to improve the code at hand, but rather everything else that needs to happen around it. In fact, I might also go so far as to say that for any large software project, the little things rarely matter; coordinating complex changes is the biggest challenge of all.
Refactoring at Scale is my attempt at helping you figure out those difficult pieces. It’s the culmination of many years of experience carrying out all sorts of refactoring projects of various scales. During my time at Slack, many of the projects I’ve led have allowed the company to scale dramatically; our product has gone from being able to support customers with 25,000 employees to those with a whopping 500,000. The strategies we developed to refactor effectively needed to tolerate explosive organizational growth, with our engineering team growing nearly sixfold during the same period. Successfully planning and executing on a project that affects both a significant portion of your codebase and an increasing number of engineers is no small feat. I hope this book gives you the tools and resources you need to do just that.
Who Should Read This Book
If you work in a large, complex codebase alongside dozens (or more) of other engineers, this book is for you!
If you’re a junior engineer seeking ways to start building more senior skills by making a difference at your company, a large refactoring effort can be a great way to achieve that. These kinds of projects have broad, meaningful impact extending well beyond your individual team. (They’re also not so glamorous that a senior engineer might snap it up right away.) They’re a great opportunity for you to acquire new professional skills (and strengthen the ones you already have). This book will teach you how to navigate this kind of project smoothly from start to finish.
This book is also a valuable resource for highly technical senior engineers who can code themselves out of any problem, but are feeling frustrated that others aren’t understanding the value of their work. If you’re feeling isolated and are looking for ways to level-up others around you, this book can teach you the strategies you need to help others see important technical problems through your eyes.
For the technical managers seeking to help guide their team through a large-scale refactor, this book can help you understand how to better support your team every step of the way. There isn’t a substantial amount of technical content contained within these pages, so if you are involved with a large-scale refactor in just about any capacity (engineering manager, product manager, project manager), you can benefit from the ideas herein.
Why I Wrote This Book
When I set out on my first large-scale refactor, I understood why the code needed to change and how it needed to change, but what puzzled me most was how to introduce those changes safely, gradually, and without stepping on everyone else’s toes. I was eager to have cross-functional impact and didn’t pause to acknowledge the ramifications the refactor might have on others’ work, nor how I might motivate them to help me complete it. I simply plowed through. (You can read about this refactor in Chapter 10!)
In the years that followed, I refactored many, many more lines of code and ended up on the receiving end of a few ill-executed refactors. The lessons I’d learned from these experiences felt important, so I began speaking about them at a number of conferences. My talks resonated with hundreds of engineers, all of whom, like me, had experienced problems effectively refactoring large surface areas of code within their own companies. It seemed clear that there was some sort of gap in our software education, specifically around this core aspect of what it means to write software professionally.
In many ways, this book attempts to teach the important things that aren’t covered in a typical computer science curriculum, simply because they are too difficult to teach in a classroom. Perhaps they cannot be taught in a book either, but why not give it a try?
Navigating This Book
This book is split into four parts and organized in rough chronological order of the work required to plan and execute a large-scale refactor, outlined as follows.
Part I introduces important concepts behind refactoring.
Part II covers everything you need to know about planning a successful refactor.
Chapter 3 provides an overview of the many metrics you can use to measure the problems your refactor seeks to solve before any improvements are made.
Chapter 4 explains the important components of a comprehensive execution plan and how to go about drafting one.
Chapter 5 discusses different approaches to get engineering leadership to support your refactor.
Chapter 6 describes how to identify which engineers are best suited to work on the refactor and tips for recruiting them.
Part III focuses on what you can do to make sure that your refactor goes well while it is underway.
Chapter 7 explores how best to promote good communication within your team and with any external stakeholders.
Chapter 8 looks at a number of ways to maintain momentum throughout the refactor.
Chapter 9 provides a few suggestions for how to ensure that the changes introduced by your refactor stick around.
Part IV contains two case studies, both pulled from projects I was involved with while working at Slack. These refactors affected a significant portion of our core application, truly at scale. I hope these will help illustrate the concepts discussed in Parts I–III of the book.
This ordering is not prescriptive; just because we’ve reached a new phase doesn’t mean we shouldn’t revisit our previous assumptions if necessary. For example, you might be kicking off your refactor with a strong sense of the team you’ll be working with, only to discover halfway through drafting your execution plan that you’ll need to bring in more engineers than you had initially anticipated. That’s ok; it happens all the time!
Conventions Used in This Book
The following typographical conventions are used in this book:
Indicates new terms, URLs, email addresses, filenames, and file extensions.
Used for program listings, as well as within paragraphs to refer to program elements such as variable or function names, databases, data types, environment variables, statements, and keywords.
This element signifies a tip or suggestion.
This element signifies a general note.
Using Code Examples
Supplemental material (code examples, exercises, etc.) is available for download at https://github.com/qcmaude/refactoring-at-scale.
This book is here to help you get your job done. In general, if example code is offered with this book, you may use it in your programs and documentation. You do not need to contact us for permission unless you’re reproducing a significant portion of the code. For example, writing a program that uses several chunks of code from this book does not require permission. Selling or distributing examples from O’Reilly books does require permission. Answering a question by citing this book and quoting example code does not require permission. Incorporating a significant amount of example code from this book into your product’s documentation does require permission.
We appreciate, but do not require, attribution. An attribution usually includes the title, author, publisher, and ISBN. For example: “Refactoring at Scale by Maude Lemaire (O’Reilly). Copyright 2021 Maude Lemaire, 978-1-492-07553-0.”
If you feel your use of code examples falls outside fair use or the permission given above, feel free to contact us at firstname.lastname@example.org.
O’Reilly Online Learning
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.
Our unique network of experts and innovators share their knowledge and expertise through books, articles, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, visit http://oreilly.com.
How to Contact Us
Please address comments and questions concerning this book to the publisher:
- O’Reilly Media, Inc.
- 1005 Gravenstein Highway North
- Sebastopol, CA 95472
- 800-998-9938 (in the United States or Canada)
- 707-829-0515 (international or local)
- 707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/refactoring-at-scale.
Email email@example.com to comment or ask technical questions about this book.
For news and information about our books and courses, visit http://oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://youtube.com/oreillymedia
Writing a book is not an easy task, and this one was no exception. Refactoring at Scale would not have been possible without the contributions of many people.
First, I’d like to thank my editor at O’Reilly, Jeff Bleiel. Jeff turned an inexperienced writer (me) into a published author. His feedback was always spot-on, helping me organize my thoughts more cohesively, and encouraging me to cut whenever I was being too wordy (which is something that happened quite frequently). I simply can’t imagine working with a better editor.
Second, I want to thank the handful of friends and colleagues who read early versions of a few chapters: Morgan Jones, Ryan Greenberg, and Jason Liszka. Their feedback assured me that my ideas were sound and would be valuable to a wide range of readers. For the words of encouragement and thought-provoking conversations, thanks go to Joann, Kevin, Chase, and Ben.
I’d like to thank Maggie Zhou for all her help cowriting the second case study chapter (Chapter 11). She is one of the most thoughtful, intelligent, energetic coworkers I’ve ever had the pleasure to work with and I’m thrilled for the world to read about our adventures together!
A huge thank you to my technical reviewers, David Cottrell and Henry Robinson. David has been a close friend since university and has led a number of large-scale refactors in his many years at Google. He’s since founded his own company. Henry is a colleague at Slack who’s made countless open-source contributions and seen explosive growth at Silicon Valley companies firsthand. They are both incredibly conscientious engineers, and the book greatly benefited from their guidance and wisdom. I am endlessly grateful for the many hours they spent verifying its contents. Any inaccuracies in the final manuscript are mistakes of my own.
Thank you to everyone who’s ever refactored something with me. There are too many of you to name, but you know who you are. You all have had a hand in shaping the ideas in this book.
Thank you to my family (Simon, Marie-Josée, François-Rémi, Sophie, Sylvia, Gerry, Stephanie, and Celia) for cheering me on from the sidelines.
Finally, thank you to my husband, Avery. Thank you for your patience, for giving me the time, space, and encouragement to write. Thank you for letting me hijack countless afternoons to talk through an idea or two (or three or four). Thank you for believing in me. This book is just as much yours as it is mine. I love you.