• Friday, September 5th
Maxine scans her to-do list, slowly shaking her head in frustration. It’s been two days and she is determined to perform a Phoenix build on her laptop, like any new developer should be able to do. This has become her mission. But according to her list, there are over a hundred items that she’s missing, and no one seems to know where to find them.
She has done nothing on her list. Except for updating and sending out her résumé. Many friends replied to her right away, promising to look for positions she might be interested in.
Maxine asked her guide, Josh, about all those missing build items, but he didn’t know anything about them. The build team used to know these things, but the details are either out-of-date or missing entirely, the knowledge scattered across the entire organization.
She is frustrated, every turn she takes leads to a dead end. There is nothing fun about this challenge. What she is doing, she’s pretty sure, is the exact opposite of fun.
She’s an engineer at heart, and she loves challenges and solving problems. She’s been exiled smack in the middle of probably the most important project in the entire company’s history. And somewhere, there’s code—almost certainly millions of lines of code written by hundreds of developers over nearly three years. But she can’t find any of it.
Maxine loves coding and she’s awesome at it. But she knows that there’s something even more important than code: the systems that enable developers to be productive, so that they can write high-quality code quickly and safely, freeing themselves from all the things that prevent them from solving important business problems.
Which seems to be completely missing here. Maxine is one of the best in the game, but after four days she still has almost nothing to show for it. Just endless clicking around, reading documents, opening tickets, scheduling meetings with people to get things she needs, trapped in the worst scavenger hunt ever.
For a moment, Maxine wonders if she’s the only person having this problem. But she sees developers all around her struggling, so she quickly pushes away any feelings of self-doubt.
Maxine knows her kung fu is amazing. Many times in her career she’s had to solve problems that seemed hopeless and impossible. Often in the middle of the night. Sometimes without any documentation or source code. One of her most famous escapades is still known as the “Maxine Post-Holiday Save,” where all the in-store systems that handled refunds crashed spectacularly on the Friday after Christmas. It’s one of the busiest shopping days of the year as people come in to return gifts from loved ones so they can buy something they actually want.
With her team, Maxine worked into the wee hours of Saturday morning to fix a multi-threading deadlock in a database vendor’s ODBC driver. She had to manually disassemble the vendor library and then generate a binary patch. By hand.
Everyone said it couldn’t be done. But she pulled it off, to the amazement of the scores of people who had worked that outage for over seven hours. The database vendor professional services team was in awe and immediately offered her a job, which she politely declined.
The legends about her kept growing after that. She’s classically trained as a developer, and in her career, she’s written software to stitch together panoramic graphic images and chip layout algorithms for CAD/CAM applications, back-end servers for massive multi-user games, and, most recently, the ordering, replenishment, and scheduling processes that orchestrate thousands of suppliers into a plant production schedule for their MRP systems.
She routinely lives in the world of NP-complete problems that are so difficult to solve they can take more than polynomial time to complete. She loves the Papers We Love series, revisiting her favorite academic papers from mathematics and computer science.
But she has never seen her job as just writing application code, working only pre-deployment. In production, when theory meets reality, she’s fixed wildly misbehaving middleware servers, overloaded message buses, intermittent failures in RAID disk arrays, and core switches that somehow kept flipping into half-duplex mode.
She’s fixed technology components that were spilling out their guts in the middle of the night, having filled up every disk and log server, making it impossible for teams to understand what was actually happening. She led the effort to systematically isolate, diagnose, and restore those services based on decades of intuition and countless production battles.
She’s deciphered stack traces on application servers that were literally on fire, racing to get them safely backed up before the flooding water, halon extinguishers, and emergency power shutdowns destroyed everything.
But deep down, she’s a developer. She’s a developer who loves functional programming because she knows that pure functions and composability are better tools to think with. She eschews imperative programming in favor of declarative modes of thinking. She despises and has a healthy fear of state mutation and non-referential transparency. She favors the lambda calculus over Turing machines because of their mathematical purity. She loves LISPs because she loves her code as data and vice versa.
But hers is not merely a theoretical vocation—she loves nothing more than getting her hands dirty, creating business value where none thought it could be extracted, applying the strangler pattern to dismantle decades-old code monoliths and replacing them safely, confidently, and brilliantly.
She is still the only person who knows every keyboard shortcut from vi to the latest, greatest editors. But she is never ashamed to tell anyone that she still needs to look up nearly every command line option for Git—because Git can be scary and hard! What other tool uses SHA-1 hashes as part of its UI?
And yet, as awesome as she is, at the height of her soaring powers and skills honed over decades, here she sits in the middle of the Phoenix Project unable do a Phoenix build, even after two days. She found where two of the four source code repositories are, and she’s found the three installers for some of the proprietary source code management (SCM) tools and compilers.
However, she is still waiting for license keys for the SCM, and she doesn’t know who to ask to get license keys for the two other build tools. She needs credentials for three network shares and five SharePoints, and no one knows where to get the ten mysterious configuration files mentioned in the documentation. When she emailed the person who wrote the docs, it bounced. They had long left the organization.
She is stuck. No one responds quickly to her emails, her tickets, or her voicemails. She’s asked Randy to help, to escalate her requests, but everyone says it will take a couple of days because they’re so busy.
Of course, Maxine never just takes “no” for an answer. She has made it her mission to do whatever it takes to get a build running. She’s hunted down almost all of the people who have promised her something. She’s found out where they sit and has pestered them, even camping out at their desks, willing to stay there until they get her what she needs.
Sometimes she got what she needed: a URL, a SharePoint document, a license key, a configuration file. But more often than not, the person she hunted down didn’t have what she needed—they would have to ask somebody else, so they would open up a ticket on Maxine’s behalf. And now they were both waiting.
Sometimes, they had a promising lead or clue on who or where Maxine needed to go to next. Most times, though, it was just a dead end, and she was right back where she started.
Trying to get a Phoenix build going is like playing Legend of Zelda, if it were written by a sadist, forcing her to adventure far and wide to find hidden keys scattered across the kingdom and given only measly clues from uncaring NPCs. But when you finally finish the level, you can’t actually play the next level—you have to mail paper coupons to the manufacturer and wait weeks to get the activation codes.
If this really were a video game, Maxine would have already quit, because this game sucks. But Phoenix isn’t a game—Phoenix is important, and Maxine never quits or abandons important things.
Maxine sits at her desk looking at the calendar she’s printed out and pinned to the wall.
She turns back to her computer and runs her finger down her ever-growing list of to-dos again—each item a dependency she requires to get her build going.
She just added two more SharePoint credentials she needs to get from two different Dev managers who, for some reason, run their own Active Directory domains. They’re rumored to contain some critical build documentation with some of the information she seeks.
Randy sent her a ton of Word docs, Visio diagrams, and marketing PowerPoint presentations, which she quickly skims for clues. They may be helpful to marketing people and architects, she supposes, but she’s an engineer. She doesn’t want to see brochures for the car they’ve promised to build—she wants to see the engineering plans and the actual parts that they’re going to assemble the car from.
These documents might be useful to someone, so she posted them on the wiki. Moments later, someone she doesn’t know asks her to take them down because they might contain confidential information.
Looking further down her to-do list, she reads:
Find someone who can give me access to Dev or Test environments.
These were referenced in some of the documentation she read yesterday, but she has no idea who to ask to get access.
She has crossed off one item:
Get account for integration test environment.
This was less satisfying than she had hoped. She poked around the environment for two hours, trying to gain an understanding of the giant application. But in the end, she found it too bewildering—it was like trying to picture the layout of an enormous building by crawling around air ducts without a map or a flashlight.
She types out a new to-do:
Find someone who’s actually doing integration testing so I can shoulder-surf while they work.
Watching someone use the Phoenix applications might help orient her. She’s baffled that no one knows of an actual person who uses Phoenix. Just who are they building all this code for?
Scanning her to-do list again, she confirms that she in fact has nothing to actually do—she has already pestered everybody today and now she’s just waiting for people to (not) get back to her.
It’s Friday, 1:32 p.m. Four and a half hours to go until five, when she can finally leave the building. She tries hard not to sigh again.
She looks at her to-do list. She looks at the clock.
She looks at her nails, thinking that she needs a manicure.
She gets up from her desk with her coffee mug and walks to the kitchen, passing by groups of people wearing hoodies, huddled together, talking in hushed urgency. Just to have something to do, she pours herself another cup of coffee. She looks down at her mug and realizes that she’s already had five cups today, to satisfy the need to be doing something. She pours her coffee down the drain.
Along with her ever expanding to-do list, Maxine has kept a daily work diary on her personal laptop for the last decade. In it, she tracks everything she’s worked on, how much time she spent on it, any interesting lessons she learned from it, and a list of things to never do again (most recently, “Don’t waste time trying to escape spaces in file names in Makefiles—it’s too difficult. Use anything else instead.”).
She stares at her huge to-do list and her recent work diary entries in disbelief. She has never met a system she couldn’t beat. Is it possible that the utterly mediocre Phoenix Project, totally incapable of anything, is actually defeating me? Over my dead body, she vows silently, then turns back to her work diary entries.
4 p.m.: Waited around for Josh this afternoon who was supposed to walk me through his setup so I can replicate it. He’s dealing with more nightly build problems.
I have a ticket to get access to build server, but was told by security that I need authorization from my manager. Sent email to Randy.
I’m reading every developer design doc I can find, but they’re all starting to look the same to me. I want to see the source code, not read design docs.
4:30 p.m.: In one of the design docs I found the most succinct description of Phoenix: “Project Phoenix will close close the gap with the competition, allowing our customers to do the same things online that they can do in our 900 stores. We will finally have a single view of our customers, so in-store employees can see their preferences and order history, and enable more effective cross-channel promotion.”
The scope of Phoenix is a little frightening. It needs to talk to hundreds of other applications across the enterprise. What could go wrong?
5 p.m.: Calling it a day. Chris stopped by, reminding me to not rock the boat or call too much attention to myself. And to not deploy anything into production.
Yeah, yeah. Sheesh. I can’t even get a build going or log into network shares. How would I be able to push anything into production?
Bored out of my skull. Going home to play with new puppy.
9:30 a.m.: Yes! They’ve given me accounts on a couple more wikis. I’m eager to dig in. This is progress, right?
10 a.m.: Seriously? This is it? I found some QA docs, but this can’t be all of it, right? Where are all the test plans? Where are the automated test scripts?
12 p.m.: Okay, I met William, the QA director. Seems like a nice guy. We were able to meet just long enough to get me user accounts for their network share. Millions of Word docs filled with manual test plans.
I emailed William asking whether I can meet some of his test team. How do they execute all these tests? Seems like they need a small army. And where do they put the test results? I’m on his calendar. In two weeks. Madness.
3 p.m.: I found out where the big daily project stand-up is held: it’s 8 a.m. by the whiteboards. I missed it today, but I won’t miss it tomorrow.
5 p.m.: I’ve gotten so little done in two days. Everything I try to do requires an email, a ticket, or trying to find someone. I’m now resorting to asking them out for coffee. Maybe I’ll get more responses.
10 a.m.: The “15-minute stand-up meeting” went for almost 90 minutes because of all the emergencies. I don’t know how I missed this meeting yesterday—seems hard to miss because of all the yelling. Wow.
OMG. Almost no one else can build Phoenix on their laptops, either. They’re supposed to deploy this into production in TWO WEEKS! (No one is worried. Crazy. They think it will be delayed again.)
If I were in their shoes, I’d be losing my shit. Oh, well.
2 p.m.: I found a bunch of contractor developers brought in two months ago. They can’t do builds, either. Shocking. I took them out to lunch. What a disappointment. They know even less than I do. At least the salad was okay.
I shared everything I know with them, which they were extremely grateful for. Always good to give more than you get—you never know who can help you in the future. Networking matters.
Note to self: I must curb my coffee intake. Must have drunk 7 cups yesterday. This is not good—I think I’m getting heart palpitations.
At 4:45, Maxine packs up her things. There’s no chance anyone will get her anything this late on a Friday.
She almost makes it to the staircase when she runs into Randy.
“Hi, Maxine. Bummer we couldn’t get further on the Dev environment. I escalated a bunch of issues and will make some phone calls before I leave today.”
Maxine shrugs. “Thanks. I hope that lights a fire under some people.”
“Whatever it takes, right?” Randy smiles. “Uh, I have one more thing that I need?”
Uh oh, thinks Maxine. But she says, “Sure, what’s up?”
“Umm. Everyone on the Phoenix Project is required to submit timecards,” Randy says. “We’ve got to show utilization levels or the project management people take our people away. I sent you the link to our time-carding system. Could you fill it out before you leave? It should only take a couple of minutes.”
He looks both ways before whispering, “I especially need your hours because when it comes to budgeting next year, it will help me backfill your position.”
“No problem at all, Randy. I’ll take care of it right now before I leave,” Maxine says agreeably, but she is not happy. She understands the budgeting games that need to be played, but that’s not what bothers her. Instead, it’s that she already knows exactly what she worked on all week, because she keeps such meticulous notes. Absolutely zero real outcomes accomplished. Zilch. Nothing. Nada.
Back at her desk, she logs into the time-carding system. By her name are hundreds of project codes. They’re not the project names. Instead, they’re project codes that all look like airline reservation numbers—ten characters long with capital letters.
Looking at Randy’s email, she copies the project code he gave her (PPX423-94-10) into the field and then dutifully puts in eight hours into each field from Wednesday to Friday and hits submit. She frowns. It won’t let her submit until she describes what she did each day.
Maxine groans. She writes something for each day, basically saying some variant of “Working on Phoenix builds but waiting for entire universe to get me something.” She spends five minutes modifying the text so that each entry is sufficiently different from each other.
It felt bad enough to be sitting on her butt getting so little done this week, despite her very best efforts, but it feels far worse to have to lie about it in writing.
Over the weekend, Maxine continually scans her phone for updates on her tickets, but only sees them being transferred from one person to another. When her husband, Jake, asks her why she is brooding, she refuses to admit that it’s because of the timecard she filled out—it was like rubbing salt in the wound of non-productiveness. She allows herself to be distracted by their new puppy, Waffles, and thoroughly enjoys seeing her kids play with him.
By Monday morning, Maxine has successfully convinced herself to be cheerful, upbeat, and optimistic as she files into the big auditorium for the Town Hall that the CEO of the company, Steve Masters, hosts every other month. She’s enjoyed attending these ever since she joined the company. Her first one made a big impression on her because it was the first time she had seen executives directly address an entire company, taking questions from any of the nearly seven thousand employees.
Steve usually presents with Dick, the CFO. About a year ago, Steve also started co-presenting with Sarah Moulton, the SVP of retail operations. She has profit-and-loss responsibility for retail, the second largest business unit generating over $700 million in revenue per year. While Steve and Dick exude a certain amount of trust and authenticity, Sarah seems less trustable and believable. Last year, she had a different pitch every Town Hall, promising a totally different transformation then what had been presented before, causing lots of confusion, organizational whiplash, and eventually ridicule.
Maxine sees Steve preparing off stage, writing last-minute notes on a folded sheet of paper. Someone hands him a microphone, and he walks onto the stage to polite applause. “Good morning, everyone. Thank you so much for joining us today. This is sixty-sixth Town Hall that I’ve had the privilege of hosting.
“As you know, for almost a century, our mission has been to help our hard-working customers keep their cars running so that they can conduct their daily lives. For most of our customers, that means driving to work so they can collect a paycheck, take their kids to school, and take care of their loved ones. Parts Unlimited helps our customers in many ways. We are one of the world’s most admired manufacturing organizations, making the high-quality and affordable parts that our customers need to keep their cars running. We also have over seven thousand world-class employees directly helping our customers in nearly one thousand stores around this great nation. We are often the only thing that keeps their cars out of expensive service stations.”
Maxine has heard this almost fifty times from Steve at these sessions—it’s obviously important to him to remind everyone who their customers really are. When things go wrong with Maxine’s car, she usually takes it to her car dealership because it’s still under warranty. But the vast majority of their customers don’t have that luxury. Their cars are older, sometimes older than her kids—in fact, their customers may be driving the same make, model, and year of the car she drove as a teenager. They often have little discretionary income. When something goes wrong with their car, it can wipe out whatever savings they have (if any). And when their car is at a repair shop, they not only deplete their savings but they also can’t drive to work. And that means they can’t provide for their families.
Maxine appreciates these reminders about their customers—when engineers think of “the customer” in the abstract instead of as a real person, you rarely get the right outcomes.
Steve continues, “For almost a century, that mission has remained unchanged, although the business environment certainly has. On the manufacturing side, we now have fierce overseas competitors that undercut our prices. On the retail side, our competitors have opened up thousands of stores in the very same markets we serve.
“We are also in a time of incredible economic disruption. Amazon and the other e-commerce giants are reshaping our economy. Some of the most famous retailers that many of us grew up with are going out of business, such as Toys“R”Us, Blockbuster, and Borders. Just down the road here from corporate headquarters, many of us drive past that space where the old Blockbuster used to be, and that space has remained vacant for over a decade.
“We are not immune to this. Our same-store sales are continuing to decline. Many of our customers would rather order their windshield wipers from their phone from someone else than go into one of our physical stores and talk to someone.
“But I believe people don’t want just automotive parts; they want help from people they trust. And that’s why our store associates are so important. That’s why we invest so much in training. And the Phoenix Project will allow us to bring that expertise and trust to our customers in the channels they want to use, whether it’s our physical stores or online.
“Sarah will talk about the progress of the Phoenix Project later, and how it supports the three metrics I care most about: employee engagement, customer satisfaction, and cash flow. If all our employees are excited to come to work each day, and if we’re delighting our customers through constant innovation and great service, cash flow will take care of itself.
“But before we go through our top annual goals, let me first talk about something that is probably on all your minds,” Steve pauses for several moments. “Recently, I sent out an email that Bob Strauss is taking over as chairman of Parts Unlimited. As many of you know, I’ve been here for eleven years, and for the first eight of those, I had the privilege of working for Bob. He was the person who hired me back when I was head of sales at another manufacturer. I’ll always be grateful to Bob for giving me a chance to be COO of this company and for mentoring me over the years. When he retired, I took over for him as CEO and chairman.
“Effective last week, the board of directors has re-appointed Bob to be board chairman,” Steve says, his voice starting to quaver. Maxine watches with amazement as he wipes a tear from his eye. “Of course, I support this move and I look forward to working with Bob again. I’ve asked Bob to come out and share some words with us and tell us what it means for the company.”
Until this moment, Maxine hadn’t realized how much of a setback this was for Steve. She had heard that it was a demotion, but to be honest, she didn’t really understand or care much about these types of changes at the executive level. Executives came and went, often without much impact on her and her daily work. But she’s riveted by the drama unfolding in front of her.
A slightly stooped older man with white hair and a wry smile walks on stage and stands next to Steve.
“Hi, everyone. It’s great to be here in front of you after so many years. I even see some familiar faces, which makes me very happy. For those of you who don’t know me, my name is Bob Strauss. I was CEO of this company for fifteen years, back when dinosaurs roamed the earth. And even before that, I was an employee at this great company for nearly thirty years. As Steve mentioned, it was with great hope and pride that I recruited him away from another company many years ago.
“Since retiring, I’ve continued to serve on the board of directors. The job of the board is very simple: to represent the interests of the company shareholders, which includes almost all of you. We want to ensure that the company’s future is secure. If you have a pension or have been part of our employee retirement stock purchase plan, this is probably as important to you as it is to me.
“We do this primarily by keeping company executives accountable, by hiring, and, umm, occasionally firing the CEO,” he says plainly. Maxine’s breath catches slightly—until that moment, Bob seemed like a friendly grandfather. Apparently, he has a stricter side.
“Just by looking at the stock price, you know that the markets don’t think we are performing as well as we should be. When our company’s stock price goes down while our competitors’ shares are going up, something has to change.
“I like to think that companies have two modes of operations: peacetime and wartime. Peacetime is when things are going well. This is when we are growing as a company and can continue business as usual. During these times, the CEO is often also the board chairman. However, wartime is when the company is in crisis, when it is shrinking or at risk of disappearing entirely, like what’s happening to us now.
“During wartime, it’s about finding ways to avoid extinction. And during wartime, the board will often split the roles of CEO and chairman.” Bob pauses, squinting into the bright lights, looking across the entirely silent audience. “I want everyone to know that I have complete confidence in Steve and his leadership. And if all goes well, we’ll figure out how to get him the chairmanship again so I can go back into retirement where I belong.” The crowd laughs nervously as Bob waves and makes his exit.
Steve steps up to the front of the stage and says, “Could everyone give a round of applause for Bob Strauss?”
After a muted applause, Steve resumes, “The company goals this year were to stabilize our business. Our manufacturing operation makes up two-thirds of our revenue, which has remained flat but still profitable. This has been the mainstay of our business for almost a century, and we’ve been able to fend off our very fierce Asian competitors.
“However, our retail operations continue to underperform. Our revenue is nearly five percent lower than last year,” he says. “Our biggest quarter is still coming up, so there’s hope. But hope alone is not a strategy, and you can see how Wall Street has reacted to our performance so far. However, I remain confident that the Phoenix Project will help us adapt to these new market conditions.
“So, without further ado, I’ll turn it over to Sarah Moulton, our SVP of retail operations, to describe why the Phoenix Project is so important to the future of the company.”
Sarah walks onto the stage wearing a strikingly beautiful royal blue business suit. Whatever Maxine’s opinion of Sarah, she grants that Sarah always looks fabulous. In fact, she would look right at home on the cover of Fortune—intelligent, aggressive, and ambitious.
“As Steve and Bob mentioned,” Sarah begins, “we are in a time of incredible digital disruption in retail. Even our customers order online and through their phones. The goal of the Phoenix Project is to enable our customers to order however they want, whether it be online, in our stores, or even through our channel partners. And wherever they order, they should be able to have their product delivered to their homes or to pick it up in one of stores.
“This is what we’ve been trying to do for years. Right now, our stores are still in the dark ages. That was Parts Unlimited 1.0. The Phoenix Project will create Parts Unlimited 2.0. There are so many efficiencies we can create to help us compete against the e-commerce giants, but we must innovate and be agile. In order to remain relevant, people need to view us as a market leader creating new business models—what worked for our first century may not work for our second.”
As always, there’s some validity to what Sarah says, Maxine grudgingly acknowledges, but she can be so condescending.
“The Phoenix Project is the most important initiative for our company, and we’re betting our survival on it. We’ve spent nearly $20 million on this project over three years, and customers still haven’t seen any value,” she continues. “I’ve decided that it’s time for us to finally get in the game. We will be launching the Phoenix Project later this month. No more delays. No more postponements.”
Maxine hears an audible gasp from the whole audience and a loud buzz of urgent murmuring. Sarah continues, “This will finally give us parity with the competition, and we will be poised to regain market share.”
Maxine sighs in frustration. She understands Sarah’s urgency, but it doesn’t change the fact that there are over a hundred developers who are nowhere near as productive as they should be, struggling to perform routine builds, spending too much time in meetings, or waiting for things they need. Sarah’s speech sounds like listening to a general tell you how important winning the war is and then finding out that all the soldiers have been stuck in port for three years.
On the other hand, at least Sarah hasn’t pitched something completely new today.
Steve thanks Sarah and then quickly reviews the company financials, and an injury that happened in one of the manufacturing plants last month. He talks about Hannah, who had her finger crushed by a stamping machine, and how they’ve replaced that machine with one that has a sensor preventing the plates from closing when anyone is the danger area. He applauds the team for not waiting for budget to act on this, “Remember, safety is a precondition of work.”
Maxine loves these report-outs and has always been impressed and emotionally moved by how much Steve cares about employee safety.
He says, “That almost concludes our report-out. We have about fifteen minutes for questions and answers.”
Maxine’s attention wanders as people ask Steve questions about the revenue forecast, the performance of the physical stores, the recent issues in manufacturing … But when someone asks about the payroll outage, she’s jolted alert before shrinking back into her seat while straining to hear every word.
“I apologize to everyone who was affected by this,” Steve replies. “I understand how disruptive this was to everyone, and rest assured that we’ve taken very specific actions to make sure that it never happens again. It was a combination of technical problems and human error, and we think we’ve remedied both of them.”
Maxine closes her eyes, feeling her cheeks turn bright red, hoping no one is looking at her. She can’t see how her exile to the Phoenix Project could possibly be considered a remedy.