Huey
December 27, 2024

charity.wtf

On Versioning Observabilities (1.0, 2.0, 3.0…10.0?!?)

Hazel Weakly, you little troublemaker.  As I whined to Hazel over text, after she sweetly sent me a preview draft of her post: “PLEASE don’t post this! I feel like I spend all my time trying to help bring clarity and context to what’s happening in the market, and this is NOT HELPING. Do you know how hard it is to try and socialize shared language around complex sociotechnical topics? Talking about ‘observability 3.0’ is just going to confuse everyone.”...

7 days ago

charity.wtf

“Founder Mode” and the Art of Mythmaking

I’ve never been good at “hot takes”. Anyone who knows anything about marketing can tell you that the best time to share your opinion about something is when everyone is all worked up about it. Hot topics drive clicks and eyeballs and attention en masse. Unfortunately, my internal combustion engine doesn’t run that way. If anything, my fuel runs the other way. If everybody’s already buzzing about something, I feel like chances are, everything that needs to be said is...

10 days ago

charity.wtf

There Is Only One Key Difference Between Observability 1.0 and 2.0

Originally posted on the Honeycomb blog on November 19th, 2024 We’ve been talking about observability 2.0 a lot lately; what it means for telemetry and instrumentation, its practices and sociotechnical implications, and the dramatically different shape of its cost model. With all of these details swimming about, I’m afraid we’re already starting to lose sight of what matters. The distinction between observability 1.0 and observability 2.0 is not a laundry list, it’s not marketing speak, and it’s not that complicated or hard to understand. The distinction is a technical one,...

about 1 month ago

charity.wtf

How Hard Should Your Employer Work To Retain You?

Recently we learned that Google spent $2.7 billion to re-hire a single AI researcher who had left to start his own company. As Charlie Brown would say: “Good grief.” This is an (incredibly!) extreme example. But back in the halcyon days of the zero interest rate phenomenon (ZIRP), smaller versions of this tale played out daily. Many rank-and-file engineers have stories about submitting their resignation, or threatening to quit, and their managers plying them with stock or cash or promotions...

3 months ago

charity.wtf

Is It Time To Version Observability? (Signs Point To Yes)

Augh! I am so behind on so much writing, I’m even behind on writing shit that I need to reference in order to write other pieces of writing. Like this one. So we’re just gonna do this quick and dirty on the personal blog, and not bother bringing it up to the editorial standards of…anyone else’s sites. If you’d rather consume these ideas in other ways: I gave a keynote at SRECon in March Here is a slide deck of...

5 months ago

charity.wtf

Pragmatism, Neutrality and Leadership

Every year or so, some tech CEO does something massively stupid, like declaring “No politics at work!”, or “Trump voters are oppressed and live in fear!”, and we all get a good pained laugh over how out of touch and lacking in self-awareness they are. We hear a lot about the howlers, and much less about the practical challenges leaders face in trying to create a work environment where people from vastly different backgrounds and belief systems come together in...

5 months ago

charity.wtf

Generative AI is not going to build your engineering team for you

Originally posted on the Stack Overflow blog on June 10th, 2024 When I was 19 years old, I dropped out of college and moved to San Francisco. I had a job offer in hand to be a Unix sysadmin for Taos Consulting. However, before my first day of work I was lured away to a startup in the city, where I worked as a software engineer on mail subsystems. I never questioned whether or not I could find work. Jobs...

7 months ago

charity.wtf

The Cost Crisis in Observability Tooling

Originally posted on the Honeycomb blog on January 24th, 2024 The cost of services is on everybody’s mind right now, with interest rates rising, economic growth slowing, and organizational budgets increasingly feeling the pinch. But I hear a special edge in people’s voices when it comes to their observability bill, and I don’t think it’s just about the cost of goods sold. I think it’s because people are beginning to correctly intuit that the value they get out of their...

11 months ago

charity.wtf

Questionable Advice: “My boss says we don’t need any engineering managers. Is he right?”

I recently joined a startup to run an engineering org of about 40 engineers. My title is VP Engineering. However, I have been having lots of ongoing conflict with the CEO (a former engineer) around whether or not I am allowed to have or hire any dedicated engineering managers. Right now, the engineers are clustered into small teams of 3-4, each of which has a lead engineer — someone who leads the group, but whose primary responsibility is still writing...

12 months ago

charity.wtf

Becoming An Engineering Manager Can Make You Better At Life And Relationships

Original title: “Why Should You (Or Anyone) Become An Engineering Manager?” The first piece I ever wrote about engineering management, The Engineer/Manager Pendulum, was written as a love letter to a friend of mine who was unhappy at work. He was an engineering director at a large and fast-growing startup, where he had substantially built out the entire infrastructure org, but he really missed being an engineer and building things. He wasn’t getting a lot of satisfaction out of his...

about 1 year ago

charity.wtf

LLMs Demand Observability-Driven Development

Originally posted on the Honeycomb blog on September 20th, 2023 Our industry is in the early days of an explosion in software using LLMs, as well as (separately, but relatedly) a revolution in how engineers write and run code, thanks to generative AI. Many software engineers are encountering LLMs for the very first time, while many ML engineers are being exposed directly to production systems for the very first time. Both types of engineers are finding themselves plunged into a...

over 1 year ago

charity.wtf

How to Communicate When Trust Is Low (Without Digging Yourself Into A Deeper Hole)

This is based on an internal quip doc I wrote up about careful communication in the context of rebuilding trust. I got a couple requests to turn it into a blog post for sharing purposes; here you go. In this doc I mention Christine, my wonderful, brilliant cofounder and CEO, and the time (years ago) when our relationship had broken down completely, forcing us to rebuild our trust from the ground up. (Cofounder relationships can be hard. They are a...

over 1 year ago

charity.wtf

Helicopter Management and Other Mistakes

You are a freshly minted manager. You come full of rage and frustration at the poor management you’ve endured and witnessed in tech, and you are god damn determined not to repeat all of those mistakes. You are tired of reporting to a manager who isn’t transparent with you, who hoards critical information and isn’t forthcoming about changes that impact you. You are tired of not being listened to or treated like a cog, so you swear to really listen...

over 1 year ago

charity.wtf

Questionable Advice: “How can I drive change and influence teams…without power?”

Last month I got to attend GOTO Chicago and give a talk about continuous deployment and high-performing teams. Honestly I did a terrible job, and I’m not being modest. I had just rolled off a delayed redeye flight; I realized partway through that I had the wrong slides loaded, and my laptop screen was flashing throughout the talk, which was horribly distracting and means I couldn’t read the speaker notes or see which slide was next. Argh! Anyway, shit happens....

over 1 year ago

charity.wtf

Choose Boring Technology Culture

Honeycomb recently announced our $50M Series D funding round. We aren’t the type to hype this a lot; Emily summed it up crisply as, “Living another day on someone else’s money isn’t business success, even though it is a lovely vote of confidence.” Agreed. The vote of confidence does mean more than usual, given the dire state of VC funding these days, but…raising money is not success. Building a viable, sustainable company is success. Whenever we are talking to investors,...

over 1 year ago

charity.wtf

Questionable Advice: “People Used To Take Me Seriously. Then I Became A Software Vendor”

I recently got a plaintive text message from my magnificent friend Abby Bangser, asking about a conversation we had several years ago: “Hey, I’ve got a question for you. A long time ago I remember you talking about what an adjustment it was becoming a vendor, how all of a sudden people would just discard your opinion and your expertise without even listening. And that it was SUPER ANNOYING. I’m now experiencing something similar. Did you ever find any good...

over 1 year ago

charity.wtf

Architects, Anti-Patterns, and Organizational Fuckery

I recently wrote a twitter thread on the proper role of architects, or as I put it, tongue-in-cheek-ily, whether or not architect is a “bullshit role”. I don't know if this is a troll or not, but with all apologies to my many wonderful, highly skilled "Architect" friends, I tend to think it's a bullshit role. I believe that only the people building software systems get to have opinions on how those systems get built. https://t.co/aUShYfyIKY— Charity Majors (@mipsytipsy) February...

almost 2 years ago

charity.wtf

Deploys Are The ✨WRONG✨ Way To Change User Experience

This piece was first published on the honeycomb.io blog on 2023-03-08. …. I’m no stranger to ranting about deploys. But there’s one thing I haven’t sufficiently ranted about yet, which is this: Deploying software is a terrible, horrible, no good, very bad way to go about the process of changing user-facing code. It sucks even if you have excellent, fast, fully automated deploys (which most of you do not). Relying on deploys to change user experience is a problem because it fundamentally confuses and scrambles up two very...

almost 2 years ago

charity.wtf

How to Throw A Company Offsite In A “Post-COVID” World

Earlier this month we had our first Honeycomb all-hands offsite in three years … our first one since February of 2020, before the plague. It was wonderful and glorious and silly and energizing and so, so SO much fun. It was a potent reminder of the reality that no virtual activity can compare with the energy of being physically present with people you care about. I was talking with Paul Biggar last month, telling him about all the things we...

almost 2 years ago

charity.wtf

Every Achievement Has A Denominator

One of the classic failure modes of management is the empire-builder — the managers who measure their own status, rank or value by the number of teams and people “under” them. Everyone knows you aren’t supposed to do this, but most of us secretly, sheepishly do it anyway to some extent. After all, it’s not untrue — the more teams and people that roll up to you, the wider your influence and the more impact you have on more people, by...

about 2 years ago

charity.wtf

The Future of Ops is Platform Engineering

First published on 2022-09-30 at https://www.honeycomb.io/blog/future-ops-platform-engineering. Two years ago I wrote a piece in The New Stack about the Future of Ops Careers. Towards the end, I wrote: The reality is that jack-of-all-trades systems infrastructure jobs are slowly vanishing: the world doesn’t need thousands of people who can expertly tune postfix, SpamAssassin, and ClamAV—the world has Gmail. (…) Building infrastructure and operational expertise used to be bundled together into a single role. But the industry is now bifurcating along an infrastructure...

about 2 years ago

charity.wtf

The Hierarchy Is Bullshit (And Bad For Business)

My friend Molly has had an impressive career. She got a job as a software engineer after graduating from college, and after kicking ass for a year or so she was offered a promotion to management, which she accepted with relish. Molly was smart, driven, and fiercely ambitious, so she swiftly clambered up the ranks to hold director, VP, and other shiny leadership roles. It took two decades, an IPO and a vicious case of burnout before she allowed herself...

over 2 years ago

charity.wtf

Rituals for Engineering Teams

Last weekend I happened to pick up a book called “Rituals For Work: 50 Ways To Create Engagement, Shared Purpose, And A Culture That Can Adapt To Change.” It’s a super quick read, more comic book than textbook, but I liked it. It got me thinking about the many rituals I have initiated and/or participated in over the course of my career. Of course, I never thought of them as such — I thought of them as “having fun at...

over 2 years ago

charity.wtf

Live Your Best Life With Structured Events

If you’re like most of us, you learned to debug as a baby engineer by way of printf(3). By the time you were shipping code to production you had probably learned to instrument your code with a real metrics library. Maybe a tenth of us learned to use gdb and still step through functions on the regular. (I said maybe.) Printing stuff to stdout is still the Swiss Army knife of tools. Always there when you reach for it, usually...

over 2 years ago