Ep. 05 - Phillip Carter, Principal PM, Honeycomb | Can AI Improve Observability?
AI in observability: What works, what doesn’t. Phillip Carter shares where humans still lead and where AI fits in.
Jim Bennett is head of developer advocacy at Pieces, focusing on enabling developers to be more productive by leveraging contextual awareness of not only the code they write but the content they read and the conversations they have.
Phillip Carter is a Principal Product Manager, at Honeycomb who focusses on making the developer experience great. Lead for AI-related efforts, creating the industry's first LLM-assisted data querying tool for Observability. Helps driving Honeycomb's OpenTelemetry focus, the Open Source presence, and several other developer tool integrations.
🎧 Listen on Spotify and Apple Podcasts
Insights
🟢 Phillip reflects on his initial skepticism and optimism around AI's role in coding back in 2022, especially its potential in data pattern recognition.
🟢 AI shows promise in spotting patterns in observability data and could eventually help preemptively identify incidents – but it’s not quite there yet.
🟢 AI tools can assist developers in adding observability features like logging and metrics, especially in legacy systems, making older codebases more transparent.
🟢 AI struggles to understand critical business context, like cost constraints and tradeoffs. This makes human judgment irreplaceable in many observability decisions.
🟢 Unlike tools like Pieces, most AI dev tools don’t integrate collaboration context (e.g., why a decision was made), which can lead to misguided code suggestions.
🟢 There’s potential for AI agents to enforce observability best practices – but they'll need far more context-awareness to do it reliably.
🟢 If AI replaced entry-level devs, we’d lose the pipeline of future senior engineers. The mentorship and growth path remains critical in dev teams.
Timeline
0:00:00 - Intro
0:03:38 - Phillips initial thoughts in AI in coding from 2022
0:04:44 - Can I help spot patterns in observability data
0:13:12 - A future where AI can preemptively find incidents
0:19:35 - How can AI developer tools help add observability to your code?
0:23:51 - AI is great for adding decent logging and metrics to legacy apps
0:26:43 - AI doesn’t understand your business context and constraints like cost
0:28:22 - AI developer tools lack context of decisions in collaboration tools (unlike Pieces!)
0:31:35 - Could AI agents ensure observability is included in your code?
0:32:46 - Humans are more important than AI
0:37:42 - Adding code to Honeycomb that AI would fail to do right
0:42:34 - AI cannot replace junior engineers because where would senior engineers come from?
