Pieces Long-Term Memory agent: interview with Pieces CEO, CAIO, & ML Engineer

To celebrate our latest Product Hunt launch (we got #1 Product of the day 🥳), I sat down with Pieces CEO and technical co-founder Tsavo Knott, Chief AI Officer Sam Jones, and ML Engineer Kieran McGimsey to break down what makes Pieces Long-Term Memory Agent (LTM-2) different from previous versions of LTM, how it actually helps developers, and what's coming next.

Let's get into the interview. 

What sets Pieces apart from other AI tools?

Sam highlights how Pieces differentiates itself from other tools by "automatically contextualizing your chat," "storing and resurfacing items from your past workflow," and "allowing you to choose a range of on and offline models to power our features."

The bottom line? 

The majority of AI tools start from scratch every time you use them. 

You explain the same context over and over (lots of copy and pasting!). Pieces doesn't need you to do that. It will remember what you've been working on and automatically bring that context forward (in the Workstream Activity view). 

This saves you time and keeps you focused on what matters, which is your work, not reexplaining your situation to the LLM.

Kieran points out another key aspect:

"LLM chats are the norm now but they are a bit fatiguing due to needing to write a semantic message and navigating around hallucinations and inconsistencies in responses. Having these more deterministic, manual interactions with LTM (like searching through a timeline of past work or pulling out specific text/links/resources) will help keep users in their flow state."

What Kieran's talking about is escaping the chat-only interactions that are commonly seen in tools like ChatGPT and Claude. 

We've all felt that frustration of writing the perfect prompt only to get a response that is completely irrelevant.

Pieces solves this (or at least tries its best to) by letting you directly access your work history. 

Need that GitHub link from Tuesday?

You can just grab it from your Workstream Activities, even when using GitHub with Pieces.

The goal is to keep your work organized and accessible, so you can spend more time building rather than prompt engineering.

How does Pieces capture and resurface workflow details?

According to Sam:

"we are very selective about what is added to the LTM. By analyzing where the user's focus is and how it relates to the big picture of their workstream, we can prevent a lot of corruption at the source."

Pieces only pulls in data from where you're focused, using native tooling. Here's how it works:

• It builds a local database of text extracted from your workflow.

• It generates periodic summaries of that data, which you can reference, share, or even chat with

• When you ask the copilot a question, Pieces checks if any past summaries or events are relevant and surfaces that context in the chat.

Kieran adds,

"Long-term memory is now accessible outside of chat responses. Users can browse, search, and interact with LTM data in a dedicated memories view. Seeing the memories grow will encourage users to keep the LTM engine on, even if they rarely chat with the context."

That means if you ask something related to a past project, Pieces can automatically connect the dots—without you needing to remember specific keywords or filenames.

How can users interact with their stored memories?

One of the standout features of LTM-2 is the ability to interact with stored memories beyond chat responses. Users can:

• Explore a timeline of past work: Seeing days, weeks, or even months of historic data encourages users to keep LTM on, as they can visually track their workflow.

• Extract specific details: Users can pull out URLs, message drafts, or snippets from their timeline without needing to phrase a full chat prompt.

• Initialize chat from stored summaries: Instead of always starting a conversation from scratch, users can launch a chat from a summary, automatically setting relevant time ranges for a more stable and accurate response.

This provides an alternative to traditional AI chat interactions, reducing fatigue and making it easier to access relevant past work.

How does Pieces know what to remember?

Pieces determines what's worth keeping based on "what the user is focusing on and how that focus changes to infer what information is useful to retain."

That information then gets surfaced in two main ways:

• "By an agent to generate summaries."

• "Via RAG (Retrieval-Augmented Generation) within the copilot."

Kieran suggests another potential approach (something that could come out in future releases): 

"Summaries could have a user-defined intention. (For example) The user could set a seed prompt in the timeline like 'keep track of all GitHub issues I interact with today' or 'take notes on all machine learning papers I read this week' and we'll maintain a dedicated summary for this prompt through the day, focused on the intention they set.

These intentions could also be inherited from a PM tool like Jira where we use their active tickets as seeds and try to aggregate documentation related to resolving those tickets. Down the line, we're going to try to infer these intentions ourselves, but giving more control to the user lets them tailor Pieces to their unique needs."

How do you deal with the issue of memory contamination?

One of the biggest challenges with long-term memory systems is preventing memory contamination (when unrelated contexts blend together), leading to incorrect or misleading information.

Sam mentions that "the issue of memory contamination is complex and a core challenge in designing features like this one."

Pieces deals with this problem at three levels:

• On entry: "We are very selective about what is added to the LTM. By analyzing where the user's focus is and how it relates to the big picture of their workstream, we can prevent a lot of corruption at the source."

• On roll-up: "When we roll up memories into periodic summaries, our agent looks for narratives and themes across workflow elements. When we find contradictions, we resolve them by comparing those narratives to cut out random chatter and keep focus on core tasks."

• At query time: "When you interact with your workstream data, through the copilot or the summaries, those interactions are used to infer which aspects are useful/truthful and which are not. This allows us to elevate quality information while demoting the noise."

Pieces also periodically cleans stored memories to remove contamination, making it "much more resistant to context corruption than other solutions out there."

What about privacy & security?

Pieces follows a local-first approach, meaning everything it captures is processed and stored directly on your device. Sensitive details are automatically filtered out, and if you're using a local LLM like Mistral, your data never leaves your machine.

Sam notes that "data is stored locally on the user's machine," giving users full control over their information. Additionally, "users have a range of options to power their experience, from fully local through to fully in the cloud."

For more info on how data is handled, check out the privacy page in our docs.

Can users control what Pieces captures?

Yes. "Using the allow/deny list functionality," in the Long-Term Memory access control, users can specify exactly what Pieces can see and store.

How does Pieces manage performance and system resources?

One of the biggest concerns with AI tools running locally is performance. 

Pieces handles this by using "small, highly optimized embedding models that execute well in low-resource settings." Additionally, "we use native OS-level functionality to tap into efficient OS native code."

What this means for you: Pieces can run relatively smoothly even on older machines without draining your battery. 🤝

What's next for Pieces?

According to Tsavo, "We're continually working to enhance the experience, making it even more magical and, importantly, extending it to teams and formalizing the product (and brand) beyond just developers."

He adds,

"LTM-2 was actually designed from the start to benefit everyone, and technically, digital workers at large can use it today and leverage the power of artificial memory in their own unique workflows."

Looking ahead, LTM-2.5 will bring major upgrades to retrieval and memory navigation. 

"You'll experience intuitive, dynamic summary generation based not on fixed intervals, but tailored around topics, tags, specific time ranges, and more". It will also make "effortless sharing and retrieval of memories across individuals and teams" much more seamless. He notes that "we're roughly six weeks out from the official release of LTM-2.5 and can't wait to hear your thoughts!"

But it doesn't stop there – LTM-3 is already in development and will take long-term memory even further.

"LTM-3 will push the boundaries even further, focusing on extremely deep recall capabilities—perfect when the exact timing or source of context is unknown”.

Tsavo makes it very clear that Pieces is moving fast: "Anyway, know that we're hustlin' & stay tuned 🙌"

Final thoughts

LTM-2 is changing how we work with information. Period.

Developers spend countless hours trying to remember where they saw that solution, why they made that decision, or what that random function actually does. Those days are over.

With Pieces, your workflow gets its own memory. Your context stays intact. Your focus remains unbroken.

And this is just the beginning. LTM-2.5 and LTM-3 are right around the corner, promising even more powerful ways to capture and recall what matters in your work.

The technology to augment your memory is here. The real question is: how much more productive would you be if you had an AI that remembers everything?