In this episode of 'The Next Next,' host Jason Jacobs interviews Sara Remsen, CEO and co-founder of Melodi. They discuss Melodi's mission to assist AI companies by enhancing their models with human feedback and delve into Sara's background in product design at PTC and MIT. The conversation covers the rise of AI agents, the tools available for building them, and the complexities involved in understanding and improving end-customer experiences. They explore Melodi's current position, its use of AI in product development and market strategy, and the broader implications of AI in various industries. Sara highlights the importance of operational processes and domain expertise as competitive advantages in the rapidly evolving AI landscape. They also touch on the challenges and opportunities in building AI-driven businesses and the future direction of AI in both commercial applications and daily life.
Fostering AI Adoption: A Deep Dive with Melodi's Sara Remsen In this episode of The Next Next, host Jason Jacobs talks with Sara Remsen, CEO and co-founder of Melodi, a startup helping AI companies improve their models with human feedback. They discuss Melodi's journey, AI's current transformative state, and its potential opportunities and hype. The conversation covers the rise of AI agents, the tools available for building them, the pros and cons of those tools, and the challenges companies face in developing in-house agents. Sara explains Melodi's role in measuring digital resolution rates for AI-powered chatbots, detailing their approach to improving customer interaction insights. The discussion also touches on Melodi's origin story and developmental journey, the impact of generative AI on traditional product feedback methods, and the future of AI integration in businesses. Towards the end, Sara shares practical advice for founders and established companies on incorporating AI thoughtfully and strategically.
00:00 Introduction to Sara Remsen and Melodi
00:20 Overview of the Episode's Topics
01:24 Introduction to The Next Next Show
02:03 Sara Remsen Joins the Conversation
02:09 Challenges in Podcast Recording
02:29 Sara's Background and Connection with Jason
03:31 Founding Melodi and Its Mission
03:40 Understanding Melodi's Functionality
04:38 The Origin Story of Melodi
08:05 Exploring RAG Systems
14:52 Challenges and Opportunities with AI Agents
24:53 The Future of AI and Digital Workforces
27:07 Prioritizing Tool Sets and Tasks
27:23 Melodi's Integration and Functionality
28:22 Task Agnosticism and Monitoring
28:46 Criteria for Melodi's Market Fit
29:12 AI Maturity and Company Size
29:31 Melodi's Ideal Customer Profile
36:31 Challenges in the AI Market
37:52 Future of Software and AI
43:56 Melodi's Sales and Customer Acquisition
45:42 Incorporating AI in Operations
49:16 Advice for Founders and Companies
52:56 Melodi's Future Plans
54:23 Creative Uses of AI in Daily Life
55:09 Conclusion and Farewell
Jason Jacobs: Today on The Next Next, our guest is Sara Remsen, CEO and co founder of Melodi. Melodi is a venture backed startup that assists AI companies in enhancing their models through human feedback. Before founding Melodi, Sara led product design at PTC, where she was VP of product experience for Vuforia Augmented Reality.
We have a great discussion in this episode and gosh, we cover a lot of ground about AI, the transformative power of it, where it is on the journey today, how much of it is hype and where the real opportunities lie. We talk about the rise of agents and the tools that are available to build these agents, the pros and cons of those tools and why a company might choose to build agents in house.
And we also talk about some of the challenges that companies who do decide to build these agents in house come across in terms of Understanding the end customer, what kind of experience they're having, and how to improve that experience over time, and that's exactly why Melodi exists. We then talk about where Melodi is in the journey, and how much they've been using AI to build their own product, how much they're planning to use it when they get more aggressive at building out a sales force and going to market, and we also talk about how they're incorporating it into their offerings, what kinds of customers they're serving, where they're getting the most traction, etc.
I don't want to spoil the whole discussion, so why don't we get into it. But before we get started.
I'm Jason Jacobs, and this is The Next Next, it's not really a show. It's more of a learning journey to explore how founders can build ambitious companies while being present for family and not compromising flexibility and control, and also how emerging AI tools can assist with that. Each week, we bring on guests who are at the tip of the spear on redefining how ambitious companies get built.
And selfishly, the goal is for this to help me better understand how to do that myself. While bringing all of you along for the ride. Not sure where this is going to go, but it's going to be fun.
Okay, Sara Remsen, welcome to the show.
Sara Remsen: Thanks so much for having me. I'm excited to be here
Jason Jacobs: Thanks for coming. I feel so bad that you went and booked this podcast studio at Staples and expected it to have all the equipment there and it didn't have the equipment. And then you went and bought headphones and then, or a mic, and then you can't get the mic to work. So now you're just on your AirPods, but here we are.
Sara Remsen: with technology that's been working pretty well for a few years. So I think we're good.
Jason Jacobs: It's funny. I. I reached out to you cold actually because I saw that you put on an AI event here in Boston and it looked like a good event with a great attendees and format and I wish I had gone actually and and then I looked more into Melodi and it looks super relevant to the kinds of things I'm learning about so I reached out and it turns out we've met before but only when you were focused on climate stuff.
Sara Remsen: Yes. Yes. No, I've been a fan of your MCJ podcast forever when I was exploring business ideas in the carbon market. And yeah, we met at that dinner. And so I was really excited to hear that you're working on AI and continuing your journey, similar to what you did for climate now for AI. And I know For me at the time, it was really formative and so I hope that this can be helpful for people who are interested in AI and trying to get into it and figuring out how it all works and how they fit in.
So I'm excited. I'm excited to be here.
Jason Jacobs: We'll see. Yeah, as we were talking about before recording, it's early days for the show. I know I'm having a blast and having great and building great relationships and learning a ton from from the show. Yeah. On the content side, the audience is growing, but it's still, it's early days.
We'll see what happens, but. But jumping into it, maybe just start by framing the discussion. What is Melodi and how did it come about?
Sara Remsen: Yeah. So Melodi is essentially a tool that product and operations teams can use to measure their digital resolution rate for AI agents. So in this new world where we have agents instead of apps, a lot of those agents are effectively chatbots that are responsible for answering questions about Different parts of the business, whether that's product questions, sales questions, customer success questions, and all of those questions come from large knowledge bases, whether that's, documentation, FAQs training sets, things like that.
And so what Melodi does is help those teams understand what people are asking and how well the AI assistant is responding to those questions. So those teams can then report back to, executives and other. People in a company that are making these big investments and big bets on AI that their investments are actually working.
Jason Jacobs: Huh. And I have a ton of questions about that. But before we dive more into Melodi maybe just talk a bit about the origin story. When did it come about? How did it come about? Why did it come about?
Sara Remsen: Oh, yes. I have been, I've been working in human centered design my entire career. I've been really focused on building great products that provide real value to users. And when I was at MIT for grad school, I did some work with IDEO. I was at the MIT Media Lab, and I was So I'm really experimenting with a lot of different ways that emerging technology can solve existing problems.
So at the time, it was augmented reality and spatial computing, but I see a lot of the same parallels now with AI. And so after I left my last job. I was exploring ideas in the climate space. I got connected with my co founder, Kevin, who was the former head of hub spot. And he was actually 1 of the 1st product managers to build products at hub spot, which then were shipped out to millions scaled out to 1000s of customers.
And so he really saw 1st hand. All of the challenges of building a great product and building something that customers really needed. And so we teamed up because I was really excited about the opportunity for AI to make technology useful again, for people to solve real problems in a way that haven't, really like we haven't seen, like everything looks the same.
All the UIs look the same. All the tools look the same. I think there's a big stepwise function for. What AI technology can do for a lot of people. So Kevin and I teamed up. We brought in our third co-founder, who incidentally was the best friend of my former co-founder and CTO. I knew Greg through Varun.
Part of it, I guess part of the story there is that when I was at mi at, when I was at MIT, I started my first company called Waypoint. We worked on that for a few years. It was specifically around. Augmented reality workforce training, so all about capturing expertise and knowledge from the heads of frontline workers, people who worked in factories and transforming that into documentation for compliance and training.
So anyway, that's a long winded way of saying I got connected with Kevin and Greg. We were really excited about. The opportunity to help teams build more helpful and we're drawing a lot on a lot of concrete experience from our careers. And, the last almost 2 years now of working really closely with teams that are at the forefront of this.
Jason Jacobs: And when you initially got to chatting with Kevin about the opportunity, what was the initial germ of an idea? Was there pain that you were observing that people were feeling? Were there opportunities that you saw that were coming? Like how like what was the first foot forward, if you will, or the first step?
Sara Remsen: Like all good AI companies, we started out as a carbon company. So when we were, when I was exploring ideas in climate and in the carbon space, Kevin and I first got together because we were really excited about providing transparency to carbon credits, using machine learning to understand all of this information that's in these massive reports for carbon projects.
And. I have many thoughts about the carbon market, and there's many reasons why we didn't pursue that as a business, but we realized that with. With generative AI, people were turning to these rag systems retrieval augmented generation. And so it was really and as we were starting to build one of those ourselves, we realized that there was no good way to get the information that you needed about whether it was working.
And once, Kevin, especially because he'd lived this as a PM at HubSpot, once you reach a certain scale and you go beyond. Demo into production when you're out in the wild and you have incredibly high scale usage there's no good way to understand what people are trying to do where they're having success where they're struggling and We knew that we could surface those insights and performance metrics to teams
Jason Jacobs: So I'm going to take a stab at explaining RAG because someone just explained it to me the other day. I think it was yesterday even. So it sounds like you've got these LLMs and then you've got the the end user. And if the end user has a, or I guess it could be an agent as, as well. But you have either an agent or a human that makes a request of the LLM and the RAG sits in the middle and says, before we're going to send the request to the LLM.
Or in addition to sending the request to the LLM, we're going to complement that with anything generally available that we can find on our own that's outside of the LLM's purview? Is that right?
Sara Remsen: yeah, the way that I think about rag and I'm a super visual person so I'll do my best to explain this is like Rag is imagine if you have a knowledge base for your company, like, how to use different products. Let's take this microphone that I just thought from staples. Like, how do I use this microphone?
All of the information about that microphone and about my business are stars in space. They're like constellations in every direction. So over here in 1 galaxy, you have information about how to set it up. Over here in this galaxy, you have information about how to troubleshoot it. And what RAG does is I ask a question, how come my mic isn't working?
And it basically goes and looks throughout your galaxy or your solar system for the information that might be most relevant in answering that question. So that then when it does answer your question, it is pulling from a specific source the way that it finds that I think is really interesting. That's like how embeddings work.
And there's a lot of interesting techniques that teams are doing in order to make that process more efficient and more effective.
Jason Jacobs: So are there tools that provide RAG, or is that something that gets built in house, or how where does it come from?
Sara Remsen: Yeah, tools like OpenAI have in house RAG systems. You can just upload files as part of their API playground and get a RAG chatbot pretty much out of the box. What we're seeing teams do though is more sophisticated, which is using a specific inventings model, making sure that they have all the data and the right sources that are referenced in there, building and scaffolding a pipeline that, Has LLM calls along the way, but is often more complex than just help me troubleshoot this.
It's often based on what What product line the person is talking about or where even in the flow of a conversation that question comes up. I think what's what I think is really interesting about this right now is that. I think that domain expertise and operational know how will become the new flywheel for companies.
So everyone used to think that it was going to be data and it was going to be labeled data and the more data you had, the more successful you were going to be. But now I think the ubiquity and quality of these general purpose LLMs make it so that they can do almost Anything and that you could do with labeled data.
It's obviously not true in all contexts, especially in medical ones, but. These RAG systems are actually the ways that companies are able to offer a differentiated solution from just like general purpose chat GPT. If staples wanted to offer a chat bot for troubleshooting all of its products, like that would be better than anything that chat GPT could offer because they would have the domain expertise about every single product in their inventory.
Jason Jacobs: Got it. And it sounds like the LLMs can provide information that can be found out In the on the web, essentially but there's a lot of information like that, that, so in the Staples example, is that information just in people's heads or is it actually written and categorized and organized in a way that's AI friendly?
Sara Remsen: Yeah, that's another misconception. I think everyone's I have the data. I can make it useful for AI. And the fact of the truth is that you have to do a fair amount of work to make it useful. In the staples example, that the information that would be used in a rag system has to be documented somewhere, probably in text, probably in like your FAQs, or, holding this product manual it needs to be somewhere that the LLM, when it's making its call can go find.
Jason Jacobs: And in the Staples example, is that information not publicly available on the web?
Sara Remsen: Sometimes it is, but in, for a lot of our customers who are building more sophisticated rag use cases, some of it's publicly available, but the majority is internal because it's all about internal expertise about their own customers or their own processes. That haven't been made public.
Jason Jacobs: Can you give an example of that and feel free to anonymize whatever aspects of it you need to not betray any confidentiality?
Sara Remsen: Yeah. We can keep going with maybe the staples example, since that's where I am. And I don't to my yeah, physically,
Jason Jacobs: listeners, she means she's physically sitting in a Staples recording studio. Yeah.
Sara Remsen: they have a coworking space. Little known fact. It's actually lovely. They've got great snacks. So if staples were to operate customer support. AI agent to anyone who's buying their equipment, what they would do is they would take their internal knowledge base about. How all of their products work, how to troubleshoot every conversation they've ever had about, this little red light blinking, they would take their external facing knowledge base and things that you find on websites, you embed all of that.
You then hook it up with an embeddings model. You hook all of that up to your pipeline and a large language model, and then you surface that as a chatbot to your customers. So when they ask a question, it's getting processed by that LLM. It might be getting classified or categorized along the way, and then bringing up the relevant information to answer the question correctly.
Jason Jacobs: Got it. And where's the pain in that example?
Sara Remsen: The pain is that When teams do that, they're in the dark. They don't know what people are asking and how
Jason Jacobs: People being the end customer who's making the initial query?
Sara Remsen: Yes. So let's say at staples, you've got an AI operations manager who's responsible for the customer success organization. They're probably trying to do something like increase.
Digital deflection, or increased digital resolution rate, meaning what percentage of my customer inquiries can I resolve quickly with AI? The challenge is that when you move to a world where people can now talk to these agents. There's no the volume of data is much higher. The complexity of those conversations is it's a conversation.
It's it's, it can be often hard to tell if that person's question was answered correctly or not, whether that was. Was the right information surfaced? Was the did the answer have enough detail? Did the person then go on their merry way? And as these teams are trying to shift and to automate some of these processes, and the thing that they care about is this digital resolution, right?
There's no way for them to measure how that actually is working today. No way for them to understand what people are asking and how well the agent response.
Jason Jacobs: So in this example, is chatbot and agent, are chatbot and agent synonymous?
Sara Remsen: Yes, in this case
Jason Jacobs: and did chatbots exist before agents?
Sara Remsen: yes,
Jason Jacobs: And when chatbots existed before agents, were companies also in the dark?
Sara Remsen: no, because chatbots used to be designed like phone trees. It was like, press one for troubleshooting, press two for billing information. And when companies designed chatbots for customer success in the before times, you would outline the 10 things that you would expect customers to want to be able to do. Then you probably had a couple of key words and phrases that would automatically classify a question into one of those ten buckets. And so it was easy to understand, okay, how many people are actually asking about troubleshooting the microphone. And it was easy for them to make sure that the response was always good.
Because it wasn't, it was very deterministic. It was like, pick your own adventure, one through ten. And the challenge that we now have with generative AI, which is both the challenge and the unique opportunity, is that it's effectively infinitely flexible, meaning it will respond to any human query that gets put out there.
And the way that people interact with it is Like no one can ever really truly predict. So the other thing we see a lot is teams who are working on demos. It's easy to show a great demo of someone doing one of those 10 things, but then you put it out in the wild and someone says, sell me a car for a dollar or give me a discount on this price or something else.
And the agent does its best to respond, but it's prop it's. It's unlikely that team has seen that situation before. And so what all of this means is that it's really hard for these teams to understand this digital resolution rate. What are people actually asking? Because now they're asking in 100 different ways instead of just 10.
And is the agent responding to those in a way that's expected and good?
Jason Jacobs: What was broken about the 10, like the 10 category way? Like, why, is this just technology for technology's sake, or does it actually help make the customer experience better?
Sara Remsen: I think some of it, yes, honestly, and I think the best teams are thinking about using generative AI as if it's a phone tree, meaning they're thinking about the 10 things that they want or expect customers to be able to do, and they make sure that their AI assistant or their chatbot does those 10 things really well.
The opportunity that people see is again, this like flexibility. So if the chatbot of yesterday could support 10 inquiries. Now, the chatbots today can support 100. And so before you might have only been able to outsource 20 percent of your customer success questions to your AI assistant, because its use case was rather narrow.
And now you can do almost all of it. And cool. I think there's a shift from like having the human as the default and seeing like where you could expedite that with a, with an AI chatbot. And now I think the default is going to be it's an AI chatbot. And at what point do you escalate to a person?
Jason Jacobs: And these chatbots, or agents, if you will I actually just did an episode yesterday with Sean from Coworked. I don't know if Coworked, but they're they've got a hundred internal agents that that take on different functions of a product. Project manager to extend the capabilities of a PMO office to drive way more efficiency without.
To get way more done without increasing your staff essentially. And and one of the things we talked about is that they looked at the capabilities of the off the shelf agent builders, and they were insufficient, so they actually took it on to do it in house to their chagrin. What are you seeing out there in terms of these agents?
How much of them are getting done? But what is the landscape? Let me, yeah, let me stop asking a leading question. What does the landscape look like for getting these agents built today?
Sara Remsen: Yeah, we are in the very early days where the technology is changing really fast. The use cases are changing really fast. I see all kinds of different things. So I think for some companies. There are starting to be very good off the shelf, full stack AI assistants or AI chatbots, if you want to call them for specific verticals or specific functions, I should say so customer success.
There's a bunch of companies in that space now. Like Maven is 1 Decagon. In keep. I think they're all doing a great job, and there's going to be a lot of businesses for whom their solutions are perfect. Exactly what they need. Hook it up and you get the automated customer success that you need.
There are other businesses, which are more of the companies that we work with, who are investing in AI as a strategic lever for their entire business. And so what that means is that they're probably investing in an AI team. They probably have an AI operations team. Person, and they are building these AI assistants in house for customer success sales, direct customer facing, like directly into their product.
And those are the teams that we work with, because they're the ones who are really on the hook for demonstrating that they can solve these problems with AI assistance and they need analytics and performance metrics to be able to report that back to their. Executives. So I think we're still in summary.
We're still in early days between like, where are the use cases where you can just buy a solution off the shelf versus like, when do you actually need to build it yourself?
Jason Jacobs: If you take the Maven example or the other vendors that you mentioned that have the more turnkey offerings are the are the in house teams still blind in those cases or are those companies providing the eyes, if you will, that you're providing for the people that are DIY?
Sara Remsen: They do have, as far as I know simple metrics. They're extremely focused on their particular use case, like their vertical of customer success. Let's say one thing that we're seeing at Melodi, which. I think will be an interesting trend as the market matures is that these conversations are a goldmine of information.
So when chat GPT first came out and took the world by storm and took off and grew insanely quickly, it proved that people were actually really excited about talking to robots. It was the same tech. It was like the completion model. So it was like, is the sunset blank? And it was red. And instead they just flipped that on its head.
And so now you could say, what color is the sunset? And it would say, the sunset is red. And we just see people like talking and talking and talking to these assistants. And I think that, the way that people talk to these assistants will provide a ton of useful information about customers, about competitive intelligence.
It ends up being almost like this form of conversation intelligence. So as an example, one company that we're working with, they've got an internal agent for sales. So like for their sales reps to ask questions and one thing they're looking at and they're using Melodi for is to identify in which regions are sales reps asking about their competitors the most, which means like if.
Let's again, let's go back to the staples example. Let's say sales reps for staples are asking about office max in the northeast. Now, that team knows that they need to update the battle cards against office max, for example. And so there's almost like this, like data enrichment layer that we're providing that is helping.
All different parts of an organization become more efficient over time.
Jason Jacobs: One analogy that comes to mind, and I'll throw it out there because I'm curious to see how you react, is I remember when companies had websites, they were all dialed in with Google Analytics or whatever other tools they were using, and then when native mobile apps came out and they ported their experience, whether it was commerce or whatever it was, into mobile, all of a sudden they're blind in so many things that they could see on the web.
Is this similar with agents?
Sara Remsen: Yes, it's exactly that shift. We went from web to mobile. Now we're at chatbots and very soon we will be at, I think the true vision of agents, which is not just a chatbot that you have to go back and forth with, but. An agent that can actually go act and execute tasks on your behalf.
And in that world, it's going to be even more important to understand what those agents are running around doing and how people are interacting with them.
Jason Jacobs: What's funny is that when I hear this, I keep thinking about the intersection of also what's happening in robotics, where, especially with so many companies returning to in person work, it's like, are you going to have a robot co worker who's just like there at your beck and call to go and do whatever it is you tell it to do?
Or a fleet of them?
Sara Remsen: Yeah, I think so. I think you're going to have people are going to be managing digital workforces. It's going to be I think the best version of that is that you're going to have. Assistance for the parts of your job that are the most boring and the most tedious that you never wanted to do. Anyway I cannot wait to have someone and AI that can do all of my taxes.
But. Yeah, there will be, like, fully functioning agents that can go out and execute things independently.
Jason Jacobs: So when it comes to actually entering the market, given that it's early days, I would imagine it's messy in terms of different stacks are compatible with different things and not compatible with other things. And you have to make decisions. And I'm putting words in your mouth. I'm throwing this out, I'm saying it as a statement, but asking it as a question but you have to, I would imagine you need to make decisions about, we're going to support this stack and we're not going to invest in supporting that stack yet because it's not as widely adopted yet, or this one is not as widely adopted, but we can perform better on it.
So we're going to start there and get a beachhead somewhere before we expand. Like what, how do you think about that in terms of how to prioritize where to focus? And I mean that both in terms of what tool sets or stack to support, but also what tasks to support as well.
Sara Remsen: Yeah, we are Melodi is a layer that sits on top of these conversations. The way that we work with our customers is they send transcripts to us. Back and forth, multi turn conversations between users and assistant and they can also do things like send tool calls or sources or other metadata that can
Jason Jacobs: And is this, is it sent in an automated way or is it actually, yeah. So it's like a, like an API that, that that enables it.
Sara Remsen: exactly. We've got an SDK. Again, I think there actually are a ton of parallels to web analytics and monitoring. It's it's all this SDK and then all of your data gets piped to Melodi and then we can layer the important analytics on top of it. And because of that, we are agnostic for any of these large providers.
You can use whatever provider you want. You can switch. Models whenever you want. A lot of these things go through so many processing steps too, and we're really focused on the end result. What does the customer actually see? And so that's part of what we focus on.
Jason Jacobs: Are you are you task agnostic then? As long as they if they send it to you, you can do it? Or does it work better with certain tasks today? Or types of tasks?
Sara Remsen: The task is really this rag Q and A. It's what are people asking about? Are you getting a response? And we can monitor that, determine what people are actually asking about, and then where they're running into problems.
Jason Jacobs: And then, since so many companies are now at least in some form looking to or in the process of building, agents, there are small companies, there are big companies, there are companies in different sectors, there are DIY ones, there are ones that use off the shelf products. I mean it sounds like one criteria is DIY in terms of what's a good fit for Melodi.
What other criteria are important in terms of your initial go to market?
Sara Remsen: Yeah it's definitely an interesting time because what is most important to us is actually where companies are on this AI maturity journey versus the specific. Industry or a style of team. So like company size, I would say. But what we do know is that the companies that are the best fit for Melodi's offering are have invested in AI as a strategic initiative. Probably as an external product, but also have invested internally and having operations roles, whether it's called AI operations or just an operations manager who's responsible for overseeing implementation of AI initiatives in house, they're probably building some of it themselves instead of picking an off the shelf option again, in part, because they're making a strategic bet to upscale their own AI team, or because they have Such complex systems that the off the shelf just doesn't doesn't quite cut it.
They're probably already using a product analytics tool, like mix panel or heap or Pendo full story. 1 of those to monitor and understand how people are using their non products, and they're probably already using what's now called ops. That's a Lang Smith, Lang Fuse, BrainTrust the really, the tools that are focused for engineers and developers to understand like key engineering performance metrics, so like latency accuracy, things like that.
Jason Jacobs: And that makes them a good fit because it's like they're dialed. It's almost it's like you it's if you buy a house and you renovate all these different rooms and then there's that one room over there that you chose not to, and then it just like gnaws at you every day because you know what renovated feels like.
And so now it like stands out even more that it's not. So it's like sophisticated in other places. Therefore, if you're flying blind in this place, it's going to bother you more acutely.
Sara Remsen: I think it's yeah, there's somewhere to go with that metaphor and I want to think about it for a second, but it's like The first question that these teams are asking themselves is, can I make it work? And it's yeah, I have it. I built it. It exists. And then the next question is is it working? Is it working for the people who are actually using it?
Does it ladder up to the business metrics and the ROI that I actually care about? And that I made. Public commitments to my investors about, and that's where Melodi comes in. So it's more like, to get back to the house metaphor, these teams are buying these LLM ops platforms because that's about scaffolding.
That's about building the structure of the house and the plumbing and the electric and whatever, but Melodi is about how do you design the kitchen so that someone can actually reach the fridge and the stove and the kitchen island in the right way?
Jason Jacobs: You mentioned that that one of the criteria or a good sign would be if they're either building AI, incorporating AI into their products, or if they're using it in house or both. I'm just curious does it tend to be all or nothing? Are there companies that are building AI products that aren't leaning hard on it?
And then conversely, are there companies that are leaning hard on it in house that aren't building AI products, or do they tend to go together? Yeah, so
Sara Remsen: Yeah, that's a really good question.
Such a range and it changes, honestly, so fast. I hear from a lot of companies. Oh, we have 25 demos and I think where we are in the market is everyone's experimenting so much with. Where does this actually work? Where does this actually provide value?
Where should I continue to invest? And so I do think that teams that are the most successful with external facing AI offerings often have one in house that's working pretty well because it's safer to test with your own employees, they're not going to churn. You're not going to risk revenue because they got a bad answer and so these teams often build the stuff in house before they go out. Oh, that's such a good question. And you mean the. like for companies that are offering agents as part of their offering.
Jason Jacobs: Coworks because he came on yesterday. They're essentially, it's like An outsourced, I don't know if these would be his words but it's outsourced project management augmentation through agents. But it's on a per project basis, not on a per seat subscription basis and in his mind as, And again, I don't want to put words in his mouth, but I think what he was saying was that as as models like his get more prolific and mature and adopted it will put pressure on the per seat alternatives.
Sara Remsen: Yeah. If you think about the way that we price work today, you either pay someone by the hour, you pay someone by the project, or you pay someone on retainer or as a salary. And so I think we're going to see those types of models for different types of agentic work where. The way that we price that Melodi today, at least is based on usage.
So it's like, how much or how much data do you have that you need analyzed? And then our pricing skills with that. I think for teams that are looking at full replacements for people again, you could fall into any 1 of those buckets. It depends on how much. Of that work you need done. So I don't think it'll be, I think there, there may still be some per seat pricing for AI agents, but it may look more like usage.
Jason Jacobs: So are you priced more, it sounds almost like electricity. The more you use, the more you pay.
Sara Remsen: Yeah. I don't have any surge pricing though, but yeah. Variable rate.
Jason Jacobs: Got it. And I'm not looking for you to share anything confidential that you don't want to share but to the extent you can just talk about, just for context for listeners of just how long you've been in market, what phase you are, like where are you with customer adoption and it's less about what are your revenue numbers and it's more just about like how how much have you been out there for.
Yeah, how deeply penetrated are you and how much are you learning from like real world feedback?
Sara Remsen: Yeah. Yeah. So we're a little under two years old. We. Have been we've got early traction, early revenue. We're working with some really great companies. And what's really exciting is within those companies, we're having great success. So we're getting a lot of usage from a lot of different teams within those companies that said the market is really frothy.
And this startup is very different from my last startup where I have two variables that are changing all the time instead of just one. So with my last company, Waypoint, we were solving an existing problem with a new technology. So because of this new technology, augmented reality, and the sensors that were available, we could now do things that were not possible before and solve an existing problem of documentation 10 times better.
The challenge in the AI market right now. Is that both the technology is changing, but also the problem space. And that's maddening because the way that people talk about the problems that they have changes conversation week to week. And that's why this AI maturity curve is so important because we're starting to see patterns about the types of problems that people talk about having.
And those shift over time. And some of those phases are really short and some of those are more durable and long term. So I think it's a really exciting time to build a company. The only way to learn any of this is to start building it and talking to people who are also building it and seeing if you can solve some of their problems, but there is a lot of ambiguity and uncertainty and so many directions that all of us can go.
Jason Jacobs: So I want to ask one soft, just general software question and then an agent question. So the software question is, I've heard Sahil Lavangia came on the show he was an earlier guest and he was talking about how he thinks that as these tools proliferate and lower the barriers and the cost and increase the speed, et cetera, of building software that it will get so cheap and so fast that No one's going to need software vendors anymore because they're going to, they're going to have their own customized personalized glove fit software built in house.
What do you think about that and how does that align with how you see the future of software playing out?
Sara Remsen: Also a very interesting question. I think. I think that as much as we want all of this to be automated and hyper personalized, it's, at the end of the day, it's still people who are trying to do stuff and get things done and have problems solved for them. And I do think there's some interesting concepts around this idea of this convergence of economies of scale and hyper personalization, where. The whole framing for go to market for a startup is like what's your repeatable product your repeatable go to market How do you take one thing and sell it the same? The same way the same time to as many people as you can And now you have this infinitely personal customization where, yeah, I could see a world where everyone has relatively bespoke solutions to what their problems are specifically.
But I think you're still going to have people who are creating, who are going to have ideas for those solutions, people who are going to need those problems to be solved. And I think my hope is that we move to a world where And I got a little bit on my soapbox about this before, but I'm so sick of UIs.
I, I, it's like death by a thousand bad UIs. And I'm really excited about the future where we have technology that solves these real fundamental human problems in a way that is intuitive and natural and extremely helpful, where we don't have to teach people how to click around. We can just get the stuff done that we need to get done.
So I don't know how that's going to play out in terms of the number of companies and scale and things like that. I believe that we have some exciting new interaction patterns coming our way, and there's still going to be people at the center of it.
Jason Jacobs: And now I'm going to ask you the same question about agents, right? Because you talked about all these vendors providing them, you talked about companies DIY what's your view on how that plays out in the future? And selfishly for Melodi how do you hope it plays out?
Sara Remsen: Yeah, 1 thing that I think is a little scary and concerning is the hyper concentration of power in these large providers and we saw it in the last 15 years with Microsoft and Apple and Google and Amazon meta. And we're seeing it again, but at an even faster scale these companies are already so big.
They are already so out ahead. No, 1's in their right mind is like, trying to train their own foundational model anymore. Except maybe deep seek and that's a whole other podcast, but. They're running away with these, with this tooling, and on one hand, formal Melodi, that's fine because this is tooling, it's web services effectively, like everyone will have access to this.
Everyone will have access to this state of the art. The more people use it and the cheaper it gets, and the better it gets, the better it is for us, because that means more customers. The challenge is, again, it makes it really hard to compete on many different use cases, which is why I really believe that this this expertise and operational process will be the competitive advantage for any companies that are trying to provide an offering that otherwise ChatGPT could do. So I think on one hand, it's good. On the other hand, I think there's like expertise and operational processes like that has, I want it to play out like that because that's where companies will be able to provide real value in a way that creates a more diverse and interesting landscape.
Jason Jacobs: That's talking about the LLMs but what about the Mavens versus the DIY? Because either way, if it's Maven or if it's DIY, it's not the LLM, right? But so I get how we can all fear the LLM. But but for what? So should Maven fear that fear the LLM? Should the DIY hope that the LLM does this or that they don't have to DIY anymore?
Like how how do you think that it's going to play out as it relates to the agent side specifically?
Sara Remsen: Okay, that makes sense. I think it's a spectrum and there are going to be companies and people all across that spectrum that have slightly different needs, specifically convenience and customization. So people who want high convenience, low customization, they're going to go for the off the shelf tools like Maven.
People who want high customization and are willing to invest in it will continue to DIY. I, and even if we run out the sort of future vision that you mentioned before, where everything is hyper. Customized and personalized, it's. There's going to be I think it's easy to think about that level of complexity working perfectly.
But again, there's people that need to be able to understand what. Any of these systems are doing and acting, and that's actually, I think, going to put a constraint on how successful these. Agents can proliferate independently and. Solve every single problem immediately.
Jason Jacobs: Since it sounds like the customers where you have been the best fit early on are more about mindset and AI maturity than about category or stage. How do you go about targeting your selling and also how are new customers finding you? Is it an outbound model? Is it an inbound model? Is it self service?
So yeah, it'd be helpful to understand both of those. What's the sales model and also how do you prioritize given that you can't say like this vertical at this stage or something nice and clean.
Sara Remsen: Yeah, we've been looking specifically at titles. So if you have an AI product manager or an AI operations manager, that's a good signal, at least to start that you're ready for Melodi's product offering. If that company has again, an external facing offering, whether that's a strategic announcement or an actual product that's in market, that's another good signal for us.
So we've been doing a mix of outbound, we've been doing some inbound, one consequences. There's all this AI drivel everywhere. I don't know about you, but my LinkedIn inbox is so crowded with all these hyper personalized messages that I selling me stuff I don't really want. And I don't, I honestly don't know how where we're going to go from outbound.
I think we may, the pendulum may swing back for more personalized outreach for events, for webinars. Things like that. So we're, it's changing rapidly and we're still early enough that we are trying to figure out what the right motion is for us, especially as everyone's needs keep changing,
Jason Jacobs: And I have to ask, but how has Melodi incorporated AI into your external offerings and into your internal operations?
Sara Remsen: Of course. Our entire product is a machine learning pipeline, so we take all the transcripts and we analyze them. We've got our own models that we run on top of it which is really a. Clever, it's like algorithm design. So it's like a clever combination of what that input data is specific context.
If someone left feedback, and then we have classification models that run after that. It's not the model of oh, we take the data, we train on it and we have our own model. It's more thoughtful design, but. We use, I would say we're huge fans of cursor deep note, a lot of the AI coding tools I have found to be the most jaw dropping in terms of how they take, where they, the amount of productivity that we are able to get and both the amount of work and the type of work that we can do as a small founding team.
V0 from Brasel, another one, a great one for that. Superhuman and I'm experimenting with their their AI responses to various things like that's been helpful. So we're not at the scale yet where we want to automate, different parts of the business because we're still like we're still learning so fast.
I think that's actually, that's an important point about agents as they scale is. They need, the task has to be very well defined for them. The best way to do prompt engineering is to be really clear in your thinking, in terms of your expectations, your goals and guidelines. And when you're a really early stage startup, your entire.
Competitive differentiation is that you are extremely nimble. You're learning really fast and you're extremely adaptive. And so I don't want to outsource any of that to something that is inherently like a repeatable thing while I am still iterating so quickly. So we use it internally, but more for like personal productivity as compared to having a customer success agent.
Jason Jacobs: When you think about, not just software, but just industry, right? There's a lot of talk about how profound the changes will be to how we work and live, and some people say game over for humans doing any job, right? And other people say, you know what? We'll incorporate where it makes sense and we won't where it doesn't, but we don't need to force feed it across everything.
And it's overblown. It's going to be more incremental than disruptive. What do you think,
Sara Remsen: I think like any good proclamation or generalization, it's probably somewhere in the middle. I think we're going to see some creative destruction, quote, unquote, that's going to eliminate jobs for a lot of people, and I don't think that they will be able to upskill as fast as society would like them to or support.
I don't think that we have the systems in place to support that transition very well today. But that said, it's not going to, it's not going to be completely overnight. And I think we're starting to think about the things that we, like the places where we want AI in our life and the places where we don't.
And I have faith in humanity for being able to not lead us to a crazy robot overlord place, but a place where technology can actually solve these problems for us.
Jason Jacobs: For any founders out there that are out there thinking about what they might build next? What advice do you have for them in terms of how they should be thinking about incorporating AI from Day zero. And the answer might just be that's way too general. It completely depends on the situation. So I can't offer generic advice, but I don't know maybe you have something different to say like how should they incorporate it?
And also, how much should they incorporate it, if at all? And how can they tell?
Sara Remsen: Yeah, I hear a lot of people who entrepreneurs and other professionals just in a panic about AI Oh, what should I be doing with AI? How should I be thinking about it? And my advice is just get a chat GPT pro account or a quad pro account or whatever you want. And the best way is to just start experimenting with it.
And. The other day, my, my daughter's almost two, and she can navigate an entire iPad, which is terrifying. And that's just because she's been pressing on different parts of the screen constantly, and she's not afraid of making a mistake. And I think we get into these places with software in particular where everything feels like, I can't make it work the way that I want it to work, and oh, I clicked this, and I need to go back, and that's not the right way.
And the beauty of AI is that if you don't get it right over, it doesn't respond in the way that you expect, like you just try again. And a lot of that learning people have to do themselves. So I would say if you're thinking about being in this space, the best thing you can do is start building stuff, get the pro account, get your API key, start building some things and hooking it up together.
You can go. So fast with a great demo or get to a great demo so fast now. And just be aware that going from demo to production is very thorny and challenging. So just be prepared for that, but otherwise it's good product design and standard, like people forget that too. It's building a great AI product is just building a great product.
Jason Jacobs: and I'll ask the same question, but more from an organizational standpoint, if there's companies out there that have been around a long time and that do things, they've done things the same way for a long time, and that, some of them might be digital companies, some of them might be, apparel companies, some of them might be materials companies there's all kinds of companies out there that make the world go round.
Should every one of them get an AI officer or people with AI in their titles or Or yeah, like how would you think about that in terms of who should be doing stuff when they hear all this buzz getting jammed down their throat every day? And who should just filter out the noise and keep doing what they're doing?
Sara Remsen: Yeah. I think one mistake that I have seen companies make is having a no AI policy at all, like being very worried about data privacy and sensitive information, which I think is important, but there are ways to get around that, like with the right companies, like with the right account, with the right legal compliance.
So I would actually say. Companies should get smart on the data privacy side of things and then find ways to implement it internally and experiment with it and get a sense for what it is and what it can do. And then, if they are interested in investing in it strategically it is overhyped. Be, be wary of that.
And again any good product focus on a use case first. Don't worry so much about building a giant AI platform, build one thing, test it out, see if it works. And if it does, then you can iterate and scale. So we've forgotten some of those things, those lessons and the excitement about AI.
I think
Jason Jacobs: so what's next for Melodi? What are some of the key priorities if you look over the next 12, 18, 24 months?
Sara Remsen: yeah, so we've got a couple of great customers right now. We're really focused on making sure that we're providing a ton of value to them. We are looking for more customers always. For anyone who's in AI operations or AI product management I personally am excited about.
Developing a little bit more of a community around AI in Boston, especially like women in AI, designers, product, and AI. So I'll be planning some more events hopefully in the next year or so. And Yeah, I don't know. It's a exciting, wild ride and exciting time. But it feels like this is a way that we can help make AI what it should be, which is helpful for people.
Jason Jacobs: And for anyone listening that is inspired by your work, is there any one particular that you want to hear from or any homework that you want to give people who who are listening to the show, who want to learn more?
Sara Remsen: I would say if you're AI product or AI ops, I'd love to chat with you. If you're an entrepreneur thinking about this space, I do a lot of advising at MIT and other places and always happy to talk to other founders who are in the trenches. And yeah, otherwise, just excited to be part of this journey and contribute to this podcast.
Jason Jacobs: Thanks, Sara. Anything I didn't ask that I should have, or any parting words?
Sara Remsen: The only thing that occurred to me is that I use AI with my kids a lot. And I don't know if people have talked about that before, but I made like a custom coloring book for my son for his birthday. And we put chat GPT on the table and ask it to tell us jokes sometimes and I think there's just like some really fun ways that it's making me a more creative parent, which I really like.
So I would recommend trying out both those things.
Jason Jacobs: Huh, yeah, my, my kids are always trying to get me to cook more, so maybe I'll ask ChatGPT to think of some dishes if I tell them the ingredients that you know, because they're picky, so if I tell ChatGPT what my kids like, maybe it can help. Suggest me some things that and teach me how to cook.
All right. Thanks for coming on the show we really look forward to following your progress and Melodi's progress I look forward to coming to some of your events if I'm invited and to keeping this dialogue going Thanks so much Sara and best of luck
Sara Remsen: Thanks, Jason.
Jason Jacobs: Thank you for tuning into the next, next, if you enjoyed it, you can subscribe from your favorite podcast player. In addition to the podcast, which typically publishes weekly, there's also a weekly newsletter on sub stack at the next, next dot sub stack. com. That's essentially for weekly accountability of the ground.
I'm covering areas I'm tackling next and where I could use some help as well. And it's a great area to foster discussion and dialogue around the topics that we cover on the show. Thanks for tuning in. See you next week!