Slightly Caffeinated

Chris Gmyr (00:00)
Hey, welcome back to this lately caffeinated podcast. I'm Chris Gmyr Hey TJ, so what's new in your world?

TJ Miller (00:03)
I'm TJ Miller.

man, life has just been a bucket of lemons. Like, really, the last couple months have been super intense and lots of interpersonal life stuff going down. But like the last two weeks have been just mind bendingly awful. But I think I'm like working towards being on the other side of stuff.

which is nice and Just like humming along man starting to ramp up for a lara con Both unlike working on a lara con project for Geocode.io working on like getting my talk together Yeah, trying to like jam on prism there's a whole bunch of like in flight stuff with prism that I'd like to pull over the finish line like things I'd like to have for lara con so and like a big bug that I'm trying to

sort through, so we can talk about that a little later. But, you know, I'm hanging in, man. How about you? How are you doing?

Chris Gmyr (01:03)
I'm glad you're hanging in there and hopefully everything works out for the best how you want it and yeah, just keep on keeping on for sure.

TJ Miller (01:11)
Yeah, yeah, man,

it's day by day, hour by hour sometimes, but like, just, we're doing it. We're making as much lemonade as possible.

Chris Gmyr (01:19)
Yep, totally. Yeah, over here, just, I don't know, same stuff. Just kind of heads down in work mode, trying to push some projects forward. So it's just kind of all consuming all throughout the workday. But going to be doing some cool things coming up soon. So I'll have more to talk about there once we start digging into things.

TJ Miller (01:32)
Yeah.

Chris Gmyr (01:40)
And then, yeah, just wind it down on the school year. My son is on, we have a bunch of crazy school options down here. You could do different tracks and year-round school or traditional school. And so we're on a traditional calendar. So my son is out for the entire summer. So that is ending in about two weeks, two and a half weeks. And then he'll be with us full time for the summer.

and doing some traveling, doing some things, signing up for some camps for him. So just trying to keep him busy and occupied as a going on nine-year-old and just wants to be entertained all day, which is a lot with also a three-year-old and trying to work full-time and do a bunch of other stuff. So yeah, just craziness, end of the year type of stuff, but all good. Nothing too bad.

TJ Miller (02:19)
Yeah.

Yeah.

Yeah,

same over here, man. Today is actually my son's last, last day of school. So yeah, also bleeding into the summer for him. but this is his last day of elementary school. He's, he's in middle school next year. So, just had his, like his, his graduation and, yeah, also trying to like ramp up summer plans for him. So while I'm at Leracon, he's going to go with, my wife to New Mexico where my father-in-law is, and they're going to meet back up with us in Denver. And then,

Yeah, he's got a three day baseball camp I signed him up for later this month, which I think he's going to have an absolute riot at. And he just got a, from my mom, he just got a metal detector this morning. So he's, he's very excited to go comb some beaches and find some, find some stuff, you know, I'm going to have him check out the yard, see what he can find around here first.

Chris Gmyr (03:19)
Yeah, find some of those like old screws from two years ago project or something like that.

TJ Miller (03:23)
Yeah, yeah,

maybe he'll find the power line that runs from my house to the garage and I know where to not dig.

Chris Gmyr (03:29)
Yeah, yeah.

Nice. That'll be fun.

TJ Miller (03:31)
Cool man.

Yeah dude. Yeah, you wanna talk some coffee? Man.

Chris Gmyr (03:36)
Yeah, coffee talk.

So basically, like, I don't know, I've just been making a bunch of cold brew. Just keeping it up.

TJ Miller (03:41)
Yeah. Yep.

Same. Lots of cold brew here. just switched everything up. We've been doing South American beans for the last little while. And my wife was just like, we need to switch this up. She's big on the African beans over South American. So we just got like a handful of bags of some African blends. Really excited about that because I'm a big fan of those too. Like a nice Kenya's.

like maybe one of my favorites. It's always like super smooth and you know, not very acidic. So I'm a big fan. So we just kind of switched that kind of stuff up, but lots of cold brew. I think you were telling me before that your wife is trying some crazy science cocktail coffee. Like I gotta hear about this.

Chris Gmyr (04:22)
It's just protein coffee. It's not anything too crazy. So I've been trying to get some more protein in. We have a bunch of different protein powders because I've been working out pretty hard since the beginning of the year and just trying to do more research and just trying to, without sounding too bro-y, hitting those daily protein goals. But it's important. And so you've got to get it.

TJ Miller (04:25)
Hahaha

Yeah.

Chris Gmyr (04:46)
get it in where you can and it's hard to eat a lot of actual like meat protein and you know whatever on a daily basis. yeah, yeah, yeah, got to supplement with some powders and stuff like that. So especially with just dumping some protein powder into like a smoothie or even like a coffee is like an easy way to get that boost, especially in the morning keeps you like fuller for longer and like helps curb cravings and

TJ Miller (04:54)
It's a lot harder than you think, like a lot harder.

Chris Gmyr (05:13)
Just helps with so many things. So she's been doing this little concoction with the cold brew that we've been making a couple scoops of protein Some cocoa powder and soy milk, which also has a bunch of protein in it. So it's almost basically like a mocha and Just packs a little punch. It has a little bit of that kind of protein, you know flavor from

TJ Miller (05:27)
Nice.

Chris Gmyr (05:36)
uh the powder but like not too bad either we have like a vanilla flavored one so it kind of covers up most of it so it's kind of like a mocha vanilla like cold brew i don't know kind of smoothie drink it's kind of nice

TJ Miller (05:41)
Mm-hmm. Yeah.

Nice.

Yeah, dude, that's awesome. I've definitely made, like coffee smoothies before. So instead of using like a coffee flavoring, I'll use like cold, I'll put cold brew in, and that kind of gives it at least like that, like that coffee-ish flavor, a little bit of caffeine too, which is always nice. but yeah, it's, that sounds actually like really pretty nice. At first you were telling me like protein coffee. was like, I don't know about this. and.

Chris Gmyr (06:10)
Yeah, it's pretty good

TJ Miller (06:15)
Now that you describe it, I'm like, kind of want some.

Chris Gmyr (06:17)
Yeah, it's pretty good. It's it's slightly sweet if you get like the original. So I milk because they have like unsweetened too, but like the original just adds that little bit of like sweetness to make it a little bit more enjoyable instead of just like kind of bitter ish, you know, coffee and just kind of bland or like, you know, semi bland like vanilla powder. So with the.

cocoa powder and the slight sweetness from the soy milk. It's a pretty pleasant drink.

TJ Miller (06:47)
Hmm, that sounds pretty good, man. Yeah, it's the more like I finally have been. Starting to get back to the gym, like I've taken like almost a month off at this point, just with various things going on and just it's been a struggle to get back to it. But I've been back a few times, like over the last week or so, and that's been great. And like I've been starting to try and focus on at least like trying to hit.

majority of my protein goals and like, man, it is just, it is so hard. Like it is, it is a lot harder than I, you know, originally like envisioned. I'm like, I'll just like, yeah, eat some more meat and like a few other things. Like, no, you gotta add in like shakes and powders and like, you just kind of have to in order to hit those goals. Or you're just like eating an obscene amount of meat. It's like, I'm not even hungry, but like I need the protein.

Chris Gmyr (07:36)
Yep.

TJ Miller (07:37)
Gotta get it in. So

Chris Gmyr (07:38)
Yep.

TJ Miller (07:39)
it's tricky, But it pays off. Like, it does. So we'll see how that goes. But I might have to hitch you up for this recipe and start making some iced coffee, some protein coffee.

Chris Gmyr (07:49)
Yeah, yeah, for sure. Just to,

yeah, mix it up a little bit.

TJ Miller (07:54)
Yeah, just something different for sure. Cool, dude. Moving on from coffee, Claude in Anthropic has just been absolutely on fire over the last couple of weeks. ⁓ Between the V4 model launch, Claude code going general availability, all sorts of new features inside of like,

Chris Gmyr (08:04)
Yeah, I've been killing it.

TJ Miller (08:16)
Claude's desktop app, and then I think they recently just allowed you to use Claude code with your pro account before you had to use your API, which I'm spending 20 bucks a month for Claude. And then on top of it, I'm then spending whatever my spend is on my API key, which has been a lot lately.

Chris Gmyr (08:23)
Mm-hmm.

TJ Miller (08:36)
Yeah, like I buy my credits in like $25 chunks at a time just because that seems like a reasonable thing to do. And it's been like every other every couple days. I'm like, oh man, this is getting gnarly. So it's nice to be able to like switch over and use like some of my ProPlan credits too.

Chris Gmyr (08:52)
Yeah, that is nice. I just tinkered with Claude Code more the other day. And it me a little worried at first, because when it's trying to analyze everything, it's showing you how many tokens that it's going through and sending back and forth. I'm like, oh, no. And then I look in the dashboard, and it's like, you've spent $0.19. I'm like, oh, sweet. But that was only a very small action that I took with Claude Code.

TJ Miller (09:07)
Mm-hmm.

Yeah.

Chris Gmyr (09:22)
can't even imagine doing a bigger rewrite or doing in-depth analysis or a whole bunch of things every day that you're going through. yeah, chunking through those $25 worth of credits every few days makes sense.

TJ Miller (09:36)
Yeah, permanently like I got access to Claude code during preview like very early in preview. They basically announced it and I signed up like that day. So I've been in preview for a while. I like pair program with it all the time all day long. Like I basically it's rare for me to be writing code without pairing with Claude code just because like it gets through so much stuff. And you

You get to like, I don't know, I've kind of worked with it enough that I've kind of learned how to work with it. And we've been incredibly productive and it's been putting out like really solid code. But I also don't let it, for the most part, I don't let it like run wild. I'm like reviewing the code, especially for like critical pieces of the code, like that are like not boilerplate, right? Especially if I'm having to do like TDD and writing out tests.

Like I'm making sure that all of the tests that are writing are like actually covering things that are important. And then like the key bits of functionality, I'm also like reviewing that code before it writes anything to like make sure it's good. But then as it's in boilerplate stuff, I'll let it just kind of like run on its own. But lots of upfront planning, like I'll talk a little bit more about in the prism stuff, but I've had it been like working on a couple pretty big refactors.

And it's fairly well, but it's definitely like handholding. It's a pair programming process. I think if you approach it that way, it's super productive and you're not gonna get like trash code. You're like working together. So to like propose something, I'm like, don't know, maybe do it this way instead. And it goes back, does its thing. And it's like, all right, great. Like this is fine. Like let's keep this and like move on to the next piece.

Chris Gmyr (11:16)
Yeah. Interesting. like is I know with like regular chat prompting and I know like you sending us a bunch of examples before like you can get some of those like pre prompts as like multiple pages of all this crazy stuff like I don't even know what's in there, you know? And then going to clog code where it's through the CLI, it seems like you're kind of back in that kind of chat.

TJ Miller (11:17)
so.

Yeah

Chris Gmyr (11:42)
module, but it has a lot more context of whatever project that you're working with. Do you need to do a lot of prompt engineering with Claude Code, or is it more of like you're talking to an actual engineer of like, go write this test, or I don't know. What are the things that you use within those Claude Code prompts that maybe you don't have to in like an AI chat?

TJ Miller (12:05)
Yeah, I don't know. I actually, I think I do look very different. That's not even really that different. Like my prompting's kind of become fairly standard across like both things, depending on what I'm doing. Like, so let's talk a concrete example from Prism right now. There's like a streaming, there's like a streaming issue. We use Laravel's HTTP helper through all of Prism to make all HTTP requests.

And there's like some bug somewhere. I don't think it's a bug with Laravel's HTTP helper, but there's like something that's happening on two fronts. There's like a weird dependency issue. Like if you're on Laravel 12.15 and you have a particular version of Symfony's HTTP library, like there's this like random

bug that pops up that like prevents streaming output from happening. So it's like this weird, very specific dependency version that like a few contributors have figured out the issue with. But beyond that, there's some people who are running into the stream contents being consumed before the output. So like you have a streaming response and like there's something that's like buffering that and consuming the stream as I'm trying to output it. So it's being consumed before I output the stream.

with all the different pieces of content from the provider's response. And I can't figure out what it is, because I can't recreate it. It works for me, and it's always worked for me. But there are several people who have run into this issue. So I decided to spin up a little test branch, and I gave it to one of the folks that were having issues. And instead of using Laravel's HTTP helper, I just went with a straight guzzle client.

Chris Gmyr (13:32)
interesting.

TJ Miller (13:53)
And he said, these switched over to that branch and it worked immediately. So I'm thinking, like, there's a lot of magic that takes place in Laravel's HTTP helper. Like it's resolving out of the container. It's doing a few other things. There's like middleware you can attach to it. There's just like all sorts of like really nice developer experience functionality, but like, I don't know. I can't figure out like what is specifically causing this like stream to get consumed.

because I can't recreate it. If I could recreate it, I'd love to figure out what's actually going on, but I can't. Everything just works for me. So the plan that I decided to come up with is I want to just implement my own HTTP client helper. We'll use Guzzle, and I still want all of the developer experience niceties that come with the Laravel's HTTP helper, or at least a lot of them. ⁓

Chris Gmyr (14:41)
Yeah.

TJ Miller (14:44)
So I like, I set out with Claude. I'm like, all right, we're going to do this with Claude because this should be like something fairly straightforward, especially the boilerplate of like implementing the new client across all the different providers. That's like, that should be really easy for it to do. And that's a lot of like, a lot of work for me to do is like go through each provider, update. It's just like, let's just let this thing rip through it. So my approach with a lot of this stuff now is like, let's come up with a plan.

So I hopped into the Claude and I started brain dumping everything I thought about the HTTP helper. Here's the issue we're having. Here's why we're swapping out the HTTP helper. We're going to use Guzzle. I want these specific features. I want basic HTTP methods. I want helpers for streaming. I want testing things to be able to fake responses. I need error handling.

and like all that stuff. So I like basically listed out like all the features that I want to have inside of it. I got kind of descriptive about a few of those features, but like I'm just trying to give it as much information out of my head as possible so it has that as context. And then I'm like, all right, we're going to plan this out. We're going to do like just like the same approach that I've taken with Prism of kind of doing this like outside in work of like almost

documentation-driven development. Like, here's the design for how I want it to work, and then go fulfill that contract and like make all the functional pieces work. So I took that approach with Claude too. I'm like, all right, so here's what we're going to do. We're going to plan out what the abstraction looks like. And so I had it go through and like basically write up an implementation doc for how to implement this HTTP helper, like core classes, core methods that need to be on these things.

like describe the functionality of everything. So like we just came up with this big markdown document that is like, here's the thing, here's the plan for how to implement this and what it should look like. So I review all of that, make sure I'm aligned with everything. And then I go back to Claude and I'm like, all right, man, let's implement this. So here's the detailed plan for how we're going to approach this in like all of the context that it should need. And I'm like, all right, let's first like implement this thing.

So it just went off, like implemented everything, wrote all the tests for everything, made sure the tests are passing. And now I can go back and say, now that we've built the HTTP helper, let's go swap out that implementation for all the various providers. So that's the approach I've been taking with most things is like doing a planning stage upfront to like figure everything out first and then dive into the code. So instead of like just diving into the code and like,

Here's what I'd like you to do. Let's get really specific and lay out a plan first. And so I've really taken the approach of treating it like I would treat a junior dev and the knowledge that I expect them to have. And that's paid off really pretty well. So these design and implementation docs that I'm still having it right for the most part have been extremely helpful in making sure that it's actually doing what I expect it to be doing.

Chris Gmyr (17:34)
Mm-hmm.

Nice.

TJ Miller (17:48)
So there's a little bit of upfront work, but like that works great. And I mean, I take the similar approaches. I'm like chatting in like a chat interface too, is like, try to give it as much context as possible to like guide it towards the output that I expect it to have. Cause remember like behind the scenes, this is all just like a big probability engine. It's like auto-complete on steroids. So we're trying to like guide it to the right auto-complete. So the more context that you can give it, the

the closer it's going to get to what you want in the end.

Chris Gmyr (18:16)
Yeah, nice. That's awesome. And I know, like in code, comparing it to the chat interface or the app, a repo is basically like a project that you install code in or like, right? And then you can still have individual chats. So you can have this one chat going to implement your new version of the HTTP helper.

TJ Miller (18:30)
Mm-hmm.

Chris Gmyr (18:40)
Right. And then you can have like another chat of like, Hey, help me fix this like other bug over here. And you can swap between them or should you really only be running like one thread like at a time.

TJ Miller (18:50)
I have it just working on one thing at a time. There's been maybe only two or three times that I've had Claude working on two different instances of Claude code working in the same code base doing the same thing, or doing different things. Just because it's a lot for me to manage, because I'm still like...

I'm not letting it run wild for the most part. I'm still like pair programming, reviewing code. So like having to balance between all those contexts is just a lot. So I typically only do like one thing at a time. And that's, that's worked out like really pretty well, I think. I know people get like crazy with it and I've seen some like wild stuff where people have wired up like 15, 20, like coding agents, like doing stuff. And I'm like, this is just.

Chris Gmyr (19:10)
Yeah.

Then.

TJ Miller (19:33)
out of control, like let's do like one thing at a time.

Chris Gmyr (19:36)
Yeah, yeah, totally. And then I know when you install Claude Code in a repo, it has that Claude markdown file in the root. Is there anything additional that you put in there that it doesn't write for you? Because it goes through and analyzes your code base and says, hey, here's maybe some helpful commands, or here's what is in the repo, here's how it works type of thing, maybe some architecture stuff.

But is that a place where you can add additional instructions or any sort of additional contacts that you want to allow to feed into the engine? Or do you use that at all? Or what do you use that with?

TJ Miller (20:13)
Yeah, I use that. I have a PHP and Laravel coding style guide that I've been just tweaking and developing over the last few months. And that works really pretty well. And so I have that inside the Claude code, that Claude markdown file. So it loads that into context on every chat. it has my coding standards and style.

already predefined in there. And then I give it like, I let it do it's like it's a knit thing and I go and just kind of tweak it, make sure it's got the right commands and everything in there to like run the different things. But the coding style guidelines are, I think, like key to getting it to write code the way that I want it. I'm like, no final classes, define strict types, I'm like everything.

making sure, trying to guide it towards using import statements instead of inline classes, because that's something it loves to do is inline namespace classes. that's just, no, we're not going to do that. Yeah, we're not going to do that. So use imports, it talks about using pest tests and how I prefer to write pest tests. So it just gets really detailed about coding style and what I expect out of it.

Chris Gmyr (21:09)
Yeah, it's a little weird.

Nice.

TJ Miller (21:25)
So I put that in there. It's got like, it's basic build commands, some architecture things, but like really, like, I don't know, me see. ⁓

Chris Gmyr (21:33)
I

remember you mentioning the guidelines before, there's public repos for other languages too. So people are looking for other frameworks like Ruby on Rails or TypeScript or stuff like that. There's available guidelines for those out and about,

TJ Miller (21:50)
Yeah, there's some stuff out there already. This one is one that I had Claude initially come up with. So I was like, hey Claude, like I need a coding style guideline that follows Laravel's coding conventions. So that was the first thing that it did is like came up with this like style guide. And then I've just been like, I read through it, like tweak some things to my personal taste and flavors. And then I've just, as I work with it, as things like kind of crop up where it does like something weird.

Like forever it was like doing all these like, like I said, like inline namespaces. So I had like went in and like tweaked it so that it you know, specifically called out to like not use inline namespaces to use import, like use statements and stuff. what I ran into is it kept on, it kept on getting the namespace wrong for Prism. It kept on like using an old.

Chris Gmyr (22:23)
Mm-hmm.

TJ Miller (22:39)
old namespace and I have no idea where it got that from, but it keeps on using like echo lab slash prism instead of like prism slash prism. So I had to like go into that Claude file and I'm like, Hey, the namespace for this project is prism slash prism. Like stop it. Like, so just as in like, this is what I recommend to you is like, as you run into stuff using it, like go and tweak that file, like just edit it as you go. and things will just kind of get better and better.

Chris Gmyr (22:51)
You

TJ Miller (23:04)
And one of the big things too is like when I'm like kind of, when it's like done implementing something, I like to ask it to go back and like clean up after itself. Cause it'll, it'll thrash around a little bit, like creating craft or like, I've definitely found like methods that it created that are never used. And so it's like, Hey, go clean up after yourself a little bit. like anything that's like super complex, maybe make it a little bit more simple and like kind of try to get it to do that refactor cycle afterwards.

I found that to be like super helpful too.

Chris Gmyr (23:34)
Yeah, that's really cool. Did you commit that Claude markdown file to Prism or is that just for local?

TJ Miller (23:41)
No, I've got that as part of my like global get ignore.

I don't have a good reason for not committing anything, don't think. just, I don't know. I maybe should.

Chris Gmyr (23:50)
Yeah, I don't know. just thought

it was interesting. If other people are trying to use these tools, especially on AI-related things, I don't know. It could be something cool just to push out there and share. And if other people were also doing the same workflow that you are on their own branches and features, maybe it would be less work for you and or Claude to do on the back end of that for pulling down

TJ Miller (24:08)
Mm-hmm.

Chris Gmyr (24:14)
PRs, branches, refactoring things, doing code reviews, things like that. I don't know. It could be something that could end up saving you a little time.

TJ Miller (24:24)
Yeah, no, and I think that as people use Claude Code now that it's generally available to you, they'll inherit the rules that I'm using and help keep the coding style consistent across the project. So it's probably a good idea to keep that in there.

Chris Gmyr (24:40)
Yeah, I like that. Sweet.

TJ Miller (24:43)
Yeah, that's some cool stuff, Yeah, I've been loving Claude Code. Like I use it, like I said, I use it pretty much every day whenever I'm writing any code. And it's been, I've had it been crunching on quite a bit of Prism stuff too lately, especially like stuff I don't know at all, which is, I always feel like is a really dangerous place to be using AIs when you don't know anything because you can't like,

can't check its work. You end up trusting it a little too much. But the other thing is I kind of want this stuff out there, and I don't know anything about it. And I also don't want to sink the time into learning it either. So.

Yeah, it's been super helpful. I love it.

Chris Gmyr (25:25)
Yeah, and it's and it's I think fine to iterate over those things too, because like in maybe like those contacts like you're maybe more of the junior engineer, right? And then like once you get the code out there like someone else can comment it. That's the benefit of doing it open source. So other people who know more about that specific context they can chime in and help out and be like, well, if we did it this way, you're I don't know enabling this. You're bypassing this issue and you know around around we go.

TJ Miller (25:34)
Yeah.

Chris Gmyr (25:52)
So I know. I think that's totally fine.

TJ Miller (25:56)
Yeah, yeah, and I think even just getting the initial version of stuff out there is like, even if it's not great, or it's maybe doing some things in a weird way, at least it's out there. yeah, you can start getting those. If people are going to use it, you can start getting that outside contribution feedback loop back into it, which is always super helpful.

Chris Gmyr (26:14)
Yeah, totally. Sweet.

TJ Miller (26:17)
So

I guess we can lead that right into some Prism updates if you're up for it. Yeah. So I just talked a little bit about that big guzzlery factor. because I've been really prioritizing that because there are people currently trying to build with streaming output that are having issues. So I want to fix that. Because it feels if.

Chris Gmyr (26:21)
Yeah, let's hear some updates.

in

TJ Miller (26:40)
even though I can't recreate it, it feels like a big important bug in Prism. And so I want to get that resolved ASAP. So I've been focusing a lot on that.

Chris Gmyr (26:50)
Sorry to jetton. I'm assuming that you already asked this, but did you ask Claude to be like, hey, here's the issue, which you already said that you did. But where could this possibly happen in the call stack between the package application code and all the dependencies underneath? Did you ask that, or did it just not come back with anything that helpful?

TJ Miller (27:15)
didn't come back with anything super helpful. And even if it did, the problem is I can't recreate it. So I can't even like test the fixes that Claude's suggesting. So I'm just like, I'm really in the dark on this one. And it's so frustrating. Another piece to this though, is there is like, I do kind of have a little bit of a broader goal to eliminate Laravel as a dependency. Like I've built this specifically for Laravel.

And I think there's a lot of affordances in it for Laravel. But if I can decouple it from Laravel code, like specifically, then that opens up Prism to be able to be used by like the broader PHP audience. And I know that there's like some interest in that. I know there's like some people who have been talking about wanting to use it in WordPress, Symphony. So if I can like further decouple it from Laravel,

like by using just like our own HTTP class, then that's going to be pretty nice. It's like a step in that direction too. So there's a little bit beyond this. There's some like broader goals that I think this aligns with. ⁓ But yeah, like the biggest problem with this issue is I cannot recreate it. Another contributor was able to recreate it on a very specific version of like Laravel, like 12.15.

Chris Gmyr (28:22)
Yep, totally.

TJ Miller (28:35)
But then all of a sudden it started, like he was able to recreate the issue, but then all of a sudden it started working. And so like, yeah, we're like at a complete loss. ⁓ Yeah, it's such a weird issue. So yeah, so working on this Guzzle thing, I've also been super obsessed with telemetry and like metrics and being able to visualize the

Chris Gmyr (28:45)
Yeah, that's really weird. Sorry to Riel.

TJ Miller (29:01)
like the whole process that like Prism undertakes. So that's been, I've gone through a couple iterations of that. So I initially did sort of like a decorator pattern, which worked pretty well, like a driver based decorator pattern. And then someone was like, well, maybe we can try using Laravel events. And I'm like, ooh, that sounds nice being able to just like call some events, like sprinkle some events through the code base and like.

do that, which I liked also and ended up, I ended up talking about this on Twitter a bit. And a lot of people were like, yeah, use Laravel events. That'd be awesome. So I did, but I was having trouble managing, like, parent and child traces because I had to, like, drill down this, like, parent span ID, like, all the way through, like, down to, like,

all the different parts of the code base that we're trying to track. And that didn't feel good either. So I kind of went back to a decorated approach. And I like that approach, I think. But somebody else did some stuff with events. Another contributed to some stuff with events and using Laravel's context helper. So I'm wondering if...

that brings events back into play. got to look at the comment he made on the PR. It was pretty thorough. But I'm really obsessed with getting this telemetry to work. so I want to ship in Prism the telemetry functionality. I want to ship a log driver and a null driver for those. And then I've spun up a separate package for open telemetry integration. So because I'm taking a driver-based approach with it,

we can have this external package because there's like a whole bunch of dependencies to make OpenTelemetry work. And so I just decided to do that as like a separate package that you can include and then add it as like a driver using like Laravel's manager class and everything. So it works just like all the other driver-based stuff in Laravel. So OpenTelemetry is like pretty dope because you can hook that up to all sorts of like other engines. So like one of the ones I really like is

Jaeger, so it's really easy to hook up to there. And then you get the nice visualization of all the span burned down. here's how long the request is. Here's how long it spent making the HTTP request to Anthropic. Then here's how long it took calling tools. And then here's how long the tool calls actually were. And then here's the final HTTP request, where it's taking the tool results and sending it back. So you get this really nice breakdown of everything.

Which is cool when you're trying to debug an issue. Someone on Twitter reached out and was like, hey, my agent's taking a really long time to respond, and I don't know where the problem is. I like, am I dealing with slow, long requests? Or is the tool taking a long time to run? I don't know where the issue is. I'm like, try this branch out. It's got telemetry in it. And that seemed super helpful.

That's also something I'd like to be able to showcase at LeraCon. I'm like super hot on it. It's just something I feel like is really important with all of this stuff. Especially when you're using tools and having to iterate and automate things. I kind of need some sort of way to see inside the black box and visualize what's going on.

Chris Gmyr (32:01)
Yeah.

Yeah, well, there's already so much of that black box magic unknown going on in AI in general. So the more that you can put into the tooling and some of those metrics of the inputs and outputs and where it's possibly getting held up, or maybe it points you in a direction that you can clean things up or make things a little bit smoother, that is all super helpful. And minimizing hopefully the

unknowns and kind of black box-ness of it.

TJ Miller (32:41)
Yeah, yeah, I think so. And so, like, I don't know, it's just something that I feel is like a core tenant to the way that I think about AI is like having that visualization into everything. So I've just had like a bug up my butt on like doing telemetry, even though like no one's really asked for it. Like, I feel like it's just something important to have.

And so it was funny, like I had already started working on it. Then somebody asked for something for it. So it's like, feel like, I don't know, it's just, it's important to me. Um, so I want to get that in there. And then I started working on a like chat user interface to like mess with new streaming stuff. So Laravel put out their react and view hooks for streaming responses. Um, so I started playing with that a bit.

didn't really... Some things didn't work the way I expected them to. Other things worked, like, really well. I think most of my issues are that I just am, like, kinda weak on JavaScript frontends right now, and so I'm definitely, like, shooting in the dark on a handful of things. what I was saying about...

stuff that I don't know having Claude do that was dangerous. I don't know shit about open telemetry. That's where that was going. I don't know anything about open telemetry. So like it wrote all of that on its own and it works. So I'm just going to trust it and like hope that people that know open telemetry better that are using the package like have feedback for it.

But that was the thing that I recently built that I'm like, have no idea and I like don't want to spend the time learning it. I just I want to have it there. ⁓ So that's I had Claude just like run with that one.

Chris Gmyr (34:20)
Yeah.

There's a lot involved with telemetry in general. ⁓

TJ Miller (34:27)
Yeah.

Yeah, it was cool because I was like, hey, these results aren't showing, like the spans aren't showing up in Jaeger. And it like knew how to curl Jaeger's endpoint to like check to see if there was stuff in there. So it got into its own loop of like trying to fix the bug, like firing the events and then going to like check in Jaeger to like see if the spans were registered there. So it had this like really nice feedback loop that it could iterate with, which was fun to see.

Chris Gmyr (34:54)
Yeah, that's awesome.

TJ Miller (34:55)
So back to the streaming stuff, I built a little chat UI for everything and I really need to refine it a bit. But it's led me into doing some cooler stuff with streaming and getting tool chunks in place. Right now it just does text chunks, but I want explicit tool chunks to be in there so that you can call out in the UI, like, hey.

I ask it to find the latest version of Laravel and I give it a search tool, right? So it'll be like, hey, yeah, let me use the search tool. And then you'll see a box pop up that's like, oh, using the search tool with these parameters. right, search tool finished. Here's the results of the search tool. And then it continues on with its response saying, OK, yeah, based on the tool results, here's the latest version of Laravel. So kind of trying to, now that I have this chat.

playground, I'm trying to work on affordances for all of that. But then furthermore, I wanted the chat to be persistent so you could have multiple conversations and go back to those conversations. So now I've got to worry about message persistence. And this is something that a lot of people have been asking for, some affordances around rehydrating conversations, storing messages, storing the conversation. So now that I've got a

a better idea of how to approach that. I'm going to start working on some affordances for that. Because I think people want to be building, there's definitely people interested in building chat interfaces. So trying to make life easy for them too.

Chris Gmyr (36:18)
Yeah, totally. Now, that be like because I know you said you're trying to get away from using the Laravel dependencies. Would that be like another plugin for persistence that would like only be using like Eloquent and Laravel components? Or would you still go like building your own implementation or it would just like look some other way?

TJ Miller (36:42)
I think there's a couple ways to approach this in different levels of affordances. So just simply adding some standard way to serialize message classes and be able to rehydrate them. So at the very least, being able to take a message and turn it into an associative array. That way you can store it as a JSON blob or however you want to store it as a message. So at least a way to serialize messages.

Chris Gmyr (37:10)
Mm-hmm.

TJ Miller (37:13)
so some affordance there and that can be very generic to like PHP. something I've also been eyeing up is, like maybe an additional package for conversation management, right? So like being able to have, maybe some, like have a separate package.

that has some migrations already in it, some like a different like a driver-based approach where you could have a like a Redis driver, so like a cache driver, you could have a Eloquent driver or you can implement your own storage driver with some like callback hooks or something. So that way you can like resolve the conversation out of the database and then your messages are like already pre-mapped back to like message classes and then vice versa.

you know, when you go to like save that conversation. So I think there's probably a separate package in mind for that. And that's probably something I'd want to separate anyways, because it's going to be complex in its own right. And I'm trying to keep Prism like fairly focused on what it's doing and then extracting like packages for other complex things that are maybe related, but not part of like the core functionality.

Chris Gmyr (38:21)
Yeah, that makes sense.

TJ Miller (38:23)
And that's where like with telemetry, right? I'm shipping with a log driver and a null driver, right? Like base level functionality for the thing. And then for open telemetry, it's like a separate package.

Chris Gmyr (38:33)
Nice. Yeah, that makes a lot of sense.

TJ Miller (38:37)
So there's, want to keep working in the chat interface. I know Pushpack is working on a Laravel starter kit that is, it's almost a clone of OpenAI's UI. It looks really pretty cool. I haven't seen the code yet for it, but I've played around with it a little bit and it's pretty rad. I'll end up trying to like dig through that, I think, and find other opportunities to like.

make chat a little bit easier. It's not like I use third party chat interfaces for my chat experience. Like I use open web UI and I really like that. But so much of the stuff that I work on is like not really chat focused. So it's a new space for me to be like playing in expanding prism in.

Chris Gmyr (39:19)
Mm-hmm.

TJ Miller (39:26)
So hopefully we get some stuff in before Lericon to make life really easy and I can showcase some of this conversation memory management stuff there too.

Chris Gmyr (39:35)
Yeah, that'd be awesome. They still got a little time. So hopefully there's enough to get it all in.

TJ Miller (39:39)
Mm-hmm. Yeah, well,

we'll see. I think it's going to be some really in the weeds weekends and just trying to make some wild stuff happen. But thankfully, I've got Claude to work with and be able to pull some of this stuff together. if I can get to planning some of this things out, maybe I can get some contributors to help pull a few things over the finish line, too.

Chris Gmyr (40:03)
Yeah, yeah, I they'll be great. Sweet. That sounds awesome.

TJ Miller (40:06)
Cool, man.

Yeah, do we, you wanna wrap up?

Chris Gmyr (40:10)
Yeah, we can wrap up. So thank you so much for listening to the Slightly Caffeinated podcast. Show notes and all the links, social channels are down below and also available at slightlycaffeinated.fm. Thank you for listening and we'll catch you all next week.

TJ Miller (40:23)
Thank you.