Slightly Caffeinated

Chris Gmyr (00:00)
Hey, welcome back to this Slightly Caffeinated podcast. I'm Chris Gmyr

TJ (00:03)
I'm TJ Miller.

Chris Gmyr (00:05)
Hey TJ, so what's new in your world?

TJ (00:08)
man, geez. Newest thing in my world, is finally shipped the new streaming output for Prism. Like this has been, I've been working on it in earnest since LeraCon. I went through five iterations on it, like figuring out what the API was going to be, like what was I even going to ship? And then,

Chris Gmyr (00:16)
Yes.

TJ (00:30)
It just took me forever to like go through all the providers. So I found out there's nine providers we support streaming for. I had to rewrite all of their stream handlers to like emit the new events and everything. But it's finally done. It's out there. It's tagged. It's released. And so now I've just been like playing catch up on pull requests and issues and

trying to get all that stuff together. that's been like pretty all consuming.

And then like outside of that, just like life stuff, been eating my weight in apple cider donuts. Went to another cider mill because I couldn't help myself. I just, needed more cider and donuts. So we went out to a local orchard and got cider donuts, some pie, some bread, like just all the delicious fall stuff. So yeah, soaking in that.

Chris Gmyr (01:05)
you

Awesome.

TJ (01:25)
trying to get the like kiddo prepped for Halloween because he's making his own costume this year. So just trying to make sure he has like all the supplies he needs to keep chugging along making his thing. So yeah, yeah, nothing crazy. he, my son decided he wants to do science Olympiad again this year. ⁓ I told him and the head coach that I will not be coaching this year, but

Chris Gmyr (01:37)
Very cool.

nice.

TJ (01:51)
Uh, I'm happy to support, but like coaching was definitely a stretch last year and, uh, I don't think I have it in me this year, but happy to like support however I can outside of actually coaching. So, um, he's pretty stoked. So, uh, let them have at it, but like, I'm, I'm, I'm not going to be doing that this year.

Chris Gmyr (01:52)
That's a really good call.

Yep, totally. Is he going to do a similar coding track, or is he going to try something else, like a different topic or subject or whatever?

TJ (02:26)
Yeah, he tried some, he's trying some new stuff. He's doing an event called Poisons and Potions. I have no idea what that is, but that's one of the ones he signed up for. I'm assuming it's like some chemistry stuff. and then I think he also signed up for Code Breakers, which is like a cryptographic, like puzzle breaking kind of one. so that seemed pretty cool. I think he'd be into that. Like.

like ciphers and stuff. so I don't know. We'll see. I, I wasn't surprised that he like didn't choose the coding one again this year. I think it's good to like mix it up, you know, and try something different. I know one of his friends did like the same event he's done for like years. And so I think there's like,

I think there's something to be said about like kind of repeating the same event and being able to like, you know, try and get a different outcome from it. But I think there's also lot of merit in like trying something new too. So.

Chris Gmyr (03:18)
Yeah, for sure. Especially when they're young, I feel like just going wide on a bunch of different subjects might be the, maybe better for some people, the way to go. Just so they can see and test out different things. Because even though they might like one subject or activity really well, they might like a different one better. So if they never try it, then they'll never know.

TJ (03:39)
Yep. Yeah. No, and that's kind of like what I told him about like picking classes in middle school and like when he goes to pick classes in high school to like pick stuff that like you're not going to have an opportunity to take again. Like. I really appreciated all the years of like drafting that I took, but I also kind of wish that I would have taken like shop class or like wood shop or engine shop, like automotive, like whatever. Like I kind of wish I would have taken some of those too, just to kind of have like.

some of that like foundational knowledge for those things. Um, cause like, don't have another opportunity to take that stuff again, you know, like, unless I wanted to go like audit some college courses for it, but like, I have life now, you know, back, back in high school, it would have been great to take that stuff, have that like foundational functional knowledge for the rest of your life, you know, and, have the opportunities to take it. So I know it's the same approach I've kind of taken with his like,

school to like go try a variety of things like you never know what's gonna land.

Chris Gmyr (04:39)
Yep. Yeah, I like that.

TJ (04:41)
Yeah, man, how about you? Like, what's new in your world?

Chris Gmyr (04:43)
Um, yeah, I mean, just been traveling around a little bit. Last weekend we went to Virginia to visit my cousins, see her older son's football game, because he's a senior and he only had a couple more games left. So we went and saw the game and hung out for, you know, Friday, Saturday and came back home on Sunday. Just did some like yard work and stuff like that, Sunday into Monday.

Then this week we're going to go to Pumpkin Patch. go to like a big, like there's a bunch of like family farms around here. So you can do like, you know, obviously get like pumpkins. There's like treats and stuff like that. You can ride ponies. There's petting zoos. There's a bunch of activities for the kids and like corn mazes and hay rides and stuff like that. So we're going.

TJ (05:16)
Yeah.

Chris Gmyr (05:37)
the four of us and then some of the other scout families that we're friends with ⁓ are meeting us up there too. So that'll be fun. Nice little get together for Saturday morning. And ⁓ we also have the NC State Fair just kicked off today, I think. So maybe Sunday we'll hit that up and check that off for this year, which is always fun.

TJ (05:58)
Yeah, that sounds like

a ton of fun, man. It reminds me, I had to go pick up some pumpkins too.

Chris Gmyr (06:04)
Yep, getting close. So yeah, good fun things, activities, stuff like that. And then just cranking at work, doing a whole bunch of different projects, which we'll get into shortly. But yeah, everything's generally pretty good and getting cooler by the day, which is nice. So hopefully we're out of the 80 degree weather, which is a little weird to say because usually going from the Northeast, you just want that warm weather.

TJ (06:06)
Yeah. Yeah.

Yeah.

Chris Gmyr (06:32)
But down here it's like, I just need a break and be okay with going outside with a sweatshirt on or something like that. Like it's just, I don't know, need a break.

TJ (06:38)
Yeah.

Yeah, I'm like my sweet spot is like somewhere between like 65 and 75. So like I live for spring and fall. So the weather has just been like perfect. I think I went for like two walks yesterday. I'm just like, it was gorgeous out and just like soaking up that that sweet fall weather.

Chris Gmyr (06:59)
Yeah, nice. go outside and get some more walks in, take the dog or something like that, even though it's only like 15 minutes or something. working out is great, but it's only three days a week, like first thing in the morning. And then I feel by midday and especially the afternoon, just sitting down all the time or even standing at the desk, it's like you just get stiff and.

TJ (07:06)
Yeah.

Mm-hmm.

Chris Gmyr (07:22)
So it's like, sometimes I just get caught up with, you know, whatever I'm doing. And it's like, get to the end of the day and it's like, I should have went outside or walk around a little bit more. I don't know, gotta block something on the calendar or do something to get out and move around a little bit more besides just going to the gym.

TJ (07:30)
Yeah

Yep. Yeah. Especially when I find like I've not been going to the gym, which is something I need to make a change for. But, I found so much going to the gym, like just getting out for walks and like staying loose was like so nice. Cause I'd get, you know, like on gym days, like towards the end of the day, like you start, start feeling like stiff and kind of like, you know, all contracted from like working out. So like getting outside and like, just kind of like, yeah, keeping things loose was super important.

Chris Gmyr (08:06)
Yeah, yeah, for sure. Nice. We'll talk about some coffee real quick.

TJ (08:08)
Cool,

Yeah,

dude. Yeah, I don't have a ton. ⁓ I picked up a couple of bags of counterculture. ⁓ I've just been kind of on a kick. I picked up some, counterculture hologram on sale, a little while back and I just kind of got, got hooked on it. So I picked up two more bags and I've just been like living, living on that in the French press. So, thoroughly, thoroughly happy with it. but I've been, I wouldn't want to try one of these like

Chris Gmyr (08:25)
Yeah.

TJ (08:35)
sacred cup bags we've been talking about for the last couple weeks too, so might have to order some of that up today.

Chris Gmyr (08:41)
Yeah, for sure. I love counterculture, so those are great picks. But yeah, I've just been sticking with the secret cups. added ⁓ two more bags, the ones that I mentioned on previous episodes, because they still had them available. So I added it to the regular subscription that we get and just been enjoying those. And I don't know, tastes pretty good. And I enjoy them while they're still available.

So it'll probably be pretty boring on the the new coffee topics, but I know it's solid. I'll take it

TJ (09:05)
Yeah.

Yeah, man, I love it. Yeah, I want to get a couple bags of these before they disappear too, because that just sounds like, sounds really good. And I like the cause too. Like it's, it's, it's one of those purchases that like, you feel good about making also, you know.

Chris Gmyr (09:26)
Yep, yep, totally. Sweet.

TJ (09:28)
Sick. So you want to talk some, ⁓

anthropics and clod news?

Chris Gmyr (09:33)
Yeah, new Cloud news. They just launched Haiku 4.5 yesterday. I believe it was. Is that yesterday? Yeah, yesterday. So yeah, it's a much smaller model than the Sonnets. It's supposed to be a lot quicker, but still good for some tasks. So I'll put the launch announcement in the show notes.

TJ (09:41)
Yeah.

Chris Gmyr (09:56)
Yeah, they're using it in a couple of examples in like cloud code. So it's still good for coding. It's really good for quick multitasking. So they're showing it in the Chrome extension that they built out. And yeah, it seems pretty cool. haven't, I think I've only tried it like a couple of times, because it was later when I noticed it yesterday. But yeah, definitely could be doing a little bit better for some of the ⁓

multitasking or smaller code changes, instead of using the big more expensive rate limited models. So yeah, kind of tinker around with that a little bit more, but it seems pretty cool on the surface.

TJ (10:31)
Yeah, I haven't played with it a whole bunch, since it came out, but, I've used Haiku a little bit in the past, but I normally just stick with like the Sonnet series for the most part. ⁓ just striking like that really nice balance between speed and, productivity or like speed and intelligence, I guess. ⁓ but looking at their benchmarks, it's like, you know, it was a smaller model, but that trade off is the speed.

Chris Gmyr (10:49)
Mm-hmm.

TJ (10:56)
and it's the benchmarks are really pretty impressive, like landing Haiku somewhere between sonnet 4.5 and sonnet four. like definitely tracking pretty well. And you know, the Haiku model is still, you know, it's, scoring around, GPT five, and scoring better than Gemini 2.5 pro. So like, it's definitely not something to sleep on for sure.

Chris Gmyr (11:21)
Yeah, yeah. Yeah, so I got to take a more, a little bit, but yeah, definitely got some positive results so far. So yeah, and if it can like speed up those like smaller tasks, like not like huge refactorings or you know, whatever, just like smaller, like day-to-day coding things and save some money on some rate limits and tokens and such, I don't know, that'd be a good bet.

TJ (11:30)
Yeah.

Yeah, like you can kind of get into some of those workflows too, like using the more powerful model to do the planning and then leaving the execution phase to something a little bit smaller, a little bit less powerful because it doesn't have to do the planning so much. So that's something I used to do before Sonnet 4.5 was I would have like Opus do the planning and then Sonnet do the implementation. I wonder what kind of...

experience it would be having Sonnet do the planning and then Haiku do the implementation, know. Maybe something to experiment with for sure.

Chris Gmyr (12:18)
Yeah, totally. And I have started messing around in Cloud Code, changing the agents a little bit more. So just like you were explaining, doing the planning and exploration, plan, writing, like we talked about before, I save all the plans into a plans markdown file. Do all that, and then tell one of the smaller models to just go do it.

So yeah, really successful results with that. So now that I can check out IQ moving forward, I'll be tinkering with that for sure.

TJ (12:51)
Yeah, I mean, you were experimenting with agents pretty fair bit. How's that been going?

Chris Gmyr (12:56)
Good, I'm still kind of working through the workflows, but basically I'm trying to put all of the agent calls in all the custom commands. So like my custom PR review command or planning command, it says like, if you're asking about, I don't know, this chunk of code or the specific ask, then look at these agents.

So there's like a code review agent, there's a TypeScript agent, there's a DynamoDB agent, there's a testing agent. So it says like, if you're talking about tests or re-architecting something or writing tests, like use the testing agent, if you're looking at overall like architecture and like how things flow in between the layers of the application, use the architect. So trying to like tweak and change that a little bit more, but yeah, it seems pretty solid.

But with agents, does use a lot of tokens behind the scenes. I think some resources that I looked at, sometimes it takes up to 20,000 tokens to just initialize an agent and shift around the context a little bit. So you probably shouldn't be using Opus for those things because you're probably going to smash through your rate limits for the day or the week.

TJ (14:10)
Mm-hmm.

Chris Gmyr (14:10)
which

I think like a lot of people are seemingly running into like on Reddit. Like I keep on getting pinged from all this, you know, rate limit, BS go stuff, you know, on Redis and people running into problems and blasting through their weekly limit in, you know, a few hours. I haven't run into that, but I try and keep a close eye on it. And I've really only used Opus for like a handful of things or recently.

and again, trying to stay away from the real limit issues. So I'm not sure if you've since run into anything with rate limits or switching to the bigger models. ⁓

TJ (14:46)
I haven't,

I haven't hit any limits yet. and I'm on, I'm on the like 20 X max, like I'm on the max out max plan. I haven't hit any rate limits. I've been, I feel like I've been using it pretty heavily with like the re both with stuff I'm working on at work and the, streaming output refactor. feel like I really leveraged Claude quite a bit and I didn't, I hadn't hit any limits whatsoever. but I was just.

exclusively using the Sonnet 4.5 model. I haven't been playing around with any other models lately since that one dropped. I've just been straight using that one. I've been playing with agents a little bit, like nothing that complex. What I've been trying to get my workflow down to is the main agent loop.

is the like, that's the planning agent. Like that's all about like that, that I'm treating that as like you're the planner, you are the coordinator. like that's your job is to like kind of keep tabs on things at a high level and then like do all of the detailed planning. And then I have a layer of LPHP specialist agent. And then I try to get it to like farm all the workout.

Chris Gmyr (15:33)
Mm-hmm.

TJ (15:52)
to like, if you're writing Laravel, if you're writing PHP code, use this agent to do the implementation. And then that's got all the context for like coding standards and stuff like that inside of it. And that's been working out pretty well. It behaves, I have to say like, it feels very effective, but it behaves super weird. I've been seeing it like write context to temp files to like send to the agent.

So what I think it's doing is it like, we'll write a bunch of stuff to like attempt markdown file is like context. And then the first step of the agent is to like read that file till then like know what to do. And so that's been, it's like, it's really funky when it happens, but it works. Like it does what I ask it to do. ⁓ it just like does it in kind of this like,

Chris Gmyr (16:38)
Mm-hmm.

TJ (16:41)
wonky way. And so like, I don't know if you ran into any of that with your agents and stuff, but it seems really effective. But I don't know, maybe I'm just like, perceiving it to be more effective because it's more complex. Like, I don't know. It seems to do a better job. It seems to get closer to the mark of what I'm looking for. And I haven't hit

Chris Gmyr (16:57)
Yeah.

TJ (17:06)
Like because it has its own separate context compared to like the main loop, I haven't hit conversation compacting quite as much. but yeah, it's definitely like, it feels a little weird. Like it's workflow feels a little weird, but it, it's doing exactly what I'm asking it to do. So.

Chris Gmyr (17:25)
Yeah, I've had the exact opposite problem in some cases where the agents and the work and the context switching in between like the main chat and the sub agents. Some of those contacts have been so large and comprehensive and actually choose up the main context window more.

So I was actually looking for different solutions to actually instruct the agents to write to either like a file or a temp file and then have like either one, the main contacts or the sub agents read that. Cause then it keeps the context in the window low because for some like really intensive operations, the context window would be so large. And I feel like one day,

I like I compacted probably like seven, eight times in like a few hours. It's like, what is going on here? So just exploring some other options with that. there might've been like an update or something that changed like how subagents work with like higher levels of contacts. I don't know, I'm just speculating there. But I was actually thinking of implementing something within the agents of like, if you're passing a large amount of contacts back and forth,

Don't do that, basically limit it to like 200 tokens and write to a file and in your main like 200 ⁓ token limit, pass the reference to the file back to the main context or vice versa. Keep the context switching as minimal as possible and reference files. So maybe it's the better way to go.

TJ (18:59)
Yeah,

yeah, no, it's interesting. I might try to standardize it so it tries to keep the state out of the actual temp directory.

Because that breaks the like, because it's not operating the same root directory, it kind of like breaks out of the flow. And you have to approve it to run this command inside of the temp directory. it just gets kind of messy. I wonder if I can like, if you're kind of heading in that direction too, I wonder if I can try to guide it into using like temp files inside the project instead as like as context.

Chris Gmyr (19:31)
Yeah.

Irvin like saving them into like the .cloud slash context directory, making it a new one of those. Because I could see with jobs that are run all the time with the same agent, maybe on the same branch with the same workflow, like it might be more beneficial to also read something that the agent did, you know, an hour ago.

TJ (19:38)
Sure, something like that too.

Chris Gmyr (19:55)
to build up that context or summary of whatever happened more easily. So maybe if those files are longer lived, maybe they could leverage them also instead of doing all the agent work.

TJ (20:08)
Yeah, yeah, I think there's a couple ways to like maybe approach it. ⁓ I'm not

I don't know if I'm like super sold on agents yet. I don't know, where do you stand? Are you like pretty sold on it being like a solid solution or are you still in the like testing it out experimenting phase?

Chris Gmyr (20:25)
It depends on what you want to do, I think, because at least with the agents that I'm working with so far, they seem to be really comprehensive and thorough, which is good in some cases, but sometimes it's just like way too much also, like finding all of these like, you know, crazy nitpicks or something like that in, you know, a PR review where like I wouldn't really care about it, or some

it's raising some potential security thing that isn't actually an issue because of like how we have our like infrastructure set up, like constantly flagging those things where I feel like the main context that wasn't as strict about those things. So I think it depends on the job that you wanna do and how strict you're designing your agents to be.

TJ (20:57)
Mm-hmm.

Chris Gmyr (21:11)
There could be like different levels of agents. So you might have like a super strict or like super secure security agent versus, you know, a lightweight one. Maybe that's, you know, a different way to go. I haven't tinkered with like different levels of severity or complexity with that. I'm not sure how well that would work or if you can tell a specific agent, like be a little bit more relaxed because of XYZ.

It might be able to handle that, but I don't know. I'm still tinkering and kind of finding my way through it. Just in general, it seems like the agents, least the way that I have them set up, are a lot more comprehensive, which depending, again, what you're working on, could be a pro or a con.

TJ (22:00)
Yeah, something like, I feel like I did a sort of fair amount of research before I got into it. But I also know that I didn't do a ton of research into it either. so I wonder, I wonder spending a little bit more time with it, like what patterns, like I can go find a, see if we can improve it at all. I don't know. This whole like agent OS thing is also pretty interesting. ⁓

Chris Gmyr (22:22)
Mm-hmm.

TJ (22:23)
something we can maybe link. This is something that you sent me from like Brian Castle's stuff. I've had it flagged to like look a little bit closer into it, but I haven't really yet. But I like the approach, right? It's like, it's builds itself as like a system for spec-driven agentic development, which is like really kind of what I've been pushing for since the beginning of all this was like, get your spec, get your plan together first, and then

and then have it go off and implement instead of trying to do it on the way. ⁓

Chris Gmyr (22:51)
Yep, exactly.

Yeah, and this is really cool too, because he's basically using it to build new products. So he's using it to build builder methods and a couple other side projects that he has. And it's cool, like you write up the spec and then it takes the spec and all the agents kind of break that down into different technical docs, to-do lists, things like that. And then it'll just go through and chunk everything through.

And as it's like working through the specs, it'll ask questions to validate or go like in different directions. So yeah, it's really impressive of what he's done so far. And he just released Agent OS 2.0, the end of last week, I believe. So I got to tinker more with that a little bit more, but yeah, definitely check out Builder Methods and what Brian Castle's doing. He's also been doing a bunch on his YouTube channel.

And he has some paid for but relatively inexpensive workshops that he's done and moving forward to.

TJ (23:52)
Yeah, super fascinating. Like I think there's like really interesting use cases. Like, I know. I feel like I've got like two use cases, right? There's like working on an inside a fully fledged like product. And then there's also like package development. And I think those are like very different. They're sort of similar, but like very different spaces to be working inside of. And so some of this stuff like agent OS, I think seems really good for like.

building and working inside of product.

So I mean, I don't know, it'll be interesting to see like how something like that plays over to like package development and like tooling and stuff like that too. So, I dunno, I, I've definitely had it flagged to like look at it since I think you had mentioned it to me a few weeks ago.

Chris Gmyr (24:34)
Yep, and I joined his pro group. So I'm in his like Discord and chatting with some people in there, which has been really interesting. And there's some, definitely some super users in there, but it's really interesting just how he has Agent OS set up. And I think even though it's set up for more of like the product world, it's all just markdown files and agents and custom commands with cloud code.

So you could definitely take that and run with it and have it build out more like package specific tooling, which is really nice. That's what I really appreciate about the system too is that it seems pretty flexible and you could probably repurpose or get rid of a few things that you don't need.

TJ (25:18)
Yeah, no, this is interesting. Yeah, I definitely want to like read through the docs and like maybe play with it this weekend. Kind of see where it goes because I've got some upcoming work on Prism that I need to like, I've got like a big list of issues to go through. And so I wonder how much like this can help me get through some of those, like plow through some of that work. I don't know. It's interesting stuff for sure.

Chris Gmyr (25:40)
Yeah, that'd be cool. Let us know how it goes.

TJ (25:43)
Yeah, man. moving forward, I'd love to hear about some of your work, like even working with MCP servers a bit. I would love to hear a bit more about that.

Chris Gmyr (25:54)
Yeah. So for work I've been working on, how do we implement more AI in the engineering org? What are some of the tools that we need? So I spec'd out a big plan for the team to look into. So upper leadership is looking into that right now. But I have a bunch of little side projects and POCs.

that are working on kind of behind the scenes. And one of them is an MCP server or package documentation. So we have a lot of repos, a lot of different, not a lot of, but several different tech stacks. We've aligned on newer stuff being like node, TypeScript, Fastify for like API development and a handful of other things. But we have some like old like Ruby on Rails. We have some Python stuff. We have some random things, you know, here and there.

And it makes it hard to figure out what you can and can't do in different repos as you switch and move around, do some guest engineering, because someone might be on an older version of Fastify or a newer version of Xa that we use for data validation and DTOs and stuff like that. And even though AI is pretty good at

like picking out like patterns, it's not great for like what's the latest information or like how to do acts in a specific version of a package. And it makes it even harder if you're switching around to different repos and everyone has different packages and it's like, I did something in this repo yesterday. I want to do the same thing over here, but I can't because it's on a different version or some patch version or something like that that's not available one way or the other.

So what this MCP server aims to do is basically for all the repos, it's gonna have like a package that we can install on everyone's repo that will basically scan all of the unique packages, dependencies, and versions of that, send it to the server, and then the server will queue that stuff up. It'll go look at

GitHub, it'll look at NPM, it'll look at, like if you're using PHP, it'll look at like Composer, it'll find all the documentation that is available. So it'll look in like the GitHub docs directory, it'll go through the readme, it'll go out to NPM, it'll go out to see if there's like a website with docs available on it. It'll basically scan all this stuff and then put it in Postgres.

with PG Vector database and then convert all these chunks of pages into vectors to then be searched through in the database. Then we have like those packages and versions saved to the clod.md file. And then when you say like, how do I do X in Fastify, it'll look.

to see and match up that you're asking about Fastify, which is a dependency. And now that's the trigger to actually do the doc search via MCP. And it'll call out to the MCP server. It'll know the application that you're calling from and pick out the version that you have logged in the MCP server. So it'll say like application one is Fastify version five. So I'm gonna respond with ABC.

of how you do that. But if you're calling it from application two, it'll say, oh, you're on Fastify version four. So I'm going to respond differently because you're on a different version of that. So multiply that by all the different dependencies, packages, versions, all that is really cool. And doing some initial testing with my main repo that I work in, would not only bring in the

TJ (29:36)
Sick.

Chris Gmyr (29:54)
documentation of, like for example Fastify, it would also find patterns and match that to what's actually available in the repo as well. So like here's the actual docs and here's an example of where it's used like elsewhere if you're not aware or like looking at that section. So it's really great bouncing around to different repos and if you don't want to or can't find like an exact example, it'll do that for you.

TJ (30:20)
That's so sick, man. Like, I think that's very advantageous.

Dang.

That's super cool, dude. Like, I don't know what else to say. Like, that is so cool.

Chris Gmyr (30:27)
Yeah. That's pretty sweet.

TJ (30:33)
Yeah.

I want to see the implementation. love this.

Chris Gmyr (30:37)
Yeah, it's pretty cool.

It's still like early days, so it's a little rough. But really to get from, I was working on the idea with Claude Chat via web, so I could do it. when ideas popped into my head, like after hours or like on the weekend, I would just pop into a chat and be like, well, what about this? It's like, ⁓ yeah, let's change this, that, and other thing. Finally got to a point where I could say, OK.

TJ (30:57)
Yeah.

Chris Gmyr (31:01)
take all this stuff, extract it to basically a technical implementation doc that I can put into a new repo and then Cloud Code will just run with it. So the planning took, I don't know, on and off, maybe like a week, week and a half or so. But once I created the new repo and booted up Cloud Code with the implementation plan, we basically got it done in two half days of work.

So within like 24 full hours, like I had something working with just two packages. It had scanned everything and has like a queue server behind the scenes with like Valkey Redis. It goes out and just there's like different like fetching workers based on like if you're getting it from GitHub or NPM or like a specific website.

TJ (31:33)
Yeah.

Chris Gmyr (31:50)
And then goes and does like the embeddings and everything is like tracked now. So if there's like a failure, it's like, well, I needed to scan like 11 pages, but I only got like five of the way through the queue will like pick that up. then once the rate limits are raised or whatever the issue is, those will automatically like pick up and finish the job to scan all the docs and make the embeddings and put those into the database. So it's been.

Pretty cool project. I'm so glad that it actually works.

TJ (32:22)
Yeah.

No, man, that is like, it's such a cool workflow. And I think it's super valuable. you're working on, yeah, like being in the environment you're in where you've got all of these like separate teams working on separate repos with separate versions of things. I mean, it's hard enough to get like,

Claude code to stay like it's hard enough to get feed it like up to date information, let alone information for specific versions of things. So, no, I just, think this is like such a smart, smart approach to it. And like, it's going to really, I think. Make the like development experience through like Claude code so much better, like so much better for the team. Like I've seen,

I'm working on my first AI project at Geocodeo and what I did is I spun up like a fresh layer of all app to kind of like work on the stuff inside of it. I added boost and then somebody contributed AI guidelines for Prism that like integrate with boost. So I did that too. And like, man, like just having those like AI guidelines for Prism, it knocked out some like pretty wild stuff.

using Prism all the correct way and everything, which was super cool. just actually being able to witness the power of having those docs available makes the development experience a million times better.

Chris Gmyr (33:45)
Mm hmm. Yeah. And maybe that's even something to look at to include in the package somewhere is almost like a Prism agent that knows like specific ways of like how to do stuff in Prism. So like your the main contacts are like the Laravel agent like doesn't necessarily have to know the ins and outs and then that

Prism agent could change over time as you like increment the versions, as you have new features and functionalities, different ways to do things, new availability for stuff. And then you would just like pull that in or do like a Sim link or I don't know, whatever you need to do for like global development, but like the official like Prism agent to help you do implementation with cloud code. That might be pretty sweet.

TJ (34:34)
That'd

be pretty cool. Yeah, cause the, it's not a little bit of context. It's like 4,000 tokens, I think. So like the guidelines for Prism are like, I haven't really looked at them. Somebody contributed them. didn't give it too close of a look. I just think mostly skimmed them. So like, I don't know what kind of optimizations I can do, but like 4,000 tokens isn't a little bit. So,

It might be something to revisit there. Or yeah, break that into an agent and have an agent available or something. I could always slap together a Laravel command to pull the agent in and everything to you if you're using Cloud Code. Because I think Cloud Code is the only multi-agent one at the moment. I everybody else is single threaded. So I don't know. It might be something interesting.

Chris Gmyr (35:08)
Mm-hmm.

Yeah, totally. So yeah, MCP POC is going well. I'll show I'll have some more updates as we move forward in the next few weeks. I'll keep you all updated.

TJ (35:29)
Yeah, I can't wait to hear more about it, man.

Chris Gmyr (35:31)
Yeah, well I'm speaking to Prism about some updates.

TJ (35:35)
Yeah, man. Nothing super crazy outside of streaming output. I'm just so stoked to have finally got that shipped. I think there's probably some like. I know it needs some work, so like I'm going to let it sit for a little bit. Hopefully people build with it and use it and I get some feedback on it. I can like iterate on it before. But this is like the last big piece before 1.0, so.

I want to like let it simmer for a little bit, get used, get some feedback on it, iterate, and then we'll be at 1.0 like soon-ish. So pretty excited about that. ⁓ But there's, I feel, I feel bad. like going to the repo and it's just, it's nasty with like issues and PRs. And I think I've got the PRs like fairly whittled down over the last couple of days, but now I got to go.

Chris Gmyr (36:07)
Yeah, that's awesome. Yeah.

you

TJ (36:25)
target issues and I need to go implement streaming output for like the bedrock provider since that's a separate package. You do like streaming output there too. So.

Just chugging along. It's nice to be able, it's nice to be out of the streaming output refactor and be able to mentally focus on other things because that was just such a big blocker and took so much mental overhead and work that I'm just, I'm so thankful to have that out there. Yeah, and I think I've got my first feature too that I'm turning down.

Chris Gmyr (36:54)
Yeah, out of the woods.

TJ (36:59)
which is interesting. It's one of the first things that I feel like it's out of scope for the package. And just thinking about the maintenance burden of it, one of the features is some providers like OpenAI and Anthropic and think maybe XAI, they allow you to upload files to use as context.

there was a contribution aiming to add file upload handling to Prism. And like, I think I want to keep Prism just focused on like generation. But as an open source like maintainer, this is like, I think the first time that I'm like, I'm approaching a feature and just like, no. I think it's great. I think it could be a separate package to like handle this stuff, but

I've got to draw the boundaries somewhere around what Prism is. think that was a forcing function to get to a place where I'm like, Prism is about generation, right? And enabling generation. I think handling the file uploading is just slightly out of scope. there's a maintenance burden attached to it that I don't know if I'm willing to take on right now.

So it's just like, it's a one person team at the moment. So like, I kind of got to be like mindful of adding stuff to it because then like I'm on, I'm on the hook to maintain it. so yeah, that'll be a first, first, first time to handle something like that. But, ⁓ other than that, man, just playing catch up for now, like knock out the issues and then, ⁓

Chris Gmyr (38:23)
Yeah.

TJ (38:30)
kind of see what's next. I've definitely got a few features that I I want to work on, but I want to get the issue countdown for sure first.

Chris Gmyr (38:38)
Yep, totally. Yeah. Now that you're out of the woods with the streaming, at least for the most part, think you'll be able to put some decent energy into the issues and PRs and other things going on, clean it up a little bit and see what the feedback is from streaming. then hopefully that 1.0 is on the horizon pretty soon.

TJ (38:57)
Yeah,

man. think I definitely feel like it is, which is a big one. As far as I'm concerned, everything's stable right now. So like there really shouldn't be any more API changes that I can think of. So yeah, not so bad. Yeah.

Chris Gmyr (39:11)
Very cool. Yeah,

we'll keep us posted on how everything is working out, and we'll be here for the 1.0 launch when you're ready.

TJ (39:18)
Oh, for

sure, man, for sure. All right, dude, on that note, you want to wrap up?

Chris Gmyr (39:22)
Yeah, let's wrap up. Thanks for listening to the Slightly Caffeinated podcast. Show notes and all the links social channels are down below and also available at slightlycaffeinated.fm. Thank you for listening and we'll catch you all next week.

TJ (39:34)
Yes, yeah.