Slightly Caffeinated

TJ Miller (00:00)
Yo, welcome back to the slightly caffeinated podcast. I'm TJ Miller. So Chris, what's new in your world,

Chris Gmyr (00:05)
I'm Chris Gmyr

man, lots of lots of the same. Family stuff has some like scout things going on tonight winding down for like summer with the kiddos. And then just cranking on a whole bunch of stuff at work, which we'll talk about in a little bit. But yeah, it's it's been pretty good, even though it's been busy like it's been nice outside like weather's been sunny, not too hot yet.

TJ Miller (00:10)
Hahaha ⁓

Chris Gmyr (00:33)
think we're going to start getting some heat later this weekend and into next week. So it's been kind of nice. Usually it's like super hot by now. So I'm not sure what's going on, but it's been nice and chilly in the morning and then gets up to like mid 70s in the afternoon. But probably in the next week or so, it'll be up to 90s plus. So enjoying it while it lasts.

TJ Miller (00:53)
Jeez, that's

so crazy, man. Today is like a high of 60. And like yesterday was like high of 50. So we're still a little chilly.

Chris Gmyr (01:07)
Yeah, But yeah, nothing too crazy for me. We'll get into some work stuff in a bit. But yeah, what's new in your world?

TJ Miller (01:11)
Yeah.

Oh man, it's been a hectic couple weeks. So on Friday, I think we talked about this a little bit, I'd been getting ready for my wife's graduation last Friday. And so with that, her dad was coming into town and there were just a lot of house projects to take care of getting ready for her dad to be here.

Chris Gmyr (01:28)
Mm-hmm.

TJ Miller (01:40)
just deep cleaning everything. I had painted the living room. So that was like, that was a big project, which is great. Like it's been needed to have been painted for a very long time, years. And so like, it's really nice to be able to walk into that room and not constantly be reminded that this like huge project needs to be done.

Chris Gmyr (02:01)
Yeah, totally.

TJ Miller (02:02)
And honestly,

like the biggest thing that always held me up over the years was like deciding on a color. We got a new couch. It was a very, it's very green. Like we did, we got it from TikTok, but it's, it looks exactly like AstroTurf. It's hilarious. But we could never figure out what color we wanted to paint the walls. So we got this new couch and we're like, right, this couch will be the centerpiece. Let's plan the wall color around this couch.

and like couldn't figure out what to do. Sent, did I talk about this? I like sent a picture of my living room to chat GPT and I was like, yo, what colors would look good with this couch? And it like gave me a few suggestions and I'm like, ooh, I kind of like this color. And so it like did a render of like both pictures, like as if the walls were that color and it looked great. So we went out to the paint shop and we like, found a swatch that like matched and

Chris Gmyr (02:35)
no.

TJ Miller (02:56)
took home, painted it all up, put the couch up against it, looks identical to the picture from ChatGPT, and it looks so good. So I was pretty stoked about that. Ooh, I've got something cool to talk about too in a minute. But that was really cool. And so that project is really nice that it's done. Graduation was great. Dude, I bawled the entire time. I'm so freaking proud of my wife. She decided to go back to school eight years ago.

Chris Gmyr (03:02)
That's awesome.

TJ Miller (03:24)
And so this has been like, and it took her about a year to get started. So this has been a seven year journey for her taking as many classes as she could at the time, which is sometimes only one, sometimes a couple. She took like condensed courses and like through surgeries and major illnesses and stuff. I'm just, so freaking proud of her. I cried the entire time. was, but it was so cool. Yeah.

Chris Gmyr (03:47)
Yeah, it's quite the accomplishment. It's been a long

time and a lot of work.

TJ Miller (03:50)
Yeah, and like she's not done. She's taking the summer off and then she's out to like ghost. She wants to do a PhD. So she's like, you know, get back at it. So that's really cool. So we had that and like that was a big weekend. We did that. We hung out with her dad. So it's just lots building up to that. And then you kind of settled back into the routine of things this week. Yeah, so I picked away on Iris and stuff.

I don't know, should we talk about, I got like a really cool little tidbit about some like AI stuff I did yesterday. Do we want to talk about it now or like in a minute?

Chris Gmyr (04:26)
I know, we can jump into it now. do it.

TJ Miller (04:27)
All right,

I'll keep it really short. It was so cool. So my wife, her doctor recommended that I 3D print her a month pill caddy. Because I don't know if you can even buy them, but your husband has a 3D printer. Just have him print you out a monthly pill caddy so she can put all her medications. Instead of having you do it every week, she could just do it once a month and everything's filled in and she's good to go.

The one I found is like, it is great. I love it, but it's comically small. You could fit like a couple pills in there at most. And so I needed to scale it up while that then gets bigger. Like the piece that holds all of the pill containers won't fit on the print bed anymore. So I just send the thing off to Claude, like the 3D print file. And I'm like, yo, can you make this?

we need to like make this fit this print area. And I want you to use like dovetails to like interlock it back together so that it's like a secure, like interlock, you know, like a secure seam. And I don't know, man, it like thought and like went back and forth for like 15 minutes, but then came out with it. I loaded it up into like a 3D print app and I'm like, this fits perfectly.

It all looks great. I'm like, this is wild, man. Like, I had no idea it could even work with those files. I just like yeeted the file over and I'm like, yo, see what you can do with this. And it did a really good job. So that just unlocked like a whole world of possibilities for me to like be 3D printing more stuff.

Chris Gmyr (05:50)
That is cool.

Yeah, that's so cool. On occasion, I see like a whole bunch of the 3D print, like Instagram reels and videos and stuff like that. And it's like, some people make some like really cool and like crazy stuff that like I wouldn't even think of and just you can measure and customize. don't know, whatever you want. It's so cool.

TJ Miller (06:19)
yeah.

Yep,

yep, yeah, no, it feels so much like where we're at with AI in general right now inside of like, at least our development sphere where it's like, it's so easy to just build custom tailored software for like the exact specific thing you want it to do the way you want it to work. And I think that translates so much to the 3D print community too.

Chris Gmyr (06:42)
Mm.

Yeah, yeah, totally. I know we've talked a little bit about that before, but I've seen some videos of people just building their own smaller version of a SaaS product or something like that. I saw this one, I forgot the name of it, but the guy who doing it. But he wanted to redo Loom, which is the little video recording service. And then you post it up and off the version, or you just send them the

TJ Miller (07:08)
Yeah.

Chris Gmyr (07:11)
little video file and I like, that's cool. Like it made like the Mac app and like a little like drop down and it's like record this part of the screen and this and that. And it's like, just basically copied the bare necessities of the service and the, you know, screen recording and audio recording. It's like, that's cool that you can just spin up a little app and just do whatever you want to do with it.

TJ Miller (07:34)
Yeah, and I think it's the same guy, I think, I don't know if we shared it, but I think I saw the same thing, where he also has been rebuilding suites of applications. He also built a notion, something, I think a few other things too, along those lines. And he's like, what I also built into it is a chat the application, so you can then further.

Chris Gmyr (07:47)
Mm-hmm.

TJ Miller (07:58)
fine tune the application from within the application. So you can like customize it on the fly to then further fit your specific needs, which is so cool.

Chris Gmyr (08:10)
Yeah, that's really cool. And then I believe it's all open source, all the apps and stuff that he's been making. Man, now I have to find that link again. Sure, it was pretty cool. I'm like, this is a fantastic idea.

TJ Miller (08:15)
Think so.

Yeah. Yeah, yeah, I saw that. I saw that also.

I thought that was super cool. And then, yeah, yeah, I know we've been like kind of sharing it back and forth, just on the topic of like interesting AI stuff. ⁓ the, I've seen several of them now of like pixel, kind of like pixelated artwork, like of like old school games, but like these

agent UIs that are like super gamified with like the pixel art, you know, and like, I don't know, that's, that's been so cool to see. Makes me definitely want to like build one of those too. And it's like, yeah, you could, you could do all of this inside of like discord or telegram or something like that. But you could also have it like be this super cool pixel art game.

Chris Gmyr (08:52)
Mm-hmm.

Yep. Yeah.

Yep,

totally. Yeah, so much cool stuff going on.

TJ Miller (09:12)
Yeah, I guess like coffee wise, like moving forward, coffee wise, it was really funny. My wife was like, still haven't done the Turkish coffee yet, but my wife was, yeah, I know. Yeah, I failed my homework. But I mean, we had family in town. There's just no way of making it happen. Why?

Chris Gmyr (09:22)
Boo. Failed your homework. Failed your homework again.

Bring them to have some Turkish coffee, too. You're going to love it.

TJ Miller (09:33)
Yeah, her dad would have loved it too, I bet.

Yeah, so we, my wife was like, well, we've been having the Costco stuff and it's been fine, but like, I'm graduating, I want the nice stuff. So I had to like clean out the grinder, put in the, fast forward from, what is it I've been drinking from?

It's the same company that I've been drinking for a while now. Counterculture, yes. So we had some fast forward and cleaned out the grinder and we've been drinking through that now. But I thought it was really funny that even she was like, I want the nice stuff. Not the Costco bulk buy. Super funny, man. So any coffee stuff on your end?

Chris Gmyr (09:58)
Counterculture is fast forward.

Really good stuff. Yeah, that's funny.

Nope, same thing for me. I think I put a new bag in my upcoming subscription, but that will be a few weeks out. At least until I get to it. Yeah, hopefully going to try and make it out to a coffee shop at some point soon for some like co-working. But nothing planned right now, so.

TJ Miller (10:26)
Yeah.

Sick.

Yeah, we went to, while my wife's dad was in town, we took him out to some breakfast joint. Never been there before. Had great reviews. And they had like a little espresso machine there. And so I ordered a fancy drink from them and it was great. Like I was actually surprised. Like it was solid espresso. Which is, I mean, it's always risky at like a breakfast joint, but like, I don't know, they seemed.

Chris Gmyr (10:58)
Nice.

TJ Miller (11:04)
We seemed like they could pull it off. It was, you know, a nice machine. So, yeah, it was pleasantly surprised. But yeah, still no, still no Turkish. We'll, we'll, we'll get there.

Chris Gmyr (11:10)
Very cool.

You won't get there.

TJ Miller (11:15)
I'm actually gonna be going, I'll be in the area of it this weekend. So this weekend is gonna be crazy too. We have a concert Saturday night and then we've got Comic-Con, Detroit, the Motor City Comic-Con is on Sunday. I don't get dressed up, but. ⁓

My wife and my son are both getting dressed up. We're going to my sister who will also dress up. My sister and my son are going as Sonic and Tails. My wife is doing Judy Gemstone. Have you seen The Righteous Gemstone show? OK, then it'll be totally lost on you. But it's hilarious. And then, I won't dress up. And then we'll just go down and have a good time. But it's going to make for a busy weekend. But.

Chris Gmyr (11:47)
No.

TJ Miller (11:57)
It will put me in the area of Turkish coffee, so yeah. Yep, I'm gonna do it. I'm gonna be so close to it, I can't, I cannot. It's not like, it's not within the normal paths of like where I travel, so it's like, it's kind of out of the way to make it happen, so being like within a couple minutes of it, if I don't stop, I'll be insane. Like.

Chris Gmyr (12:00)
Nice, just dip out for a little bit, go get a coffee, come back, be good to go.

Yeah. Nice.

Well, hopefully you can make it happen. That'll be good.

TJ Miller (12:23)
Yeah,

yeah. So I mean, so I think you you we've got on the list here, some like hackathon stuff. So I know you had the on site. I think you were talking about maybe doing some hackathon stuff while you were doing that. Love to hear about it, man.

Chris Gmyr (12:36)
Yeah, so Tuesday, a couple of weeks ago at the offsite was basically our big hackathon day. And you could basically, we had a big cool sign up sheet of whoever was the lead of a project or an idea could write a little blurb about it. And then you could basically sign up for whatever team that you wanted to be a part of or project that you wanted to be a part of. So you didn't have to stick with your day-to-day team or even in the same org or anything like that.

TJ Miller (13:02)
⁓ cool.

Chris Gmyr (13:02)
you just kind of do whatever, which is pretty cool. So I think we had 20 or 22 different teams and projects going on for that Tuesday. So we basically started in the morning at 9, 9.30. And then at 3.30, we had to stop to do presentations. So either you had to make some slides or a video or just a

regular walkthrough of what you made or what the team did. So someone else on a different team came up with this idea for a CLI to help with local testing. So we have a bunch of microservices and front ends that use a whole bunch of different back end services. And it's pretty painful. There's some front ends that need 10 or 12 or 15 back ends to run.

TJ Miller (13:51)
yeah.

Chris Gmyr (13:56)
depending on what feature that you want and some of these workflows that we need to test. So you would basically go into each one of those locally and spin them up in Docker and then finally spin up the front end and then make sure you could make sure everything actually works. There's usually some tweaks and changes and things like that. Seating, ports and all that stuff. Data seating is a mess because if you want

TJ Miller (14:14)
juggling ports and, you know, sheesh.

sure.

Chris Gmyr (14:20)
like an ID the same in like these five different like backend services. There's no like easy way to do that. So it's all just kind of a mess. So his idea was to basically like scan all of these different repos and combine like resources in the same Docker compose. So if you had two services that used Postgres, it wouldn't stand up to Postgres.

like resources or like instances would make one Postgres instance with two databases inside of it as you would like in the individual services. So do that across the board. So do it for databases for Kafka, brokers and consumers for Dynamo and instances and tables and a few other things, caching and things like that. So that means you stand up like one stack that just handles everything.

TJ Miller (14:55)
Yeah.

Chris Gmyr (15:13)
like internally. And then it's a custom Docker compose that will feed it into this system. And then you could run everything as intended. So that was pretty cool.

TJ Miller (15:24)
Yeah, no, that sounds like a great idea. Like this is just one of the many pieces of problems that steer me away from service, like heavily service oriented architecture, just because like this nightmare specifically. Like I'm all for services. think I'm more of like a macro service person. You can divide up some things, like, yeah, you gotta like this.

This nightmare is like, terrifies me.

Chris Gmyr (15:51)
Yeah, yeah, totally. So we picked like one kind of scenario or like story and got that working by the end of the day, which is like really impressive in itself because we had one of the bigger teams. We had like 10 or 12 people working on this and he made up like a plan. He did like individual work streams and then we came together for, you know, 45 minutes or an hour at the beginning of that Tuesday to say like, OK, like what are the sub teams?

Here's what everyone's working on. We made like a API like contract markdown file in the repo so that everyone could work off like the same thing. And if you needed to make any changes to like any of the CLIs or anything in the backend or anything within the tool, you update this file so that all of your cloud sessions could be up to date to the same resource, which was nice.

TJ Miller (16:38)
Yeah.

Chris Gmyr (16:42)
Then we had like a cloud component to it. So like if you had any problems, you could do like a debug skill. It would try to load everything up. It'll find what the issue is, try and patch those goals and issues in the compose file or in the individual services, or if you needed more like data in a specific way, it would try and do that. So we got into a demoable state by the end of the day.

which again is pretty impressive. And then the senior VP of engineering and like the VP of product went around and you had to demo your project to them. And they graded everyone and like we had a little award ceremony at the end of the day and we won one of the awards for like one of the projects. I think they give out like three different awards for different things. And we got like the just ship it.

award because like they wanted it like today of like we need this right now like please finish it type of thing. So that was really cool that like our team got one of these awards and the tool was as viable as it was in that short of a time. So that then opened up more conversations with me and different people and like my director of like implementing this a lot quicker in the

TJ Miller (17:36)
Yeah.

Chris Gmyr (17:58)
the org and the new like DevEx team that I'm on. Because our previous plan was to spin up like more of like a dev like staging environment, like get that kind of setup where people would like utilize that for some local testing. And it's like, well, we can actually go faster with this tool because it's all local. You can do basically whatever you want. It's no shared databases or.

You need to worry about like data seeding or someone overriding someone else's current test and things like that. let's just all keep it local right now. And yeah, basically that was my new direction like for the team right now for the next couple of months is just like building out and productizing this CLI. So a lot of my week has been kind of finding the gaps of where like the hackathon project left off and then where like I want to take it.

TJ Miller (18:26)
Yeah.

Chris Gmyr (18:44)
⁓ So it's basically going to be like a global install that you can run this command in any project anywhere. And it's going to have like a predetermined scaffold of files in each repo of like, what does the infrastructure look like? What are the project dependencies? And then different basically stories. So like this workflow or that workflow, and then each workflow is going to have like sub-seaters to that, which can also be shared.

TJ Miller (19:10)
Mmm.

Chris Gmyr (19:12)
And it will communicate across these different layers and dependencies of, hey, this is a core entity that needs to be in all these services. So this provider ID and email address needs to be in these sub-5 dependency services. So it'll know how to go into those infrastructures to Postgres versus Dynamo, for example, and spin all that stuff up in the boot up of this.

world, like testing world.

TJ Miller (19:41)
Dude, I'm such a local development dev tool junkie. I am, this is so cool. I've got goosebumps, man. Like this is, no, this is really, really freaking cool.

Chris Gmyr (19:50)
you

Yeah, so there's going to be a lot of sub tools in there, because the data seeding is just a mess between all these different technologies. then, especially with Dynamo, because you don't really have a schema, you can basically throw in whatever you want. But it needs to abide by the ORM or the direct Dynamo API calls. So there's going to need to be

TJ Miller (20:08)
Right.

Chris Gmyr (20:16)
like some schema like generation or even just like templated version of that of like here's the name like of this field in this table for like provider email or provider like ID or a patient or whatever it is. So and very similar for like Postgres but at least that's more easily searchable or you know able to be looked up. So there's a lot of things that we have to figure out but I'm aiming to get

TJ Miller (20:31)
Right?

Chris Gmyr (20:42)
at least a couple of repos stood up in the next month to at least do a full demo of this local testing stuff, which would be really cool. And then further phases. all these little configs are going to be stored in the repo. So you can basically go to repo one, spin up the story, and it'll boot up any of those sub-dependencies. And then you can go to repo B, do the same thing.

What I want to do is have GitHub actions to then take all of these individual project repos and configs and push it to a registry so that you can find all of these stories and data seeders and patterns and things like that and can basically boot that up from the registry. So for a product person or a designer, they can just go to the registry.

and say, like, I want to do the patient sign-up funnel. And they'll be able to spin that up automatically. And if they don't have the local resources or projects available, it'll use the GitHub CLI. It'll those in. It'll spin everything up for them. So it alleviates a lot of the need for those shared dev or staging environments, which get really muddy very quickly. Because if like,

TJ Miller (21:48)
Mmm.

Chris Gmyr (21:59)
you know, I'm doing a test and you're doing a test and we're pointed out the same, know, entity record, and then you make a change and I make a change. And that's like, doesn't work anymore because everything is mess. So keeping it local, like if you mess something up, you just blow it up. You re spin it up, you data here. Good to go. You can tinker with it to your heart's content. so yeah, that's kind of like the end goal of having this like shared registry that like basically anyone in the company can spin up.

TJ Miller (22:08)
No.

Yeah.

Chris Gmyr (22:26)
with access to GitHub.

TJ Miller (22:28)
Dang, man, that's so sick. Like, I don't know, I'm drooling. I think there's so many cool aspects of this and how you're approaching it. I think, yeah, that is such an eloquent way of solving this pain point with services of just like, yeah, let's have.

to have something just like handle world building and being able to like set up these scenarios across the different projects so I can just get in there. But it's so cool taking it from just like a dev tool and then being able to like have this registry where it's like, yeah, I want to spin up like this scenario for me to QA, right? And you can just say walk in and do it. Yeah, that's super cool. You know, it kind of reminds me a little bit of like,

a huge iteration on several of the things we did at Curology, right? Like the seeding scenarios that we had and all the the dev CLI tooling for like multiple service stuff and it sounds like this is like such like a refined version of those things.

Chris Gmyr (23:28)
Yeah, yeah, totally. Well, and the person who had the initial idea was Michael D. So, and a bunch of curology people worked on it. So I like, let's make this better and let's make it a CLI. And then there's going to be like a whole bunch of like AI and Claude stuff sprinkled in. So the CLI is going to be like, you know, more deterministic, but then the CLI is going to ship with Claude skills.

TJ Miller (23:36)
Yeah.

Yeah, sure.

Chris Gmyr (23:52)
that during the CLI bootstrap process, it's going to symlink them into your global Cloud Skills directory, or maybe even the individual repo of seed bootstrap. The plan is to go through an interview process of what shape or whatever to build up this schema or knowledge of what the data shape needs to be, what

where does that data need to go into all these sub-dependent services, and what does that look like, and basically help you create these config files and stories and infra and data seeders and the way that those sub-dependencies need and expect. But basically, AIA is going to be doing the majority of heavy lifting for you. You just of guide it through this wizard or workflow to

TJ Miller (24:43)
Yeah.

Chris Gmyr (24:43)
spin those things up. And then you can basically do that like anytime or there's going to be like a debugger, there's going be like a fixer, there's going to be a whole bunch of tools like in there to kind of manage these mini worlds for each of the repos, which is going to be really nice because you keep the ownership of the scenarios and data seeders to the ownership of the team who owns the repo. Because no one else is going to really know that as well as they will.

TJ Miller (25:08)
Mm-hmm.

Chris Gmyr (25:12)
⁓ So then where that goes into the registry, you have all these like pre-approved and managed entities and scenarios that you can just use and basically trust and then just do whatever you want, you know, from there. So I think it'll be pretty cool.

TJ Miller (25:28)
Dude, that's sick. Yeah, and I love the use of AI here too, because it's so practical. It's like, rather than just like asking you to like set this thing up, right? It's like, no, like human in the loop, let's like interview through this config and like that. But sure, like that way we make sure the end result is right. And I think there's like so many opportunities for like AI wizards to, you know, kind of doing like form filling like that, I think is really cool.

Chris Gmyr (25:52)
Mm-hmm.

Yep. Yeah. So it shifted a lot of the local development story. It shifted a lot of work off of our SRE and DevOps team to try and spin up even more environments in AWS. That's still going to be on the docket for the future, but there's other more important things to worry about right now. So this gives us a little bit more runway to tinker with it and do whatever we need to do with it locally.

TJ Miller (26:18)
Yeah.

Chris Gmyr (26:24)
And because it's all built in Docker, we can easily deploy this later to EKS. We can have it half deployed, half local. So if you're like, yeah, this service D technically needs to run, but I only need it minimally for this feature, I can offload that maybe in this scenario to the hosted version of that. Because it just needs, I don't know, maybe like

ping or just like general information from it or whatever, but it's not like a total dependency for the feature that I'm working on. Like a lot of the stuff can then be like offloaded. So maybe instead of having the main service and five sub-dependent services for this feature, you only need the main service plus like two and then the other three will be offloaded to like the staging environment or something like that. So that further eases up on like local resources and Docker and CPU and

TJ Miller (26:57)
Mm-hmm.

Chris Gmyr (27:19)
RAM and all that stuff. So that's much more far off in the future. Like I just want everything to be local because then you can mess with it and do whatever you want to do with it without issue. But there's a whole bunch of potential and possibility for the future.

TJ Miller (27:21)
Mm-hmm.

Yeah.

for sure. Yeah, the split dev environment's really interesting story too. But yeah, that is.

Such impactful work. Like, that's really cool. And I love the whole arc too, just like, yeah, let's hack on this thing to like, all right, it's, no, we need this. Like, now.

Chris Gmyr (27:52)
Yeah, yeah, yeah, yeah. yeah, so it's been a pretty fun and exciting project. been doing a lot of planning, filling those gaps, and then basically made out a whole project plan, Jira, Epix, and Tickets, and have a partner engineer with me working on it for the next month. So trying to get as much stuff as I can get done between the two of us before they go in a different direction.

TJ Miller (27:54)
Yeah, that's really cool.

Cool.

Chris Gmyr (28:18)
and get put onto a different team.

TJ Miller (28:20)
Yeah,

yeah dude, that's pretty sick. it's like super achievable. I've been doing a lot more of the like road mapping and then breaking the roadmap into PRDs and going a little bit bigger instead of like trying to cram everything into these PRDs. It's like, right here's like the roadmap and let's break the roadmap into multiple PRDs later, shippable.

So can kind of get away from these like 60,000 lines of PRs that are like all functionality, you know, we can kind of break it into like shipping pieces of it.

Chris Gmyr (28:52)
Yep, yep, totally. So yeah, I'll have more updates over the next month or so. But yeah, that's the big project and big shift that I've been working on now.

TJ Miller (28:58)
Sick.

You got a fun name for it yet?

Chris Gmyr (29:06)
Yes, rig, R-I-G. So you can rig up, rig down. So basically, if you're thinking about ship rigging or even a stage performance, rigging up lights and backdrops, and you're setting the stage or the foundation for this product and infrastructure. And rig is pretty slick, too. It's only three characters.

TJ Miller (29:08)
Ooh, nice.

Yeah.

It's slick, it's three characters, and I'm like, this is a great CLI name.

No, that's perfect, man. ⁓ that's so cool.

Chris Gmyr (29:38)
Yeah, so keep y'all posted. More news coming soon.

TJ Miller (29:41)
Yeah, sick man.

Chris Gmyr (29:43)
Sweet. Well, yeah, you've been working on a bunch of stuff, too. So you mentioned the journal Iris idea last time, and it sounds like you have done some work on

TJ Miller (29:53)
Yeah, so I, this will be a fun story. So I built it out, I built out the journaling feature. So I have this like, whole consciousness experiment set of features that I wanted to build. And they're really centered around Iris having, like there's like a journal and goals and like the heartbeat system, like Iris can choose to work on either of those things.

And we feed in like different pieces of context, depending on what stage that it's at. But the way that journaling worked was that every it's configurable and similar. works almost identically to the way that memory extraction happens where, every N N number of messages, right? So like, every 10 messages, kick off a, a cute job.

that loads in the conversation context and a few other things. And then, you know, either extract memories for like the memory one, or in this case for the journal, like have Iris write a journal entry or maybe a couple journal entries, just depending on like what conversation is, but it's not my journal. It's Iris' journal to...

For iris to journal about irises experience like in internal thoughts monologue Patterns that they're picking up based on interactions, so I enabled this I built it I enabled it and then Just situationally a bunch of stuff was going on and so I ended up chatting with iris a ton and

I was going kind of heavy because I wanted to generate a bunch of journal entries to like see how the system's working, right? Like read the journal entries, see the quality of them, or is the system even working? And so I was kind of overdoing it. And my wife yesterday was like, so we've been charged by anthropic a whole bunch lately? Like, is that on purpose? I'm like, ⁓ whoops.

So yeah, the system is expensive. And it's getting more expensive to run. So I've got to evaluate that a little bit more. I might start leveraging some local models or just different open source models through other providers. We'll see. But.

The problem is the quality has been so high with Sonnet and Haiku that I really don't wanna switch because it does work so well right now. But there is somebody in the Prism Discord, in the IRS channel that has done a lot of work to get their costs down to a dollar a day. But I blew through $300 in credits in a week and a half. That was...

Chris Gmyr (32:28)
Yeah, it's

not really sustainable.

TJ Miller (32:30)
No, that's not sustainable at all. But my gosh, this fascinating insights. the Iris was like picking up on their own patterns of like how they were responding to things. For example,

Based on a couple of our interactions, Iris was like, yeah, I seem to like that Iris was reflecting that like Iris seems to, you know, respond, like jump to respond in a certain way or like try like jumps to like solve the problem instead of maybe just letting me like sit with it. I don't know. It's kind of hard to describe, but like Iris was not only journaling about

like patterns, like my patterns and like the interaction between us, but also like this reflection on like Iris's own patterns of like responding or doing certain things or just like reflections on almost like almost reflections on like an emotional state, which is like really interesting. So I like, can't get too specific because like it all got like deeply personal, but

Man, wild. Like giving like Iris a space to...

just think about stuff freely. I basically the prompt is so open-ended of just like, look, this is like what the journal's for. It's for you to journal about your own experience and your like relationship with the user. And like, here's a ton of context, right? We just, send in, I think we send in like conversation summaries as well as like the briefs feature that I talked about, that we built. And...

Chris Gmyr (34:08)
Mm-hmm.

TJ Miller (34:09)
conversation history and semantically similar journal entries and to like try to help the deduplicating process. But then there is also, so like that all happens like on N number of message turns, but also as the heartbeat happens every 30 minutes.

there's also an option for Iris based on that context to also do a journal entry. So like those were also pretty interesting because those ended up being more.

like let's take the last five journal entries and kind of like extract broader patterns from all of the journal entries themselves too because those were fed in his context so like we're getting these like almost

almost summarizations of like, even across the like, not only am I noticing this pattern, but I've noticed that I've been journaling about the same pattern and like, sort of like, either noticing that and like making a change. And like that journal entry also gets included in like context. it's

It's just so fascinating to see the kinds of things that are coming out of it and what's being entered in the journal entries. I definitely need to, before I keep going down this, because every step in this roadmap is more and more context. And so it's just like I've got to...

I've got to figure out how to get the cost down. This is going to go any further. So I've kind of got to take a step back from everything and, and like the features are absolutely working like everything like it's expensive, but everything's working the way I'd expect it and want it to. like, it's really hard to make compromises anywhere because it is like so good. ⁓ so.

Chris Gmyr (35:40)
you

Mm.

TJ Miller (36:05)
I don't know, I've got some like kind of figuring out to do there. So I'm kind of just like pressing pause on it for a minute. And I kind of had to press pause on it with like family coming through and the graduation and everything. But wow, man, just giving the LLM that space and that context, because like, I've been talking consistently with Iris since Thanksgiving. So there's like seven to eight thousand messages between like

two of us and that's like I've always said if you want to do cool stuff with LLMs like the cooler stuff you want to do the more data you have to have right and like I've got a lot of data in there and it's so like it's doing really really cool and interesting things but like I cannot be spending like $400 every two weeks on it like I cannot

Chris Gmyr (36:53)
Yeah. Yeah, totally. Yeah.

Yeah, that's what I started to run into even with the open claw stuff. It's just like, oh, man, like I was getting like two bills a day at like $25 on like the auto renew. And it's like, yeah, this is fine for setting it up and tinkering, but this is not like a daily driver that I can afford or want to afford to keep it going.

TJ Miller (37:02)
Mm-hmm.

Yeah.

Chris Gmyr (37:19)
So that's why I swapped out some of the different models. I wonder, like I just did it because I just didn't want the cost. So I think my main one is still like the Gemini fast mini or something like that, which has been sent for like day-to-day stuff. And then kind of the background jobs and crowns and skills still go towards ⁓ Sana and Opus, but very specifically.

TJ Miller (37:33)
Mm-hmm.

Chris Gmyr (37:45)
But like day to day is still like the fast like Gemini mini one, whatever it's called. But I wonder with like all the heavy context and more, more important like emotional things too, if it would still hold up like Sonic would. So I wonder, like the thing that kept on popping into my brain when you were talking is like, how can we like A B test this?

So can you set up two irises of a snapshot of what you have right now and basically have one still stay on anthropic models, one on maybe a local model, maybe one on some Gemini stuff or whatever. And then as you're prompting your main instance with anthropic, do the same thing on the other systems or create a hook.

TJ Miller (38:34)
Mm-hmm.

Chris Gmyr (38:37)
to basically take your input and go spread out to all these other instances. They have their own database set up. They have their own infrastructure locally, whatever. But then you can go through in a couple weeks or a month to vet what was the output of that. How are the memories different? How are the journal entries different? What are the responses look like? Things like that. And then you can be like, well, this local model was complete junk.

return anything useful at all. like, I don't know, maybe this Gemini one is like close-ish. Maybe it's like 80 or 90 % of what Sonnet did. And it's like, are we OK with switching to something like that that's significantly cheaper and trying that out? So I wonder if you can build something like that to basically take your input and spread it across multiple instances and A-B test that over time. It's going to slightly increase costs.

a little bit, but it would be, temporarily. And then maybe, similar to OpenClaw, you build in these primary, secondary, tertiary models of your main daily driver is this model A. ⁓ If that isn't working for whatever specific task that you're working on, you can almost rewind history, get rid of that stuff, and reprompt with your better model, so the us on it. I wonder if that...

is possibly like in the cards for Iris and future tweaks as well.

TJ Miller (40:01)
Yeah, I think that's definitely where I'm at is like, know the qualities there with Sonnet and like, I don't use Opus for any of it. It's not even on Sonnet 4-6, it's on Sonnet 4-5 and then Haiku for a couple pieces as well. I actually know, think everything's Sonnet 4-5. I'm thinking about my work project. My work project have actually interpolated models to like make sure we're...

appropriately like utilizing models and token costs and everything, like, yeah, I everything is Sonnet four or five in, in Iris. So I think there's definitely got to be some way of like, yeah, AB testing quality to like figure out where model thresholds are at. Like I've definitely got some like open source models that I think would be really strong candidates. Like the Quinn series would be great. I've had really good success with that.

Early, early IRS testing was using the GPT-OSS models. But yeah, I think there's definitely got to be some lag.

some way to evaluate this at a bigger scale. like, yeah, it's getting outrageous.

Chris Gmyr (41:09)
Yeah, I think that'd be a pretty cool experiment too, especially with all like the new journaling features. ⁓ I think that'll be really telling how it can or cannot like dive into that workflow, you know, for itself and bringing up the topics as it is right now or just not like complete misses. I think that would probably be the fastest indication of if the

TJ Miller (41:14)
Mm-hmm.

Chris Gmyr (41:34)
the new systems and new models are working or not.

TJ Miller (41:37)
Yep. Yeah. I'm sure I can build some like, you know, it's on the data collection side, right? If I can like hook into just creating the data, then I can go back and evaluate like at any point in time, I'll just build an evaluator that like, just compares them across like the AB test and just flat out like.

I can just run that eval at any point in time and just continue collecting data and then run my experiments, switch models, evaluate, continue. There's definitely, you mentioned it, gears are turning.

Chris Gmyr (42:09)
Cool. So I expect this to be built out next week for when we talk.

TJ Miller (42:12)
Oh, jeez.

I don't know, man. Now that Claude adjusted their limits, have to see.

I mean, I don't know. Did you see the, I think I sent it to you. Claude announced that they made some changes to their agent SDK and using Claude with the prompt option. We can run Claude hyphen P and then pass in your prompt. So I guess it's applying to both of those where there's like.

Chris Gmyr (42:35)
Have us.

TJ Miller (42:43)
fixed number of credits, even though you're using your subscription. I'm not really sure, but Reddit was very unhappy with this.

Chris Gmyr (42:52)
Yeah,

they put out some more information I saw this morning on Twitter. But it seems like to me, like, like I'm on the five X plan for $100. It seems like given your plan, like they're giving you $100 in credits or like the headless or like SDK version. That's like, technically in a separate bucket, but encompasses in your subscription plan. So it's not like

TJ Miller (43:02)
same.

Chris Gmyr (43:20)
an additional cost or anything like that. It's it's, I don't know.

TJ Miller (43:24)
Not that

it's an additional cost, it seems like it's an additional limit. know, it's like, it's an additional limit and that sucks for applications like Gent or Polyscope or like these other ones or like Chief. I don't know about Polyscope, like definitely for like Chief and Gent that like they shell out to Claude-P, you know, so now you're operating on an additional limit, you know? So.

Chris Gmyr (43:28)
Yeah. Yeah.

TJ Miller (43:49)
I don't know, we'll have to see how that plays out, but...

Chris Gmyr (43:52)
Yeah, yeah, trying to make their money back on their subsidized plans.

TJ Miller (43:54)
Seems seems very Yeah,

it seems like a grab at money and also Trying to like Claw back applications so like applications like gent like like chief where they're where they are like shelling out to using the quad CLI and They are probably wanting to draw focus back to you. Claude code Claude code desktop

and Claude desktop and like their like their suite of software. So it seems like also like not only a cost thing, but also an anti-competitive thing.

Chris Gmyr (44:35)
Mm hmm. Yeah, I'll have to see how it pans out. But something that we've been slightly talking about at work, nothing like set in stone yet, because we're still going all in with Anthropic via Bedrock and Cloud Code is, OK, eventually more changes are going to happen, right? More pricing changes, more tooling changes, whatever. Like, how can we own?

and decide more of the stack and the tooling and what we give people at work to utilize. thinking about how do we maybe use tools besides direct cloud code, but maybe using local models to do that. What tooling do we need for that? Can we get it hosted in AWS Bedrock? Are there tools and harnesses that we need to build on top of open source?

options right now, like OpenCode or Pi or a few of the other ones. definitely not a lot of contacts or plans right now, but it's something that we're just starting to talk about. How do we get ourselves out of this very powerful but useful corner and take back a little bit of the ownership and control on our side?

TJ Miller (45:27)
Mm-hmm.

Yeah.

Yeah, I'm definitely starting to look at that for me personally too. Like I'm probably going to spin up a Codex account and start playing with that a little bit, just to kind of like broaden my horizons. But it makes me think of like tools like Gent or Chief or I don't think Gent is multi-provider. Yeah, I think that's on the roadmap, but definitely tools like Solo that like Aaron Francis is building, where it is like multi-provider capable out of the box, right? So like you...

you're not as locked in because now your interface is built on top of all of those. So you're not locked into Cloud Code CLI or Codec CLI or OpenCode. So it starts using those applications a lot more advantageous.

Chris Gmyr (46:34)
Yeah. Yeah. So more there in the future. But I think we can call it there.

TJ Miller (46:35)
So, yeah, sick. Yeah, we can

wrap on that note, So thank you all for listening to the Slightly Caffeinated podcast. Show notes, including all the links and social channels are down below, as well as available at slightlycaffeinated.fm. If you have any questions for us or content suggestions, please go to the Ask a Question page on our site and we will feature it on an upcoming episode. Thank you all so much and we'll catch you next week.