Slightly Caffeinated

TJ Miller (00:00)
Yo, welcome back to the Slightly Caffeinated Podcast. I'm TJ. Hey Chris, man. So what's new in your world after our couple week hiatus here?

Chris Gmyr (00:04)
And I'm Chris.

I know it's been a couple weeks, been a long time. Missed chatting on a weekly basis. And a lot has been going on. had Thanksgiving week. Luckily was off all week, which was nice. So had a couple of days to kind of prep and got ready to travel. Went up to Virginia to visit my cousin and her family and stayed there for a couple nights. Came back for the weekend. Thanksgiving was super fun. Got to hang out with.

TJ Miller (00:17)
Yeah.

Chris Gmyr (00:37)
all the family and played like a bunch of games, stayed up late, ate some good food. Luckily the turkey and all the stuff that we brought and did came out fantastic. Everyone enjoyed it and some of her like neighborhood friends came over too. So it was like a good group and had entertainment for the kids. Her kids are much older and the neighbors were much older. So they just kind of took our two under their wings and let them have a good time and let us kind of relax for a little bit.

which was nice. So yeah, and then after that, came back and prepped to get ready for a company trip to San Francisco. So went out there on Tuesday morning, came back Friday afternoon, was jam-packed. Yeah, it was an offsite for talking about a big project that we're working on, getting some vendors, partners involved in a bunch of conversations and planning.

was really helpful to be all in the same room compared to, you know, 50 plus people being in a zoom call, trying to hash stuff out. so didn't get to see the city too much. Unfortunately, we did do some like walking around, but it was kind of like packed with work stuff. And then I was out, Friday morning, super early, to get back before bedtime on the East coast. So yeah, we'll have to go back and do some more sightseeing and SF. but.

Luckily, it was nice there, not too cold, a lot warmer there than it was here. So I was a little worried about that being by the ocean out there. But yeah, not too bad at all.

TJ Miller (02:07)
That's sick, man. That sounds like a really nice, like chill Thanksgiving. Definitely envious of getting to go out to go like San Francisco. never been, always like somewhere I'd love to go hang out. Closest I've been is like San Diego. I was out there for a wave PHP and I, I fell in love with the place. I'd, think I'd move there at an instant if I could, but, it's like, just like, it's like my sweet spot for, for.

like temperature and everything like I don't know man like shorts and sweatshirt is my my jam if I can get away with it.

Chris Gmyr (02:39)
Yep, totally. I love San Diego too. And my brother-in-law is in one part of San Diego. So we've gone out there a handful of times over the last couple of years. So it's just always super nice out there. And like you said, weather is just perfect. If it dips any colder, know, you just, everyone just stays inside. So if it gets colder or if it rains or something like that, everyone just stays inside. So yeah.

TJ Miller (02:58)
Yeah.

Yeah, man. That's cool, dude. Glad the kiddos had fun. And I had no doubt that all your turkey and everything would turn out. You're always, you know, during the summer posting like sick barbecue pics and that's awesome, man.

Chris Gmyr (03:19)
Yeah, yeah, came out great. So yeah, how about you? What's been new with you in the last couple weeks?

TJ Miller (03:24)
man. you, you mentioned Thanksgiving because I totally have like just been in a blur the past like two weeks. So I, forgot that even happened. it was, we had a really chill Thanksgiving. We, my parents moved away. my mom's siblings also moved away in the last like year or so. So, it was like a first, first Thanksgiving with my grandpa flying solo. So my sister and I made, made a point of.

taking him out to eat and we had just like a really nice time. and I know he super appreciated it. So it was, it was cool. Like my sister and I, we don't have any beef, but we're also just like not super close. We're just, we're just different people and, we're cool with that, but it was cool to get to like hang out with her and her fiance and cause we don't get to see, like, even though they live like pretty close, we just don't get to see them very often. work a ton.

Chris Gmyr (03:54)
Yeah, that's awesome.

TJ Miller (04:14)
so was really nice getting to hang out and see everybody and it was good food. over Thanksgiving weekend, we cooked like my grand, my grandfather didn't want to make a big fuss. So we like made reservations at a restaurant and did that, that thing. so we actually made our turkey over the weekend and the last two years we've done a like, like a brined turkey. And this year we just did a like.

YOLO throw it in the oven, like season it up and it was, it turned out really good. so yeah, it was, it was nice to still kind of do something like semi-traditional around here. Cause I don't know what I'd do without my like green bean casserole and some turkey, you know? Like that's my jam. yeah, I, I, no trips for me, unfortunately. just kind of like jamming around the house and.

Chris Gmyr (04:42)
Nice.

Yep, totally.

TJ Miller (05:02)
working on Prism and hanging out with the kiddo got, got our, I've been coordinating Christmas gifts. got everything like coming in, like got stuff for my wife coming in the mail. I'm just hoping everything makes it here on time. And, the, the dual, I'm super excited. My wife and I decided to get a 3d printer for the gift for myself and my son, cause he's like a little, he's like a little STEM engineer. Like he loves that stuff. So.

Chris Gmyr (05:23)
Sweet, that'll be fun.

TJ Miller (05:29)
He's always like making 3D stuff with like perler beads like the you know like the ironing like the beads you put on like little pegs and then you iron them like sort of flat and they like melt together. He's like he'll he makes like 3D things out of it. It's nuts. So he'll like engineer stuff that like snaps together and like it kids kids pretty bright. So I think this is gonna like take it up a notch.

Chris Gmyr (05:47)
Wow, that's really cool.

Wow.

Yeah, that's impressive because those things are so small. And in order to make pieces and a model that fits together or something, that's pretty impressive. That's super cool.

TJ Miller (06:09)
Yeah, I mean, he's got thousands of Legos, doesn't play with those, but like, Perler beads, he's like inventing stuff, so like...

Chris Gmyr (06:15)
Yeah, the Legos now, those are done for you. There's instructions. Like, it's too easy. Like, gotta do your own thing.

TJ Miller (06:21)
Yeah, see, actually, it was weird. I had a conversation with him about it, because now I've got to try to figure out what to do with these Legos. Some of them are new, his Legos, but the vast majority of them are my Legos from when I was a kid. So I've got this emotional attachment to them, they've been sitting in bins for years. So I'm in this weird place, but I was talking to him about it. like, why don't you like, he's like, I'm bored. I'm like, why don't you go build with Legos? He's like,

Nah, man, like I like building kits and like we don't have kit, like they're all apart now. And like, you know, I'd have to sort through everything and, know, rebuild the kits. And he's just, that's not fun for me. And I'm like, you're so inventive and creative. Like why, what's wrong with freeform Legos? And he's like, I don't know. I just prefer the kits. I'm like, so yeah, there's, I don't, there's this weird disconnect. I don't know, but.

Chris Gmyr (07:13)
Yeah.

TJ Miller (07:14)
We've been talking about 3D printers for the last like eight months, I think. Every year for Christmas, my father-in-law gets him these like monthly STEM kit subscriptions. So he had like, when he was younger, he had like a little kid's one. For a little while, he moved over to like a cooking one, because he got like really into that. And then he recently got on like, because now he's

super digital and on YouTube and all over the place. He got a bug up his butt about getting the Mark Roper kits, the Crunch Labs kits. And so he talked my father-in-law into changing his subscription over. And they're super cool kits. First one was a disk launcher. The second one was a coin spinner.

It teaches you about ratchets and it's got a ratcheting arm and then you pull the trigger and it flicks a coin and spins them. Pretty cool. And it comes with a kit of two and a little plastic things, almost like a little plastic bowl so you can kick them in and battle each other with them. And one of the cool things that I really appreciate about it is

Chris Gmyr (08:09)
That's really cool.

TJ Miller (08:24)
All the kits come with, like you can go download the 3D printed instructions, like 3D printer files for them. So like for the disc shooter, you need more discs, like go grab the STL file and go 3D print yourself some more discs or like, hey, the coin flicker, the part that actually hits the coin, it's like just laser printed cardboard. So it's like getting dense in it from hitting the coin so many times.

You can go like 3D print your own plastic one if you wanted to too. So he's been like way into that because he's like, I wonder like, I lost all my discs under the couch. I want to make more discs. like, one, you could just get them out from under the couch, but two, yeah, but it'd be real cool if we had a 3D printer for it. Like being all facetious because there's one sitting down here in the basement, like waiting for Christmas. I didn't.

Chris Gmyr (08:54)
That's cool.

you

Yeah. Nice. That's so awesome.

TJ Miller (09:18)
I didn't want to wait for Christmas for it. Like I was like, we can just like make a big deal about it being an early Christmas gift. Cause like, want to start playing with it. My wife was like, she shut that down so fast. She was like, nope. That is like Christmas day, homie. So I'm, I'm. Yeah, I'm super excited about it. know, I know his, his first print he's been talking about is a rocked up.

Chris Gmyr (09:29)
the

That's right. You know, tons of time to play with it, I'm sure.

TJ Miller (09:46)
So like Dwayne the Rock Johnson head with octopus tentacles. He's just, he's been talking about it for like two or three months now. He's just like, when we get our 3D printer, whenever that happens, first thing I'm going to print is a rock to put us like, okay, man. So.

Chris Gmyr (09:50)
Thank

That's so funny. He's already planning to have the 3D printer in all of his projects and stuff.

TJ Miller (10:12)
yeah, no, he's been making plans and doesn't even like, it's, he's been making plans for like the, the someday when we, when we eventually get one. This is going to be awesome.

Chris Gmyr (10:23)
That is awesome. That's great. I'm looking forward to seeing what you guys print and build and all the cool projects.

TJ Miller (10:29)
And I'm Mr. Utility over here. Of course, I've got some fun things I want to print, but I'm like, man, I've got this bracket for my oven drawer, the one underneath it that broke. Well, I'm already in AutoCAD trying to make a new one. So it's like, want a rock to put some, I want to fix things around the house.

Chris Gmyr (10:49)
Yeah, that's awesome. I've seen so many people online just make whatever they need. They have stands for, I don't know, keyboards, tech things, fixing stuff around the house, like you said, just cool things. It's like, this would be cool if I could do this. And you just print it and do it, and it's out in a couple hours. And off you go. And like,

TJ Miller (11:12)
Yeah, I want to mix.

Chris Gmyr (11:13)
on like, I don't know, some super custom thing. It's like just print whatever you want.

TJ Miller (11:18)
Yeah, dude, think once I get like, took a bunch of AutoCAD in high school and like, took four years of drafting and then a year of like independent study in AutoCAD in high school. Like I was super into it. so I'd love to get back into it. And I know one of the things I'd love to do is I to bang out like a three-dimensional like Laravel logo. I would love to do like a cool, like one of the

like LiveWire, like little squid guy things. I've seen someone on, it's been going around like socials, like Brickit, there's like Lego, Laravel logo, like LiveWire and Tailwind logos. But I'd like, I'd love to pop those into like, make them three dimensional and print some out. I think it'd be fun.

Yeah, speaking of Christmas gifts, I just got back from an early Christmas gift surprise with my wife. She's like, hey, it's Friday the 13th. You want to go get like like all the tattoo shops are doing like Friday the 13th, like quick flash specials. And I'm like, yeah, like why not? You know, I haven't gotten tattooed in like 10 years. Let's go do it. That'd be a fun little couples afternoon thing to go do.

Chris Gmyr (12:31)
Yeah, totally.

TJ Miller (12:33)
She's like, I also need to get a gift certificate. I'm like, what do need a gift certificate for? She's like, I don't know, Christmas. Like, OK. So we go in and we end up waiting for an hour and a half. And there's still like five more people in front of us. So this is going to take forever. So Laurel's like, let's just bail. Let's go get your gift certificate. So we're talking to the owner of the tattoo shop. And he's like, so.

like, when are you looking to get this done? I'm like, whatever, man. I'm pretty flexible with my schedule. And he's like, well, it looks like my tomorrow just got cleared out. like, bet. So I, yeah, guess tomorrow I'm going and getting tattooed. I'm very excited about it.

Chris Gmyr (13:11)
you

That's cool.

TJ Miller (13:22)
You got some tattoos right, man.

Chris Gmyr (13:23)
Yeah, have a handful. Both to my arms, across my shoulder blades. And then the latest one is on my left calf, which latest was, I don't know, 10, 11 years ago now. So in the same boat with you, haven't gotten anything done yet. So yeah, not really sure. Probably more when, not if. I do want to get some more, or don't know, get some color on.

TJ Miller (13:35)
Yeah.

Chris Gmyr (13:48)
the one on my leg or something like that, not really decided on anything right now. Just kind of like, I don't know, whenever something, you know, hits me that, you know, I want, then, you know, I'll figure it out. But yeah, no plans as of yet, but definitely want to get some more in the future.

TJ Miller (14:04)
Yeah, I've been like itching to get some done, but everything I've got right now is like bigger projects, like finishing up my left sleeve, which I've only got about like half, like the lower forearm is done, but I got to do like the whole rest of everything. And then my right arm, I don't even know what I want to do. I've got low key a little, a little regret on my right sleeve. So we'll see. I don't know what's going to happen with that.

but yeah, I'm getting my knuckles tattooed tomorrow. I'm just so super excited. It was, I've been wanting to do it for years, but just couldn't think about what to do. And it's kind of a running joke in the house, like for.

Jeez, like almost 11, 12 years now. I've remapped Caps Lock, the key on my keyboard to escape. Like you tap it for escape, you hold it for control. I've had it for like 11 or 12 years. And so, you know, like maybe 10 years ago, my wife sits down to use my computer to do something and like everything just starts closing like on her. Like she's just like, it's exiting like input fields.

Chris Gmyr (15:07)
Thank

TJ Miller (15:11)
closing windows and she's like, what the hell? Like, how do you use your computer? Like what the hell is going on over here? I'm like, what are you, what are you doing? She's like, I'm just typing on your keyboard and like everything's getting all messed up. So I'm like, all right, just type, let me watch what you're doing. And she's like, I'm like, the only way this stuff's happening is because you're hitting escape. what, what are you, are you yelling? Why are you using caps lock?

And so come to find out she uses caps lock as the shift key. So like, yeah, like I've never heard of this before ever. And so she like, yeah, if you're like writing the sentence, you're like, you know, capitalizing the first letter of the sentence. And she like, well, hit caps lock, type the letter and then toggle caps lock again. like, you know, the sh like the

Chris Gmyr (15:40)
jeez.

That's definitely keystrokes. That's not how it's supposed to be.

TJ Miller (16:02)
The shift key exists and it's like one key lower. Like, what are you doing, man? So it's just been like a running joke for, for years. And so, I don't know. We were just like sitting around. were like, I stumbled across a Reddit post where someone talked about their wife doing it also. And we, I just like dawned on me. I'm like, my God, I need to get caps lock tattooed across my knuckles.

Chris Gmyr (16:23)
Yeah, that's awesome.

TJ Miller (16:26)
And so we were telling the tattoo artist what we were going to do. He's like, that is amazing. And so as we're filling out the gift certificate, he's like, so have you thought about what font you wanted in or anything? He's like, all lowercase, right? It totally blew right over my head. just a minute later, it dawned on me. like, my god. Yes, lowercase, no.

Chris Gmyr (16:47)
That'd be hilarious. I love that it just has such a great story attached to it. And then more recently with the tattoo artist today, just the story just keeps on continuing. It's fantastic.

TJ Miller (16:57)
It just keeps going and so I absolutely love it. think we're, I think we kind of settled on like a typewriter font. So it'll be even a little kind of funnier with that in there. So I'm just, I'm absolutely geeked about it. And I had no idea it would be be tomorrow either, especially when he's just like, yeah, I got kind of a busy schedule. And he was like, I guess tomorrow at 11?

bet. Let's go.

Chris Gmyr (17:27)
Yep, let's do it.

TJ Miller (17:29)
So yeah, it all worked out pretty well. just, like I keep saying, I'm just geeked.

Chris Gmyr (17:34)
Yeah, that'll be awesome. have to definitely share some photos in the telegram and socials, I'm sure. So can't wait to see it.

TJ Miller (17:42)
Yeah, yeah, I'll definitely be posting about it when it's done tomorrow. be, I think, yeah, folks will probably maybe see pictures before this is out. I don't know. It might be right around the same time if we're, publish tomorrow. So yeah.

Chris Gmyr (17:55)
Yeah, published tomorrow, so yeah, maybe around the same time.

TJ Miller (17:59)
Yeah, that'll be fun. That'll be cool.

So yeah, man, I'd love to just kind of look at some of the stuff we thought about talking about here today. I'd love to hear some updates on like the GraphQL SDK. And then think after that, maybe we can talk a little Prism.

Chris Gmyr (18:12)
Yeah, totally. So I took your idea, well, I guess to rewind a little bit. And a few episodes ago, we were talking about interacting with a GraphQL API and just it being very different from Rust. And typically, when we've made our own SDKs, you make all the DTOs, the resources, and everything is strictly typed in GraphQL.

is very open. You can ask for anything. You can do mutations. It just looks and functions very different from a RESTful API. So what I wanted to do was utilize some sort of SDK or strict types or creating strict types out of a GraphQL response. And also when constructing data to push to do updates, creations via mutations.

And we talked a little bit about kind of constructing and merging these fragments of data. And somehow that would be typed to certain things. And it was just a little wonky. And your suggestion was push that stuff to the DTOs, because you'll already have typing. Maybe the DTOs can be like a mechanism in between some of this functionality. And that's what I ended up doing.

So it's working and functioning really well so far, and I'm super happy about it. So how I have it set up is there's a resource. So this is kind of like the high level entity. So like a customer, if we're talking about like an e-commerce type site. So a customer you can create, you can update, you can delete, you can do a handful of other things with certain settings. But basically like first name, last name, email address.

phone number for basics and a customer ID. So for getting a customer and also updating a customer, you have a customer DTO. So you have to specify an ID. And that's also a value object because it has some filtering and some checks and balances within that. And then it's just basically a couple of strings for the names and then a different value object for

an email address and also a phone number. So all of these are constructed in the customer DTO. And I added some traits to do some additional magic on that. Because when you send data to a GraphQL, you also indicate what response you want back. So if I'm updating a customer with ID, first name, last name, email and phone number, I generally

like would get those things back in a more restful type of API. So basically what I'm doing is using some reflection, grabbing the names of the properties on the DTO. So ID, first name, last name, email, phone number, and casting that into a GraphQLable string as the response ask. And then the data of the input

for the mutation gets grabbed also from the DTO. So it's mashing up the ID equals the first name equals, all that stuff is computed and put into a different part of the GraphQL request. But essentially all engineers have to do or client code is basically call this customer resource and passing through a DTO.

just like you would in a more restful type of way. And you get the same response. It's using that same DTO as the input and generating the output from there. So the DTO is kind of like controlling the input, the output, the casting, and everything else like that within the resource. So it's very little that the client code needs to do. And it's

so user friendly already. You don't have to worry about any of the fragments or dropping any of the typing or anything else like that. It's really nice. So it's basically resource versus DTO and good to go from there. So I know that was a lot of explaining code on podcasts, which is not always the best. But hopefully you followed me through that journey.

But if not, like I can definitely dive into it a little bit.

TJ Miller (22:22)
Yeah, dude, that sounds super cool. Like I think it's, it's kind of coming back to me now that it was, yeah, like my recommendation or my suggestion, I guess, was to kind of treat it like a rest API and kind of push that stuff to the DT like through the DTO as in value objects. So that like, yeah, your, your request is like sending in this, like this DTO and then that's like handling all the, like the complexity. And then you're kind of getting like a.

rustish response back out of it, but something that's like predictable. if you do have like, so whatever, if you've got like a couple different user DTOs, but they just have like different attributes on them, you know, I think that's, I think that's totally fine. You've got like, I think it could, you gotta, gotta watch it. That could get out of hand having, you know,

a user with a field, like one extra field, like I think you just kind of take a step back and you do that, like that pre-planning ahead of time. Like I'm, I'm a big proponent of like spec driven API development. And I think that that kind of like sort of applies here to you where you kind of define the spec of like what you want as far as your inputs and outputs. And then you design DTOs around those like inputs and outputs. Like.

You lose a little bit of the magic and the like advantages of GraphQL, but I think that's fine. Like.

Chris Gmyr (23:46)
Yeah, I think that's fine too. You're replacing the.

I don't know the all the the functions and features and flexibility of the graph QL endpoint for more strict typing and known data in ease of use, you know with that and I think that's a pretty good trade off. Does yeah are some endpoints going to be called with a little bit more data than has always used? Yeah, but I don't know. I think that's fine and to your point, as long as these things aren't calling.

you know, 30 sub resources or something crazy like that. That's easy enough to look out for to, you know, review and pull requests and, you know, keep an eye out for, you know, in the future and make any make as many details as you want to do whatever you want in the future. But I think our use cases for this endpoint are pretty minimal. Like it's not that we're going to have to go like too crazy with the data that we're

pushing and pulling, I don't think it would really come to having an issue with using two generic of DTOs to get more data than what's actually needed. But just the typing is a huge bonus from all this.

TJ Miller (25:00)
Yeah, I feel like that's like the biggest conundrum with like GraphQL is like, I really prefer like stricter typing. I don't know, I've come full circle. I was definitely like big anti-types for a long time. And now I'm like big into like love having types, you know, and having some type safety and, you know, just having a really nice developer experience inside of

various IDs, No, I think it's like a really good approach, man. I think.

No, I think that's like a great way to handle it.

GraphQL is wild. I've honestly, I've not really worked with it and I've put in effort to avoid it because of these exact problems or I guess needing to like come up with some solutions for these things. Like, I don't know, just kind of stayed away.

Chris Gmyr (25:48)
Yeah. Yeah, it's really weird to like. It seems like from my research and I'm definitely not like a graph QL expert. It's just looking at a whole bunch of different libraries and places that do offer graph QL and points that especially the mutations are super wonky. Because if you have. Piped validation rules on input that's almost like a.

function within a function over the actual mutation that you want to call. It's almost like a named alias before you even get to the mutation function, which is really weird. This probably doesn't make any sense over a podcast. But if you look at Shopify has a big GraphQL admin API endpoint.

So if you look at like customers on there, there's like multiple levels of a mutation and that's because they're passing through some strict typing on what can be pushed in via input. So like the, the email, the first name, last name, email, things like that, it's all validation roles that they have on their side. That's strictly type within the schema. So just adding that level of strictness on the API side forces you to kind of build up

more of that structure and the schema to push to GraphQL to make that call. So it's just a little confusing and more loops that you have to go through to construct all this stuff for a GraphQL endpoint. it's just, I don't know, it's very different than Rust, which I'm definitely more used to as well. So.

Luckily, everything is looking good right now. I know it'll shift and adjust as we expand on the project and start to do more with the data. But just trying to get the basics and the foundation working right now. And so far, it's been working out very well.

TJ Miller (27:37)
Yeah, sick, man. Nice job. That's like a, it's like a, it's a really tricky, it's like a really tricky thing to have to approach. And I'm stoked to you for like, came across something like I, I, I smell a couple of blog posts and the conference talks somewhere in there, man.

Chris Gmyr (27:38)
Thanks, and thanks for the suggestion.

Yeah, I was just thinking it's like, man, maybe when I get this a little bit more buttoned up, I'll throw at least a blog post and some examples and some code to construct all these GraphQL entities before passing it on to the actual API. So yeah, that's definitely a good call out. I'll have to write something up in, I don't know, next month or so.

TJ Miller (28:15)
Yeah, dude, if PHP tech still had their call for papers open, I would be all over you to like turn that into a talk and submit it. Cause I think, I think there's people who, other people who would definitely like need to find some way of like approaching this. And I'm sure, other people appreciate like the strict typing and everything. So yeah, definitely will be like on the lookout for a blog post at least.

Chris Gmyr (28:21)
Thank

Yeah, totally. So yeah, how about Prism? How's everything going with error? Any updates that you want to share?

TJ Miller (28:44)
yeah, man, I've made a mess. No. Yes, I think in both fronts, things are moving along like really pretty well. Let's see here. I'll just pull up the. Let me pull up the release notes.

Chris Gmyr (28:49)
Productive mess or just a mess?

TJ Miller (29:00)
and see what's been happening because there's been some stuffs. Big thing was we got out embeddings. I don't know if we had talked about that at all, but I ended up getting embeddings out for a couple of providers. And that's super powerful stuff for being able to enable retrieval, augmented generation, like rag workflows.

So you could take a PDF, long text, turn it into vector coordinates with embeddings, and then store that in a vector database. And then you do like semantic queries back to that. So you ask a question to a large language model. They take that question, turn that into embeddings, then go back to that vector database and query for similarities. And then it will like pull back that content from

like that you stored in there and then inject that into the prompt. And then now the LLM has that as context to finish like completing the response and like responding back to you. So pretty common like way to think about this would be like take the Laravel documentation, turn that all into vector coordinates and then store that stuff. And then you'd be like chatting with a like Laravel.

large language model agent and then it would like reference the documentation to actually like give you answers to your questions. So we got like embeddings out and like the big one was adding

Let's see, was it? Big one was adding structured output. So this is something that I've been wanting to get to for a while. Like I kind of took a little bit of a new focus on Prism. So instead of like taking a feature or a, like basically taking a feature set and then doing it for all providers at once, I'm doing like taking a feature set like embeddings or structured output. And I'm doing it for like

one to three providers, and I'm trying to nail providers that are very different in the way they handle things. So typically that means I'm going to do OpenAI and then I'm going to do Anthropic because they're two vastly different APIs. And then on top of it, they handle everything different. So I did structured output for OpenAI, which is great. They've got several different ways of handling that.

through like a JSON mode and a structured output mode. I'm not going to dive into that, but they've got like two different modes depending on the model that you're using. And then I did Anthropic and that went pretty smooth. They don't officially support structured output, but I've kind of like finagled a way to get it rather consistent. So we've got like adapted structured output for Anthropic and then

I did Olamas as well. Cause I was like, you know, I'm in there, like, let me, let me play with that. I'm a big fan of Olamas. So like, that's always something that I kind of prioritize. So for me, the like primary three that I'm building for and testing things out on is like open AI, Anthropic and Olamas. So getting to like Olamas, like a launch structured output for all three of them. And then later that day.

Olamma announced that they have official support for structured output. So I had done a similar adaptation that I did for Anthropic. And so like the Prism docs, I'm calling it adapted support. So it's like things that the providers don't officially support, but we've kind of figured out a clever solution to make it work anyways. So I launched adapted support for Olamma.

Hours later, they post on like socials, hey, we have official structured output, like ready to go now. Like, all right, cool. Like, let me go like adapt that. And what I had discovered is both for the way I handled it, Anthropic and natively with open AI, they both work with structured output and tool use. So like.

That's something that I wanted to continue to support. So it's like you can have the full functionality of the Prism library, tools, prompting, messaging, everything, and then get structured output at the end. Olamma is like and or. So like OpenAI, I can throw the tools at it, everything, in the schema for the structured output, all in a single request.

And then I can iterate through tool calls, send back the tool results, but at the same time sending the schema with every request. And it just knows that when it's done getting through its request loop, here's structured output. The methodology that I adapted for Anthropic, that kind of worked anyways because we just go through all the request loop with tools and everything, with tool.

like sending back the tool responses, like tool output. And then at the very end, we were like sending a message to say like, hey, respond in structured output. That works. But for...

Well, I was sending that user message along the way for like Anthropic anyways. But like with Olamma, I have to do all the requests loop first. like send the messages, send the tools. If the message comes back that like, hey, the language model wants to use a tool, I process the tool, send the tool response back.

And then what I need to do is basically wait for the model to say, like, hey, I'm done with output. We've processed all the tool calls. Like here's your final output. And then I need to send one additional request at the end saying, all right, now take all of that context that you have from all of the other requests and then give me structured output.

That breaks everything. Like it just, it, it, it works, but it breaks all the abstractions and all of the patterns that I have in place because what I'm trying to do is handle all of the language models and like all of the providers in a standardized way. Like I've built up all these abstractions and then I'm now using these abstractions to like have a standardized request loop for handling all the providers.

well now this isn't standardized anymore because I need to wait until we get the like I'm finished signal and then send one last message.

I don't want to like do that. don't want to like, I know I need to do that similarly for Anthropic. Open AI doesn't need it. I don't know what all the, like, I don't know if I'm to have to do that for any other providers yet or not, but like, I don't want to force myself to do that for like open AI if it doesn't need it. Like if it can just handle everything I throw at it, then I just want to let it do its thing. So this led me to a massive.

Chris Gmyr (35:31)
Yeah.

TJ Miller (35:35)
massive internal refactor that I've been, I feel like I've been kind of pushing up against this refactor for a while. And I'm still on the fence about it. Like I've been wrestling with it for like a week and a half and I'm like still kind of on the fence about it. I've pulled in a few people into like

a chat channel and I've been like going back and forth with folks there. I think everyone thinks it's fine. I'm, I'm, I'm the holdout and like, I don't know how I feel about this. the new, the new way I'm handling things in the refactor, like right now when you call like structured output or text or embeddings, like that returns what I've been calling a generator.

And that generator is responsible for handling all the fluent building of the request, then taking that request and iterating through calling the different providers. So we'll iterate through the generator handles, listening for tool calls, invoking those tools, taking the response from those tools, the output, and then sending it back and then waiting for that stop signal.

and then returning the final response. And it's also responsible for formatting that final response and tracking state through all of those request loops. So they're this centralized request loop thing. I don't know. That's why I call them generators. It generates the inputs, the outputs, the requests, does it all. And so it's been nice to like, I really like

having all of that logic in one place, that's like one spot to reason about it all.

But now, with this refactor, the way that it works is you call prism, you call structured output, and that returns what I'm calling, I'm stealing all these things from different libraries. So if you, with Laravel, you have their HTTP helper, you invoke that, you get a pending request. Well, I'm stealing that. I'm calling this a pending request too, because that's kind of what it is.

And it gives you all the fluent methods to build things up where it's like, you're using this provider in this model, these max tokens, like here's the tools that I want to send. It's got this fluent builder for all this stuff. And then the generate method, which is the final method you call to kick all this stuff off, that still lives there. But all that does is defer to the provider. So it calls this provider.

structured and then like passes the request, like the fluently built request is the argument. And so now the provider is like responsible for the whole damn thing, like managing the request loop, all of that stuff. So I've now shifted the responsibility from managing that from this generator class to each provider. And I feel like there's going to be a bunch of

duplicated code all over the place to manage that. But I think that's mostly because so many providers provide some version of an OpenAI compatible API. if I take that away, don't know. We work with a whole bunch of distinct providers that like

like our like anthropic versus open AI where it's like totally different endpoints, totally different way of handling things. Maybe there's not so much duplicated code like in all reality and like the duplicated code makes sense even for like the open AI compatible endpoints because all of them are actually using deprecate like stuff that open AI is already deprecated. So they, kind of need them to all be

separate code bases for each provider because who knows at what point they're going to be up to date with open AI or like be falling behind or introduce something custom that like, you can use the open AI compatible API, but we also do this like other thing that like now diverges a little bit. So I don't know. That's a lot of talking about code on a podcast, but

Chris Gmyr (39:27)
Yeah.

TJ Miller (39:45)
That's what I've been struggling with is I don't know if it's dry enough.

And also like this, all this refactor has been for structured output. like even within the provider, I feel like there's going to be like a bunch of duplicated code between like text completion requests versus structured output versus, and embeddings is just a, it's its own thing. like, but structured output and text requests, like the only difference between the two really is like you're sending

some additional fields or you're like sending an additional message at the end to like say, hey, like do this as structured output.

Chris Gmyr (40:27)
Mm.

TJ Miller (40:28)
So it's...

I think...

I think I need to just like push forward with the refactor. And like, I think this is where I'm at. I think I'm just going to push forward with the refactor, even though I don't feel great about it because at the end of the day, it gives me like ultimate flexibility. Worst case per each provider, there's going to be potentially a fair amount of duplicated code between text requests and structured output requests.

Chris Gmyr (40:35)
Yeah.

TJ Miller (40:59)
but that doesn't break my rule of three even. So like, I've got a rule of three, like I don't really, I try not to like go to abstractions until I'm facing, like I've done the same thing three times. So I'm still only in the like, I've got duplicated stuff twice. Maybe even, like, I don't know, I haven't done a text request yet. So I don't know.

I don't even know how much overlap there's gonna be between the two, but.

Chris Gmyr (41:22)
Yeah. I think, as you mentioned, things in these different AI providers start diverging. And I can even see them specializing in certain use cases. And then it would make the API diverge even more. So I wonder if this is a good preventative measure to making those steps easier in the future.

TJ Miller (41:44)
So what it does is right now, I guess another way of thinking about this is you have your request for structured output or text, and that returns a generator. The generator returns a structured output response, and the provider returns a provider response. So that's the world.

before the refactor. The world after the refactor is the prism class you call structured output that returns a pending request. And then that pending request just returns a structured output request. the provider now returns a structured, or sorry, the like prism structured output returns pending request. The pending request returns a structured output response and

the provider now returns the structured output response. So we're getting rid of this concept of a provider response. And the provider is just responsible for returning the actual end response for the whole thing.

I think it starts to treat all the providers as mini isolated packages of functionality. I think this whole thing was modeled after how Versel built their AISDK. That was the initial inspiration for all of this. And that's very much what they do. Each provider is literally its own package.

So if you want to use the Versel AI SDK, you're pulling in the SDK and then you're also pulling in each provider package individually based on the providers that you want to use. So you'd be including the like AI slash open AI, AI slash Anthropic if you wanted like the Anthropic models. So they're very much treating them like as little isolated packages. And I think you can even use each provider.

on its own without using the AISDK. even if I take a step back and look at my original inspiration, they're even handling it much more similarly to the way I'm refactoring things now, which I guess kind of gives me a little bit of an indication that maybe this isn't the worst thing. But it's a big refactor.

Chris Gmyr (43:46)
Interesting.

10.

TJ Miller (44:10)
And I'm just, think I'm going to be stuck waffling on the fence about it until somebody tells me to just like, it's good, go for it. You know? Yeah. I think, I think I'm just going to push forward on it at this point because, I, I need to move on. I've got PRs that people have submitted that I need to get to. And I've just been like stuck in this thing like this morning.

Chris Gmyr (44:20)
Good, go for it. Ship it.

TJ Miller (44:34)
I woke up at five o'clock this morning to like try out, I went to bed last night with a new idea, woke up this morning, like knocked through like three quarters of it and then hit a brick wall and said, maybe this isn't the way to go either. I was using pipelines. I'm like, we'll use pipelines and each provider can like define an array of like handlers to like pass this like state object through. And then.

But then I realized I was gonna end up, I shifted the duplication to the making the requests. Because now the requests have to be duplicated in each handler that needs to make a request. I'm like, that's even worse. That's even worse duplication.

Chris Gmyr (45:16)
I do like pipelines though. Might not be the best use case for you in this instance, but in general.

TJ Miller (45:21)
It was pretty clean. actually really liked if I could have figured out a better abstraction for managing requests than maybe, but.

I think that even though this is like the refactor that I'm working on currently is like maybe more verbose. think it's really easy to reason about where it's just like, Nope, the provider is responsible for the whole thing. the request loop is really not that bad. It's just like a little bit of recursion. it's not like I'm trying to like talk about this like generator doing this like huge complicated request loop. It's just like some recursion man. Like if it, if it, if it ends in like.

Chris Gmyr (45:57)
Not too bad.

TJ Miller (45:58)
If the response ends in like tool calls, then you call some tools and then you just like recurse until it usually keep recursing until it says like ends in like reason stop. Great. Like it's not that complicated. So I don't know. I'm just going to do it and hope it's good. And then I'm sure in like two months, I'll want to refactor the whole thing again. So.

Chris Gmyr (46:25)
Well, you know, that's how goes.

TJ Miller (46:27)
I'll be two months smarter in two months, so.

Chris Gmyr (46:31)
That's right, you know more and that's one of the rules of refactoring, right? Like if you know something new or different or better, then it's okay to refactor.

TJ Miller (46:41)
Yeah, it's just a heap of work. Because I've got to do all the providers. Because I'm not just going to do it for structured output. I'm going to do it for structured output, probably tag and ship that, and then go do all the text requests for all the different providers. And tag and ship that, do embeddings, and then hopefully by the end of the weekend, I'll be all refactored.

Chris Gmyr (47:04)
Nice. Well, that sounds like a good plan. Just real quick, because I know we're getting a little late on time. The only other thing that came up when you were explaining everything is, I think you said, like, Olam has a final step before you can get the structured output back. Have you thought about or implemented any sort of either lifecycle hooks or generator hooks? So you can say, like, hey,

On the generator, can append on a callable or something like that, or even built on directly to the provider. It's like post maybe this type of request. It has to append this as a hook at the end. So if you need to do any follow-up calls or anything like that to close the loop, you could do that either on the generator or on the provider. Not sure.

where it might be like better suited. But I wonder if doing almost like a life cycle hook or request type hook somewhere would allow you to have some of those options conditionally.

TJ Miller (48:09)
You know, I definitely thought about that. was thinking about like, because another feature I want to add in the future is is middleware where you have like an opportunity before I send a request to like modify stuff if you want to, which could be interesting for like rag workflows or other things, but like.

that is going to require some sort of hook system. Like, maybe I just make a hook system, and providers could optionally hook into after stop, right? Because that's what I have to do. I have to wait until Olauma gives me the finished reason stop. And then I send one last request after that that says, hey, give me structured output. So I could have an after stop hook and somehow figure out

a way to have the provider like register itself, like listen for that hook or something.

I, there's a couple pieces to that mental model that I like, I haven't been able to figure out. And so I just kind of like table that cause I couldn't think of it, but maybe, maybe I'll give that another shot tonight and like, see if I can figure out how to hook because that'd be, that'd be a nice way to, think, keep the central like, like request loop in place, but then kind of allow for.

flexibility for providers to be able to still do their own thing.

Chris Gmyr (49:27)
Yeah, I don't know. Check it out. Just what popped into my head. It sounded like it could be a good possible option.

TJ Miller (49:34)
Yeah, I also kind of started reaching for the like my design patterns dictionary and was like looking at some different like strategy patterns, but I don't want to like necessarily have a bunch of bespoke strategies that are only being used for one provider. And now I've got this like namespace strategy that's like just for one provider, but isn't necessarily living with the provider because the generator is like part of the package.

Chris Gmyr (50:01)
Mm-hmm.

TJ Miller (50:02)
And I've always just kind of naturally thought about there being some separation between like the package and a provider. I don't know why. I think it comes from the, like, the initial inspiration being Vercell. so I don't know. think I'll explore some hooks, I think tonight, see how it goes. And depending on what happens, like I'm not leaving this weekend without it being done. So like, whatever I decide to do, I'm,

I'm putting the pressure on myself to finish it this weekend because I've got a bunch of other stuff that I want to tackle. And this is holding up. This is now starting to hold up a fair amount of stuff and it's giving me anxiety. So I'm doing it this weekend, man.

Chris Gmyr (50:41)
Yep, call it good, cut it, like, publish it, good to go. Because there's always going to be more stuff to work on. So.

TJ Miller (50:46)
Yup.

100%, man, 100%. This is a never-ending thing for the near future. Cool, man. Well, we can probably wrap on that.

Chris Gmyr (50:55)
Totally.

Yeah, let's wrap up.

TJ Miller (51:00)
Cool. Thank you all for listening to the Slightly Caffeinated podcast. Show notes, including all the links from our show and social channels will be at the bottom of slightlycaffeinated.fm. Thank you all for listening. We will catch you next week.

Chris Gmyr (51:15)
See you all next week. Thanks.