Slightly Caffeinated

TJ Miller (00:00)
Hey, welcome back to the Slightly Caffeinated Podcast. I'm TJ Miller. so Chris, what's new in your world,

Chris Gmyr (00:05)
and I'm Chris Gmyr

Yeah, kind of quiet week for work stuff. This week my wife had to travel up to New York to help her mom out with some health stuff. So I've been playing single dad from home the whole week. doing took the week off doing the whole like school pick up and drop off hanging out with the toddler during the day and naptimes and you know all all that. So it's been kind of a different week.

TJ Miller (00:20)
Hahaha

Chris Gmyr (00:36)
busy as far as just family stuff going on. And we had a little bit of a stomach bug come through the kids last night, which was not super fun. And we'll get into the details of that, spare everyone those details. But yeah, was up pretty late with the toddler and not feeling 100 % today, but we're just going to roll with it. And luckily, my wife is coming back home tomorrow.

TJ Miller (00:43)
Mmm.

the gory details.

Chris Gmyr (01:00)
afternoon, Friday afternoon. So get back to somewhat normal, but just a huge sense of appreciation and the acknowledgement of the work that she does. I've done a long weekend with them or something like that, but this is the first time that we've gone six or almost seven days with her being gone and me taking the week off. So you can basically do anything for a couple days.

TJ Miller (01:20)
yeah.

Chris Gmyr (01:25)
You know, not too bad, but if you have like an extended time of doing that, feel like more things crop up and you're just energy level wanes and is completely different, you know, at the end of that timeline. So to do this job like every week, week in and week out and for, you know, months and years that she's she's been doing it with my son originally and then still at home with the both of them.

is just totally different from, you know, what I do going to work and being able to change that up on a daily basis and have a work in like adult life and then dad, father, husband, family life. So, the fact that she doesn't have that sort of difference of, of lives and like energy and resources, it just that much harder to get through. So.

TJ Miller (02:14)
Yep.

Chris Gmyr (02:15)
you know, just

TJ Miller (02:15)
Yeah.

Chris Gmyr (02:15)
even more of a appreciation of, you know, the work that she does, even if she's not getting like actively like paid like monetize like from that work, like it's hugely important to do. yeah, yeah, totally. And as a dad and family man yourself and with the kid and the dogs and the general zoo that you have, you know, I know that you've had

TJ Miller (02:27)
Yeah, doesn't mean it's not work, man.

Chris Gmyr (02:41)
you know, a lot of experience, you know, doing that too. So it's just what we need to do and it's necessary and important work.

TJ Miller (02:49)
Yeah, yeah, man. Even when Laurel steps away for a couple days or something, she just comes back and I am so appreciative of all the work she does around here because it is so much to do with one person and that work just is very unforgiving sometimes. So, no, massive appreciation for Laurel. yeah, just because they're not like...

Chris Gmyr (03:09)
Yep.

TJ Miller (03:15)
getting paid doesn't mean that it's not hard work, you know?

Chris Gmyr (03:17)
Yeah, yeah, 100%. So yeah, looking forward to her getting back and seeing how everything's going with family up there. And yeah, since the last podcast, I did publish the YNAB CLI first article of the series. I plan to do a couple more. Maybe even I'll crank out one tonight or tomorrow, maybe before.

this podcast airs, because I'd like to do at least a few more with it. But man, I like writing blog posts, but it seems like it took me probably five times longer to write the article than to just crank out the code. And granted, it's not like the most beautiful code is just of like hacked together as an experiment and a demo right now. And I hope to do some refactoring.

TJ Miller (03:54)
Yeah, always.

Yeah.

Chris Gmyr (04:06)
edits to not only the code in the Share repo, but also through the blog, kind of talking through that. just lapping together the code, it seemed like, I don't know, took like an hour, hour and a half or something like that. And then it just felt like hours to get all the information in the blog, make it understandable, make it easy to read, easy to understand, and stepping through all the things that I thought about and worked through the code.

to give a good understanding of what was done. And it just seemed like so much more work to do the blog post than to do the actual code that was spit out. So yeah, just interesting for sure.

TJ Miller (04:40)
how it always is, man.

That's how it always is. every blog post, every technical blog post I've ever written takes significantly more effort than the code that I'm talking about, like every time. But the thing is, is if you think about it, like the code, you can always go back and refactor, right? Like the article is kind of publish it once and let it ride. So you got to kind of take that extra time upfront that you're not going to be putting back in later as like a refactor or something, but

I have to hand it to you, That was a great article. I really enjoyed it. I thought it was really well done, put together, really well organized. That was good. I'm looking forward to the next one for sure.

Chris Gmyr (05:18)
Yeah, thank you. And the next one will be a little bit shorter because it's just like checking a category balance. And then I think we'll probably do some a little bit of refactoring after that because there's a little bit of duplicate code throughout both of the commands. So that'll be an interesting third or fourth follow-up article. So I know. I tend to like that progression through articles and series like that. And it gives people a glance into

TJ Miller (05:41)
Mm-hmm.

Chris Gmyr (05:45)
your thought process and what's available with the tooling and frameworks that you have. And might give some newcomers or beginners some other ideas on how to make their code work a little bit better or how to refactor things or make it a little bit cleaner. So I know I always enjoy reading those type of blogs. So hopefully other people enjoy it as well.

TJ Miller (06:06)
Hey man, it makes it like, I think there's also something about like making it more relatable to you, right? Like that's, we don't always crank out perfect code the first time around. that like refactoring is very important and necessary part of the process. And I think that's something that gets like lost on newer folks sometimes like reading stuff from more experienced people is like, wow. They just, they wrote this like perfectly the first time around. like, no, like this is awful code. Like I need to go back and refactor it. Like just.

It's part of the process. Yeah.

Chris Gmyr (06:36)
Yep, 100%.

And it's OK not to have perfect code for some instances. Like this one is just a hacky little side project. I just want to make it work and maybe make it a little bit prettier later. But yeah, it's not going to go out to anyone. No one else is going to work on the code besides me. Maybe some people will pull down the repo and use it for their own purposes. But it's just an experiment and something fun to work on. So not everything needs to be perfect with that.

TJ Miller (07:02)
Yeah. No, it's good stuff, man. I really appreciate you putting that out there.

Chris Gmyr (07:06)
Thanks. Yeah. So yeah, that's about it for me. What's up in your world?

TJ Miller (07:12)
man, life has been kind of chaotic. Mostly good, but kind of chaotic. So I haven't had, unfortunately, like a ton of time to put into Prism. Hopefully next week is going to be a little different. We'll kind of see how that shakes out. yeah, man, just been like hunkered down, taking care of life. Had some like really cool developments at work that we can talk about. I mean, Lericon EU happened.

managed to catch a little bit of that, which was pretty cool. Yeah, it feels like a lot and also not a lot has happened in the past week. I'm exhausted, but I guess on a personal note, I'm on week... I think this is either week four or maybe week five of being back at the gym.

Chris Gmyr (07:45)
You

TJ Miller (08:00)
three, four times a week. like yesterday, yesterday and the day before were like back to back days, which I haven't done in a long time. Normally I've lately I've been doing like Tuesdays, Thursdays and Saturdays. and yesterday and the day before were the first like back to back days that I did. and for me at my age and like level of fitness and everything, like recovery is brutal for me. So to get to the point where I

Chris Gmyr (08:00)
That's awesome.

TJ Miller (08:25)
could do to back to back days and not be totally dead is it feels like a, feels like an achievement, like a milestone, you know? Like I earned my badge for getting back to back to back days. And I'm at that point too, where I feel like there's always this tipping point of it's like a pain in the ass to get up and like get the motivation and the drive and like go to the gym and do the thing. It's like, it's really,

Chris Gmyr (08:35)
Yeah.

TJ Miller (08:53)
really rough to start. But at some point, there's like a shift and your body starts craving it. And so that like that's kind of what ended up happening is I've hit that tipping point where like I'm craving that activity. I'm craving like going to the gym and doing something. And that's kind of how like the back to back back to happen. Like my wife has been starting to get back into the gym a little bit too. And she was like, like I had already gone to the gym.

on Tuesday, like I normally do, and yesterday being Wednesday, she was like, I'm thinking about going to the gym. Like, I'm with you, let's go. And so we do very different things at the gym, but it's still kind of nice to go. Like, even though we get there, we'll like warm up on the treadmills together, and then we typically will go our separate ways and kind of like check in with each other throughout the session. But it's nice to go with somebody, you know, even though it's not like lifting buddy kind of.

Chris Gmyr (09:28)
Yeah.

Yeah.

TJ Miller (09:47)
activity, it's still nice to just know you've got someone there and it's nice to be able to check in. So I'm feeling so good about it. Like it's also with that tipping point too, like you get that motivation because your body craves it, but then you also like I'm at that point where like four or five weeks in, like you're starting to see those changes. Like I'm definitely starting to see like

Chris Gmyr (09:54)
Yeah, 100%.

TJ Miller (10:10)
more definition, I'm feeling a lot better. I'm feeling a lot stronger. My recovery times have like gotten so much better. They're like, that's also motivation to like, you you start seeing some of the results from the work. It's like, all right, I like that. Like, let's keep going. And I, I think before you get to that point where either of those two things happen, like before you start seeing results and before your body kind of starts like wanting that activity is just, it's a slog man.

but like being on the other side of it is just like, let's go. Like I hope I don't get sick and like lose the momentum or something, but.

Chris Gmyr (10:45)
Yeah, 100%. And yeah, like I've been since the beginning of the year, I've been trying to go back to but it seems like the last couple weeks between like snowstorms or people into town and now like my wife is out of town so I can't really go. I think like and I haven't gone at all this week or the tail end of last week. So I've definitely felt the letdown of that which you touched on and just like feeling that I need some sort of

TJ Miller (10:57)
Yep.

Yeah.

Chris Gmyr (11:10)
like energy outlet or just feeling like extra tired, like a little like touch of depression, like, you know, whatever, like this, you know, doesn't matter. Like I'll just eat, you know, something that I don't really care about or something like that. And yeah, definitely looking forward to getting back to it next week and building that back up again. Like I'm not too far off, but just wanting to get back into it and starting to feel better. And I think I've come to.

TJ Miller (11:16)
Yeah.

Chris Gmyr (11:36)
the conclusion that instead of going like three days a week, I've been trying to go like Monday, Wednesday, Friday, like first thing in the morning. The gym that I go to opens at five. So I usually get there at like 515, 520, something like that, which is pretty brutal in the morning, especially when it's like cold. But like that's the most consistent time that I have to do it. Cause if like, you know, trying to take like an extended lunch or something like that, it's like meetings come up, things to do, like other, other stuff going on. So it's just like now I

TJ Miller (11:47)
Yeah, it is.

Yeah.

Chris Gmyr (12:01)
tried that a little bit, it just doesn't work out. So we just have to go in the morning and come home before the kids wake up. But I think I want to try doing five full days and just doing either lifting, like those same three days, and then having Tuesday, Thursday as maybe cardio or cardio and some sauna.

because my gym also has a wet and a dry sauna. So usually doing the dry sauna is just as good as getting on the treadmill for 20 to 40 minutes. So I think to build up just the momentum of going and just flipping around my circadian rhythm to ensure that I'm up and have energy that early instead of.

TJ Miller (12:24)
Ooh, nice.

Chris Gmyr (12:42)
bouncing back and forth between days, like getting up super early, getting up a little bit later, or like, you regular time. Just keeping that flat and consistent day to day. And I don't know, it'll be a little experiment and see how I feel like after a couple weeks of doing that. And I think if I had to miss a day because we were traveling or something happened or, whatever, then it wouldn't be like as big of a negative impact of missing one of the three days and then having to wait, you know, X amount of time to...

to get back to it. So I don't know. We'll see how it works. But that's awesome that you guys have been going so consistently. And you're feeling great and seeing those changes too. That's awesome.

TJ Miller (13:21)
Yeah, man,

it's and I think something you touched on is like part of the really important aspect for me too is like the mental health aspect of it all. Like it's been so good, like actually seeing the results of like the physical changes too. But man, like the mental health aspect is profound. And yeah, I'm just I'm so appreciative to like

be fortunate enough to be able to do it. like, I don't know, man, I'd like to get back to, I'd like to get to like a five day a week flow, but like, I didn't want to blow myself out in the beginning either. And then like crush my momentum. Cause I will. Like I, I, I used to go to the gym quite a bit. I was significantly stronger. Like it's been a number, it's been difficult on the numbers game because last time I was at the gym, I was putting up probably double what I'm doing now. And that has been.

Chris Gmyr (13:52)
Totally.

TJ Miller (14:07)
painful to like go and see the night I've had to be very very conscious of like not blowing myself out and then like losing all the momentum so it's been like even though like two weeks ago I wanted to go more than three days a week and like I can't and I shouldn't because like I'm gonna hurt myself I'm gonna blow myself out and then then what am I gonna do I'm gonna have to like wait a week to recover from everything you know so but yeah the mental health aspect of it has been like

Chris Gmyr (14:18)
Yeah.

TJ Miller (14:35)
profound for sure.

Chris Gmyr (14:37)
Totally. Are you doing like a specific program or just doing some activities or exercises that you just enjoy doing?

TJ Miller (14:44)
No, loose, loose plan. And that's actually something I just reached out to, a trainer that I know and, you know, was inquiring with him of just like, Hey man, like instead of doing personal training with you, will you just put together routines for me? Like I will. Like I'm already right now, like recording all of like the weights that I'm doing for each exercise and everything. And like what exercises I'm doing each session.

So, and I'm actually probably going to slap together a Laravel app to like do all of that for me. Like just something simple. Like I'm doing it right now in Apple notes and like just a table, like copying the tables and like just filling stuff in. And it's been really simple of just like, here's the exercise, here's the weight, here's the like rep configuration. And I'm either one of two statuses. I'm either comfortable doing it or I'm pushing doing it. And that's just like, I wanted to.

put together a super simple system of either it was hard for me to do this weight or it was comfortable for me to do this weight. And then that kind of informs what I do the next session. If I was pushing, then I'm going to stay with that weight. And once I get comfortable with that weight, that's when I'm like, if I was comfortable this session, next session I'm going to do a 10 % increase. And just kind of keeping that train. it's like I do 10%. If I'm still comfortable the next time, I do another 10 % on top of that.

And I keep going to the point where I'm like, all right, now I'm pushing again. All right, I'll sit here until I'm feeling comfortable to go up another 10%. So that's kind of like the loose game plan that I have. And then I just kind of mix things up a little bit, sort of based on how I feel. I'll go in and be like, all right, I'm doing upper body today, lower body today, a little bit more cardio focused, kind of figuring where things are, and then trying to mix stuff up inside of there.

If upper body last time I like mostly focused on shoulders, maybe this time I do like little less shoulders, but like focus more on like back and pecs, you know, just I try to add some variants in there and like move the focus around a little bit. And that helps going more days too, because then you have more variety and you're kind of focusing on like bigger, bigger, you get more surface area coverage, you know, so.

So I'd like to get to a place, like it's very loose. It feels good, but it's very loose. I'd like to get with a trainer and just put together some routines, at least for like maybe a month or two until I kind of get the swing of it and then be able to run with it on my own. I've thought about sinking like a couple hours into trying to get like a AI to put together workout routines for me. But if I didn't know

a couple trainers to reach out to. That's probably what I would do is I'd spend some time trying to get like a language model to spit things out. But because I do know people, I'd rather hit them up and throw them some money to help them out, help me out and do something. So yeah, man. You want to talk a little tech stuff? All right, so.

Chris Gmyr (17:41)
Totally. That sounds awesome.

Sure, let's do it.

TJ Miller (17:46)
Let's talk about ClickHouse, the database, little bit, just because this is what I've been living at work and I'm really tickled because yesterday I hit a milestone with it and so I'm just kind of riding on it. I work at Geocodeo. We do geocoding. We do lots of geocoding. As of a couple months ago, over the span of 30 days, we processed 17 billion requests.

lots of traffic. we log, like the reason why we know we did 17 billing requests is because we log every single request for a couple of reasons. Usage-based billing and like debugging purposes. You know, those are the two primary reasons we log every request. So I think we've touched a little bit on this on like a previous episode, but I'll recap. Right now we're using MariaDB paired with the Toku database engine underneath it.

to handle high frequency inserts. So that's our current setup. TokuDB has been deprecated for a while. So part of one of the projects that I got put on coming in was like, what are we going to do about that? Do we migrate to a different database engine? Is there a better solution out there? And first thing that came to mind for me was Jess Archer's click house talk. I was like, this is probably the perfect thing for it because

What are other people like one, what are other people using ClickHouse for? And two, like what did Jess talk about in there? And it was like, all right, doing like analytical queries. Well, that's mostly what we're doing with this data is like usage based billing. We're like doing different calculations on like all the number of requests. It can handle large, very large amounts of data. We have large amounts of data over like 17 billion records.

Chris Gmyr (19:27)
Yeah, that's

TJ Miller (19:29)
large amounts

Chris Gmyr (19:29)
a lot of data.

TJ Miller (19:29)
of data. But then you also look at who's using ClickHouse and how are they using it. So if you think about Cloudflare, they're basically doing the same thing that we're doing. They're logging all the requests. You talk about Laravel Nightwatch. What are they doing? They're logging a bunch of requests and what are they using? ClickHouse. I'm like, my God, this has got to be the ticket. did a little research. I went through ClickHouse's online training. They've got

actually excellent online training for all of it. So I went through their like self-paced online course, learned as much as I could about it, started getting some configuration stuff together. Last week we launched it and we launched it to like 30 of our hundred something servers. So just a small percentage. I just kind of wanted to see how things were going to go.

So I launched it to like 30 of them, let it sit overnight, came in in the morning, had exceptions. The exceptions were basically, without getting too technical, the exceptions were basically telling me that we were inserting more records than it could keep up with. Which I knew was probably going to be a problem going into this because that was like part of our bottleneck in the first place.

was we're just inserting so much data. there was like, I went and just the first thing I did just like Googled the exception, right? And what came back to me was a page dedicated to this exception on the ClickHouse docs website. I'm like, this is fantastic. And they basically prescribed like the solution there with Mike, their ultimate advice was batch.

do batch inserts. And that's really what ClickHouse prefers is instead of yeeting a bunch of single requests to it, it prefers you do big batch requests with thousands or tens of thousands of records. That's just not an option because we record the request as part of Terminable Middleware. So we send the response, then we record the request. unless I introduce a new piece of architecture,

Chris Gmyr (21:31)
Mm-hmm.

TJ Miller (21:37)
I don't have the ability to like buffer and batch the request. Well, Clickhouse also has solutions for this. And so part of their other, like their ultimate advice was do batch requests. But they're like, well, if you can't do batch requests, use a buffer table. So it's a table with a specific engine. like database, you have your table and then you have the underlying database engine.

They have a buffer type database engine. I'm like, oh, that's cool. And then furthermore, since we're using high availability, we've got a cluster of three ClickHouse servers. They have what's called a distributed table. what a distributed table does is it, at least my understanding of how I'm going to explain it, I could totally be wrong here. But as I understand it,

It load more or less like load balances that insert across all of your nodes in your cluster So for example if I say I've got click house server click house one click house to click house three if I send an insert request without the distributed table To click house three it's gonna get inserted on click house three But if I send it to a distributed table on click house three it that may forward that insert request to click house two because that's what's

like based on its algorithms, like that's really where that should get inserted. And then eventually that gets like synced across all three nodes because we have like replication enabled on all three of them. So I ended up having to create the buffer table and then you create the distributed buffer table for that. So now I've got three tables to like handle this thing, which I guess isn't, it sounds insane.

but it's, I guess, not so insane. So we have our end storage table, which is our request table. Then we have the request buffer table, which we insert everything into and then click house based on our configuration will eventually take the records from the buffer and then insert them into the request table. And then we have the distributed table on top of the buffer table. And all that does is then load balance the request into the buffer table. And then again, the buffer table to the request table.

So I set all of that up on like Monday. So today's Thursday. I set that all up on Monday. Tuesday, I deployed to the same 30 nodes of our, like 30 of our servers. Let that sit overnight. Came in in the morning. I'm like, hey, like no exceptions. Like this feels awesome. Like let's now.

go for it and like deploy it to all of our like 100 overall. We've got like, I don't know the exact number off the top of my head, but it's, it's less than 200 and more than 140. So we have a lot of servers. Um, I won't go into why, but we just, do. Um, and I deployed it to everything. that was, uh, Wednesday I deployed it to everything.

Let it sit overnight, came in this morning, had a couple exceptions. They were just timeout exceptions this time around, which is, you know, could have just been a network blip, you know, but it handled peak traffic and it was only like 13 exceptions or something overnight. So like not even anything super crazy. And then I went and checked the count and there's already like

25 million records inserted into ClickHouse. I'm just like, this is insane traffic. But we already have millions in there. And so a query that I would be terrified to run on our MariaDB setup would be query the top five users and give me the total number of records for each of those users. So it's like,

selecting user ID and then doing a count selected, select all of them by user and then grouping by user, limit five. I can't remember the query off the top of my head. But anyhow, you can do it all in one query, right? And so I'm terrified to run. I would be terrified to run that on MariaDB. But I ran it on here and it came back in 20 milliseconds with the top five users.

one of them be having like 18 million records. we, you know, was just currently with the MariaDB setup that if I went and looked at that user right now, that top number one user, it crashes Nova. So like the query is like just go and count all of the records for this user ID crashes Nova.

Chris Gmyr (26:07)
Yeah.

TJ Miller (26:09)
because the query is massive. The fact that I can do that for the top five users in a single request in 20 milliseconds is so promising that this is going to alleviate the issues we've been experiencing with the MariaDB setup. And I am just so tickled that we're inserting it's...

holding, like the boat is holding water and it is like looking so promising for like the hopes and dreams of the project that I'm, it's, it's just so fresh that I'm, I'm on cloud nine about it right now.

Chris Gmyr (26:46)
Yeah,

that's really sweet. that you can still have the granularity of having those single requests being logged and being able to grab totals and aggregates of things very quickly for basically whatever sort of query that you want or need in the future.

TJ Miller (26:48)
Big projects.

Yeah, could absolutely

still like that's and that's an enhancement that I think we're talking about internally to like deliver to all of this right now. As a customer, you can go to your dashboard and see the total number of like month to date requests. But you can't really, there's no like line item display for like each individual request. So if you as a customer wanted to go through and debug your requests.

that's not really feasible right now. Because honestly, if we're doing usage-based billing for the month, then we're storing all of your requests for that month. So as the month progresses, that's going to be harder and harder to load all of your data. So it just it's.

It's just not feasible with what we do now. And that's something that I think we've been talking about wanting to do now is be able to give you a paginated table of all of your requests and be able to actually see all of the details. And I think we're going to be able to accomplish it with this. I've just been super impressed with ClickHouse. And I'm sold, man. This is really cool. And it's been tough. ClickHouse is just fundamentally different than

Chris Gmyr (28:09)
Yeah.

TJ Miller (28:11)
everything I'm used to.

Chris Gmyr (28:13)
Yeah, totally. And I was thinking, as you were explaining everything, I'm like, how would this be different than having maybe an event in aggregate-based system to listen for new requests coming in and then updating accounts table or something like that or whatever tables or views that you have set up. But having to maintain that and especially at

like a high scale, high availability, like you could run into like a whole bunch of conflicts with that or have other sort of things that you need to work through with that. Having this platform do all the heavy lifting for you, especially being able to handle that scale and numbers, you know, hour to hour is just like so much better than having to kind of build your own thing or like

worry about handling this in a not ideal database system. And like you said, it opens up the possibility of bringing this data and different views into the customer UI and being able to do whatever you need to do with that. Like you said, debugging calls. It's like, hey, I see this call failing for the zip code or something like that. Being able to search for that zip code or any sort of address data.

TJ Miller (29:27)
Mm-hmm.

Chris Gmyr (29:31)
that you have, or if you're able to tie it into a specific client. you have maybe one of your customers has a web client versus an API or mobile client. Maybe that's tagged in the request somehow, being able to see those different data points in there and be able to narrow down where the issues are would be hugely important to customers moving forward.

TJ Miller (29:54)
Yeah,

like we track user agent and everything too. So yeah, if they were trying to debug an issue with their, like depending on how they tag their user agents and stuff, because you can have custom user agents on your request. They could be doing some sort of tagging of like, it came from this server. And now they're able to diagnose that like, this server is having issues for whatever reason. Or like trying to debug multi-availability, like multi-availability zone kind of stuff. I don't know. I think just being able to offer that would be

huge for the developer experience. it was an intimidating project to come into being new at the company. This is foundational to how we build, how we track stuff.

It was a big project. was very intimidating at the beginning and, um, you know, was new, all new stuff for me, you know, it's new stuff for, for everyone there. None of us have ever even looked at ClickHouse before. Um, so it was definitely like intimidating and, you know, last week when I started getting those exceptions, I'm like, Oh man, like, I really don't, I don't know what I'm going to do about this, you know? And luckily there were some good docs and,

I mean, Claude had more or less given me very similar advice too. So, you know, went, I went to both like searching Google and I'm like, Claude, here's my problem. Like what, what do you got for me? introducing a buffer table was one of its suggestions. So like, this was a very valid way to go. and so I'm just, you know, coming from it being very intimidating and very new to a place where, I'm like super excited about it. And like, it's, it's working has been.

has been a journey and I'm glad to be at this point in it. And now I get to sit back and monitor it, make sure nothing goes crazy and...

pat myself in the back, have a little win, you know? It's been a few week project, man. I've been working on this for maybe close to two months now. So like it's been a sizable effort and to kind of be at the point where stuff's working, you can see the light at the end of the tunnel. know, we'll ride out, I think we'll ride out this month and then next month since we'll be able to like start fresh with collecting.

Chris Gmyr (31:40)
Yeah, yeah, that's awesome.

TJ Miller (32:02)
data from day one of the month. We can even start playing around with updating the reads and everything sometime next month. But we're doing dual entry to both databases. And I'll probably continue that for the next month or two and just make sure that in case we have to fall back, we can fall back. But things are looking good now.

Chris Gmyr (32:20)
Yeah, nice. I was going to ask you that too. How's your migration plan looking forward and still doing the dual rights or changing the primary eventually to click house?

TJ Miller (32:30)
Yeah, we'll do, we'll use dual rights, I think, um, for the next couple months. think at least that would make me comfortable if we did it for the next couple months, just to like really make sure that like ClickHouse is not going to fall over and that we've configured everything correctly. Um, cause like I said, it's, it's all, you know, some of it was me guessing on the configuration to be completely honest, me and Claude, just like making some assumptions and guessing on how to configure things correctly.

Cause the courses were great, but there's a lot of decisions to make. And I knew that we were going to be doing something that ClickHouse wasn't, they tell you to batch insert. Like that's the big, it's a big core tenant of ClickHouse. And like, I knew we weren't going to be able to do that. So I was kind of going into it like a little, well, how are we going to like do the thing they tell you not to do and do it in a reasonable way.

So yeah, we'll keep dual entry around for a bit just in case we need to fall back. But I think sometime next month we'll update all the reads to also come from ClickHouse and start to see the benefits of that side of things too.

Chris Gmyr (33:39)
Nice, yeah, that'll be super exciting to make that switch. And then you can, I know, I can see huge impact for not only you guys, but also the customers and what they're able to do in the future in their dashboards, like you said, with reporting, more granularity, debugging, like, I don't know, that's gonna be awesome.

TJ Miller (33:58)
Yeah. And I just massive, massive shout out to Jess Archer for giving that talk and putting that on our radar. Cause honestly, there's a good chance it could have ended up being another like Maria DB with some, some other engine under the hood. But the literally the first thing that came to mind, I was, I'm like, Jess Archer click click house. Let's, let's go. Like, let's look into this. There's, there's, there's something here.

Chris Gmyr (34:13)
Yeah.

Yep. Let's check it out.

TJ Miller (34:24)
So speaking of Jess Archer and talks about ClickHouse and everything, Lericon EU is a thing that just happened. They had a bit of a live stream. know neither of us were able to make it out there, unfortunately. Were you able to catch much of it? I know you've been super busy. I can't imagine you had any bandwidth for it.

Chris Gmyr (34:33)
Yeah, just rip it.

Yeah, unfortunately, I haven't been able to see much of the live streams. did catch, I believe, the tail end of Taylor's talk for L'Ereux Ville Cloud, which looked fantastic. And going live at the end of February, which is pretty sweet. Looking forward to seeing how that works out with some of my side projects, because I have all Forge, Envoyer, DigitalOcean, that whole setup, which has been working great.

TJ Miller (35:01)
Yeah.

Chris Gmyr (35:12)
Still a decent amount of maintenance, and I feel like I haven't been able to put the time into the maintenance of the droplets and all the things that I should have been doing all the way along. if I can offload that and it works out for the side projects on cloud, then that would be fantastic and condense a few of the workflows and the things that I've set up. So looking forward to that. Yeah, and that's basically the only part of the.

the talks that I caught for two days. I did hear a little bit of Aaron Francis' fusion, writing PHP and JavaScript. So I have to look into that a little bit more. Sounds really interesting. Not sure I would have a use case for it, at least right now. But I don't know, once I dive into it a little bit more, there might be more of a use case there.

TJ Miller (35:40)
Haha.

Yeah.

Chris Gmyr (36:04)
And then I heard that we have a native PHP iOS app approved and in the store from Simon. That is awesome. So yeah, looking forward to looking into that a little bit more and hearing more about that. yeah, that is pretty sweet.

TJ Miller (36:09)
Woo! Crazy.

Yeah, talking about this gym workout little appy thing I want to build for myself. That was definitely something I wanted to put on cloud. But at the same time, I'm like, ooh, maybe I can put this in iOS now and just put it on my phone. That'd be super dope. I was super heads down on this click house stuff trying to get that deployed.

Chris Gmyr (36:34)
Yeah.

TJ Miller (36:44)
testing that. So I only really made time to catch Taylor's talk. I have to say, all right, cloud, much hype, absolutely going to be using it. I think it's transformative to the space. I've got stuff I want to host over there. I think it's an absolute game changer for PHP, for Laravel. But to be honest, like I, the end of his talk had me

pretty misty eyed man. know, the end of it, I'm getting a little choked up just thinking back on it. was, you know.

very encouraging and like inspirational and I

I don't know, man. It was very meaningful. So if you don't watch anything from Lara County, you go catch the Tail Under Taylor's talk. Yeah. Yeah, that was really good. I'm excited for the live stream. I don't know if it's even still up at this point, but I'm excited to go back and catch a couple of the other talks. I definitely want to catch Simon talking about iOS. I know there was some like Nightwatch talk to you.

Chris Gmyr (37:33)
It was fantastic.

TJ Miller (37:48)
I want to catch up on that because I think that's definitely something that I'm interested in. Just like hearing more about that. I'd really honestly, there were just so many talks there that I would like to go and see. think. Was it Marcel who did had a talk on like slow database queries? I think it was Marcel that I'd like to catch that one for sure also. Some interesting stuff in there and then Fusion, man.

So, geez, it's so cool. Like I didn't catch Aaron's talk, but he did release a video at some point after, his, basically his walkthrough of Fusion. And I caught maybe the first third of it. Wild stuff, man. Being able to like write PHP in your view components. And I know he recently said like React's coming, React support will be coming, but like taking...

single file view components to a whole new level of like being able to write PHP in your view component and like it not be a like WASM process. So it's not like PHP running in your browser.

I don't have a use case for it, but my mind's blown.

Chris Gmyr (38:55)
Yeah, and just crazy that you can even do stuff like that. And then now we have options to make that even easier for whatever use cases that you need that for. Just really interesting projects that people are coming up with.

TJ Miller (39:07)
I'm such a proponent of the low, no JavaScript, really minimal use of it where you actually have to have it. I know there's use cases out there that are profound and you gotta use JavaScript for this, just because that's the way you want the interactions to work or the style of app that you're building. But I think there's so much that can be accomplished without JavaScript or minimal JavaScript.

Chris Gmyr (39:18)
Yeah.

TJ Miller (39:34)
like sprinkling in like Alpine live wire stuff like there's I Think that's like that's definitely my my preference for style of building apps So like I don't think that this is super applicable to me at all But boy, is it cool?

Chris Gmyr (39:48)
Yeah, it's super cool.

TJ Miller (39:50)
Yeah, I know

I've heard him talking a bit about it on mostly technical and writing and having AI write AST parsers, recursive AST parsers to parse the PHP to do stuff, which is just mind-boggling. But hey, this is also coming from a guy who built an entire terminal emulator in PHP. Aaron is just some sort of wizard. I don't know.

Chris Gmyr (40:15)
Yeah.

TJ Miller (40:16)
very cool the stuff that he's been putting out.

Chris Gmyr (40:18)
Yeah, it's super cool.

TJ Miller (40:19)
So, man, do you want to, I don't know, do you have anything else you want to talk about, Lerika, on EU? I feel like I haven't watched enough of it to, like, talk too much more about it.

Chris Gmyr (40:27)
Yeah, same same here. Hopefully over the course of the next week I'll catch a few more talks and then maybe we can jump on some updates or takeaways next time.

TJ Miller (40:36)
Yeah.

So you got enough time for a quick Prism update? Cool, man. So I haven't had a ton of time to work on Prism, unfortunately. I know I have a little bit of a backlog. All of y'all who have been commenting on things, discussions and issues and poll requests, I am so sorry. I see it all happening. I will 1,000 % get to it at the latest this weekend. I have feedback and thoughts for everyone.

Chris Gmyr (40:41)
Yeah, let's do it.

TJ Miller (41:05)
I've just been so buried. But I was able to kind of push out an update, an internal refactor to Olamma specifically. So Olamma has two APIs. They have their OpenAI compatibility layer, and then they've got like their own API. I was struggling with some of the features in the OpenAI compatibility layer.

I wasn't getting the consistency that I thought I should be getting out of structured outputs and there was an open request for

there was like an open request for someone who wanted us to add like the Olamma internal options API, which can only do, which I think is absolutely should be added, but it was only available through the like the Olamma API. so I decided to make that refactor. Like I was kind of hoping someone else would contribute it, but, with the deep seek R1 demo, I had to make that refactor. And so I locked

I think we talked about this a little bit on the last episode. I did finish out that refactor. I shipped it. We've got some cool stuff in the pipeline for Anthropic. We've got, know somebody who's working on contributing AWS Bedrock provider. So we've got like a whole bunch of stuff in the works, which I'm really excited about. And hopefully this weekend and over the course of next week, I can like find a little bit more time to sink into it.

But we're definitely cooking. And I'm super stoked about it. But that's all I got for updates. We got some cool stuff coming. And Olamma is now using the internal API.

Chris Gmyr (42:26)
Nice.

That's awesome. Yeah. mean, you've

been cranking, and the rest of the contributors have been cranking for a while. So totally, totally fine and understandable to take a little quiet week.

TJ Miller (42:42)
Yeah.

Yeah, I honestly, so this big refactor, this epic of refactoring that I've been talking about for like months, I've now come back full circle on it again and I'm really thinking I have to make this refactor. The whole generator, not generator, providers doing stuff. I think, I guess to recap it very briefly, the way Prism works is you have Prism.

then you decide whether you want to do text structured or embedding output. And then you chain all your options on from there. Well, let's just take text generation. So you go prism, double colon text, and that creates a pending request. So that's where you add in all your fluid building options. And then you call generate. That hands off to a generator class, and there's one for each type of completion. So we have a text completion generator.

And that handles all of the request cycles. So that handles interacting with the provider class to make the request. And then it handles the response from the provider. handles building up all of the like, we have a pretty rich response object, handles building up that rich response object, and then ultimately compiling and returning the final result. Well, the change that

I think really needs to happen is basically to eliminate the generators and put all of that responsibility on each provider. As Bedrock has been being worked on and as I'm looking at some, like after the Olama refactor, there's some things that just are getting difficult to do at like this bigger abstraction, like generator level. So the refactor really consists of

eliminating the generator and now putting all of that responsibility on the provider to handle the entire request loop, like making the request, building up the response object, invoking tools, adding the tool responses back into the request flow. The reason why I've abandoned it so many times is it starts making the providers really complex. And I've struggled with how to architect them.

There are these core concepts that are kind of universal that end up getting a little bit duplicated in every provider, but I'm really starting to find that it's getting harder and harder to do the things that are being asked of me reasonably for like Prism to do. and the things that I want Prism to do is just getting harder and harder to like fit that all into like a universal abstraction, like a generator.

And we're not talking like generators like yielding responses. They're just, I called them generators because they generate the things. But, which is also like a confusing concept as I'm starting to work on streaming and actually using like yielded generators. like, it's starting to get even kind of confusing having the generator classes in there. So one of the things I thought about doing was no longer providing

Chris Gmyr (45:28)
Yeah.

TJ Miller (45:47)
providers internal to Prism. So Prism actually ships with zero providers. if you, like you then have to compose or require each provider. So you would compose or require Echolabs dev open AI, you know, or Echolabs dev Anthropic for like either of those providers. And look, I'll make it super easy. I'll give you an artisan command that you could run like phpartisan prism.

install like add provider or something, you know, and then I can just give you a list of all the providers available and you can just, you know, grab whatever ones then the artisan command can do the composer require and everything. I'll make it super easy to do. So I've even thought about going that far as to like separating them from Prism just to make it easier to reason about and like

I don't know why that separation just makes me feel more comfortable to make the providers as complicated as they need to be, but putting them in the Prism code base where there's now this apparent shared functionality and everything, I feel like I don't want the providers to get too complex, but ultimately I think they need to be complex and manage their own stuff. Just to give like...

ultimate flexibility for each provider to do anything that that provider like may need to do. A really good example is OpenAI and Olamma. If you want to add a system prompt to an OpenAI completion, you can use the with system prompt message or you can like provide an array of messages. But ultimately what happens is even if you use that with system prompt method, what

it's doing under the hood is adding to that messages array a message type of system prompt. Now, if you look at Olama and you want to add a system prompt, that's actually not part of the message chain. that there's a top level request item for system prompts. So in the main body of the request, there's like a field for messages and there's a field for system prompt. And so it's

getting hard to reason about those types of things because it's just so different for each provider. It's like, well, for Olamma, do I strip out all the system messages from the messages field and then like concatenate and add them to the system prompt field? like Anthropic doesn't allow system messages in the message chain, period. Olamma still does. So like what...

how do I handle these three providers that are fundamentally different? How do I handle that in a semi-universal way? If I just make them responsible for their own stuff, that's fine. Then it just kind of works. But now I've got to start thinking about individual providers and the generator abstraction. I'm like,

I don't, like that's a boundary that shouldn't be crossed. So if I make the provider responsible for everything, then it doesn't matter. Like it can do whatever it needs to do to satisfy itself. And just like you pass the provider, the request object, which has a field for system prompt and has a field for messages. Like you just let that ride. And like those providers will figure it out on their own. OpenAI can add it to the message chain.

Like, Olama and Anthropic can add it to the body of the request in the same way.

Chris Gmyr (49:09)
Yeah, I think I definitely get the reasoning. I'm hesitant to say, break that out into additional packages, especially as things are right now with everything kind of in flux a little bit. Because to introduce changes and then dependency changes and then if you have to change directions, now you're updating multiple repos and having to manage different.

TJ Miller (49:22)
I'm a little hesitant to do that to you.

Chris Gmyr (49:35)
subdependencies of those things, like different version numbers. And it could be just painful for everyone to keep up with. I I guess, suggest you can still do the complex stuff in the same code base and keep the boundaries more secure. But I'd be very hesitant to extract anything out right now until the API, the functionality, the tooling in there was a little bit more

solid and where you want to keep it over time.

TJ Miller (50:02)
Yeah, I don't know why I've just got this like massive mental block to like treating the providers as their own, like basically internally treating the providers as if they were their own separate repo inside of the Prism repo. Like there's just this weird mental block that I've been fighting, but like I think ultimately I need to get to a place where...

I am treating each provider as if it was a separate repo, even though it's in the Prism code base, just I treat it on its own. like the pain of the refactor is really having to go through basically like rewrite each provider. And we've got like four or five providers. I think we got like five providers now. So it's not a little bit of work to like.

recontextualize yourself with each provider and then, you know, make everything work. So it's a doozy, but I'm...

The more advanced that prism is getting, the more I keep thinking that that needs to change.

Chris Gmyr (51:04)
Yeah, and I can definitely see that in the future, breaking them out into their own repos and dependency installations. But I don't know, right now, I think it would be still lot easier to manage in the same repo and just keeping the boundaries a little bit better. But you can still do all the complex things and just make the assumption or whatever that.

TJ Miller (51:17)
yeah.

yeah.

Chris Gmyr (51:30)
you know, the shared things that you're doing right now is not going to be able to be done, you know, in the future. And, I don't know, just keep things simple for yourself right now. Keep it in the same repo.

TJ Miller (51:40)
Yeah, and at the end of the day, each provider is unique. Even if they implement the OpenAI contract, they're still unique. They're not going to be keeping up with all the changes that OpenAI is going to be making. And so really, if I just treat each provider as a black box and yeet a request object into it and expect a response object out of it,

just let each provider like figure itself out on its own. I think I'm so convinced that like this refactor needs to happen, but I am. I really don't want to do it. I'm going to have to like round up some folks and see if we can like knock it out in a weekend or something.

Chris Gmyr (52:22)
Yeah, yeah, it's definitely a lot of work for sure.

TJ Miller (52:24)
Yeah. All right, man. think, I think we should probably wrap up from there. Cool. Thank you everybody for listening to the Slightly Caffeinated podcast. Show notes, including all the links about things we talked about and our social channels are at slightlycaffeinated.fm. Thank you all for listening and we'll catch you next week.

Chris Gmyr (52:28)
Yeah, let's wrap it up.