Overcoming Adversity - Anthropic Edition
First off, let me empathize with one of the most valuable companies in existence: Even they probably didn’t see us coming.
Obligatory GenAI Artwork, obviously thieving IP…
Yes, as you all know I’ve been Claude-pilled, whatever that means. And my timing could not have been more… interesting, given recent events. If you’re already up to speed on the recent change to Anthropic’s Subscription changes from today, skip ahead unless you like classic car analogies, custom gaming computers, and possibly doing wild shit with them and language models. Oh, that’s my reader base? Awesome. You’re amongst friends.
For those not in the know, OpenClaw lives locally on a piece of hardware you have at home. You can just have your chatbot of choice teach you how to set it up. Many folks use a Mac Mini or even a weak-ass Raspberry Pi. Personally, I tossed it on a Linux box built on an old gaming rig that somewhat controversially aged out of Windows support last fall. It was laying around with a Linux flavor already on it called Pop OS. Pop-OS works great with Steam, by the way! So, yeah, I have been waiting for Tekken 7 to go on sale so my boys and I could relive the glory days of my youth in the basement on the big tv. No need to go buy a new computer for this little side project, I figured.
Gratuitous Old School Reference!
Since it’s a retired gamer rig, it has a liquid-cooled Nvidia GeForce 2080 Super, a notorious yet powerful 2019 vintage, which in GPU years makes it the equivalent of a late 1980s Pontiac Firebird: Past its prime, leaks fluids, (literally). Noisy… Probably causes cancer. Still reasonably fast and pretty frickin’ cool though!
Now, having an under-sized local model has been fine for everyone, and many folks don’t even bother hosting one, because an OpenClaw server isn’t typically actually asked to do all that much. It’s more of an orchestrator, a front end, or a harness, that you plug in to productivity webapps, maybe host some files, whatever you can come up with that seems interesting, and then for all the heavy lifting it can offload that to Anthropic’s Claude API.
Or it could, until today. The big change today was all of us OpenClaw users was that you now have to attach the API meter to it. Kind of a pay-as-you-go model instead of all-you-can-eat. Uncapped.
The repercussions here are quite scary for power users. Certainly when I use Claude Code, which is subscription-based and has zero risk, I am prone to going ham. Yet when I burned through $1.61 in API calls Saturday morning before my second cup of coffee, well, I got a little stingy.
So, as one does, when faced with adversity, one Adapts. Improvises. Overcomes. Love that meme!
Now, you may know this about me already… I play games with my two young boys, aged 10 and 6 as of today, on big ole beefy computers that I usually build, nay, lovingly craft, myself.
This one is my latest - Finn’s Icepick
They’re always liquid-cooled and they’re state-of-the-art at the time they’re built. So, naturally, I have to fend off the crypto crowd, and now the AI crowd, for good GPUs to put in them.
It turns out, these aren’t too bad for running what I’d call the medium tier of models. Big enough to reason, yet cheap enough they don’t need to run on server clusters. And it so happens that I have three of them heating my office in the winter… and the summer. (A topic for another blog, perhaps…)
So, anyway, I started' blastin’ and by the time I was done, I had built out a 4 node cluster of inferencing awesomeness. Why? Well, because I can. And because, in some weird, obviously convoluted sense, I can be a curmudgeonly cheap guy when it comes to avoiding uncapped cloud spend on my own personal dime.
Is it not adorable?!
What next? Well, over the weekend I built a handy little chatbot thing for Discord for my guild mates. I spent about 90% of the time on the cute emoji-inspired icon. Can you guess which two emojis I mashed up in Gemini?
The chatbot pulls in data for a character from the warcraftlogs.com api, and compares it to some cleaned up data that Claude scraped from wowhead and has hosted in a SQLite database on my brains, and then spits out a pretty and formatted result. It can make recommendations for upgrades to seek and where to find them, and it can give you reasonably decent strategies for a given encounter on the fly upon request. I’m working on extending it to share some custom macros for encounters, and I hope to have it doing other odd jobs with a bit more autonomy form the clustered horsepower I’ve given it. I had spent most of the weekend really struggling because all the v1 abilities were limited by the very modest 8GB VRAM on the 2080, so with a bit more horsepower, I should be able to flex a bit more and reason through higher-order problems.
Honestly, the biggest problem I have right now, is that I’m slamming through the backlog so fast that creativity is the new bottleneck. And that, my friends, is quite the Excession. Good book and I suggest it.