Hey there! I'm a day late with my notes this week, but I promise it's for a good reason. For those of you who celebrated, I hope you had a great holiday!

LLM at Home

Besides Thanksgiving, the only thing I did last week was figure out how to deploy an LLM using an RTX 3060 Ti that I found in a box.

It was a bit of a journey, but I'm pleased to say that I accomplished the feat. I have a Qwen3-4B-Thinking-2507 model deployed now, in addition to Open WebUI and LiteLLM deployments for my friends to chat with it and self-serve API keys.

In lieu of writing up a post-mortem and posting it here, I put together a Leaflet Whiteboard to share what worked, what didn't work, things I learned, and links that I found useful in the process.

You can check it out at my main Leaflet:

Honestly, that's it for this week. Now I'm going to figure out what to do with this poor, anxious thing. Have a great first week of December!