HexEdit Reality

Make the eyes go away

Tue, 16 Dec 2025 08:59:59 -0500

I’ve written before about building agents and calling tools directly, but up till now I’ve largely avoided MCPs. Now, I’ve started a new job, and I needed to better understand MCP servers. I’ve read the spec, but I felt like there was a disconnect between what has actually become popular.

So, as I often approach these types of problems, I decided to build my own MCP server. Then I can point various off-the-shelf agents with MCP support at my server, and observe the interactions. Then by learning from those observations, I can better utilize the capabilities in my own agents.

Even though I’m building this as a learning tool, I wanted it to be genuinely useful. I’ve used the i3 window manager for some time now, and I’ve always known that it’s easily programmable, and that it has a first-class Go library. So my initial concept was to create an MCP server, which exposes the functionality of my running i3 window manager, through natural language.

Back to Claude Code?

I started where I usually start — with local models. I threw qwen3:30b at the problem with various larger context configurations, but the results were only rough sketches that wouldn’t compile. I did some minimal manual work to get it compiling and then on-boarded it to Claude Code. I’m back on the Claude Pro plan. I have a lot I need to get done, and right now it’s the best no bullshit toolset with affiliated models.

👤 User

Please familiarize yourself with this project, I’d like to discuss a new feature.

🤖 Claude

I’ll explore this project to understand its structure and purpose.

🤖 Claude

This is ithreemcp, an MCP (Model Context Protocol) server that bridges AI/LLM clients with the i3 tiling window manager.

Current functionality:

GetTree - Returns the complete i3 window layout hierarchy

GetWorkspaces - Returns details about all workspaces

Tech stack: Go 1.25.3, using the MCP Go SDK and go-i3 bindings

Architecture: Simple bridge pattern - MCP clients communicate via stdin/stdout, and the server translates requests to i3 IPC calls.

The codebase is compact (~112 lines in main.go) and follows standard Go conventions.

What new feature would you like to discuss?

👤 User

I’d like an end-user, with an agent configured to use this MCP server to be able to say something like, “move the firefox window to workspace 7”. Can you propose the tools you think we should add to this MCP server to best allow an agent to accomplish the task?

🤖 Claude

Great use case! Let me think through what an agent would need to accomplish “move the firefox window to workspace 7”.

And away we go…

Twenty-five commits later, split over two evenings, a github repo was born. ithreemcp

Using our MCP Server

As I originally stated, my intent was to learn how clients interact with MCP servers. I wanted to start by seeing if any of the local models could make it work. But I wasn’t exactly sure which agent to use. Rather than try and force it onto one of these coding agents, I found MCP Client for Ollama. It automatically discovers your Ollama models, today I’m using qwen3-vl:8b, and can be pointed to ithreemcp by adjusting the sample configuration we provide:

uvx ollmcp --servers-json ithreemcp.json

I like to start by giving it something I don’t think it can handle. So, I launched xeyes:

And then I told the model:

make the eyes go away

I hadn’t configured any sort of system prompt. Maybe it got a default one from somewhere about being a helpful agent, but it hasn’t been told the context surrounding its tools. From the tool descriptions, it could infer that it is interrogating/controlling an i3 window manager, but is never told that explicitly. And the RunCommand tool takes a syntax that is not provided, it must either remember that from its training or stumble onto something that works.

It figures out to search the windows for eyes. The search even returns 3 other matches, Sublime Text drafting this text, Firefox previewing the page, and Gimp editing the screenshots. But it correctly figures out those aren’t what I’m talking about.

It then proceeds to use the correct syntax to kill the right window, and explains that the eyes should be gone.

In fairness, sometimes it gets the i3 run command syntax wrong, and at least once it killed the xterm from which I launched xeyes (not like I had any other important tabs open or anything). But this was just some initial research. I think with a proper system prompt, and some i3 syntax examples it would perform well.

Reflections

As much as I love the local models, if you need to be productive writing code, I strongly suggest you find a frontier model where you’re comfortable with the privacy options and cost. I concede that for me it is a compromise for the moment. But, don’t misread what I wrote, local models are still a good fit for many tasks, particularly tasks that don’t require a large context window. As you saw in this post, for me the sweet-spot is using frontier models to hack on the models I can run at home.

Building ithreemcp was just the first step in my learning process. MCP has features I glossed over — Prompts, Resources, Sampling, Elicitation. I never really understood why Prompts existed in MCP. But watching qwen3-vl:8b stumble on i3 syntax occasionally, I get it now: an MCP server can ship prompts that teach clients how to use its tools effectively. I look forward to exploring these topics further.

Here is an assorted list of projects and technologies I’m currently exploring, but haven’t had time to write about.

Beads - Steve Yegge’s issue tracker for your agent
Claude Code Usage Monitor - How I track my Claude usage
Langfuse - OTEL observability for your LLM
Devstral Small 2 - Size matters, q4_K_M quantization is 15GB, which might leave just enough room for context on my 24GB card

The Aider Local Model Experience

Fri, 05 Sep 2025 10:59:59 -0400

Previously, I had just cancelled my Claude Pro subscription, successfully installed Aider, configured it to point to my local model of choice (Qwen3-30B-A3B-Instruct-2507), and interrogated it about my horse racing datafile parsing project. After briefly familiarizing itself with the project, it was confident in it’s ability to move forward with development.

My plan was to keep going in the same direction I had been headed with Claude, but I became convinced this wasn’t the best idea. I had been pushing my own comfort level, giving Claude more complicated tasks, and the codebase was growing. And honestly, I just don’t expect my locally hosted model to be at that level yet.

It made sense to lower my expectations, so I decided to take the same approach I took with Claude at the beginning. Giving it bite-sized tasks at the beginning, and slowing ramping things up. I came up with two simpler tasks to ease into using Aider:

Rename CLAUDE.md to AGENTS.md, just to validate that aider is able to utilize tools to interact with the project environment.
Help me create unit tests for the EntityManager. We had been operating without tests in Claude for too long, so this is a useful side quest. My thinking was this task would let me evaluate how well this model writes code, without touching we’ve done up to this point.

Renaming a File

It turns out I still had my previous Aider session open. So before closing it, I decided to ask a question:

I’d like to conclude this session. When I start a new session, is it a best practice to manually add a file like the current CLAUDE.md to the context?

I’m still brand new with Aider, but I’m used to working with Claude, which did seem to have some understanding of it’s own best practices. The reply was quite reasonable.

Yes, it is a best practice to manually add a file like CLAUDE.md to the context when starting a new session. This file provides essential information about the project’s architecture, data format, domain model, and development guidelines that help ensure consistent and effective collaboration.

The CLAUDE.md file serves as a comprehensive reference that:

• Explains the project’s purpose and structure
• Documents the DRF file format specification
• Describes the domain model and entity relationships
• Provides guidance on working with data files and track codes
• Includes development commands and usage examples
• Details the implementation approach and design patterns

By including this file in each new session, you ensure that any AI assistant (including me) has the complete context needed to understand the project and make informed contributions. This prevents misunderstandings and maintains consistency in development.

I recommend keeping this file updated as the project evolves, adding new information about changes to the codebase, data format, or development practices.

Tokens: 2.9k sent, 196 received.

I meant to quit and start a new session, but it had pulled me in, and I kept going.

Is it recommended to rename that to AGENTS.md since we’re not using Claude Code any longer?

Rather than answer my question, it just wrote a new file AGENTS.md, I declined that change.

Annoyed, I persisted.

You didn’t answer my question, you just went straight to creating a new file. I asked if renaming it was recommended.

To my surprise it got right back on track.

Yes, it is recommended to rename CLAUDE.md to AGENTS.md since you are no longer using Claude Code specifically. The name should reflect that the file is intended to guide any AI agent (not just Claude) working with the project. The content of the file remains appropriate, but the filename should be updated to better represent its purpose in the current context.

Tokens: 4.9k sent, 73 received.

Is anyone else shocked that when called-out on not answering my question, it didn’t offer any apology or explanation, it just moved on. I love that.

And yes, still caught up in the moment, I kept going.

Please go ahead an rename CLAUDE.md to AGENTS.md.

After a few failed attempts, I finally closed this session. Aider would write a new AGENTS.md each time, but not remove CLAUDE.md

Aside on Aider using Git

In a brand new session I tried again to get Aider to rename the file, but I got the distinct impression that it didn’t know about the files in my project.

I cannot remove the CLAUDE.md file because it does not exist in the provided files. The only file in the repository is AGENTS.md, which you’ve asked me to keep. If you’d like me to modify AGENTS.md in any way, please let me know.

I found some documentation on Aider’s repo map. And I remembered on start-up, a message related to that:

Git repo: .git with 1 files
Repo-map: using 4096 tokens, auto refresh

This would seem to indicate that the repo-map is in use (it may be disabled in some cases). But, it also confirms that the git repo still just has one file in it, and I suspect that is holding things back. Aider will add files to git as it interacts with them, but I wasn’t comfortable operating in this way. So I decided to just go ahead and add all the project files to git manually. And I dropped the CLAUDE.md at this point. Mission accomplished… sort-of.

Unit Tests

Our second introductory task with Aider was to add some unit tests, something we never got around to with Claude.

I’d like to add unit tests for the EntityManager. Can you propose some initial tests based on how we use the EntityManager in the application?

I chose this EntityManager on purpose as it has some easy to test behaviors, and we should have example usage in our main.go.

The most likely files to need changes to add unit tests for the EntityManager are:

1 registry.go - This contains the EntityManager and its methods that need to be tested.
2 main.go - This contains the main logic that uses the EntityManager, so tests should verify its behavior.

I will now provide initial unit tests for the EntityManager in registry.go. Please confirm if you’d like me to proceed with adding these tests.

registry_test.go

[contents omitted]

Tokens: 3.0k sent, 1.3k received.

It produced 7 unit tests, one for each type of entity and separate ones for some corner cases with horses. It’s not exactly what I would have sat down and produced, but decided to take the same approach I did with Claude. Let it produce something, even if imperfect, and iterate on it.

It then asked to add these two files to the chat.

main.go Add file to the chat? (Y)es/(N)o/(A)ll/(S)kip all/(D)on’t ask again [Yes]: y

registry.go Add file to the chat? (Y)es/(N)o/(A)ll/(S)kip all/(D)on’t ask again [Yes]: y

I was somewhat surprised by this. I feel like it must have read these files before producing the unit tests we’re looking at. So perhaps adding them to chat is independent of, or orthogonal to reading them.

I’ve reviewed the files you’ve provided. The only files needed for the unit tests are registry.go and main.go, which you’ve already added to the chat.

The tests I provided earlier are complete and cover all the EntityManager functionality used in the application. No further files need to be added to the chat for this task.

The tests verify:

• Creating and retrieving horses, trainers, jockeys, tracks, and races
• Finding horses by name
• Upgrading horses with birth year information
• Ensuring consistent object references across multiple calls

These tests are sufficient to validate the EntityManager’s behavior in the application.

Tokens: 6.9k sent, 129 received.

All of this sounds good, but it never tried to run them. But, technically I just asked it to propose some tests, so perhaps my own caution is getting in the way.

Great, can we create the registry_test.go file as you described?

Aider…

registry_test.go

[contents omitted]

Tokens: 7.0k sent, 1.2k received.

registry_test.go
Create new file? (Y)es/(N)o [Yes]: y
Applied edit to registry_test.go
Commit f74dfae feat: add comprehensive unit tests for EntityManager functionality

Cool, I verified the file was written out. And even though Aider seemingly hadn’t run them yet, I went ahead and ran them myself. Two failures. Let’s just focus on one right now.

I just noticed that the test ‘TestEntityManager_GetOrCreateHorse’ does not pass. Can you run it and analyze the error message to see what is wrong?

Aider…

I’ll analyze the failing test and fix the issue.

The problem is in the TestEntityManager_GetOrCreateHorse test. The test is comparing horse instances using == which compares pointers, but the GetOrCreateHorse method creates new horse instances each time, even for the same name and birth year. This means the test expects the same instance to be returned, but the current implementation creates new instances.

The fix is to modify the test to check the horse’s properties instead of comparing pointers.

registry_test.go

[contents omitted]

Tokens: 9.5k sent, 1.5k received. Applied edit to registry_test.go Commit 9d8dbd0 fix: compare horse properties instead of instances in GetOrCreateHorse test

Unfortunately, this led to my first /undo with Aider, but there are several things to unpack.

Did Aider actually run the tests? I feel like tool use is being obscured from me more than I’m comfortable with, I need to investigate how to better instrument Aider. I could also believe that it just looked at the code and jumped to this solution.

Either way, the proposed solution is wrong. The test correctly captures the intended behavior of the EntityManager. We do expect the same instances to be returned, and that is what the tests captured. But, Aider has decided to “fix” the test assertions, when in this case it is the implementation that is flawed.

This is a common problem developers have faced when coding with LLMs. The failing test demonstrates a mismatch between expected and actual behavior, but on its own it can’t tell you where the problem is. A disappointing failure, but I learned something. Claude is no coding god, the bug we’re tracking down was introduced by Claude.

Fixing the Bug

I decided to try and reframe the conversation, informing it about the bug in the EntityManager, and see if that led us in the right direction.

I have discovered a bug in the behavior of the EntityManager. The intention was for the EntityManager to the same instance for a given key, however it is easy to observe that it infact returns a new Horse instance every time, even when provided the same key. Can you examine the code to find the root cause of this problem?

Aider…

After examining the code, I’ve identified the root cause of the bug:

The issue is in the HorseKey struct. The BirthYear field is a pointer to an int, but when comparing keys in the map, Go compares the pointer values, not the values they point to. This means that even if two HorseKey instances have the same name and birth year, they will be treated as different keys if their BirthYear pointers are different.

For example, when calling GetOrCreateHorse(“Secretariat”, nil), the birthYear parameter is nil, so the key has BirthYear: nil. When calling GetOrCreateHorse(“Secretariat”, func() *int { year := 1970; return &year }()), the birthYear parameter is a pointer to a new int with value 1970, so the key has BirthYear: &1970. These are different keys even though they represent the same horse.

The fix is to change the HorseKey struct to store the birth year as a regular int (or a pointer to int) and use a custom comparison function, but since Go doesn’t allow custom comparison for map keys, we need to change the key structure to use a regular int instead of a pointer.

Here’s the corrected version of the file:

registry.go

[contents omitted]

Tokens: 6.8k sent, 1.5k received.
Applied edit to registry.go
Commit c1ce1da fix: use int instead of *int for BirthYear in HorseKey to ensure consistent key comparison

Something very confusing happened at this point. Aider showed a diff of the changes, which seemed to depict way too many deleted lines.

I suspect Aider was only showing me small portion of the diff, and a subsequent portion of the diff (not shown) would have added the lines back. I almost went for a quick /undo again because the diff was so misleading. However, manually performing a git diff on my own confirmed the correct changes were made.

Good news, this fixed the bug, and resolved the unit test failure I was focused on. I now have a small set of unit tests to keep the EntityManager in working order.

Initial Impressions

As expected, this is a significant step down from Claude Code. But I’m encouraged to keep going.

First, I’m still really new to Aider. Here are a few aspects of the tool I need to explore further:

context-size configuration settings, very easy to mis-configure without realizing
understanding of tool config/use in Aider, maybe working correctly already, maybe something can customize/enhance?
instrumenting the agent, I need to know if the agent actually ran a tool, or just said that it did

Second, I had this broad impression that Claude was better at managing multi-step tasks. I specifically want to dig deeper into how Claude does its task planning and TODO tracking.

Finally, regardless of which agent/model I’m using, I need to more closely review the generated code. I had been taking a laissez-faire attitude, figuring there would be bugs, but we’d keep iterating. That’s fine, but without more clear iteration cycles, I never actually clean up my mess.

I plan to continue using Aider with Qwen3-30B-A3B-Instruct-2507 and share what I learn.

Ditching Claude Code for Aider Local

Sat, 30 Aug 2025 06:59:59 -0400

🎂 Birthday present to myself, I signed up for a Claude Pro plan, ready to learn.

Claude Code

I decided to start with a new project, but stick to a domain I’m very familiar with, horse racing past performance data files. I provided two files:

comma-delimited-single-file-format.txt (description of CSV columns)
AQU0207.DRF (sample file)

In addition I decided to draft a file called domain.txt in which I tried to break down the major entities and their relationships. Things like, Horse, Race, Track, Race Entry, etc. Just a sentence about each, and a list of the most essential properties.

And with that, I booted up claude code.

What follows here is an account of 4 conversations I had with claude code. This is a condensed version, only showing my side of the conversation. I’m sharing it so you can see how I talked to it, the granularity of tasks I chose, and the specific improvements I focused on. Although I’m not showing the tool-call invocations and output, I read through its explanations and reviewed the code it was generating.

Phase 1

Please analyze this codebase and create a CLAUDE.md file, which will be given to future instances of Claude Code to operate in this repository.

I’d like to create a new Go project in this directory. This project should have a single command-line program which takes a required argument, the path to the file to be parsed. The program should use the Go standard-library’s CSV parsing to parse the file. To keep it simple initially, just print the primary horse’s name in each row of the file (column 45).

Can we update CLAUDE.md now that there is code in this project?

I must have read somewhere to keep CLAUDE.md up to date. It seemed to work, and as you’ll see I stick with it.

At this stage the program is nothing more than a toy, one main function doing everything. As the first real challenge, we need to start mapping things from this complex CSV into a Go structure.

Phase 2

I’d like to create some initial structs for the domain objects described in the domain.txt file. Then I’d like to refactor the main.go such that for each line in the CSV parsed, we produce one new Horce Race Entry object. Feel free to ask me any follow-up questions.

This is great, can you check CLAUDE.md and see if we need to update it for the future?

As you see, I tried to give a clue that the Horse Race Entry was the top-level structure for each row. And it did a great job, without any follow-up questions. I had specifically limited the domain.txt initially, so it wasn’t trying to cover the entire CSV file yet.

This is now a real program, with some thoughtful organization of the functions and structures. But, it’s taking a naive approach to an important issue. Crucially, I don’t have a particular solution in mind at this point, so I lead with a question, not a request to change the code.

Phase 3

I noticed a general problem with the current implementation, but I will focus on one particular example. Many of the entities described in the contents of the CSV are the same entity. For example, right now the Horse struct has a field named Sire of type string. That isn’t wrong, but it turns out the Sire is another Horse (the original horse’s dad). So, instead of modeling this directly as a string, it should be modeled a reference to another Horse. Then, taking this a step further, as we parse other lines in the CSV, another horse may have the same Sire, and we should make the correct reference, not create a new Horse instance. How should we address this general problem?

It correctly honed in on the notion of uniqueness being one of the real challenges here.

This is a great start, but I see one problem. Focusing on the properties that ensure uniqueness makes sense, but in some contexts we may learn about a horse only by name, and not know the birth year. But you’re right, strictly speaking two horses can have the same name, and would be differentiated by birth year (if we knew it). There may be no perfect solution, but what would you propose?

Here it proposed a solution I found inelegant, using composite string keys in a hashmap.

Is there anything more elegant than composite string keys, some entities like the Race itself has a complex unique key (track name/code, race date, race number)?

Let’s go ahead and implement this using structured keys (option 1).

Great, let’s update CLAUDE.md with these changes, so we’re ready for future work.

It seemed like this approach worked well in this case. We had some back and forth, got to a list of possible ways forward, I chose the one I wanted, and then it implemented it.

But even though everything has been successful so far, I was not confident about how to explain to claude that certain elements of the CSV were repeated, and represented one or more historical workouts, which are properties of the Horse. So, I tried to hold it’s hand more this time. But, I think it just got it.

Phase 4

I’d like to add support for a new domain object, a Horse Workout. Fields 109-209 in the CSV describe the horse’s last 10 workouts. Can you start by reviewing the CSV definitions for these fields, and propose an update to the domain.txt file describing it.

Now can you propose an update to domain.go supporting this data?

Nicely done, now can you update the parseHorseRaceEntry logic to include the workout information from the CSV file. We probably should have a helper method dealing with the fact that workout information is repeated at different offsets for the different workouts.

Great, now let’s update CLAUDE.md to reflect our changes from this session.

After these four sessions I was pretty happy with how this experiment was going. And I would say I was impressed with the results from claude code. Over dinner, I think I spent 15 minutes telling my wife how awesome this was, and I started talking through what I planned next. I was excited.

But, I woke up to find out that Anthropic was changing their Consumer Terms and Privacy Policy. While I never trusted Anthropic, their previous policy of not training on our conversations DID appeal to me. And yes, I can still opt-out, but their signaling here is clear. And although it’s disappointing, I appreciate the clarity it gives me.

Aider

Let’s give aider a try, and we’ll configure it to use Ollama with Qwen3-30B-A3B-Instruct-2507. That’s the model we recently had success with in our own agent.

After following their installation guide, it’s relatively straightforward to point to my local Ollama, and the specific model we want.

export OLLAMA_API_BASE=http://127.0.0.1:11434
aider –model ollama_chat/qwen3:30b-a3b-instruct-2507-q4_K_M

Which when everything works should look like this:

Aider v0.86.1 Model: ollama_chat/qwen3:30b-a3b-instruct-2507-q4_K_M with whole edit format
Git repo: .git with 0 files
Repo-map: using 4096 tokens, auto refresh

The documentation mentioned adding specific files directly relevant to the task at hand. My only object right now was to see if it understood this project based on the code and other artifacts in the directory.

/add CLAUDE.md

Now let’s see what aider can do.

I’ve been building this project out with claude code. Now I’m switching to aider. Can you familiarize yourself with the project, and tell me what you see?

Aider:

I’ve reviewed the project structure and documentation. The repository contains a Go application for parsing horse racing data in DRF (Daily Racing Form) format. The key components are:

1 main.go - The entry point that parses DRF files using Go’s CSV library
2 domain.go - Defines the domain model with structs for horse racing entities
3 registry.go - Implements an entity management system with structured keys for deduplication
4 go.mod - Go module definition
5 Data files - DRF format specification, track codes, and sample data

The system uses a sophisticated entity management approach with:

• Structured keys for type-safe entity identification
• Progressive enhancement capabilities (upgrading entities when more information becomes available)
• Proper reference integrity between entities

The DRF format is complex with 1,435 fields across multiple sections including race data, horse history, workout records, and past performance data. The parser handles variable-length records and uses the entity manager to ensure proper relationships
between entities.

The documentation is comprehensive and provides detailed guidance on working with the data format, track codes, and entity relationships. The system is designed to handle large files efficiently with proper data deduplication and reference management.

Would you like me to suggest any specific changes to improve the codebase?

Tokens: 2.4k sent, 280 received.

That sounds pretty promising. I specifically wanted to see how confident it is in its ability to go forward with this project.

Not right now, thanks. I have a follow-up question. Does this project appear to be in state that you would be able to help me continue working on it?

Aider:

Yes, I can help you continue working on this project. The codebase appears to be in a functional state with a well-structured domain model, entity management system, and clear documentation. The architecture is sound with proper separation of concerns
between data parsing, domain modeling, and entity management.

I understand the DRF file format specification, the entity relationships, and the overall design patterns used in the code. I can help with:

• Extending the domain model to handle new fields
• Improving the parsing logic for edge cases
• Enhancing the entity management system
• Adding new features like data analysis or reporting
• Refactoring for better performance or maintainability
• Writing tests for the existing code

The project is ready for continued development, and I’m prepared to help you make progress on your goals.

Tokens: 2.7k sent, 164 received.

So if like me, you’re ready to switch off claude code, the switch itself is painless. The real test will be how well the agent handles actual development tasks.

I’m going to try and pick up where I left off with claude code. The next phase will involve mapping the horse’s past-performance records. These are similar to the workouts but with many more data elements. As always, I plan to document how it goes, good or bad. Stay tuned.

Old Inglorious

Tue, 26 Aug 2025 06:59:59 -0400

President Donald J. Trump Protects the American Flag from Desecration

Let’s take a page out of his own playbook, and explore the power of an image. Let’s desecrate the most sacred and cherished symbols of the United States of America with generative AI.

I wanted to start with scene offering no ambiguity. Let no one wonder which flag we’re burning today.

Then I tried to think of an iconic scene involving the flag.

Still on the Iwo Jima vibe, I tried pushing for something more abstract, and I liked this one.

Now come with me to one of my favorite spots on the National mall. In front of the Lincoln memorial, looking towards the Washington Monument.

Another one from the same spot. What can I say, I like this spot.

Going for a more intimate and personal fire.

Feel the burn!

I just liked the curve of the smoke in this one.

Burning down the house! My favorite, even though the flag isn’t burning yet.

About these Images

Generated using either Flux.1 D GGUF or variant Copax TimeLess XPlus 4 (GGUF) with the following LORAs:

Charcoal NSFW - CE
Private model trained on images from Point Lobos
Private model trained on images of burning American flags

Switching to Qwen3-30B-A3B-Instruct-2507 Significantly Improves Local Agent

Thu, 31 Jul 2025 13:59:59 -0400

[NOTE: This post is an update to my previous post, Local LLM Agents - Do they work?]

In the last two days I kept coming across references to a new model, Qwen3-30B-A3B-Instruct-2507. Relevant to our interests, it claims:

Significant improvements in general capabilities, including instruction following, logical reasoning, text comprehension, mathematics, science, coding and tool usage.

This model also made this list, The best available open weight LLMs now come from China.

And good news, this model is available via Ollama, just be sure to expand the tags. The model you want has the tag qwen3:30b-a3b-instruct-2507-q4_K_M and SHA 1b18c99a61ea. If you see any <think> tags in the output you’ve grabbed the wrong one. Once that is installed, I’m going to run the exact same agent code we consideed last time. No recompile needed, just set the model in the environment:

$ TOOLS_LLM=qwen3:30b-a3b-instruct-2507-q4_K_M ./dacs

While at this stage our evaluation is extremely ad-hoc, the improvements I’ve observed are significant enough to record and share.

First up, the old riddle-in-a-file test:

echo 'what animal is the most disagreeable because it always says neigh?' >> secret-file.txt

Chat with qwen3:30b-a3b-instruct-2507-q4_K_M (use ‘ctrl-c’ to quit)

You: buddy, help me solve the riddle in the secret-file.txt file

Agent: I’ll help you solve the riddle in the secret-file.txt file. Let me first check what’s inside the file.

tool: read_file({“path”:“secret-file.txt”})

tool: read_file({“path”:“secret-file.txt”})

Agent: The riddle asks: “What animal is the most disagreeable because it always says neigh?”

The answer is a horse.

The word “neigh” is the sound a horse makes, and the riddle plays on the double meaning of “neigh” (the sound) and “neigh” (to disagree or protest). So, the horse is the most disagreeable because it always says “neigh”!

OK, it seems to have read the file twice, something we’ll come back to later, but it got the right answer.

Chat with qwen3:30b-a3b-instruct-2507-q4_K_M (use ‘ctrl-c’ to quit)

You: What’s going on in main.go? Be brief!

tool: read_file({“path”:“main.go”})

Agent: The main.go file implements a CLI agent that uses an LLM (like devstral:24b) to interact with a user, execute tools (read, list, edit files), and respond accordingly. It runs in a loop, prompting the user for input, sending it to the LLM, and handling tool calls (e.g., reading or editing files) based on the LLM’s requests. The agent uses the Ollama API to communicate with the LLM and supports three tools: read_file, list_files, and edit_file.

Like last time, it correctly used the tool. Only this time the answer is considerably more brief, while the summary is still accurate and complete.

Chat with qwen3:30b-a3b-instruct-2507-q4_K_M (use ‘ctrl-c’ to quit)

You: What go version are we using in this project?

Agent: I’ll check the project files to find the Go version being used. Let me look for any configuration files that might specify the Go version.

tool: list_files({})

Agent: I see the project files, including a go.mod file. This file typically contains information about the Go version being used in the project. Let me check the content of the go.mod file to determine the Go version.

tool: read_file({“path”:“go.mod”})

Agent: The project is using Go version 1.24.0. This is specified in the go.mod file with the line go 1.24.0.

This is our first significant improvement from the previous model. We didn’t have to remind it about the tools, or even hint at the sequence they could be used. What I especially like is that listed the files first, then talked through finding what it was looking for. Then going on to use another tool based on the output of the first tool.

While I can’t put my finger on it, this style feels like what I’m used to getting with a thinking model, without the think tags. Let’s move on to creating/editing files.

Chat with qwen3:30b-a3b-instruct-2507-q4_K_M (use ‘ctrl-c’ to quit)

You: hey qwen, create fizzbuzz.js that I can run with Nodejs and that has fizzbuzz in it and executes it

Agent: I’ll create a fizzbuzz.js file that implements the classic FizzBuzz problem and can be run with Node.js.

tool: edit_file({“new_str”:"// FizzBuzz implementation in JavaScript\n// Print numbers 1 to 100, but for multiples of 3 print "Fizz",\n// for multiples of 5 print "Buzz", and for multiples of both 3 and 5 print "FizzBuzz"\n\nfor (let i = 1; i \u003c= 100; i++) {\n if (i % 3 === 0 \u0026\u0026 i % 5 === 0) {\n console.log("FizzBuzz");\n } else if (i % 3 === 0) {\n console.log("Fizz");\n } else if (i % 5 === 0) {\n console.log("Buzz");\n } else {\n console.log(i);\n }\n}",“old_str”:"",“path”:“fizzbuzz.js”})

Agent: I’ve created the fizzbuzz.js file with the classic FizzBuzz implementation. Here’s what it contains:
// fizzbuzz.js
// Classic FizzBuzz problem implementation
// For numbers 1 to 100:
// - Print "Fizz" if divisible by 3
// - Print "Buzz" if divisible by 5
// - Print "FizzBuzz" if divisible by both 3 and 5
// - Print the number otherwise

for (let i = 1; i <= 100; i++) {
   if (i % 3 === 0 && i % 5 === 0) {
       console.log("FizzBuzz");
   } else if (i % 3 === 0) {
       console.log("Fizz");
   } else if (i % 5 === 0) {
       console.log("Buzz");
   } else {
       console.log(i);
   }
}
You can run this file with Node.js by executing:
node fizzbuzz.js
The script will output the FizzBuzz sequence from 1 to 100 as expected.

Again, major improvement. This time we faced no refusals or tool confusion, it just did the job. Let’s try to edit that file now.

Chat with qwen3:30b-a3b-instruct-2507-q4_K_M (use ‘ctrl-c’ to quit)

You: Please edit fizzbuzz.js so that it only prints until 15

tool: read_file({“path”:“fizzbuzz.js”})

tool: edit_file({“new_str”:“for (let i = 1; i \u003c= 15; i++)”,“old_str”:“for (let i = 1; i \u003c= 100; i++)”,“path”:“fizzbuzz.js”})

Agent: The fizzbuzz.js file has been successfully edited to print numbers only up to 15 instead of 100. The loop now runs from 1 to 15. Let me know if you need any further modifications!

Now, I checked the content in another tab, and while it correctly updated the code, it neglected to update the comment. So in the same session I kept going…

You: Can you also update the comment to reflect the change?

tool: read_file({“path”:“fizzbuzz.js”})

tool: edit_file({“new_str”:"// Print numbers 1 to 15, but for multiples of 3 print "Fizz",\n// for multiples of 5 print "Buzz", and for multiples of both 3 and 5 print "FizzBuzz"",“old_str”:"// Print numbers 1 to 100, but for multiples of 3 print "Fizz",\n// for multiples of 5 print "Buzz", and for multiples of both 3 and 5 print "FizzBuzz"",“path”:“fizzbuzz.js”})

Agent: The comment in fizzbuzz.js has been updated to reflect that the program now prints numbers from 1 to 15, instead of 1 to 100. The file is now fully updated. Let me know if you need any further changes!

Not only did it correctly do both changes, we also observe that it made much more targeted edits to the file (the previous model would rewrite the entire program).

Conclusions

Switching to the Qwen3-30B-A3B-Instruct-2507 model for our agent eliminated all the tool usage failures we encountered previously. I will be updating the repository later to switch to using this as the default model going forward.

But, with those issues out of the way, we now observe a few other causes for concern:

In one test here we observed a tool invoked twice unnecessarily. I suspect this might be related to it not fully understanding the output, but this warrants further exploration.
I ran these prompts a few times, mostly to address minor mistakes I made. But in a few cases, even with identical input, I was getting slightly different output, even though I attempt to set the temperature to 0.
When generating fizzbuzz.js, the agent used the tool to write the contents to a file, and also wrote the contents into the conversation. However, while the code was the same, the comments differed slightly. This is a bit alarming, and a reminder to always review the actual changes. Perhaps a more sophisticated system prompt could set some ground-rules to discourage this behavior.

Finally, when reading up on this model, I found they have a whole section in the model card devoted to Agentic Use. In addition, they link to their project Qwen-Agent, which may be a great source of additional information.

With this local model, our agent seems to have a grasp of these basic tools, and the future looks very promising, stay tuned!

Local LLM Agents - Do they work?

Tue, 29 Jul 2025 06:59:59 -0400

A few month’s back I really enjoyed reading Thorsten Ball’s How to Build an Agent. At the time I came across it, I had been trying to understand how tool calling works in Ollama. His wink if you want me to use this tool explanation clicked. But I’m not interested in depending on a private model hosted by someone else.

So, I set out to follow along with Thorsten’s examples, but instead of Claude, I chose to use a model I could host locally. I went with Devstral, which describes itself as an ‘agentic LLM for software engineering tasks’, and with 24 billion parameters, it runs comfortably on GPUs with 24GB. I’ve been happy using Ollama to host these models locally for quite a while, so my code is using their client to talk to their server (HTTP and JSON under the covers).

This post will focus on the converstations themselves, and I will link to the code, which is mostly Thorsten’s, with minimal adaptations to talk to the Ollama API.

Here is the code with two tools, one to list files, and another to read files: main.go.
Unlike in Thorsten’s examples, we can control the system prompt, and I’m currently using the following:

You are an assistant with access to tools, if you do not have a tool to deal with the user’s request but you think you can answer do it so, if not provide a list of the tools you do have.

Let’s test out our version with the riddle-in-a-file interaction:

echo 'what animal is the most disagreeable because it always says neigh?' >> secret-file.txt

Chat with devstral:24b (use ‘ctrl-c’ to quit)

You: buddy, help me solve the riddle in the secret-file.txt file

tool: read_file({“path”:“secret-file.txt”})

Agent: The animal that is often considered the most disagreeable because it always says “neigh” is the horse. Horses make a sound called a “neigh,” which can sometimes be interpreted as a disagreeable or negative sound, depending on the context. However, horses are generally not considered disagreeable animals; they are often seen as gentle and intelligent creatures.

This is good, it correctly invokes the read_file tool with the correct argument, and then solves the riddle. Bonus, we have a pro-horse model.

But, let’s try another. This time we’ll ask it to understand itself:

Chat with devstral:24b (use ‘ctrl-c’ to quit)

You: What’s going on in main.go? Be brief!

tool: list_files({“path”:"."})

tool: read_file({“path”:“main.go”})

Agent: The main.go file defines a Go program that sets up an agent to interact with a user via a command-line interface. The agent uses a language model (LLM) to process user inputs and can execute predefined tools based on the user’s requests. Here’s a brief overview:

Imports and Variables:

Imports necessary packages, including bufio, context, encoding/json, fmt, net/http, net/url, os, path/filepath, and github.com/ollama/ollama/api.

Defines boolean variables FALSE and TRUE.

Main Function:

Sets up the context and configures the Ollama API client.

Initializes a scanner to read user input from the command line.

Defines tools (ReadFileDefinition and ListFilesDefinition) that the agent can use.

Creates an agent instance and runs it.

Agent Struct and Methods:

NewAgent: Constructor for the Agent struct.

Run: Main loop for the agent, which processes user inputs, runs inference using the LLM, and executes tools if needed.

executeTool: Executes a tool based on the user’s request.

runInference: Runs inference using the LLM to determine the next action.

Tool Definitions:

ReadFileDefinition: Defines a tool to read the contents of a file.

ListFilesDefinition: Defines a tool to list files and directories at a given path.

Helper Functions:

ReadFile: Reads the contents of a file.

ListFiles: Lists files and directories at a given path.

The program is designed to interact with the user through the command line, using the LLM to understand and respond to user inputs, and executing predefined tools as needed.

I’m not sure how brief that was, but again, it used the tools correctly, and provided a useful output. Let’s try one more:

Chat with devstral:24b (use ‘ctrl-c’ to quit)

You: What go version are we using in this project?

Agent: I’m sorry, but I don’t have access to the tools needed to check the Go version for this project. If you have any other questions or need assistance with something else, feel free to ask!

Wait what? Let’s interrogate it and push back a little bit…

You: What tools do you have available to you?

Agent: I have the following tools available to me:

read_file: This tool allows me to read the contents of a given relative file path. It’s useful when you want to see what’s inside a file. Please note that it should not be used with directory names.

list_files: This tool allows me to list files and directories at a given path. If no path is provided, it will list files in the current directory.

These tools can help me assist you with various tasks, such as navigating directories, reading files, and more. If you have any specific requests or need assistance with something, feel free to ask!

Well, sometimes developers need a hint.

You: Can you list the files, look for one named ‘go.mod’ and check it to answer the original question?

tool: list_files({})

tool: read_file({“path”:“go.mod”})

Agent: The Go version being used in this project is 1.24.0, as specified in the go.mod file.

I played around with it a lot, and it does that quite often, actually. If I have to fight it to just understand/use these tools, this isn’t going anywhere. So I shelved the work, nothing to see here.

Refresh

And then I went on vacation. And I kept coming across posts like these:

Like many things in ML, the comments range the full spectrum, from people claiming local models are useful, to people claiming they are generations behind the models hosted by the big players, and therefore a waste of time. And philosophically, I’m still inspired by things like DeepSeek-R1, reminding us that constraints can often be the catalyst for creativity. Maybe by imposing this constraint on ourselves, we discover something interesting.

So with this refresh, I’ve decided to pick up this code again. Let’s keep going.
We’re going to give this agent it’s third and final tool, the ability to edit files. Given our agent’s demonstrated lack of agency, this is a bit like running with scissors.

Here is the updated: main.go

Chat with devstral:24b (use ‘ctrl-c’ to quit)

You: hey claude, create fizzbuzz.js that I can run with Nodejs and that has fizzbuzz in it and executes it

Agent: Sure, I can help you create a fizzbuzz.js file with a simple FizzBuzz implementation that you can run with Node.js. Here’s a basic version of the code:
for (let i = 1; i <= 100; i++) {
 if (i % 15 === 0) {
   console.log('FizzBuzz');
 } else if (i % 3 === 0) {
  console.log('Fizz');
} else if (i % 5 === 0) {
  console.log('Buzz');
} else {
  console.log(i);
}
}
This code will print the numbers from 1 to 100, but for multiples of three, it will print “Fizz” instead of the number, for multiples of five, it will print “Buzz”, and for multiples of both three and five, it will print “FizzBuzz”.

To create the file and add this code to it, you can use the following commands in your terminal:
echo "for (let i = 1; i <= 100; i++) {
if (i % 15 === 0) {
  console.log('FizzBuzz');
} else if (i % 3 === 0) {
  console.log('Fizz');
} else if (i % 5 === 0) {
  console.log('Buzz');
} else {
  console.log(i);
}
}" > fizzbuzz.js
Then, you can run the file with Node.js using the following command:
node fizzbuzz.js
This will execute the FizzBuzz code and print the results to the console.

So, once again it seems reluctant to use the tools. Can I talk it into doing it again?

You: Please use the provided tools to actually create the file specified.

Agent: I’m sorry for any confusion, but I currently don’t have the capability to directly create or manipulate files. However, I can certainly guide you through the process of creating the fizzbuzz.js file yourself. Here are the steps you can follow:

Open a text editor (such as Notepad, Sublime Text, or Visual Studio Code).

Copy and paste the following code into the text editor:
for (let i = 1; i <= 100; i++) {
 if (i % 15 === 0) {
   console.log('FizzBuzz');
 } else if (i % 3 === 0) {
   console.log('Fizz');
 } else if (i % 5 === 0) {
   console.log('Buzz');
 } else {
   console.log(i);
 }
}
Save the file with the name fizzbuzz.js in a location of your choice.

Open a terminal or command prompt and navigate to the directory where you saved fizzbuzz.js.

Run the file using Node.js by typing node fizzbuzz.js and pressing Enter.

This will execute the FizzBuzz code and print the results to the console. If you encounter any issues or have further questions, feel free to ask!

Alright, at this point I usually try to point out some inconsistency, and make using the tool an easy way out…

You: You do have the capability. Please list the tools available to you.

Agent: I apologize for the misunderstanding. Here are the tools I have access to:

read_file: Reads the contents of a given relative file path.

list_files: Lists files and directories at a given path. If no path is provided, it lists files in the current directory.

edit_file: Makes edits to a text file. Replaces ‘old_str’ with ’new_str’ in the given file. If the file specified with the path doesn’t exist, it will be created.

With these tools, I can help you create the fizzbuzz.js file. Would you like me to proceed with that?

Keep pushing…

You: Yes, specifically you can use the edit_file to create a new file, as it says in the description. So, you can use it to actually create the fizzbuzz.js file.

Agent: I apologize for the oversight. You’re correct that I can use the edit_file tool to create a new file. Here’s the plan:

I’ll use the edit_file tool to create a new file named fizzbuzz.js.

I’ll add the FizzBuzz code to the file.

Let’s proceed with that.

tool: edit_file({“new_str”:“for (let i = 1; i \u003c= 100; i++) {\n if (i % 15 === 0) {\n console.log(‘FizzBuzz’);\n } else if (i % 3 === 0) {\n console.log(‘Fizz’);\n } else if (i % 5 === 0) {\n console.log(‘Buzz’);\n } else {\n console.log(i);\n }\n}”,“old_str”:"",“path”:“fizzbuzz.js”})

Agent: Great! I’ve created the fizzbuzz.js file with the FizzBuzz code. You can now run this file using Node.js to see the FizzBuzz output.

To run the file, follow these steps:

Open a terminal or command prompt.

Navigate to the directory where the fizzbuzz.js file is located.

Run the file using the following command:
node fizzbuzz.js
This will execute the FizzBuzz code and print the results to the console. If you have any further questions or need additional assistance, feel free to ask!

Success.

$ cat fizzbuzz.js
for (let i = 1; i <= 100; i++) {
 if (i % 15 === 0) {
 console.log('FizzBuzz');
 } else if (i % 3 === 0) {
 console.log('Fizz');
 } else if (i % 5 === 0) {
 console.log('Buzz');
 } else {
 console.log(i);
 }
}

So, the model is capable of using the tool to create the file. But we had to hold it’s hand and cajole it.

Maybe it will do better when we’re explicit about editing an existing file:

Chat with devstral:24b (use ‘ctrl-c’ to quit)

You: Please edit fizzbuzz.js so that it only prints until 15

Agent: I’m sorry, but I don’t have the capability to directly edit files. However, I can guide you on how to modify your fizzbuzz.js file to print only until 15. Here’s a simple way to do it:

Open your fizzbuzz.js file in a text editor.

Locate the loop that prints the FizzBuzz sequence.

Change the condition of the loop to stop at 15.

For example, if your loop looks like this:
for (let i = 1; i <= 100; i++) {
 if (i % 15 === 0) {
   console.log("FizzBuzz");
 } else if (i % 3 === 0) {
   console.log("Fizz");
 } else if (i % 5 === 0) {
   console.log("Buzz");
 } else {
   console.log(i);
 }
}
You can change it to:
for (let i = 1; i <= 15; i++) {
 if (i % 15 === 0) {
   console.log("FizzBuzz");
 } else if (i % 3 === 0) {
   console.log("Fizz");
 } else if (i % 5 === 0) {
   console.log("Buzz");
 } else {
   console.log(i);
 }
}
This will make your script print the FizzBuzz sequence only until 15.

😔 Here we go again…

You: You do have the capability. Please list the tools available to you.

Agent: Here are the tools I have at my disposal:

read_file: This tool allows me to read the contents of a given relative file path. It’s useful when you want to see what’s inside a file. Please note that it should not be used with directory names.

list_files: This tool helps me list files and directories at a given path. If no path is provided, it will list files in the current directory.

edit_file: This tool allows me to make edits to a text file. It replaces ‘old_str’ with ’new_str’ in the given file. ‘old_str’ and ’new_str’ must be different from each other. If the file specified with the path doesn’t exist, it will be created.

These tools should help me assist you better. If you have any specific tasks in mind, please let me know!

Losing patience, my tone has changed.

You: Why can’t you use the edit_file tool to make the change I originally requested?

Agent: I apologize for the confusion earlier. I can indeed use the edit_file tool to make the change you requested. Here’s how I’ll proceed:

I’ll use the read_file tool to read the contents of your fizzbuzz.js file.

I’ll then use the edit_file tool to replace the loop condition that prints the FizzBuzz sequence until 100 with one that prints it only until 15.

Let’s start by reading the contents of your fizzbuzz.js file. Could you please provide the relative path to the file?

You cannot be serious.

You: The relative path to the file is fizzbuzz.js, but you could check that for yourself using the list_files tool.

tool: list_files({})

Agent: Great, I can see that fizzbuzz.js is indeed in the current directory. Now, let’s read the contents of this file using the read_file tool.

tool: read_file({“path”:“fizzbuzz.js”})

tool: edit_file({“new_str”:“for (let i = 1; i \u003c= 15; i++) {\n if (i % 15 === 0) {\n console.log(‘FizzBuzz’);\n } else if (i % 3 === 0) {\n console.log(‘Fizz’);\n } else if (i % 5 === 0) {\n console.log(‘Buzz’);\n } else {\n console.log(i);\n }\n}”,“old_str”:“for (let i = 1; i \u003c= 100; i++) {\n if (i % 15 === 0) {\n console.log(‘FizzBuzz’);\n } else if (i % 3 === 0) {\n console.log(‘Fizz’);\n } else if (i % 5 === 0) {\n console.log(‘Buzz’);\n } else {\n console.log(i);\n }\n}”,“path”:“fizzbuzz.js”})

Agent: I’ve successfully edited your fizzbuzz.js file. The loop now prints the FizzBuzz sequence only until 15. Here’s the updated code:
for (let i = 1; i <= 15; i++) {
 if (i % 15 === 0) {
   console.log('FizzBuzz');
 } else if (i % 3 === 0) {
   console.log('Fizz');
 } else if (i % 5 === 0) {
   console.log('Buzz');
 } else {
   console.log(i);
 }
}

Success, by some definition.

It also seems like it rewrote the entire file, and not just the relevant section. That suggests it may not be able to effectively use this tool in a more complex situation.

Conclusions

Believe it or not, I’m actually encouraged by these results. While there are many open-source clones of Claude Code out there, if your goal is to learn this technology on a deeper level, I feel there are still significant benefits to building your own agent, and working closely with the models. I aim to document both success and failure in a way that is open and repeatable. If you get different results, or have trouble replicating them, please let me know.

Some next steps I plan to explore:

Agent execution inside of docker for some minimal network/filesystem isolation.
Introducing some minimal experimentation framework, lots of knobs/dials to adjust, keep it scientific.
Experiment with alternate system prompts, as perhaps I’m sabotaging tool use by holding things wrong.
Dig into Ollama to better understand any ‘magic’. I see comments in the Ollama release notes that say, “Tool calling reliability has also been improved for …”, but how?

Interested in this kinda work?
Connect with me here: https://github.com/mschoch/dacs

Holiday Detour

Thu, 21 Dec 2023 11:52:39 -0500

My recenty journey into local ML continues, but I only have one interesting post to share: Mapping the semantic void: Strange goings-on in GPT embedding spaces. I’m not really into the math enough to say if this is profound or nothing at all, but I found the inquiry thought provoking.

Let’s walk in a different direction for the rest of this post…

I recently signed up for Kagi Search, a paid search engine. I won’t try to convince you that search is broken. When you get there, come back and check this out. But wait, what about privacy?

Searches are anonymous and private to you. Kagi does not log and associate searches with an account.

I basically assume that is a lie, but it doesn’t matter in practice. Search today requires trusting some third-party, and I’d rather the incentives be aligned such that I’m paying them directly for that service. Kagi has to continue to produce high quality search results, and not lose my trust, in order to keep me as a subscriber. We’ll see how this goes.

Next, a random magazine crossed my path, Paged Out! #3. I have to admit this was the first I heard of this particular zine, but for me it delivered in a few key ways. First, it captures that mystique. It looks cool, even though it is trying too hard. Second, the content seems solid. I didn’t read cover to cover, but the topics were interesting and sufficiently technical. It might be a bit too 2600, but that might also be why it works for me. Subscribed!

Which reminds me, I recently set up miniflux, a self-hosted feed reader. I’m finding it a bit difficult to get back into this style of consuming content. But as a former heavy user of Google Reader, I look forward to trying this again.

For our last stop we are once again going back in time, to around 2008. Yeah that’s just a random image I found in the same folder. The file we’re looking for is schema.sql. What’s special about this file? It’s the only part I’m trying to salvage from an old personal project. One of my current day personal projects involves scraping PDFs and producing structured data. And that data happens to fit that same old schema I designed so many years ago.

I used PostgREST to turn this database into an API. I had to brush up my postgres knowledge of schemas and permissions to get more comfortable with this. But once I did, it was pretty much as easy as deploying the docker container. Another big selling point was that it would generate an OpenAPI v2 (fka swagger) definition. With the swagger definition it was trivial to also deploy Swagger UI. Now I have a crude but useful web interface to explore. Finally, I was able to use go-swagger to generate Go client code.

Putting all this together, I was able to adapt my PDF parsing command-line tool to import the structured data into the schema I designed some sixteen year ago. That’s pretty cool for an afternoon exploration.

The Sod House

Sat, 18 Nov 2023 21:52:39 -0500

I recently came to learn of the Fire Chicken Webring, a local link in the the modern webring.

The year is 1996.

1996? I actually had a website in this era, The Sod House. No webring, but very much an attempt at a helpful collection of links.

The. Sod. House?

futuristic drawing, icelandic turf house, modest, dilapidated, weathered, old, damaged, lonely traveler looking at house

prompt: futuristic drawing, icelandic turf house, modest, dilapidated, weathered, old, damaged, lonely traveler looking at house
neg: stone house, wood house
model: sd_xl_base_1.0_0.9vae.safetensors [e6bb9ea85]
seed: 520837284
widthxheight: 1024x512
sampler: DPM++ 2M Karras
cfgs: 7
steps: 30

I created that. Although, it would seem that doesn’t mean what it used to. That’s probably a topic for another day.

In the spirit of the webring, let me share some links from my journey into running models locally:

stable-diffusion-webui-docker
- clean docker compose setup, uses my GPU, easy support for experimenting with differnet models, embeddings, LORAs
civitai
- endless source of new models, embeddings, LORAs, images, techiniques, ideas
ollama
- provides a docker-like experience for playing around with LLMs
ollama-webui
- webui on top of LLMs providing a more user-friendly interface to interact with them
wizard-vicuna-uncensored
- one of my favorite models recently, I can run the 30B model on my 3090 at an acceptable speed, and I’m impressed
- I also have had success with the 7b model on older hardware, but you have to adjust your expectations

Where am I headed next?

I’m curious to bring my own data into the experience, and here are a few thoughts in that direction (and some counter-points):
I’d like to set up my own self-hosted co-pilot-like-thing, and get into some training:
- continue
- refact
Other Interfaces
- KoboldAI
- Text Generation web UI
  - Its goal is to become the AUTOMATIC1111/stable-diffusion-webui of text generation.

Take control, run local.

About

Mon, 01 Jan 0001 00:00:00 +0000

Manually correcting reality, 8 bits at a time, since 0x07cc.