← All writing

Checking capacity before routing work

Revising my routing preferences after correcting an assumption about available model usage.

I changed my preferred routing order twice on October 28. Earlier in the day, I wanted more suitable work going to Gemini Flash and local models reserved for very small tasks. Later, after correcting my understanding of the available quota, I asked to rebalance it.

The first decision partly depended on what I thought was abundant capacity. That made Gemini look like a sensible place to send more work. Once that assumption changed, the order needed another look. The task hadn’t changed just because my understanding of the allowance had.

This exposed a weakness in how I was thinking about model selection. Available quota can influence where work goes, but it doesn’t say whether the model can finish that particular task reliably. A route also needs to be reachable in the current environment. Having a model listed in a configuration file doesn’t establish either of those things.

I still wanted to use local models where they fit. The adjustment was about finding suitable work for them within the actual limits of the system. Sending a task locally only helps if the result is useful enough to justify the time spent running and checking it. The same requirement applies to a cloud route with usage available.

The distinction between a routing change and a result was important here. The earlier thread reported configuration changes, but also said the new order hadn’t been exercised under a real workload. The later planning work wasn’t all finished either. I had changed the intended behaviour; I didn’t yet have evidence that the revised arrangement saved time or reduced paid usage.

That leaves a less tidy account than saying I found the best routing order. My preferences were provisional, and one of the inputs was wrong. Correcting it meant revisiting the decision rather than treating the previous configuration as settled.

For the next implementation step, the work was narrowed to a small local slice. That gave me a more manageable thing to check than trying to validate the entire routing system at once. A broad preference for a provider could wait until there was evidence from the tasks I actually wanted it to handle.

Written by . Follow via RSS ↗