Queue-Based AI Workflows in Laravel: Jobs, Retries, and Cost Control

The problem: a request that costs money and takes a minute

The agent from the last post runs inside the HTTP request. That is fine when the whole loop is one lookup and two seconds. It stops being fine the moment a tool takes thirty seconds, or the model wants five rounds, or the user asks for a report over a million rows.

You get the obvious failure first: PHP-FPM timeouts, an nginx 504, a user staring at a spinner. Then you get the expensive one. The request died, but the API call did not - you were billed for tokens nobody ever read. The user hits refresh. You get billed again.

Moving the loop into a queued job fixes the timeout. It does not fix the billing, and it introduces a new way to lose money: a retry is a new set of tokens.

The shape

POST /reports  ->  dispatch(GenerateReport::class)  ->  202 + run_id
                          |
                   queue worker
                          |
              +-----------+-----------+
              |  agent loop, max 5    |
              |  rounds, budget in    |
              |  tokens, not seconds  |
              +-----------+-----------+
                          |
              record spend, broadcast
                          |
                    client polls / listens

Three properties matter, and only one of them is about queues:

1. The job owns a budget, not a timeout.
2. A retry must not re-run the work already paid for.
3. Something outside the job can stop all of it.

The job

class GenerateReport implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable;

    public int $tries = 3;
    public int $timeout = 180;

    // Retries are 1 minute, then 5, then give up. An LLM provider having a
    // bad thirty seconds is the common case; hammering it is not the fix.
    public array $backoff = [60, 300];

    public function __construct(public AiRun $run) {}

    public function handle(Agent $agent): void
    {
        if ($this->run->isTerminal()) {
            return; // already finished by an earlier attempt
        }

        $agent->run($this->run, budget: new TokenBudget(
            maxTokens: 40_000,
            maxRounds: 5,
        ));
    }

    public function failed(Throwable $e): void
    {
        $this->run->markFailed($e->getMessage());
    }
}

$tries = 3 with no other change is the trap. Attempt two starts from scratch: same prompt, same tools, same tokens, same bill. Three attempts of a job that costs forty cents is a dollar twenty for one report.

The isTerminal() check is the cheap half of the fix. The expensive half is the run record.

Make the run the unit of work, not the job

Schema::create('ai_runs', function (Blueprint $t) {
    $t->id();
    $t->foreignId('user_id');
    $t->string('status')->default('queued'); // queued|running|done|failed|capped
    $t->unsignedInteger('rounds')->default(0);
    $t->unsignedInteger('input_tokens')->default(0);
    $t->unsignedInteger('output_tokens')->default(0);
    $t->unsignedInteger('cached_tokens')->default(0);
    $t->decimal('cost_usd', 8, 5)->default(0);
    $t->json('transcript')->nullable();
    $t->timestamps();
});

Every round appends to the transcript and increments the counters in the same transaction as the API call's result. A retry then resumes: the transcript is the state, so attempt two continues from round three instead of round zero. And when someone asks "what did that report cost?", the answer is a column, not an estimate.

Estimating token counts client-side is how a spend cap silently stops capping. Read the usage block the provider returns and price that.

public function record(AiRun $run, Usage $usage): void
{
    DB::transaction(function () use ($run, $usage) {
        $run->increment('rounds');
        $run->increment('input_tokens', $usage->inputTokens);
        $run->increment('output_tokens', $usage->outputTokens);
        $run->increment('cached_tokens', $usage->cacheReadTokens);
        $run->increment('cost_usd', $this->pricer->usd($usage));
    });
}

The budget, checked between rounds

final class TokenBudget
{
    public function __construct(
        public readonly int $maxTokens,
        public readonly int $maxRounds,
    ) {}

    public function exceededBy(AiRun $run): ?string
    {
        if ($run->rounds >= $this->maxRounds) {
            return 'round limit';
        }

        if ($run->input_tokens + $run->output_tokens >= $this->maxTokens) {
            return 'token budget';
        }

        return null;
    }
}

A round limit is not a nicety. An agent with a tool that returns an error the model does not understand will call it again, then again, politely, forever. Five rounds is the number I ship; it has never been the reason a real request failed, and it has repeatedly been the reason a bad one cost forty cents instead of forty dollars.

When the budget trips, the run ends with status = capped and whatever partial answer exists. A capped run is a product decision, not an exception: the user sees "I got this far", which beats both a spinner and a stack trace.

The account-level cap

Per-run budgets bound one report. They do nothing about a thousand reports.

public function handle(Agent $agent, SpendGate $gate): void
{
    if ($gate->monthlyCapReached()) {
        $this->run->markCapped('monthly cap');
        return; // NOT release() - re-queueing a capped run just moves the queue
    }
    // ...
}

Note what it does not do: release(). Putting the job back on the queue when the cap is hit gives you a queue that fills up and retries into the same wall until the month rolls over. Fail the run, tell the user, keep the queue clean.

I run this pattern on the terminal assistant on this site, where the cap is a hard ten dollars a month, enforced in the route before the model is reached. The endpoint is public and unauthenticated, so the cap is not a safety net - it is the only thing between a bored visitor and my card.

Two things to know about in-memory counters, since that is what the simple version uses:

• They are per-instance. On serverless, your real cap is the cap times the number of warm instances. Redis or the database is the honest version the moment you have more than one worker.
• They reset on deploy. If your deploy cadence is faster than your billing cycle, the counter is decorative.

Retries that don't re-bill

The combination that works:

| Failure | What should happen | |---|---| | Provider 429 / 5xx mid-round | Retry with backoff, resume from the transcript | | Tool threw | Feed the error back to the model as a tool result, do not retry the job | | Budget exceeded | Terminal. No retry. | | Monthly cap | Terminal. No retry, no release. | | Job timeout | Terminal after $tries, transcript preserved for inspection |

The middle row is the one people get wrong. A tool that throws is information the model can use - "the order ID does not exist" is a fine thing for it to hear and recover from. Retrying the whole job for it burns every token spent so far to arrive back at the same error.

What this bought

On the assistant, the sequence of guardrails is deliberately unglamorous: a body-size gate before parsing, an input cap, a per-IP rate limit, a per-run output ceiling, and a monthly circuit breaker - four of them before the model is ever reached. Every one of those exists because the failure it prevents is cheaper to stop than to explain on an invoice.

None of that is AI engineering, exactly. It is the same thing you would do around any third-party call that bills per invocation and occasionally takes a minute. That is the point: the model is the easy part.

Next

The obvious hole above is the one I skipped: the cost numbers only mean something if you are actually reading the usage block. Next up: prompt caching - what it actually saves, including the time I found my own code claiming to use a cache it had never enabled.

I'm Aditya Kumar (adityakdevin) - Tech Lead & full-stack developer building AI-powered web products with Laravel, Vue, and LLM APIs. I do this as Laravel + AI integration work for clients. Find me at adityadev.in.