Reserve, confirm and release

Some work has a cost you learn only when it is done: a model call billed by the tokens it used, a conversion billed by the minutes of the output. Reporting afterwards would let several such calls start on the same last part of an allowance. A reservation takes an amount from the allowance before the work starts, and you settle it with what was actually used once the work is done.

How a reservation works

export async function complete(claims: TokenClaims, prompt: string) {
  const reservation = await shiprev.reserve({
    organization_id: claims.organization_id, // from the verified session token only
    capability_key: 'llm_tokens',
    amount: 4000, // the most this call may use
  });

  let answer: ModelAnswer;
  try {
    answer = await callModel(prompt, { maxTokens: 4000 });
  } catch (error) {
    // Only the work failed: nothing was used, so give the whole reservation back.
    await shiprev.release(reservation.reservation_id);
    throw error;
  }

  // The work has run. From here on, never release: settle, and retry if settling fails.
  await settle(reservation.reservation_id, reservation.expires_at, answer.usage.totalTokens);
  return answer;
}

Keep the work and the settlement apart. Release only when the work itself failed. Once it has run, a failure of confirm() is not a reason to release: that would give back an amount that was used. If the work can fail after using part of the amount, confirm what it used instead of releasing. When confirm() fails shows settle().

A confirmed reservation is counted against the allowance like reported usage. Do not also call reportUsage() for the same work, or it is counted twice.

Choose the amount

Reserve the most the work can use, and bound the work by it, as maxTokens does above. Before reserving, a write ruling with the same quantity tells you whether the amount fits in what is left of the allowance:

const ruling = shiprev.adjudicate({
  organization_id: claims.organization_id,
  capability_key: 'llm_tokens',
  quantity: 4000,
  operation: 'write',
});

The region checks the amount again when it reserves. If it does not grant the reservation, reserve() throws an error whose status is 400; do not start the work. The reservation itself is a request to the region, so reserve() also throws while the region is unreachable, and with 403 and the code UPSTREAM_LEVEL_RESTRICTED while your region restricts new writes (see If the link between your region and ShipRev is interrupted). Treat every failure of reserve() as a refusal.

Confirm what was used

confirm(reservationId, actualAmount) replaces the reserved amount with actualAmount:

• Less than reserved. The difference goes back to the allowance.
• More than reserved. The excess is counted as well. It is not refused, because the work has already run; beyond the allowance it is cleared like any other usage, as Allowances and how usage is cleared describes.
• Zero. Nothing is counted; the reservation is closed.

Without actualAmount, confirm() settles the amount this client reserved. A reservation made by another process, for example a worker that picked up the job, has to be confirmed with an explicit amount.

When confirm() fails

confirm() is a request to the region, and it can fail after the work has run: a timeout, a 5xx answer, or an answer lost on the way back after the region already settled. Keep the reservation_id and the amount used, and confirm again:

export async function settle(reservationId: string, expiresAt: string, used: number) {
  let sentBeforeExpiry = false;
  for (let attempt = 1; ; attempt++) {
    try {
      if (Date.now() < Date.parse(expiresAt)) sentBeforeExpiry = true;
      await shiprev.confirm(reservationId, used);
      return;
    } catch (error) {
      const status = (error as { status?: number }).status;
      if (status === 409 && sentBeforeExpiry) {
        // Closed while an attempt was in time: most likely that attempt was settled and its answer was lost.
        logger.warn({ reservationId, used }, 'reservation already closed; check in reconciliation');
        return;
      }
      if (status === 400 || status === 409 || attempt === 5) {
        // Expired before it was settled (409 once the region has reclaimed it), or still failing: record it for reconciliation, never release.
        logger.error({ reservationId, used, status }, 'reservation not settled');
        throw error;
      }
      await sleep(attempt * 1000);
    }
  }
}

What each outcome means:

• No status, or 5xx. The request did not reach the region, or failed there. Confirm again with the same reservation_id and amount, before expires_at.
• 409. The reservation is no longer open: it was confirmed, released or reclaimed. If an attempt was sent before expires_at and its answer was lost, this usually means that attempt was settled. If no attempt was sent before expires_at, the reservation was reclaimed and the work is not counted: handle it as an expired reservation, below. No call of the Server SDK, the Management API or the public API reads the state of a single reservation, so do not release it or confirm it in another way. Log the reservation_id and the amount you meant to confirm.
• 400, or 409 with no attempt before expires_at. The reservation expired before it was settled: confirm() answers 400 until the region reclaims it, about a minute later, and 409 after that. The work is not counted. If it must still be counted, report the amount used with reportUsage(), using an ID derived from the reservation_id, so that a repeat of that report is counted once.

For reconciliation, compare the amounts you logged with the organization's usage in the usage summary or the Management API; those show totals per capability, not single reservations. Choosing a lifetime that leaves room for retries keeps the expired case rare.

Release what you do not use

release(reservationId) gives the whole reserved amount back to the allowance. Release every reservation whose work did not run: on an error of the work, on a cancelled job, and on a timeout of your own before the work started. Until it is confirmed, released or reclaimed, the reserved amount is not available to the organization's other work.

The lifetime of a reservation

Every reservation has an expires_at. Its lifetime is ttl_seconds if you pass it to reserve(), otherwise presets.reservation_ttl_s from the client's options, otherwise the region's default of 300 seconds. Choose a lifetime longer than the work can take.

Confirm before expires_at: after it, confirm() is refused with 400, and with 409 once the region has reclaimed the reservation. A reservation that is neither confirmed nor released by expires_at is reclaimed by the region shortly after: it ends as RELEASED, and its amount goes back to the allowance. Until then release() is still accepted, so release a reservation as soon as you know the work will not run rather than wait for it to expire.

If your process can stop between reserve() and confirm(), keep the reservation_id with the job. The process that picks the job up again can then confirm it with the amount used, or release it.

Errors

reserve(), confirm() and release() throw an Error with a numeric status when the region answers with an HTTP error, and the message carries the response body.

confirm() without an amount, for a reservation this client did not make, throws before any request. The full list of what each method returns is in the Server SDK reference.

Where to go from here