<!-- cmdz — Deploy FastAPI with a vector database for RAG. Source: https://www.cmdz.com/blog/fastapi-with-a-vector-database-for-rag -->
# Deploy FastAPI with a vector database for RAG
> A retrieval-augmented app end to end: FastAPI on Uvicorn, pgvector, where the embedding cost actually lands, and why the spending limit matters more for this shape of app than any other.

A retrieval-augmented app end to end: FastAPI on Uvicorn, pgvector, where the embedding cost actually lands, and why the spending limit matters more for this shape of app than any other.

3 Jun 2026 · 3 min read · cmdz

Retrieval-augmented apps are the shape most likely to produce a surprising invoice, because the expensive part is not the part you are watching.

## The app

```toml
[build]
runtime_version = "3.12"
package_manager = "uv"

[apps.api]
start = "uvicorn app.main:app --host 0.0.0.0 --port 8000"
```

Detection handles most of this on its own from `pyproject.toml` and `uv.lock`. Pin the Python version explicitly anyway — a silent minor version change is the sort of thing that costs you a Tuesday.

## The vector store

We run Postgres with `pgvector`, which for the overwhelming majority of applications is the correct choice over a dedicated vector database. One system to back up, one system to query, transactional consistency between your embeddings and the rows they describe.

```bash
cmdz services create postgres --size small --extensions vector --attach api
```

The extension is enabled at creation. `DATABASE_URL` is injected.

Vector index storage is metered separately at 200 credits per GB per month — € 0.20 — because an HNSW index is genuinely larger and more expensive to keep resident than an ordinary B-tree. A million 1536-dimension embeddings with an HNSW index is roughly 9 GB, so about € 1.80 a month. That is the honest number, and it is smaller than most people expect.

## Where the cost actually is

Here is the part worth internalising: **the platform is not the expensive component in a RAG app.**

For a typical document assistant, the monthly bill breaks down something like:

- Your embedding provider, on ingestion and on every query
- Your inference provider, on every answer
- cmdz, for the compute and the storage

The third line is usually the smallest by a wide margin. We are not going to pretend otherwise to make our own pricing page look more important.

What our limit does for you is bound the *second-order* costs — the compute that runs the loop, the storage that accumulates, the workers that retry. If your ingestion job goes into a loop at three in the morning, the limit stops the compute. It cannot stop your embedding provider's meter, and no hosting platform can.

Which is an argument for keeping the ingestion loop on the platform with the ceiling, and for putting your own budget check around the provider call.

## Ingestion is a job, not a request

Do not embed documents inside an HTTP handler. Put it in a worker:

```toml
[apps.ingest]
start = "python -m app.worker"
type  = "worker"
```

```bash
cmdz scale ingest --replicas 0     # off between batches
cmdz scale ingest --replicas 4     # on for the run
```

Scaling a worker to zero means it costs nothing while idle. For a workload that runs for two hours a week, that is the difference between € 0.40 and € 14 a month.

## The query path

Two things that matter more than the model choice:

**Cache the embeddings of queries.** The same question arrives repeatedly in any real application. Valkey with a hash of the normalised query as the key removes a provider call and about 200 ms.

**Set a timeout on the provider call, and mean it.** A hanging inference call holds a worker, and enough of them hold every worker. Ten seconds and a clear error beats waiting.

## Health checks with a warm-up

Loading a tokenizer or a local model at import time makes the first request slow. Answer the health check before that finishes:

```python
@app.get("/health")
def health():
    return {"ok": True}
```

Keep it free of database and model access. A health route that queries pgvector turns a slow index into a failed deploy.

## Backups

Daily, included, with point-in-time recovery, and restoring costs nothing. That covers both your rows and your embeddings, which is the thing a separate vector database would have made into two problems.

Re-embedding a corpus because you lost the index is a real cost with a real invoice attached, and it is the sort of thing that only becomes obvious afterwards.

## What it costs

Two API replicas at 0.5 vCPU and 1 GB, a small Postgres with pgvector, 9 GB of index, 20 GB of documents, and an ingestion worker that runs two hours a week:

```
compute (api)      € 2.19
memory  (api)      € 2.19
ingest worker      € 0.36
database           € 1.46
storage 20 GB      € 3.00
vector index 9 GB  € 1.80
backups            € 0.29
────────────────────────────
                  € 11.29 / month
```

Set a limit that covers a bad ingestion run. Then set a separate budget with your embedding provider, because that one is not ours to cap.

## Set your limit and start.

One click with a passkey, then you verify a payment method once to start your 14-day free trial (€ 10 of credit). After that it is prepaid pay-as-you-go — you only ever spend credit you have already bought, and no invoice ever arrives above the amount you set.

- [Create account](https://app.cmdz.com/signup)
- [Read the docs](https://docs.cmdz.com)