Make a job queue retry safely

Step 1: Name the side effect

Write down the one action that must not happen twice, such as charging a card or sending a notice. Everything else can usually run again.

Checkpoint

You can point at the side effect, and you can say where its result will be recorded.

Step 2: Record the result before you rely on it

Store a row that says the side effect finished, using the job id as the key. If the row is already there, the worker stops.

insert into job_results (job_id, finished_at)
values ($1, now())
on conflict (job_id) do nothing;

Checkpoint

Running the worker twice for one job id leaves one result row.