Skip to main content

Scheduled jobs

Cron that runs where your data already is

Nightly reconciliation, digests and cleanups, running in the same deploy as the database they work on.

Cron, queues and database in one deploy · Data stored in the EU

Trivial until it needs state

A scheduled task is a one-liner until it has to touch data. Then the machine running the cron needs credentials to a database somewhere else, it has to remember where the last run finished, and it has to survive failing halfway through without processing everything twice.

What started as a cron expression turns into an architecture: a scheduler, somewhere to run the job, a database connection across a network boundary, and a way to track what the last successful run actually completed.

Codehooks runs the schedule inside the same deploy as the data it works on. The cursor lives next to the records, so a failed run can resume instead of starting again.

Where the complexity actually accumulates

Cron host plus an external databaseCodehooks
Credentials crossing a network boundarySame deploy as the data
Cursor state kept somewhere elseCursor in the same store
A partial failure means starting overQueue-backed, so a run can resume
Scheduler and runtime billed apartBoth included
Logs in a third placeLogs beside the run

A nightly reconciliation, complete

The whole thing, including the cursor. Notice there is no database connection string, no scheduler to configure, and no second service in the file.

index.js
import { app, Datastore } from 'codehooks-js';

// Every night at 03:00
app.job('0 3 * * *', async (req, res) => {
const db = await Datastore.open();

// No cursor on the very first run
const last = await db.get('lastRun');
const since = last || new Date(0).toISOString();

// One queued job per record that never synced, at any age
const { ticket } = await db.enqueueFromQuery('orders', {
synced: false
}, 'resync');

console.log(`queued ${ticket.count} orders since ${since}`);

// Only move the cursor once the run has succeeded
await db.set('lastRun', new Date().toISOString());
res.end();
});

export default app.init();

The 'resync' worker handles each record separately, so one failure does not stop the rest of the run.

Questions worth asking first

Starting with the one where a free scheduler is the better answer

GitHub Actions runs my cron for free. Why would I move?
For a stateless job you shouldn't. If the task is "call this endpoint every morning" or "run this script", a scheduled workflow is the right tool and it costs you nothing. The argument here only holds once the job needs the data: a cursor between runs, a query across your own records, or a partial failure that should resume rather than restart. That is when a cron runner and a separate database start costing more than they save.
What happens if a run overlaps the previous one?
There is no automatic locking, so handle it in your own code. The usual approach is a lock key in the key-value store: write it at the start of the run with a TTL longer than the job takes, check for it before doing any work, and delete it at the end. Designing the work to be idempotent is worth doing anyway, because a resumed run will revisit some records.
Can I trigger a job manually?
Yes. Put the work in a worker function rather than directly in the job, then have the cron job start it. You can start the same worker on demand with schedule.run({}, 'reconcile') from an ordinary endpoint, which gives you a manual trigger without duplicating the logic.
Do failed items retry automatically?
Not at the queue level, and it is worth being precise about that. A queue is persistent, so enqueued work is not lost, but what happens when your worker throws is your code's decision: catch the error, record it on the record, and let the next scheduled run pick it up. That is the pattern in the example above. If you want retries handled for you, with a retry count per step and automatic resume from the last completed step after a crash, that is what the Workflow API is for. It supports a maxRetries setting per step and persists state between them.
What if the job takes longer than expected?
Enqueue the work rather than doing it inline, which is what the example above does. The job itself stays short: it finds what needs doing, queues one item per record, and finishes. Workers then process the queue independently, so a single slow record cannot time out the whole run. Paid plans can also raise the worker timeout.

Generated code has less to misconfigure here

The schedule, the cursor and the data sit in one file, so there is no cross-service wiring for a coding agent to get subtly wrong. That is the usual failure mode when generated code has to span a scheduler, a network boundary and a separate database.

Put the schedule next to the data

Cron, queues and a database in a single deploy, so a nightly job is a function rather than an architecture.

Data stored in the EU