fix: prevent schedule() row accumulation across DO restarts#1154
Merged
threepointone merged 3 commits intomainfrom Mar 23, 2026
Merged
fix: prevent schedule() row accumulation across DO restarts#1154threepointone merged 3 commits intomainfrom
threepointone merged 3 commits intomainfrom
Conversation
Introduce idempotent scheduling to avoid duplicate schedule rows and add safety warnings.
- Add options.idempotent?: boolean to schedule(); cron schedules are idempotent by default, delayed/scheduled types are opt-in.
- Deduplicate by querying existing cf_agents_schedules (callback+payload, and cron for recurring) and return existing schedule when found.
- Warn when schedule() is called inside onStart() without { idempotent: true } (track _insideOnStart and _warnedScheduleInOnStart to avoid spam); explicit idempotent:false suppresses the warning.
- Emit schedule:create as before and add new observability event schedule:duplicate_warning when many stale one-shot rows for the same callback are processed.
- Add SQL checks and early returns to prevent creating duplicates; keep existing behavior when idempotent is not set.
- Add comprehensive tests and test agents covering onStart warnings, cron/delayed/scheduled idempotency, and alarm duplicate warnings; update wrangler test config and test worker env types.
🦋 Changeset detectedLatest commit: d0acbd0 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Made-with: Cursor
agents
@cloudflare/ai-chat
@cloudflare/codemode
hono-agents
@cloudflare/shell
@cloudflare/think
@cloudflare/voice
@cloudflare/worker-bundler
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
We hope this addresses the issues raised in #1153 — looking for feedback on the approach before we ship.
Context
#1153 reported that
schedule()called during initialization creates a new row on every Durable Object restart. In crash-loop scenarios, hundreds of stale rows accumulate incf_agents_schedules. When the alarm fires, it processes every stale row individually — 200 rows = 200 callback executions in a single cycle. The reporter's cost: ~$1,650 in DO SQL row reads from a system with 1 user.The root cause is two-fold:
schedule()always generates a freshnanoid(9), soINSERT OR REPLACEnever replaces — it always insertsalarm()processes all stale rows without any awareness of duplicatesscheduleEvery()already got idempotency in #1049. This PR extends similar protection toschedule().Changes
1. Cron schedules via
schedule()are idempotent by defaultCalling
schedule("0 * * * *", "tick")multiple times with the same(callback, cron, payload)now returns the existing row instead of creating a duplicate. This mirrorsscheduleEvery()'s behavior.This is the most important fix — cron rows created via
schedule()are never deleted (they're updated with the next execution time), so duplicates accumulate permanently. After N cold starts, you'd get N executions per cron tick, forever.Set
{ idempotent: false }to opt out if needed.2. Opt-in
idempotentoption for delayed/scheduled typesDedup key is
(type, callback, payload). Different payloads create separate rows. Default remainsfalsefor delayed/scheduled types to preserve backward compatibility — there are legitimate reasons to schedule multiple one-shots for the same callback (e.g., different payloads representing different work items).3.
onStart()warningWhen
schedule()is called insideonStart()without theidempotentoption (for non-cron types), aconsole.warnfires:The warning is:
onStart()cycle (no log spam)idempotentis explicitly set (true or false — user knows what they're doing)4. Alarm-time observability
When
alarm()is about to process ≥10 stale one-shot rows for the same callback, it emits:console.warnwith actionable guidanceschedule:duplicate_warningevent viadiagnostics_channelThis catches the amplification problem at the point of impact without changing execution semantics — all rows are still processed.
What we deliberately did NOT do
alarm(). Silently dropping scheduled work is worse than executing duplicates. The reporter's scenario involved callbacks with the same payload, but in generalschedule()rows with different payloads represent different work items. Deduping at execution time would cause silent data loss.alarm(). Doesn't actually fix the amplification when callbacks reschedule themselves (the row count stays constant across cycles). And it delays legitimate batch processing.Breaking changes
The only behavioral change is cron idempotency becoming the default. This is technically a semver concern, but the previous behavior (N duplicate cron rows → N executions per tick) was almost certainly always a bug. The escape hatch is
{ idempotent: false }.All other changes are additive: new option, new warnings, new observability event.
Test plan
idempotent: falseidempotent, suppressed with{ idempotent: true }, suppressed with{ idempotent: false }Closes #1153
Made with Cursor