Repository navigation
Feature: Per feed refresh frequency - #245
Conversation
|
Added translations to languages I speak. |
| if params.RefreshInterval != nil { | ||
| if *params.RefreshInterval == -1 { | ||
| setClauses = append(setClauses, "refresh_interval = NULL") | ||
| } else { | ||
| setClauses = append(setClauses, "refresh_interval = :refresh_interval") | ||
| args = append(args, sql.Named("refresh_interval", *params.RefreshInterval)) | ||
| } | ||
| } |
There was a problem hiding this comment.
Changing an interval should also reschedule the feed. This code updates refresh_interval but leaves next_check_at unchanged. For example, changing a feed from 24 hours to 15 minutes can leave it skipped for the remainder of the old schedule.
There was a problem hiding this comment.
Good catch, addressed on the latest commit.
| "feed.add.refreshFrequencyLabel": "Aktualisierungsfrequenz", | ||
| "feed.add.refreshFrequencyDefault": "Standard ({interval})", | ||
| "feed.add.refreshFrequency.900": "Alle 15 minutes", | ||
| "feed.add.refreshFrequency.1800": "Alle 30 minutes", | ||
| "feed.add.refreshFrequency.3600": "Alle 1 hour", | ||
| "feed.add.refreshFrequency.7200": "Alle 2 hours", | ||
| "feed.add.refreshFrequency.21600": "Alle 6 hours", | ||
| "feed.add.refreshFrequency.43200": "Alle 12 hours", | ||
| "feed.add.refreshFrequency.86400": "Alle 24 hours", |
There was a problem hiding this comment.
This sounds really nitpicky but my agent keeps complaining that the German translation is using English time units. Is it normal for German translations to do that?
There was a problem hiding this comment.
No not normal, I just didn't look at my German translations after I wrote them 🫣.
|
This PR will fail on CI because it's missing translations for |
|
Another pet peeve, time stored as number often mean UNIX timestamp or millisecond interval so I recommend adding |
- rename refresh_interval to refresh_interval_seconds across backend, frontend API types, and the feeds table migration for clarity - reschedule feed when its refresh interval changes by resetting next_check_at so the new schedule takes effect immediately - fix German refresh frequency labels to use native time units - add missing zh/fr/es/ru/sv translations
I don't speak these languages, so I just had AI do the translations.
fixed. |
Replace the dynamic min-interval ticker with a scheduler that sleeps until the earliest next_check_at across non-suspended feeds and is woken by feed mutations, so per-feed interval changes apply immediately instead of after the next global tick. Renumber the migration (003 collided with main's 003_bookmark_feed_id and would be silently skipped by the version-keyed runner), replace the -1 sentinel with an explicit ClearRefreshInterval param, clear retry_after_until when the interval changes, and expose the allowed interval set via GET /api/app.
Both dialogs now share one options source (the server-exposed whitelist, with a local fallback) and one formatInterval helper in lib/refresh-interval, replacing the per-dialog copies. formatInterval no longer renders fractional hours for arbitrary global intervals, and unexpected stored values (legacy 0) fall back to the Default option instead of showing a blank Select. Also fixes the onValueChange signature against the current Base UI Select.
Store round-trip (set/clear/absent, next_check_at reset + retry-after drop), due-feed selection and next-wake computation (retry-after dominance, past-due clamp), puller pullDue filtering and scheduleDelay floor/wake behavior, and handler lifecycle incl. validation, 0-clears semantics, GET /api/app payload, and puller wake on mutations.
|
Thanks for the feature and the clean data model, William! The per-feed column, While reviewing, we found the scheduler side needed a rework, so this branch now carries a redesign on top of your commits. This comment explains why, with pictures. Your authorship is untouched — the rework sits as four commits on top. 1. The old scheme couldn't wake upThe original rework made the global ticker dynamic (reset to the shortest interval in use), but it only re-evaluated after a tick — nothing told a sleeping puller that an interval had changed: sequenceDiagram
participant U as User
participant P as Puller (6h ticker)
U->>P: feed set to 15m (due now)
Note over P: asleep — nothing wakes it
P->>P: ≤ 6h later: pull, then ticker → 15m
The new scheme sleeps until the earliest sequenceDiagram
participant U as User
participant P as Puller (due-time loop)
U->>P: feed set to 15m + Wake()
P->>P: pull now, sleep until next next_check_at (≈15m)
flowchart LR
A["pull feeds where next_check_at <= now"] --> B["sleep until earliest next_check_at"]
B -->|timer| A
B -->|"Wake() on feed create/update/delete"| A
The key insight: 2. Load per cycleOne 15m feed + 500 feeds at the 6h global default, every 15 minutes: Two fixes rode along: an interval change now also drops 3. Migration renumber: 003 → 006 (this one was fatal)The branch predated main's own migrations. The runner keys applied migrations by the leading integer ( 4. Smaller cleanups
Backend (incl. |
|
Thanks for the PR! I'll make some adjustments based on your work. Last comment was posted by the agent, this one's from me 😆 |
The project is pnpm-based; a stray package-lock.json means someone ran npm install by accident and should surface as an untracked file, not be silently ignored. backend/fusion (local build artifact) stays.
Drop the pure-function normalize test (every branch is exercised through the handler lifecycle test), replace the circular whitelist loop and the tautological wake-receive check with sharper assertions (DeepEqual against the source list, mirrored config value, non-blocking double Wake), and remove the trailing next_check_at probe that duplicates pullpolicy/ store coverage.
…i18n The dropdown now renders bare numeric labels (15m/30m/1h/.../24h) from formatInterval instead of one translated string per preset, removing 8 keys per locale across 9 locales. The 'Default (30m)' entry is gone too: the option equal to the global interval means 'follow the global setting' (stored as NULL), everything else is an explicit override, so the list is just the /api/app whitelist merged with the global interval. An untouched select sends nothing; picking the global value clears an override, which also avoids pinning a feed to a stale fallback while /app is loading.
Base UI's Select renders the raw value in the trigger unless Root receives an items mapping — the selected interval showed '43200' instead of '12h'. Pass value/label pairs like the group select does.
The trigger collapsed to its content width (59px) next to full-width inputs; w-full aligns it with the proxy input in the same section.
This reverts commit a65c114.
…values State the precedence where the knob is documented (.env.example) and in the pull strategy section: explicit per-feed overrides always win, so adjusting the global interval only re-schedules feeds stored as NULL.
- Wake the scheduler after manual RefreshFeed/RefreshAll persist: a manual refresh can move next_check_at earlier than the sleeping timer's deadline, and without a wake the loop would oversleep (up to the 48h cache cap where the old ticker was bounded by one interval). Creation no longer wakes from the handler — the initial pull's RefreshFeed wakes instead, which also removes the create-time double-fetch window. - Cap scheduleDelay at the global interval: a stale next_check_at (missed wake, host suspend on the monotonic clock, NTP step) delays re-evaluation by at most one interval. - Split NextWakeTime's empty (SQL NULL) from due-now (0) so a legacy zero-state row cannot pin the wake time to the global interval, and backfill those rows in migration 006. - Back off to the global interval when a pass dispatched feeds but the earliest due time is still in the past (fetch-state persist failed), instead of re-fetching origins every second. - Disable the refresh select until /api/app resolves so a save cannot pin an interval chosen against the 1800 fallback. - docs: update absent-vs-0 semantics per endpoint; document the wake coverage and the sleep cap. - tests: real Start() loop test (sleeps to next_check_at, Wake interrupts), delay bounds incl. cap and persist-failure backoff, NextWakeTime ok-flag cases, wake accounting moved to the update path.
Reason for feature
Some feeds I'm subscribed to are either throttling or outright blocking me because of the high refresh frequency. I have raised the refresh frequency, but I'd still like some of my feeds to refresh more often, thus a per feed refresh frequency setting.
Summary
Adds the ability to override the global refresh interval (FUSION_PULL_INTERVAL) on a per-feed basis via a dropdown in the edit/add feed dialogues.
Screenshots
Edit feed
Add feed
Refresh frequency dropdown
Currently, these are the refresh frequency options, but I'm open to adding more options or even leaving it up to the user. I just figured having pre-set options would be more convenient.
Changes
Testing
AI-written, human-supervised.