Sets with unmappable BrickLink parts (e.g. pneumatic hoses) never clear from the "need refresh" list #163

New Issue

2026-06-16T21:36:17+02:00

2026-06-16 21:36:17 +02:00

Quick note to self, so I can fix it later.

Some sets keep showing up on the Refresh admin page even right after being freshly added or refreshed. Example: sets 8250-1 and 8462-1. Refreshing does nothing, they immediately reappear.

The list is driven by bricktracker/sql/rebrickable/set/need_refresh.sql, which flags any set having >= 1 part with a NULL in: color_rgb, color_transparent, url, bricklink_color_id, bricklink_color_name, bricklink_part_num.

For 8250/8462 the rgb/transparent/url fields are all fine. The flag comes entirely from bricklink_part_num. Every offending part is a length-specific pneumatic hose (5102c12.5, 5102c17.5, 5102c23.75, 5102c01.875, 5102c06.875, 5102c18.125, etc).

bricklink_part_num is sourced from Rebrickable's part.external_ids.BrickLink. Rebrickable has no BrickLink mapping for these hoses, so the value is NULL upstream.

So a refresh re-fetches the same Rebrickable data -> still NULL -> set reappears forever. This is a false positive: refresh can't fill data that doesn't exist upstream.

Possible fixes

A: Remove the null_bricklink_* conditions from the WHERE in need_refresh.sql. Keep the columns in the table as info only, so they no longer force a set onto the list. rgb/transparent/url remain reliable refresh triggers. This is an issue if the user has sets from before BrickLink data was added.
B: Exclude known-unmappable parts (part NOT LIKE '5102c%'). Whack-a-mole solution, not robust and relies on user feedback or my own testing.
C: Track a per-set "last refreshed under current schema" marker and only list sets never refreshed since the new fields were added. Cleaner long-term, but needs a migration + write-path change.
D: Dismiss/acknowledge button. Let the user hide a set. Most UI work. Doesn't fix the issue but hides/accepts it.

Quick note to self, so I can fix it later. Some sets keep showing up on the Refresh admin page even right after being freshly added or refreshed. Example: sets `8250-1` and `8462-1`. Refreshing does nothing, they immediately reappear. The list is driven by bricktracker/sql/rebrickable/set/need_refresh.sql, which flags any set having >= 1 part with a `NULL` in: `color_rgb`, `color_transparent`, `url`, `bricklink_color_id`, `bricklink_color_name`, `bricklink_part_num`. For 8250/8462 the rgb/transparent/url fields are all fine. The flag comes entirely from bricklink_part_num. Every offending part is a length-specific pneumatic hose (`5102c12.5`, `5102c17.5`, `5102c23.75`, `5102c01.875`, `5102c06.875`, `5102c18.125`, etc). `bricklink_part_num` is sourced from Rebrickable's `part.external_ids.BrickLink`. Rebrickable has no BrickLink mapping for these hoses, so the value is `NULL` upstream. So a refresh re-fetches the same Rebrickable data -> still `NULL` -> set reappears forever. This is a false positive: refresh can't fill data that doesn't exist upstream. Possible fixes - A: Remove the `null_bricklink_*` conditions from the `WHERE` in `need_refresh.sql`. Keep the columns in the table as info only, so they no longer force a set onto the list. rgb/transparent/url remain reliable refresh triggers. This is an issue if the user has sets from before BrickLink data was added. - B: Exclude known-unmappable parts (part `NOT LIKE '5102c%'`). Whack-a-mole solution, not robust and relies on user feedback or my own testing. - C: Track a per-set "last refreshed under current schema" marker and only list sets never refreshed since the new fields were added. Cleaner long-term, but needs a migration + write-path change. - D: Dismiss/acknowledge button. Let the user hide a set. Most UI work. Doesn't fix the issue but hides/accepts it.

FrederikBaerentsen added this to the Version 1.5 milestone 2026-06-16 21:36:17 +02:00

FrederikBaerentsen added the Kind/Bug label 2026-06-16 21:36:17 +02:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: FrederikBaerentsen/BrickTracker#163