feat: new user data structure. see docs/migration_guide
This commit is contained in:
+21
-21
@@ -41,11 +41,11 @@
|
|||||||
# Default: false
|
# Default: false
|
||||||
# BK_BRICKLINK_LINKS=true
|
# BK_BRICKLINK_LINKS=true
|
||||||
|
|
||||||
# Optional: Path to the database.
|
# Optional: Path to the database, relative to '/app/' folder
|
||||||
# Useful if you need it mounted in a Docker volume. Keep in mind that it will not
|
# Useful if you need it mounted in a Docker volume. Keep in mind that it will not
|
||||||
# do any check on the existence of the path, or if it is dangerous.
|
# do any check on the existence of the path, or if it is dangerous.
|
||||||
# Default: ./app.db
|
# Default: data/app.db
|
||||||
# BK_DATABASE_PATH=/var/lib/bricktracker/app.db
|
# BK_DATABASE_PATH=data/app.db
|
||||||
|
|
||||||
# Optional: Format of the timestamp added to the database file when downloading it
|
# Optional: Format of the timestamp added to the database file when downloading it
|
||||||
# Check https://docs.python.org/3/library/time.html#time.strftime for format details
|
# Check https://docs.python.org/3/library/time.html#time.strftime for format details
|
||||||
@@ -86,9 +86,9 @@
|
|||||||
# Default: .pdf
|
# Default: .pdf
|
||||||
# BK_INSTRUCTIONS_ALLOWED_EXTENSIONS=.pdf, .docx, .png
|
# BK_INSTRUCTIONS_ALLOWED_EXTENSIONS=.pdf, .docx, .png
|
||||||
|
|
||||||
# Optional: Folder where to store the instructions, relative to the '/app/static/' folder
|
# Optional: Folder where to store the instructions, relative to '/app/' folder
|
||||||
# Default: instructions
|
# Default: data/instructions
|
||||||
# BK_INSTRUCTIONS_FOLDER=/var/lib/bricktracker/instructions/
|
# BK_INSTRUCTIONS_FOLDER=data/instructions
|
||||||
|
|
||||||
# Optional: Hide the 'Add' entry from the menu. Does not disable the route.
|
# Optional: Hide the 'Add' entry from the menu. Does not disable the route.
|
||||||
# Default: false
|
# Default: false
|
||||||
@@ -167,9 +167,9 @@
|
|||||||
# Default: "rebrickable_minifigures"."name" ASC
|
# Default: "rebrickable_minifigures"."name" ASC
|
||||||
# BK_MINIFIGURES_DEFAULT_ORDER="rebrickable_minifigures"."name" ASC
|
# BK_MINIFIGURES_DEFAULT_ORDER="rebrickable_minifigures"."name" ASC
|
||||||
|
|
||||||
# Optional: Folder where to store the minifigures images, relative to the '/app/static/' folder
|
# Optional: Folder where to store the minifigures images, relative to '/app/' folder
|
||||||
# Default: minifigs
|
# Default: data/minifigures
|
||||||
# BK_MINIFIGURES_FOLDER=minifigures
|
# BK_MINIFIGURES_FOLDER=data/minifigures
|
||||||
|
|
||||||
# Optional: Disable threading on the task executed by the socket.
|
# Optional: Disable threading on the task executed by the socket.
|
||||||
# You should not need to change this parameter unless you are debugging something with the
|
# You should not need to change this parameter unless you are debugging something with the
|
||||||
@@ -187,9 +187,9 @@
|
|||||||
# Default: "rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC
|
# Default: "rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC
|
||||||
# BK_PARTS_DEFAULT_ORDER="total_missing" DESC, "rebrickable_parts"."name"."name" ASC
|
# BK_PARTS_DEFAULT_ORDER="total_missing" DESC, "rebrickable_parts"."name"."name" ASC
|
||||||
|
|
||||||
# Optional: Folder where to store the parts images, relative to the '/app/static/' folder
|
# Optional: Folder where to store the parts images, relative to '/app/' folder
|
||||||
# Default: parts
|
# Default: data/parts
|
||||||
# BK_PARTS_FOLDER=parts
|
# BK_PARTS_FOLDER=data/parts
|
||||||
|
|
||||||
# Optional: Enable server-side pagination for individual pages (recommended for large collections)
|
# Optional: Enable server-side pagination for individual pages (recommended for large collections)
|
||||||
# When enabled, pages use server-side pagination with configurable page sizes
|
# When enabled, pages use server-side pagination with configurable page sizes
|
||||||
@@ -327,10 +327,10 @@
|
|||||||
# Default: https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date
|
# Default: https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date
|
||||||
# BK_RETIRED_SETS_FILE_URL=
|
# BK_RETIRED_SETS_FILE_URL=
|
||||||
|
|
||||||
# Optional: Path to the unofficial retired sets lists
|
# Optional: Path to the unofficial retired sets lists, relative to '/app/' folder
|
||||||
# You can name it whatever you want, but content has to be a CSV
|
# You can name it whatever you want, but content has to be a CSV
|
||||||
# Default: ./retired_sets.csv
|
# Default: data/retired_sets.csv
|
||||||
# BK_RETIRED_SETS_PATH=/var/lib/bricktracker/retired_sets.csv
|
# BK_RETIRED_SETS_PATH=data/retired_sets.csv
|
||||||
|
|
||||||
# Optional: Change the default order of sets. By default ordered by insertion order.
|
# Optional: Change the default order of sets. By default ordered by insertion order.
|
||||||
# Useful column names for this option are:
|
# Useful column names for this option are:
|
||||||
@@ -345,9 +345,9 @@
|
|||||||
# Default: "rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC
|
# Default: "rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC
|
||||||
# BK_SETS_DEFAULT_ORDER="rebrickable_sets"."year" ASC
|
# BK_SETS_DEFAULT_ORDER="rebrickable_sets"."year" ASC
|
||||||
|
|
||||||
# Optional: Folder where to store the sets images, relative to the '/app/static/' folder
|
# Optional: Folder where to store the sets images, relative to '/app/' folder
|
||||||
# Default: sets
|
# Default: data/sets
|
||||||
# BK_SETS_FOLDER=sets
|
# BK_SETS_FOLDER=data/sets
|
||||||
|
|
||||||
# Optional: Enable set consolidation/grouping on the main sets page
|
# Optional: Enable set consolidation/grouping on the main sets page
|
||||||
# When enabled, multiple copies of the same set are grouped together showing instance count
|
# When enabled, multiple copies of the same set are grouped together showing instance count
|
||||||
@@ -389,10 +389,10 @@
|
|||||||
# Default: https://cdn.rebrickable.com/media/downloads/themes.csv.gz
|
# Default: https://cdn.rebrickable.com/media/downloads/themes.csv.gz
|
||||||
# BK_THEMES_FILE_URL=
|
# BK_THEMES_FILE_URL=
|
||||||
|
|
||||||
# Optional: Path to the themes file
|
# Optional: Path to the themes file, relative to '/app/' folder
|
||||||
# You can name it whatever you want, but content has to be a CSV
|
# You can name it whatever you want, but content has to be a CSV
|
||||||
# Default: ./themes.csv
|
# Default: data/themes.csv
|
||||||
# BK_THEMES_PATH=/var/lib/bricktracker/themes.csv
|
# BK_THEMES_PATH=data/themes.csv
|
||||||
|
|
||||||
# Optional: Timezone to use to display datetimes
|
# Optional: Timezone to use to display datetimes
|
||||||
# Check your system for available timezone/TZ values
|
# Check your system for available timezone/TZ values
|
||||||
|
|||||||
@@ -4,6 +4,30 @@
|
|||||||
|
|
||||||
### 1.3
|
### 1.3
|
||||||
|
|
||||||
|
#### Breaking Changes - Data Folder Consolidation
|
||||||
|
|
||||||
|
**IMPORTANT**: This version consolidates all user data into a single `data/` folder for easier backup and volume mapping.
|
||||||
|
|
||||||
|
- **Path handling**: All relative paths are now resolved relative to the application root (`/app` in Docker)
|
||||||
|
- Example: `data/app.db` → `/app/data/app.db`
|
||||||
|
|
||||||
|
- **New default paths** (automatically used for new installations):
|
||||||
|
- Database: `data/app.db` (was: `app.db` in root)
|
||||||
|
- CSV files: `data/*.csv` (was: `*.csv` in root)
|
||||||
|
- Images/PDFs: `data/{sets,parts,minifigures,instructions}/` (was: `static/*`)
|
||||||
|
|
||||||
|
- **Migration options**:
|
||||||
|
1. **Migrate to new structure**
|
||||||
|
2. **Keep current setup**
|
||||||
|
|
||||||
|
See [Migration Guide](docs/migration_guide.md) for detailed instructions
|
||||||
|
|
||||||
|
- **Docker users**:
|
||||||
|
- New setup uses single volume: `data:/app/data/` (replaces 5 separate volumes)
|
||||||
|
- Old volume mounts still work if you set environment variables above
|
||||||
|
|
||||||
|
#### Features
|
||||||
|
|
||||||
- Add individual pagination control system per entity type
|
- Add individual pagination control system per entity type
|
||||||
- `BK_SETS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for sets
|
- `BK_SETS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for sets
|
||||||
- `BK_PARTS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for parts
|
- `BK_PARTS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for parts
|
||||||
|
|||||||
@@ -25,6 +25,7 @@ from bricktracker.views.admin.status import admin_status_page
|
|||||||
from bricktracker.views.admin.storage import admin_storage_page
|
from bricktracker.views.admin.storage import admin_storage_page
|
||||||
from bricktracker.views.admin.tag import admin_tag_page
|
from bricktracker.views.admin.tag import admin_tag_page
|
||||||
from bricktracker.views.admin.theme import admin_theme_page
|
from bricktracker.views.admin.theme import admin_theme_page
|
||||||
|
from bricktracker.views.data import data_page
|
||||||
from bricktracker.views.error import error_404
|
from bricktracker.views.error import error_404
|
||||||
from bricktracker.views.index import index_page
|
from bricktracker.views.index import index_page
|
||||||
from bricktracker.views.instructions import instructions_page
|
from bricktracker.views.instructions import instructions_page
|
||||||
@@ -77,6 +78,7 @@ def setup_app(app: Flask) -> None:
|
|||||||
|
|
||||||
# Register app routes
|
# Register app routes
|
||||||
app.register_blueprint(add_page)
|
app.register_blueprint(add_page)
|
||||||
|
app.register_blueprint(data_page)
|
||||||
app.register_blueprint(index_page)
|
app.register_blueprint(index_page)
|
||||||
app.register_blueprint(instructions_page)
|
app.register_blueprint(instructions_page)
|
||||||
app.register_blueprint(login_page)
|
app.register_blueprint(login_page)
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
|
|||||||
{'n': 'BRICKLINK_LINK_PART_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?P={part}&C={color}'}, # noqa: E501
|
{'n': 'BRICKLINK_LINK_PART_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?P={part}&C={color}'}, # noqa: E501
|
||||||
{'n': 'BRICKLINK_LINK_SET_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?S={set_num}'}, # noqa: E501
|
{'n': 'BRICKLINK_LINK_SET_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?S={set_num}'}, # noqa: E501
|
||||||
{'n': 'BRICKLINK_LINKS', 'c': bool},
|
{'n': 'BRICKLINK_LINKS', 'c': bool},
|
||||||
{'n': 'DATABASE_PATH', 'd': './app.db'},
|
{'n': 'DATABASE_PATH', 'd': 'data/app.db'},
|
||||||
{'n': 'DATABASE_TIMESTAMP_FORMAT', 'd': '%Y-%m-%d-%H-%M-%S'},
|
{'n': 'DATABASE_TIMESTAMP_FORMAT', 'd': '%Y-%m-%d-%H-%M-%S'},
|
||||||
{'n': 'DEBUG', 'c': bool},
|
{'n': 'DEBUG', 'c': bool},
|
||||||
{'n': 'DEFAULT_TABLE_PER_PAGE', 'd': 25, 'c': int},
|
{'n': 'DEFAULT_TABLE_PER_PAGE', 'd': 25, 'c': int},
|
||||||
@@ -22,7 +22,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
|
|||||||
{'n': 'HOST', 'd': '0.0.0.0'},
|
{'n': 'HOST', 'd': '0.0.0.0'},
|
||||||
{'n': 'INDEPENDENT_ACCORDIONS', 'c': bool},
|
{'n': 'INDEPENDENT_ACCORDIONS', 'c': bool},
|
||||||
{'n': 'INSTRUCTIONS_ALLOWED_EXTENSIONS', 'd': ['.pdf'], 'c': list}, # noqa: E501
|
{'n': 'INSTRUCTIONS_ALLOWED_EXTENSIONS', 'd': ['.pdf'], 'c': list}, # noqa: E501
|
||||||
{'n': 'INSTRUCTIONS_FOLDER', 'd': 'instructions', 's': True},
|
{'n': 'INSTRUCTIONS_FOLDER', 'd': 'data/instructions'},
|
||||||
{'n': 'HIDE_ADD_SET', 'c': bool},
|
{'n': 'HIDE_ADD_SET', 'c': bool},
|
||||||
{'n': 'HIDE_ADD_BULK_SET', 'c': bool},
|
{'n': 'HIDE_ADD_BULK_SET', 'c': bool},
|
||||||
{'n': 'HIDE_ADMIN', 'c': bool},
|
{'n': 'HIDE_ADMIN', 'c': bool},
|
||||||
@@ -40,7 +40,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
|
|||||||
{'n': 'HIDE_TABLE_CHECKED_PARTS', 'c': bool},
|
{'n': 'HIDE_TABLE_CHECKED_PARTS', 'c': bool},
|
||||||
{'n': 'HIDE_WISHES', 'c': bool},
|
{'n': 'HIDE_WISHES', 'c': bool},
|
||||||
{'n': 'MINIFIGURES_DEFAULT_ORDER', 'd': '"rebrickable_minifigures"."name" ASC'}, # noqa: E501
|
{'n': 'MINIFIGURES_DEFAULT_ORDER', 'd': '"rebrickable_minifigures"."name" ASC'}, # noqa: E501
|
||||||
{'n': 'MINIFIGURES_FOLDER', 'd': 'minifigures', 's': True},
|
{'n': 'MINIFIGURES_FOLDER', 'd': 'data/minifigures'},
|
||||||
{'n': 'MINIFIGURES_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
|
{'n': 'MINIFIGURES_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
|
||||||
{'n': 'MINIFIGURES_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int},
|
{'n': 'MINIFIGURES_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int},
|
||||||
{'n': 'MINIFIGURES_SERVER_SIDE_PAGINATION', 'c': bool},
|
{'n': 'MINIFIGURES_SERVER_SIDE_PAGINATION', 'c': bool},
|
||||||
@@ -48,7 +48,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
|
|||||||
{'n': 'PARTS_SERVER_SIDE_PAGINATION', 'c': bool},
|
{'n': 'PARTS_SERVER_SIDE_PAGINATION', 'c': bool},
|
||||||
{'n': 'SETS_SERVER_SIDE_PAGINATION', 'c': bool},
|
{'n': 'SETS_SERVER_SIDE_PAGINATION', 'c': bool},
|
||||||
{'n': 'PARTS_DEFAULT_ORDER', 'd': '"rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC'}, # noqa: E501
|
{'n': 'PARTS_DEFAULT_ORDER', 'd': '"rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC'}, # noqa: E501
|
||||||
{'n': 'PARTS_FOLDER', 'd': 'parts', 's': True},
|
{'n': 'PARTS_FOLDER', 'd': 'data/parts'},
|
||||||
{'n': 'PARTS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
|
{'n': 'PARTS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
|
||||||
{'n': 'PARTS_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int},
|
{'n': 'PARTS_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int},
|
||||||
{'n': 'PROBLEMS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
|
{'n': 'PROBLEMS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
|
||||||
@@ -77,9 +77,9 @@ CONFIG: Final[list[dict[str, Any]]] = [
|
|||||||
{'n': 'REBRICKABLE_LINKS', 'e': 'LINKS', 'c': bool},
|
{'n': 'REBRICKABLE_LINKS', 'e': 'LINKS', 'c': bool},
|
||||||
{'n': 'REBRICKABLE_PAGE_SIZE', 'd': 100, 'c': int},
|
{'n': 'REBRICKABLE_PAGE_SIZE', 'd': 100, 'c': int},
|
||||||
{'n': 'RETIRED_SETS_FILE_URL', 'd': 'https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date'}, # noqa: E501
|
{'n': 'RETIRED_SETS_FILE_URL', 'd': 'https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date'}, # noqa: E501
|
||||||
{'n': 'RETIRED_SETS_PATH', 'd': './retired_sets.csv'},
|
{'n': 'RETIRED_SETS_PATH', 'd': 'data/retired_sets.csv'},
|
||||||
{'n': 'SETS_DEFAULT_ORDER', 'd': '"rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC'}, # noqa: E501
|
{'n': 'SETS_DEFAULT_ORDER', 'd': '"rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC'}, # noqa: E501
|
||||||
{'n': 'SETS_FOLDER', 'd': 'sets', 's': True},
|
{'n': 'SETS_FOLDER', 'd': 'data/sets'},
|
||||||
{'n': 'SETS_CONSOLIDATION', 'd': False, 'c': bool},
|
{'n': 'SETS_CONSOLIDATION', 'd': False, 'c': bool},
|
||||||
{'n': 'SHOW_GRID_FILTERS', 'c': bool},
|
{'n': 'SHOW_GRID_FILTERS', 'c': bool},
|
||||||
{'n': 'SHOW_GRID_SORT', 'c': bool},
|
{'n': 'SHOW_GRID_SORT', 'c': bool},
|
||||||
@@ -89,7 +89,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
|
|||||||
{'n': 'SOCKET_PATH', 'd': '/bricksocket/'},
|
{'n': 'SOCKET_PATH', 'd': '/bricksocket/'},
|
||||||
{'n': 'STORAGE_DEFAULT_ORDER', 'd': '"bricktracker_metadata_storages"."name" ASC'}, # noqa: E501
|
{'n': 'STORAGE_DEFAULT_ORDER', 'd': '"bricktracker_metadata_storages"."name" ASC'}, # noqa: E501
|
||||||
{'n': 'THEMES_FILE_URL', 'd': 'https://cdn.rebrickable.com/media/downloads/themes.csv.gz'}, # noqa: E501
|
{'n': 'THEMES_FILE_URL', 'd': 'https://cdn.rebrickable.com/media/downloads/themes.csv.gz'}, # noqa: E501
|
||||||
{'n': 'THEMES_PATH', 'd': './themes.csv'},
|
{'n': 'THEMES_PATH', 'd': 'data/themes.csv'},
|
||||||
{'n': 'TIMEZONE', 'd': 'Etc/UTC'},
|
{'n': 'TIMEZONE', 'd': 'Etc/UTC'},
|
||||||
{'n': 'USE_REMOTE_IMAGES', 'c': bool},
|
{'n': 'USE_REMOTE_IMAGES', 'c': bool},
|
||||||
{'n': 'WISHES_DEFAULT_ORDER', 'd': '"bricktracker_wishes"."rowid" DESC'},
|
{'n': 'WISHES_DEFAULT_ORDER', 'd': '"bricktracker_wishes"."rowid" DESC'},
|
||||||
|
|||||||
@@ -60,7 +60,7 @@ class BrickConfiguration(object):
|
|||||||
if self.cast == bool and isinstance(value, str):
|
if self.cast == bool and isinstance(value, str):
|
||||||
value = value.lower() in ('true', 'yes', '1')
|
value = value.lower() in ('true', 'yes', '1')
|
||||||
|
|
||||||
# Static path fixup
|
# Static path fixup (legacy - only for paths with s: True flag)
|
||||||
if self.static_path and isinstance(value, str):
|
if self.static_path and isinstance(value, str):
|
||||||
value = os.path.normpath(value)
|
value = os.path.normpath(value)
|
||||||
|
|
||||||
@@ -70,6 +70,10 @@ class BrickConfiguration(object):
|
|||||||
# Remove static prefix
|
# Remove static prefix
|
||||||
value = value.removeprefix('static/')
|
value = value.removeprefix('static/')
|
||||||
|
|
||||||
|
# Normalize regular paths (not marked as static)
|
||||||
|
elif not self.static_path and isinstance(value, str) and ('FOLDER' in self.name or 'PATH' in self.name):
|
||||||
|
value = os.path.normpath(value)
|
||||||
|
|
||||||
# Type casting
|
# Type casting
|
||||||
if self.cast is not None:
|
if self.cast is not None:
|
||||||
self.value = self.cast(value)
|
self.value = self.cast(value)
|
||||||
|
|||||||
@@ -96,9 +96,16 @@ class RebrickableImage(object):
|
|||||||
|
|
||||||
# Return the path depending on the objects provided
|
# Return the path depending on the objects provided
|
||||||
def path(self, /) -> str:
|
def path(self, /) -> str:
|
||||||
|
folder = self.folder()
|
||||||
|
# If folder is an absolute path (starts with /), use it directly
|
||||||
|
# Otherwise, make it relative to app root (current_app.root_path)
|
||||||
|
if folder.startswith('/'):
|
||||||
|
base_path = folder
|
||||||
|
else:
|
||||||
|
base_path = os.path.join(current_app.root_path, folder)
|
||||||
|
|
||||||
return os.path.join(
|
return os.path.join(
|
||||||
current_app.static_folder, # type: ignore
|
base_path,
|
||||||
self.folder(),
|
|
||||||
'{id}.{ext}'.format(id=self.id(), ext=self.extension),
|
'{id}.{ext}'.format(id=self.id(), ext=self.extension),
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -152,10 +159,21 @@ class RebrickableImage(object):
|
|||||||
# _, extension = os.path.splitext(self.part_img_url)
|
# _, extension = os.path.splitext(self.part_img_url)
|
||||||
extension = '.jpg'
|
extension = '.jpg'
|
||||||
|
|
||||||
# Compute the path
|
# Determine which route to use based on folder path
|
||||||
path = os.path.join(folder, '{name}{ext}'.format(
|
# If folder contains 'data' (new structure), use data route
|
||||||
name=name,
|
# Otherwise use static route (legacy - relative paths like 'parts', 'sets')
|
||||||
ext=extension,
|
if 'data' in folder:
|
||||||
))
|
# Extract the folder type from the folder_name config key
|
||||||
|
# E.g., 'PARTS_FOLDER' -> 'parts', 'SETS_FOLDER' -> 'sets'
|
||||||
return url_for('static', filename=path)
|
folder_type = folder_name.replace('_FOLDER', '').lower()
|
||||||
|
filename = '{name}{ext}'.format(name=name, ext=extension)
|
||||||
|
return url_for('data.serve_data_file', folder=folder_type, filename=filename)
|
||||||
|
else:
|
||||||
|
# Legacy: folder is relative to static/ (e.g., 'parts' or 'static/parts')
|
||||||
|
# Strip 'static/' prefix if present to avoid double /static/ in URL
|
||||||
|
folder_clean = folder.removeprefix('static/')
|
||||||
|
path = os.path.join(folder_clean, '{name}{ext}'.format(
|
||||||
|
name=name,
|
||||||
|
ext=extension,
|
||||||
|
))
|
||||||
|
return url_for('static', filename=path)
|
||||||
|
|||||||
@@ -0,0 +1,60 @@
|
|||||||
|
import os
|
||||||
|
import logging
|
||||||
|
from flask import Blueprint, current_app, send_from_directory, abort
|
||||||
|
from werkzeug.utils import secure_filename
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
data_page = Blueprint(
|
||||||
|
'data',
|
||||||
|
__name__,
|
||||||
|
url_prefix='/data'
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@data_page.route('/<path:folder>/<filename>')
|
||||||
|
def serve_data_file(folder: str, filename: str):
|
||||||
|
"""
|
||||||
|
Serve files from the data folder (images, PDFs, etc.)
|
||||||
|
This replaces serving these files from static/ folder.
|
||||||
|
|
||||||
|
Security:
|
||||||
|
- Only allows serving files from configured data folders
|
||||||
|
- Uses secure_filename to prevent path traversal
|
||||||
|
- Returns 404 if file doesn't exist or folder not allowed
|
||||||
|
"""
|
||||||
|
# Secure the filename to prevent path traversal attacks
|
||||||
|
safe_filename = secure_filename(filename)
|
||||||
|
|
||||||
|
# Get the configured data folders
|
||||||
|
allowed_folders = {
|
||||||
|
'sets': current_app.config.get('SETS_FOLDER', './data/sets'),
|
||||||
|
'parts': current_app.config.get('PARTS_FOLDER', './data/parts'),
|
||||||
|
'minifigures': current_app.config.get('MINIFIGURES_FOLDER', './data/minifigures'),
|
||||||
|
'instructions': current_app.config.get('INSTRUCTIONS_FOLDER', './data/instructions'),
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check if the requested folder is allowed
|
||||||
|
if folder not in allowed_folders:
|
||||||
|
logger.warning(f"Attempt to access unauthorized folder: {folder}")
|
||||||
|
abort(404)
|
||||||
|
|
||||||
|
# Get the actual folder path
|
||||||
|
folder_path = allowed_folders[folder]
|
||||||
|
|
||||||
|
# If folder_path is relative (not absolute), make it relative to app root
|
||||||
|
if not os.path.isabs(folder_path):
|
||||||
|
folder_path = os.path.join(current_app.root_path, folder_path)
|
||||||
|
|
||||||
|
# Check if file exists
|
||||||
|
file_path = os.path.join(folder_path, safe_filename)
|
||||||
|
if not os.path.isfile(file_path):
|
||||||
|
logger.debug(f"File not found: {file_path}")
|
||||||
|
abort(404)
|
||||||
|
|
||||||
|
# Verify the resolved path is still within the allowed folder (security check)
|
||||||
|
if not os.path.abspath(file_path).startswith(os.path.abspath(folder_path)):
|
||||||
|
logger.warning(f"Path traversal attempt detected: {filename}")
|
||||||
|
abort(404)
|
||||||
|
|
||||||
|
return send_from_directory(folder_path, safe_filename)
|
||||||
@@ -0,0 +1,276 @@
|
|||||||
|
# Data Folder Migration Guide (Docker Compose)
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
Starting with version 1.3, BrickTracker consolidates all user data into a single `data/` folder for easier backup, persistence, and volume mapping.
|
||||||
|
|
||||||
|
**This guide assumes you are running BrickTracker using Docker Compose with bind mounts.**
|
||||||
|
|
||||||
|
> **Note:** If you're using Docker named volumes instead of bind mounts, you'll need to manually copy data between volumes. The commands below are specific to bind mount setups.
|
||||||
|
|
||||||
|
**Backup your data before to making any changes!**
|
||||||
|
|
||||||
|
## What Changed?
|
||||||
|
|
||||||
|
### New Default Structure (v1.3+)
|
||||||
|
|
||||||
|
**All relative paths are resolved relative to `/app` inside the container.** Previously all paths were relative to `/app/static`.
|
||||||
|
|
||||||
|
For example: `data/app.db` → `/app/data/app.db`
|
||||||
|
|
||||||
|
```
|
||||||
|
Container (/app/):
|
||||||
|
├── data/ # NEW: Single volume mount for all user data
|
||||||
|
│ ├── app.db # Database
|
||||||
|
│ ├── retired_sets.csv # Downloaded CSV files
|
||||||
|
│ ├── themes.csv
|
||||||
|
│ ├── sets/ # Set images
|
||||||
|
│ ├── parts/ # Part images
|
||||||
|
│ ├── minifigures/ # Minifigure images
|
||||||
|
│ └── instructions/ # PDF instructions
|
||||||
|
└── static/ # App assets
|
||||||
|
├── brick.png
|
||||||
|
├── styles.css
|
||||||
|
└── scripts/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Docker Compose volume:** Single mount `./data:/app/data/`
|
||||||
|
|
||||||
|
### Previous Structure (v1.2 and earlier)
|
||||||
|
|
||||||
|
```
|
||||||
|
Container (/app/):
|
||||||
|
├── app.db # Mounted from ./data/ on host
|
||||||
|
├── retired_sets.csv # Mounted from ./data/ on host
|
||||||
|
├── themes.csv
|
||||||
|
└── static/
|
||||||
|
├── instructions/ # Separate bind mount
|
||||||
|
├── minifigures/ # Separate bind mount
|
||||||
|
├── parts/ # Separate bind mount
|
||||||
|
├── sets/ # Separate bind mount
|
||||||
|
```
|
||||||
|
|
||||||
|
**Docker Compose bind mounts:** 5 separate mounts
|
||||||
|
```yaml
|
||||||
|
volumes:
|
||||||
|
- ./data:/app/
|
||||||
|
- ./instructions:/app/static/instructions/
|
||||||
|
- ./minifigures:/app/static/minifigures/
|
||||||
|
- ./parts:/app/static/parts/
|
||||||
|
- ./sets:/app/static/sets/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Migration Options
|
||||||
|
|
||||||
|
### Option 1: Migrate to New Data Folder Structure (Recommended)
|
||||||
|
|
||||||
|
This is the recommended approach for cleaner backups and simpler bind mount management.
|
||||||
|
|
||||||
|
1. **Stop the container:**
|
||||||
|
```bash
|
||||||
|
docker compose down
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Create new consolidated data directory on host:**
|
||||||
|
```bash
|
||||||
|
mkdir -p ./bricktracker-data/{sets,parts,minifigures,instructions}
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Move data from old bind mount locations to new structure:**
|
||||||
|
|
||||||
|
Assuming your old `compose.yaml` had:
|
||||||
|
- `./data:/app/` (contains app.db, retired_sets.csv, themes.csv)
|
||||||
|
- `./instructions:/app/static/instructions/`
|
||||||
|
- `./minifigures:/app/static/minifigures/`
|
||||||
|
- `./parts:/app/static/parts/`
|
||||||
|
- `./sets:/app/static/sets/`
|
||||||
|
|
||||||
|
Run:
|
||||||
|
```bash
|
||||||
|
# Move database and CSV files
|
||||||
|
mv ./data/app.db ./bricktracker-data/
|
||||||
|
mv ./data/retired_sets.csv ./bricktracker-data/
|
||||||
|
mv ./data/themes.csv ./bricktracker-data/
|
||||||
|
|
||||||
|
# Move image and instruction folders
|
||||||
|
mv ./instructions/* ./bricktracker-data/instructions/
|
||||||
|
mv ./minifigures/* ./bricktracker-data/minifigures/
|
||||||
|
mv ./parts/* ./bricktracker-data/parts/
|
||||||
|
mv ./sets/* ./bricktracker-data/sets/
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Update `compose.yaml` to use single bind mount:**
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
bricktracker:
|
||||||
|
volumes:
|
||||||
|
- ./bricktracker-data:/app/data/
|
||||||
|
|
||||||
|
# Remove old volume mounts
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Remove old environment overrides from `.env` (if present):**
|
||||||
|
Delete any lines starting with:
|
||||||
|
- `BK_DATABASE_PATH=`
|
||||||
|
- `BK_INSTRUCTIONS_FOLDER=`
|
||||||
|
- `BK_MINIFIGURES_FOLDER=`
|
||||||
|
- `BK_PARTS_FOLDER=`
|
||||||
|
- `BK_SETS_FOLDER=`
|
||||||
|
- `BK_RETIRED_SETS_PATH=`
|
||||||
|
- `BK_THEMES_PATH=`
|
||||||
|
|
||||||
|
6. **Start the container:**
|
||||||
|
```bash
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
7. **Verify everything works:**
|
||||||
|
```bash
|
||||||
|
docker compose logs -f bricktracker
|
||||||
|
# Check the web interface to ensure images/data are loading
|
||||||
|
```
|
||||||
|
|
||||||
|
8. **Clean up old directories (after verification):**
|
||||||
|
```bash
|
||||||
|
rm -r ./data ./instructions ./minifigures ./parts ./sets
|
||||||
|
```
|
||||||
|
|
||||||
|
### Option 2: Keep Current Setup (No Data Migration)
|
||||||
|
|
||||||
|
If you want to keep your current volume structure without moving any files:
|
||||||
|
|
||||||
|
1. **Add these environment variables to your `.env` file:**
|
||||||
|
|
||||||
|
```env
|
||||||
|
# Keep database and CSV files in /data volume (old location)
|
||||||
|
BK_DATABASE_PATH=app.db
|
||||||
|
BK_RETIRED_SETS_PATH=retired_sets.csv
|
||||||
|
BK_THEMES_PATH=themes.csv
|
||||||
|
|
||||||
|
# Keep image/instruction folders in static/ (old location)
|
||||||
|
BK_INSTRUCTIONS_FOLDER=static/instructions
|
||||||
|
BK_MINIFIGURES_FOLDER=static/minifigures
|
||||||
|
BK_PARTS_FOLDER=static/parts
|
||||||
|
BK_SETS_FOLDER=static/sets
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Keep your existing volume mounts in `compose.yaml`:**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
volumes:
|
||||||
|
- ./data:/app/
|
||||||
|
- ./instructions:/app/static/instructions/
|
||||||
|
- ./minifigures:/app/static/minifigures/
|
||||||
|
- ./parts:/app/static/parts/
|
||||||
|
- ./sets:/app/static/sets/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Update to v1.3 and restart:**
|
||||||
|
```bash
|
||||||
|
docker compose pull
|
||||||
|
docker compose up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it! Your existing setup will continue to work.
|
||||||
|
|
||||||
|
## Configuration Reference
|
||||||
|
|
||||||
|
### New Default Paths (v1.3+)
|
||||||
|
|
||||||
|
All paths are relative to `/app` inside the container.
|
||||||
|
|
||||||
|
| Config Variable | Default Value | Resolves To (Container) | Description |
|
||||||
|
|----------------|---------------|------------------------|-------------|
|
||||||
|
| `BK_DATABASE_PATH` | `data/app.db` | `/app/data/app.db` | Database file |
|
||||||
|
| `BK_RETIRED_SETS_PATH` | `data/retired_sets.csv` | `/app/data/retired_sets.csv` | Retired sets CSV |
|
||||||
|
| `BK_THEMES_PATH` | `data/themes.csv` | `/app/data/themes.csv` | Themes CSV |
|
||||||
|
| `BK_INSTRUCTIONS_FOLDER` | `data/instructions` | `/app/data/instructions` | PDF instructions |
|
||||||
|
| `BK_MINIFIGURES_FOLDER` | `data/minifigures` | `/app/data/minifigures` | Minifigure images |
|
||||||
|
| `BK_PARTS_FOLDER` | `data/parts` | `/app/data/parts` | Part images |
|
||||||
|
| `BK_SETS_FOLDER` | `data/sets` | `/app/data/sets` | Set images |
|
||||||
|
|
||||||
|
**Docker Compose bind mount:** `./bricktracker-data:/app/data/` (single mount)
|
||||||
|
|
||||||
|
### Old Paths (v1.2 and earlier)
|
||||||
|
|
||||||
|
To preserve old volume structure without migration, add to `.env`:
|
||||||
|
|
||||||
|
| Config Variable | Value to Preserve Old Behavior | Resolves To (Container) |
|
||||||
|
|----------------|-------------------------------|------------------------|
|
||||||
|
| `BK_DATABASE_PATH` | `app.db` | `/app/app.db` |
|
||||||
|
| `BK_RETIRED_SETS_PATH` | `retired_sets.csv` | `/app/retired_sets.csv` |
|
||||||
|
| `BK_THEMES_PATH` | `themes.csv` | `/app/themes.csv` |
|
||||||
|
| `BK_INSTRUCTIONS_FOLDER` | `static/instructions` | `/app/static/instructions` |
|
||||||
|
| `BK_MINIFIGURES_FOLDER` | `static/minifigures` | `/app/static/minifigures` |
|
||||||
|
| `BK_PARTS_FOLDER` | `static/parts` | `/app/static/parts` |
|
||||||
|
| `BK_SETS_FOLDER` | `static/sets` | `/app/static/sets` |
|
||||||
|
|
||||||
|
## Benefits of New Structure
|
||||||
|
|
||||||
|
1. **Single Bind Mount**: One `./bricktracker-data:/app/data/` mount instead of five separate mounts
|
||||||
|
2. **Easier Backups**: All user data in one location - just backup the `bricktracker-data` directory
|
||||||
|
3. **Cleaner Separation**: User data separated from application assets
|
||||||
|
4. **Better Portability**: Migrate between systems by copying/moving single directory
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Images/Instructions Not Loading After Migration
|
||||||
|
|
||||||
|
1. **Check if data was copied correctly:**
|
||||||
|
```bash
|
||||||
|
docker compose exec bricktracker ls -la /app/data/
|
||||||
|
docker compose exec bricktracker ls -la /app/data/sets/
|
||||||
|
docker compose exec bricktracker ls -la /app/data/instructions/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Verify bind mount:**
|
||||||
|
```bash
|
||||||
|
docker compose config
|
||||||
|
# Should show: volumes: - ./bricktracker-data:/app/data/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Check logs for path errors:**
|
||||||
|
```bash
|
||||||
|
docker compose logs -f
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Verify no old environment overrides:**
|
||||||
|
```bash
|
||||||
|
cat .env | grep BK_
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Not Found
|
||||||
|
|
||||||
|
1. **Check database file location in container:**
|
||||||
|
```bash
|
||||||
|
docker compose exec bricktracker ls -la /app/data/app.db
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **If using old setup, verify environment variables:**
|
||||||
|
```bash
|
||||||
|
docker compose exec bricktracker env | grep BK_DATABASE_PATH
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Check host directory contains database:**
|
||||||
|
```bash
|
||||||
|
ls -la ./bricktracker-data/
|
||||||
|
# Should show: app.db, retired_sets.csv, themes.csv, and subdirectories
|
||||||
|
```
|
||||||
|
|
||||||
|
### Permission Errors
|
||||||
|
|
||||||
|
If you see permission errors after migration:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Fix permissions on bind-mounted directory
|
||||||
|
sudo chown -R $(id -u):$(id -g) ./bricktracker-data
|
||||||
|
```
|
||||||
|
|
||||||
|
### Reverting Migration
|
||||||
|
|
||||||
|
If you need to revert to the old structure:
|
||||||
|
|
||||||
|
1. Stop the container: `docker compose down`
|
||||||
|
2. Restore old `compose.yaml` with 5 volume mounts
|
||||||
|
3. Add old path environment variables to `.env` (see Option 1)
|
||||||
|
4. Start: `docker compose up -d`
|
||||||
Reference in New Issue
Block a user