feat: new user data structure. see docs/migration_guide

This commit is contained in:
2025-12-05 17:59:56 +01:00
parent 29c5d81160
commit 4d4a1aa9f9
8 changed files with 422 additions and 38 deletions
+21 -21
View File
@@ -41,11 +41,11 @@
# Default: false # Default: false
# BK_BRICKLINK_LINKS=true # BK_BRICKLINK_LINKS=true
# Optional: Path to the database. # Optional: Path to the database, relative to '/app/' folder
# Useful if you need it mounted in a Docker volume. Keep in mind that it will not # Useful if you need it mounted in a Docker volume. Keep in mind that it will not
# do any check on the existence of the path, or if it is dangerous. # do any check on the existence of the path, or if it is dangerous.
# Default: ./app.db # Default: data/app.db
# BK_DATABASE_PATH=/var/lib/bricktracker/app.db # BK_DATABASE_PATH=data/app.db
# Optional: Format of the timestamp added to the database file when downloading it # Optional: Format of the timestamp added to the database file when downloading it
# Check https://docs.python.org/3/library/time.html#time.strftime for format details # Check https://docs.python.org/3/library/time.html#time.strftime for format details
@@ -86,9 +86,9 @@
# Default: .pdf # Default: .pdf
# BK_INSTRUCTIONS_ALLOWED_EXTENSIONS=.pdf, .docx, .png # BK_INSTRUCTIONS_ALLOWED_EXTENSIONS=.pdf, .docx, .png
# Optional: Folder where to store the instructions, relative to the '/app/static/' folder # Optional: Folder where to store the instructions, relative to '/app/' folder
# Default: instructions # Default: data/instructions
# BK_INSTRUCTIONS_FOLDER=/var/lib/bricktracker/instructions/ # BK_INSTRUCTIONS_FOLDER=data/instructions
# Optional: Hide the 'Add' entry from the menu. Does not disable the route. # Optional: Hide the 'Add' entry from the menu. Does not disable the route.
# Default: false # Default: false
@@ -167,9 +167,9 @@
# Default: "rebrickable_minifigures"."name" ASC # Default: "rebrickable_minifigures"."name" ASC
# BK_MINIFIGURES_DEFAULT_ORDER="rebrickable_minifigures"."name" ASC # BK_MINIFIGURES_DEFAULT_ORDER="rebrickable_minifigures"."name" ASC
# Optional: Folder where to store the minifigures images, relative to the '/app/static/' folder # Optional: Folder where to store the minifigures images, relative to '/app/' folder
# Default: minifigs # Default: data/minifigures
# BK_MINIFIGURES_FOLDER=minifigures # BK_MINIFIGURES_FOLDER=data/minifigures
# Optional: Disable threading on the task executed by the socket. # Optional: Disable threading on the task executed by the socket.
# You should not need to change this parameter unless you are debugging something with the # You should not need to change this parameter unless you are debugging something with the
@@ -187,9 +187,9 @@
# Default: "rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC # Default: "rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC
# BK_PARTS_DEFAULT_ORDER="total_missing" DESC, "rebrickable_parts"."name"."name" ASC # BK_PARTS_DEFAULT_ORDER="total_missing" DESC, "rebrickable_parts"."name"."name" ASC
# Optional: Folder where to store the parts images, relative to the '/app/static/' folder # Optional: Folder where to store the parts images, relative to '/app/' folder
# Default: parts # Default: data/parts
# BK_PARTS_FOLDER=parts # BK_PARTS_FOLDER=data/parts
# Optional: Enable server-side pagination for individual pages (recommended for large collections) # Optional: Enable server-side pagination for individual pages (recommended for large collections)
# When enabled, pages use server-side pagination with configurable page sizes # When enabled, pages use server-side pagination with configurable page sizes
@@ -327,10 +327,10 @@
# Default: https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date # Default: https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date
# BK_RETIRED_SETS_FILE_URL= # BK_RETIRED_SETS_FILE_URL=
# Optional: Path to the unofficial retired sets lists # Optional: Path to the unofficial retired sets lists, relative to '/app/' folder
# You can name it whatever you want, but content has to be a CSV # You can name it whatever you want, but content has to be a CSV
# Default: ./retired_sets.csv # Default: data/retired_sets.csv
# BK_RETIRED_SETS_PATH=/var/lib/bricktracker/retired_sets.csv # BK_RETIRED_SETS_PATH=data/retired_sets.csv
# Optional: Change the default order of sets. By default ordered by insertion order. # Optional: Change the default order of sets. By default ordered by insertion order.
# Useful column names for this option are: # Useful column names for this option are:
@@ -345,9 +345,9 @@
# Default: "rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC # Default: "rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC
# BK_SETS_DEFAULT_ORDER="rebrickable_sets"."year" ASC # BK_SETS_DEFAULT_ORDER="rebrickable_sets"."year" ASC
# Optional: Folder where to store the sets images, relative to the '/app/static/' folder # Optional: Folder where to store the sets images, relative to '/app/' folder
# Default: sets # Default: data/sets
# BK_SETS_FOLDER=sets # BK_SETS_FOLDER=data/sets
# Optional: Enable set consolidation/grouping on the main sets page # Optional: Enable set consolidation/grouping on the main sets page
# When enabled, multiple copies of the same set are grouped together showing instance count # When enabled, multiple copies of the same set are grouped together showing instance count
@@ -389,10 +389,10 @@
# Default: https://cdn.rebrickable.com/media/downloads/themes.csv.gz # Default: https://cdn.rebrickable.com/media/downloads/themes.csv.gz
# BK_THEMES_FILE_URL= # BK_THEMES_FILE_URL=
# Optional: Path to the themes file # Optional: Path to the themes file, relative to '/app/' folder
# You can name it whatever you want, but content has to be a CSV # You can name it whatever you want, but content has to be a CSV
# Default: ./themes.csv # Default: data/themes.csv
# BK_THEMES_PATH=/var/lib/bricktracker/themes.csv # BK_THEMES_PATH=data/themes.csv
# Optional: Timezone to use to display datetimes # Optional: Timezone to use to display datetimes
# Check your system for available timezone/TZ values # Check your system for available timezone/TZ values
+24
View File
@@ -4,6 +4,30 @@
### 1.3 ### 1.3
#### Breaking Changes - Data Folder Consolidation
**IMPORTANT**: This version consolidates all user data into a single `data/` folder for easier backup and volume mapping.
- **Path handling**: All relative paths are now resolved relative to the application root (`/app` in Docker)
- Example: `data/app.db``/app/data/app.db`
- **New default paths** (automatically used for new installations):
- Database: `data/app.db` (was: `app.db` in root)
- CSV files: `data/*.csv` (was: `*.csv` in root)
- Images/PDFs: `data/{sets,parts,minifigures,instructions}/` (was: `static/*`)
- **Migration options**:
1. **Migrate to new structure**
2. **Keep current setup**
See [Migration Guide](docs/migration_guide.md) for detailed instructions
- **Docker users**:
- New setup uses single volume: `data:/app/data/` (replaces 5 separate volumes)
- Old volume mounts still work if you set environment variables above
#### Features
- Add individual pagination control system per entity type - Add individual pagination control system per entity type
- `BK_SETS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for sets - `BK_SETS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for sets
- `BK_PARTS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for parts - `BK_PARTS_SERVER_SIDE_PAGINATION`: Enable/disable pagination for parts
+2
View File
@@ -25,6 +25,7 @@ from bricktracker.views.admin.status import admin_status_page
from bricktracker.views.admin.storage import admin_storage_page from bricktracker.views.admin.storage import admin_storage_page
from bricktracker.views.admin.tag import admin_tag_page from bricktracker.views.admin.tag import admin_tag_page
from bricktracker.views.admin.theme import admin_theme_page from bricktracker.views.admin.theme import admin_theme_page
from bricktracker.views.data import data_page
from bricktracker.views.error import error_404 from bricktracker.views.error import error_404
from bricktracker.views.index import index_page from bricktracker.views.index import index_page
from bricktracker.views.instructions import instructions_page from bricktracker.views.instructions import instructions_page
@@ -77,6 +78,7 @@ def setup_app(app: Flask) -> None:
# Register app routes # Register app routes
app.register_blueprint(add_page) app.register_blueprint(add_page)
app.register_blueprint(data_page)
app.register_blueprint(index_page) app.register_blueprint(index_page)
app.register_blueprint(instructions_page) app.register_blueprint(instructions_page)
app.register_blueprint(login_page) app.register_blueprint(login_page)
+7 -7
View File
@@ -13,7 +13,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
{'n': 'BRICKLINK_LINK_PART_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?P={part}&C={color}'}, # noqa: E501 {'n': 'BRICKLINK_LINK_PART_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?P={part}&C={color}'}, # noqa: E501
{'n': 'BRICKLINK_LINK_SET_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?S={set_num}'}, # noqa: E501 {'n': 'BRICKLINK_LINK_SET_PATTERN', 'd': 'https://www.bricklink.com/v2/catalog/catalogitem.page?S={set_num}'}, # noqa: E501
{'n': 'BRICKLINK_LINKS', 'c': bool}, {'n': 'BRICKLINK_LINKS', 'c': bool},
{'n': 'DATABASE_PATH', 'd': './app.db'}, {'n': 'DATABASE_PATH', 'd': 'data/app.db'},
{'n': 'DATABASE_TIMESTAMP_FORMAT', 'd': '%Y-%m-%d-%H-%M-%S'}, {'n': 'DATABASE_TIMESTAMP_FORMAT', 'd': '%Y-%m-%d-%H-%M-%S'},
{'n': 'DEBUG', 'c': bool}, {'n': 'DEBUG', 'c': bool},
{'n': 'DEFAULT_TABLE_PER_PAGE', 'd': 25, 'c': int}, {'n': 'DEFAULT_TABLE_PER_PAGE', 'd': 25, 'c': int},
@@ -22,7 +22,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
{'n': 'HOST', 'd': '0.0.0.0'}, {'n': 'HOST', 'd': '0.0.0.0'},
{'n': 'INDEPENDENT_ACCORDIONS', 'c': bool}, {'n': 'INDEPENDENT_ACCORDIONS', 'c': bool},
{'n': 'INSTRUCTIONS_ALLOWED_EXTENSIONS', 'd': ['.pdf'], 'c': list}, # noqa: E501 {'n': 'INSTRUCTIONS_ALLOWED_EXTENSIONS', 'd': ['.pdf'], 'c': list}, # noqa: E501
{'n': 'INSTRUCTIONS_FOLDER', 'd': 'instructions', 's': True}, {'n': 'INSTRUCTIONS_FOLDER', 'd': 'data/instructions'},
{'n': 'HIDE_ADD_SET', 'c': bool}, {'n': 'HIDE_ADD_SET', 'c': bool},
{'n': 'HIDE_ADD_BULK_SET', 'c': bool}, {'n': 'HIDE_ADD_BULK_SET', 'c': bool},
{'n': 'HIDE_ADMIN', 'c': bool}, {'n': 'HIDE_ADMIN', 'c': bool},
@@ -40,7 +40,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
{'n': 'HIDE_TABLE_CHECKED_PARTS', 'c': bool}, {'n': 'HIDE_TABLE_CHECKED_PARTS', 'c': bool},
{'n': 'HIDE_WISHES', 'c': bool}, {'n': 'HIDE_WISHES', 'c': bool},
{'n': 'MINIFIGURES_DEFAULT_ORDER', 'd': '"rebrickable_minifigures"."name" ASC'}, # noqa: E501 {'n': 'MINIFIGURES_DEFAULT_ORDER', 'd': '"rebrickable_minifigures"."name" ASC'}, # noqa: E501
{'n': 'MINIFIGURES_FOLDER', 'd': 'minifigures', 's': True}, {'n': 'MINIFIGURES_FOLDER', 'd': 'data/minifigures'},
{'n': 'MINIFIGURES_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int}, {'n': 'MINIFIGURES_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
{'n': 'MINIFIGURES_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int}, {'n': 'MINIFIGURES_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int},
{'n': 'MINIFIGURES_SERVER_SIDE_PAGINATION', 'c': bool}, {'n': 'MINIFIGURES_SERVER_SIDE_PAGINATION', 'c': bool},
@@ -48,7 +48,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
{'n': 'PARTS_SERVER_SIDE_PAGINATION', 'c': bool}, {'n': 'PARTS_SERVER_SIDE_PAGINATION', 'c': bool},
{'n': 'SETS_SERVER_SIDE_PAGINATION', 'c': bool}, {'n': 'SETS_SERVER_SIDE_PAGINATION', 'c': bool},
{'n': 'PARTS_DEFAULT_ORDER', 'd': '"rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC'}, # noqa: E501 {'n': 'PARTS_DEFAULT_ORDER', 'd': '"rebrickable_parts"."name" ASC, "rebrickable_parts"."color_name" ASC, "bricktracker_parts"."spare" ASC'}, # noqa: E501
{'n': 'PARTS_FOLDER', 'd': 'parts', 's': True}, {'n': 'PARTS_FOLDER', 'd': 'data/parts'},
{'n': 'PARTS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int}, {'n': 'PARTS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
{'n': 'PARTS_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int}, {'n': 'PARTS_PAGINATION_SIZE_MOBILE', 'd': 5, 'c': int},
{'n': 'PROBLEMS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int}, {'n': 'PROBLEMS_PAGINATION_SIZE_DESKTOP', 'd': 10, 'c': int},
@@ -77,9 +77,9 @@ CONFIG: Final[list[dict[str, Any]]] = [
{'n': 'REBRICKABLE_LINKS', 'e': 'LINKS', 'c': bool}, {'n': 'REBRICKABLE_LINKS', 'e': 'LINKS', 'c': bool},
{'n': 'REBRICKABLE_PAGE_SIZE', 'd': 100, 'c': int}, {'n': 'REBRICKABLE_PAGE_SIZE', 'd': 100, 'c': int},
{'n': 'RETIRED_SETS_FILE_URL', 'd': 'https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date'}, # noqa: E501 {'n': 'RETIRED_SETS_FILE_URL', 'd': 'https://docs.google.com/spreadsheets/d/1rlYfEXtNKxUOZt2Mfv0H17DvK7bj6Pe0CuYwq6ay8WA/gviz/tq?tqx=out:csv&sheet=Sorted%20by%20Retirement%20Date'}, # noqa: E501
{'n': 'RETIRED_SETS_PATH', 'd': './retired_sets.csv'}, {'n': 'RETIRED_SETS_PATH', 'd': 'data/retired_sets.csv'},
{'n': 'SETS_DEFAULT_ORDER', 'd': '"rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC'}, # noqa: E501 {'n': 'SETS_DEFAULT_ORDER', 'd': '"rebrickable_sets"."number" DESC, "rebrickable_sets"."version" ASC'}, # noqa: E501
{'n': 'SETS_FOLDER', 'd': 'sets', 's': True}, {'n': 'SETS_FOLDER', 'd': 'data/sets'},
{'n': 'SETS_CONSOLIDATION', 'd': False, 'c': bool}, {'n': 'SETS_CONSOLIDATION', 'd': False, 'c': bool},
{'n': 'SHOW_GRID_FILTERS', 'c': bool}, {'n': 'SHOW_GRID_FILTERS', 'c': bool},
{'n': 'SHOW_GRID_SORT', 'c': bool}, {'n': 'SHOW_GRID_SORT', 'c': bool},
@@ -89,7 +89,7 @@ CONFIG: Final[list[dict[str, Any]]] = [
{'n': 'SOCKET_PATH', 'd': '/bricksocket/'}, {'n': 'SOCKET_PATH', 'd': '/bricksocket/'},
{'n': 'STORAGE_DEFAULT_ORDER', 'd': '"bricktracker_metadata_storages"."name" ASC'}, # noqa: E501 {'n': 'STORAGE_DEFAULT_ORDER', 'd': '"bricktracker_metadata_storages"."name" ASC'}, # noqa: E501
{'n': 'THEMES_FILE_URL', 'd': 'https://cdn.rebrickable.com/media/downloads/themes.csv.gz'}, # noqa: E501 {'n': 'THEMES_FILE_URL', 'd': 'https://cdn.rebrickable.com/media/downloads/themes.csv.gz'}, # noqa: E501
{'n': 'THEMES_PATH', 'd': './themes.csv'}, {'n': 'THEMES_PATH', 'd': 'data/themes.csv'},
{'n': 'TIMEZONE', 'd': 'Etc/UTC'}, {'n': 'TIMEZONE', 'd': 'Etc/UTC'},
{'n': 'USE_REMOTE_IMAGES', 'c': bool}, {'n': 'USE_REMOTE_IMAGES', 'c': bool},
{'n': 'WISHES_DEFAULT_ORDER', 'd': '"bricktracker_wishes"."rowid" DESC'}, {'n': 'WISHES_DEFAULT_ORDER', 'd': '"bricktracker_wishes"."rowid" DESC'},
+5 -1
View File
@@ -60,7 +60,7 @@ class BrickConfiguration(object):
if self.cast == bool and isinstance(value, str): if self.cast == bool and isinstance(value, str):
value = value.lower() in ('true', 'yes', '1') value = value.lower() in ('true', 'yes', '1')
# Static path fixup # Static path fixup (legacy - only for paths with s: True flag)
if self.static_path and isinstance(value, str): if self.static_path and isinstance(value, str):
value = os.path.normpath(value) value = os.path.normpath(value)
@@ -70,6 +70,10 @@ class BrickConfiguration(object):
# Remove static prefix # Remove static prefix
value = value.removeprefix('static/') value = value.removeprefix('static/')
# Normalize regular paths (not marked as static)
elif not self.static_path and isinstance(value, str) and ('FOLDER' in self.name or 'PATH' in self.name):
value = os.path.normpath(value)
# Type casting # Type casting
if self.cast is not None: if self.cast is not None:
self.value = self.cast(value) self.value = self.cast(value)
+27 -9
View File
@@ -96,9 +96,16 @@ class RebrickableImage(object):
# Return the path depending on the objects provided # Return the path depending on the objects provided
def path(self, /) -> str: def path(self, /) -> str:
folder = self.folder()
# If folder is an absolute path (starts with /), use it directly
# Otherwise, make it relative to app root (current_app.root_path)
if folder.startswith('/'):
base_path = folder
else:
base_path = os.path.join(current_app.root_path, folder)
return os.path.join( return os.path.join(
current_app.static_folder, # type: ignore base_path,
self.folder(),
'{id}.{ext}'.format(id=self.id(), ext=self.extension), '{id}.{ext}'.format(id=self.id(), ext=self.extension),
) )
@@ -152,10 +159,21 @@ class RebrickableImage(object):
# _, extension = os.path.splitext(self.part_img_url) # _, extension = os.path.splitext(self.part_img_url)
extension = '.jpg' extension = '.jpg'
# Compute the path # Determine which route to use based on folder path
path = os.path.join(folder, '{name}{ext}'.format( # If folder contains 'data' (new structure), use data route
name=name, # Otherwise use static route (legacy - relative paths like 'parts', 'sets')
ext=extension, if 'data' in folder:
)) # Extract the folder type from the folder_name config key
# E.g., 'PARTS_FOLDER' -> 'parts', 'SETS_FOLDER' -> 'sets'
return url_for('static', filename=path) folder_type = folder_name.replace('_FOLDER', '').lower()
filename = '{name}{ext}'.format(name=name, ext=extension)
return url_for('data.serve_data_file', folder=folder_type, filename=filename)
else:
# Legacy: folder is relative to static/ (e.g., 'parts' or 'static/parts')
# Strip 'static/' prefix if present to avoid double /static/ in URL
folder_clean = folder.removeprefix('static/')
path = os.path.join(folder_clean, '{name}{ext}'.format(
name=name,
ext=extension,
))
return url_for('static', filename=path)
+60
View File
@@ -0,0 +1,60 @@
import os
import logging
from flask import Blueprint, current_app, send_from_directory, abort
from werkzeug.utils import secure_filename
logger = logging.getLogger(__name__)
data_page = Blueprint(
'data',
__name__,
url_prefix='/data'
)
@data_page.route('/<path:folder>/<filename>')
def serve_data_file(folder: str, filename: str):
"""
Serve files from the data folder (images, PDFs, etc.)
This replaces serving these files from static/ folder.
Security:
- Only allows serving files from configured data folders
- Uses secure_filename to prevent path traversal
- Returns 404 if file doesn't exist or folder not allowed
"""
# Secure the filename to prevent path traversal attacks
safe_filename = secure_filename(filename)
# Get the configured data folders
allowed_folders = {
'sets': current_app.config.get('SETS_FOLDER', './data/sets'),
'parts': current_app.config.get('PARTS_FOLDER', './data/parts'),
'minifigures': current_app.config.get('MINIFIGURES_FOLDER', './data/minifigures'),
'instructions': current_app.config.get('INSTRUCTIONS_FOLDER', './data/instructions'),
}
# Check if the requested folder is allowed
if folder not in allowed_folders:
logger.warning(f"Attempt to access unauthorized folder: {folder}")
abort(404)
# Get the actual folder path
folder_path = allowed_folders[folder]
# If folder_path is relative (not absolute), make it relative to app root
if not os.path.isabs(folder_path):
folder_path = os.path.join(current_app.root_path, folder_path)
# Check if file exists
file_path = os.path.join(folder_path, safe_filename)
if not os.path.isfile(file_path):
logger.debug(f"File not found: {file_path}")
abort(404)
# Verify the resolved path is still within the allowed folder (security check)
if not os.path.abspath(file_path).startswith(os.path.abspath(folder_path)):
logger.warning(f"Path traversal attempt detected: {filename}")
abort(404)
return send_from_directory(folder_path, safe_filename)
+276
View File
@@ -0,0 +1,276 @@
# Data Folder Migration Guide (Docker Compose)
## Overview
Starting with version 1.3, BrickTracker consolidates all user data into a single `data/` folder for easier backup, persistence, and volume mapping.
**This guide assumes you are running BrickTracker using Docker Compose with bind mounts.**
> **Note:** If you're using Docker named volumes instead of bind mounts, you'll need to manually copy data between volumes. The commands below are specific to bind mount setups.
**Backup your data before to making any changes!**
## What Changed?
### New Default Structure (v1.3+)
**All relative paths are resolved relative to `/app` inside the container.** Previously all paths were relative to `/app/static`.
For example: `data/app.db``/app/data/app.db`
```
Container (/app/):
├── data/ # NEW: Single volume mount for all user data
│ ├── app.db # Database
│ ├── retired_sets.csv # Downloaded CSV files
│ ├── themes.csv
│ ├── sets/ # Set images
│ ├── parts/ # Part images
│ ├── minifigures/ # Minifigure images
│ └── instructions/ # PDF instructions
└── static/ # App assets
├── brick.png
├── styles.css
└── scripts/
```
**Docker Compose volume:** Single mount `./data:/app/data/`
### Previous Structure (v1.2 and earlier)
```
Container (/app/):
├── app.db # Mounted from ./data/ on host
├── retired_sets.csv # Mounted from ./data/ on host
├── themes.csv
└── static/
├── instructions/ # Separate bind mount
├── minifigures/ # Separate bind mount
├── parts/ # Separate bind mount
├── sets/ # Separate bind mount
```
**Docker Compose bind mounts:** 5 separate mounts
```yaml
volumes:
- ./data:/app/
- ./instructions:/app/static/instructions/
- ./minifigures:/app/static/minifigures/
- ./parts:/app/static/parts/
- ./sets:/app/static/sets/
```
## Migration Options
### Option 1: Migrate to New Data Folder Structure (Recommended)
This is the recommended approach for cleaner backups and simpler bind mount management.
1. **Stop the container:**
```bash
docker compose down
```
2. **Create new consolidated data directory on host:**
```bash
mkdir -p ./bricktracker-data/{sets,parts,minifigures,instructions}
```
3. **Move data from old bind mount locations to new structure:**
Assuming your old `compose.yaml` had:
- `./data:/app/` (contains app.db, retired_sets.csv, themes.csv)
- `./instructions:/app/static/instructions/`
- `./minifigures:/app/static/minifigures/`
- `./parts:/app/static/parts/`
- `./sets:/app/static/sets/`
Run:
```bash
# Move database and CSV files
mv ./data/app.db ./bricktracker-data/
mv ./data/retired_sets.csv ./bricktracker-data/
mv ./data/themes.csv ./bricktracker-data/
# Move image and instruction folders
mv ./instructions/* ./bricktracker-data/instructions/
mv ./minifigures/* ./bricktracker-data/minifigures/
mv ./parts/* ./bricktracker-data/parts/
mv ./sets/* ./bricktracker-data/sets/
```
4. **Update `compose.yaml` to use single bind mount:**
```yaml
services:
bricktracker:
volumes:
- ./bricktracker-data:/app/data/
# Remove old volume mounts
```
5. **Remove old environment overrides from `.env` (if present):**
Delete any lines starting with:
- `BK_DATABASE_PATH=`
- `BK_INSTRUCTIONS_FOLDER=`
- `BK_MINIFIGURES_FOLDER=`
- `BK_PARTS_FOLDER=`
- `BK_SETS_FOLDER=`
- `BK_RETIRED_SETS_PATH=`
- `BK_THEMES_PATH=`
6. **Start the container:**
```bash
docker compose up -d
```
7. **Verify everything works:**
```bash
docker compose logs -f bricktracker
# Check the web interface to ensure images/data are loading
```
8. **Clean up old directories (after verification):**
```bash
rm -r ./data ./instructions ./minifigures ./parts ./sets
```
### Option 2: Keep Current Setup (No Data Migration)
If you want to keep your current volume structure without moving any files:
1. **Add these environment variables to your `.env` file:**
```env
# Keep database and CSV files in /data volume (old location)
BK_DATABASE_PATH=app.db
BK_RETIRED_SETS_PATH=retired_sets.csv
BK_THEMES_PATH=themes.csv
# Keep image/instruction folders in static/ (old location)
BK_INSTRUCTIONS_FOLDER=static/instructions
BK_MINIFIGURES_FOLDER=static/minifigures
BK_PARTS_FOLDER=static/parts
BK_SETS_FOLDER=static/sets
```
2. **Keep your existing volume mounts in `compose.yaml`:**
```yaml
volumes:
- ./data:/app/
- ./instructions:/app/static/instructions/
- ./minifigures:/app/static/minifigures/
- ./parts:/app/static/parts/
- ./sets:/app/static/sets/
```
3. **Update to v1.3 and restart:**
```bash
docker compose pull
docker compose up -d
```
That's it! Your existing setup will continue to work.
## Configuration Reference
### New Default Paths (v1.3+)
All paths are relative to `/app` inside the container.
| Config Variable | Default Value | Resolves To (Container) | Description |
|----------------|---------------|------------------------|-------------|
| `BK_DATABASE_PATH` | `data/app.db` | `/app/data/app.db` | Database file |
| `BK_RETIRED_SETS_PATH` | `data/retired_sets.csv` | `/app/data/retired_sets.csv` | Retired sets CSV |
| `BK_THEMES_PATH` | `data/themes.csv` | `/app/data/themes.csv` | Themes CSV |
| `BK_INSTRUCTIONS_FOLDER` | `data/instructions` | `/app/data/instructions` | PDF instructions |
| `BK_MINIFIGURES_FOLDER` | `data/minifigures` | `/app/data/minifigures` | Minifigure images |
| `BK_PARTS_FOLDER` | `data/parts` | `/app/data/parts` | Part images |
| `BK_SETS_FOLDER` | `data/sets` | `/app/data/sets` | Set images |
**Docker Compose bind mount:** `./bricktracker-data:/app/data/` (single mount)
### Old Paths (v1.2 and earlier)
To preserve old volume structure without migration, add to `.env`:
| Config Variable | Value to Preserve Old Behavior | Resolves To (Container) |
|----------------|-------------------------------|------------------------|
| `BK_DATABASE_PATH` | `app.db` | `/app/app.db` |
| `BK_RETIRED_SETS_PATH` | `retired_sets.csv` | `/app/retired_sets.csv` |
| `BK_THEMES_PATH` | `themes.csv` | `/app/themes.csv` |
| `BK_INSTRUCTIONS_FOLDER` | `static/instructions` | `/app/static/instructions` |
| `BK_MINIFIGURES_FOLDER` | `static/minifigures` | `/app/static/minifigures` |
| `BK_PARTS_FOLDER` | `static/parts` | `/app/static/parts` |
| `BK_SETS_FOLDER` | `static/sets` | `/app/static/sets` |
## Benefits of New Structure
1. **Single Bind Mount**: One `./bricktracker-data:/app/data/` mount instead of five separate mounts
2. **Easier Backups**: All user data in one location - just backup the `bricktracker-data` directory
3. **Cleaner Separation**: User data separated from application assets
4. **Better Portability**: Migrate between systems by copying/moving single directory
## Troubleshooting
### Images/Instructions Not Loading After Migration
1. **Check if data was copied correctly:**
```bash
docker compose exec bricktracker ls -la /app/data/
docker compose exec bricktracker ls -la /app/data/sets/
docker compose exec bricktracker ls -la /app/data/instructions/
```
2. **Verify bind mount:**
```bash
docker compose config
# Should show: volumes: - ./bricktracker-data:/app/data/
```
3. **Check logs for path errors:**
```bash
docker compose logs -f
```
4. **Verify no old environment overrides:**
```bash
cat .env | grep BK_
```
### Database Not Found
1. **Check database file location in container:**
```bash
docker compose exec bricktracker ls -la /app/data/app.db
```
2. **If using old setup, verify environment variables:**
```bash
docker compose exec bricktracker env | grep BK_DATABASE_PATH
```
3. **Check host directory contains database:**
```bash
ls -la ./bricktracker-data/
# Should show: app.db, retired_sets.csv, themes.csv, and subdirectories
```
### Permission Errors
If you see permission errors after migration:
```bash
# Fix permissions on bind-mounted directory
sudo chown -R $(id -u):$(id -g) ./bricktracker-data
```
### Reverting Migration
If you need to revert to the old structure:
1. Stop the container: `docker compose down`
2. Restore old `compose.yaml` with 5 volume mounts
3. Add old path environment variables to `.env` (see Option 1)
4. Start: `docker compose up -d`