Clean up code #8

Closed
opened 2024-04-26 11:10:44 +02:00 by FrederikBaerentsen · 45 comments

Especially app.py

Especially app.py
FrederikBaerentsen added this to the General Improvements project 2024-12-29 16:48:29 +01:00
Author
Owner
  • app.py should utilize more functions.
  • Create one base.html file for all pages to use
  • Create a searchbar html file for each page to use (js for that search, can be on the specific page)
  • js file for each page
  • css for all pages to use in base.html
- [ ] app.py should utilize more functions. - [ ] Create one base.html file for all pages to use - [ ] Create a searchbar html file for each page to use (js for that search, can be on the specific page) - [ ] js file for each page - [ ] css for all pages to use in base.html
FrederikBaerentsen added this to the Version 1 milestone 2024-12-29 16:52:39 +01:00
Collaborator

Hello,
thank you for making this app. I wanted to add some features that would be useful to me (like who owns the set, where it is stored) but the massive app.py scared me.

I started rewriting some parts into objects, and made a bit more modular.
I also started rewriting some parts of the UI to my taste (which is my very own 😅, and is very much the default Bootstrap 5 boilerplate).

It looks like this so far on my side:
image

I'll share it in a branch if you are interested, once I'm on par with your version in terms of features.

Happy new year!

Hello, thank you for making this app. I wanted to add some features that would be useful to me (like who owns the set, where it is stored) but the massive `app.py` scared me. I started rewriting some parts into objects, and made a bit more modular. I also started rewriting some parts of the UI to my taste (which is my very own 😅, and is very much the default Bootstrap 5 boilerplate). It looks like this so far on my side: ![image](/attachments/1022c9cf-3881-48fe-aaea-e082daed06ad) I'll share it in a branch if you are interested, once I'm on par with your version in terms of features. Happy new year!
945 KiB
Author
Owner

I’m very open for new features! And who own a set (eg you or a child) would be nice. The same with the location it is stored.

The UI changes are great! Please open a PR as I love the changes you have made.

I’m very open for new features! And who own a set (eg you or a child) would be nice. The same with the location it is stored. The UI changes are great! Please open a PR as I love the changes you have made.
Collaborator

Hello, just to keep you informed, I'm still processing all your code 🙃.
I've re-implemented the sorting/filtering/searching through the sets.

It looks like this so far: image

Hello, just to keep you informed, I'm still processing all your code 🙃. I've re-implemented the sorting/filtering/searching through the sets. It looks like this so far: ![image](/attachments/3066d23d-9d89-4f3f-a7ab-7fff3bb8df52)
884 KiB
Author
Owner

That looks great. Unfortunately i haven't had time to work on this over the holidays. I hope to get back to it soon though. Do you want me to add you as collaborator and make a branch for your design?

That looks great. Unfortunately i haven't had time to work on this over the holidays. I hope to get back to it soon though. Do you want me to add you as collaborator and make a branch for your design?
Collaborator

Don't worry too much about it, I'll make my own fork whenever and do the proper merge request process.
It's going to take a while longer because I'm rewriting everything I see (whenever I can make sense of it).

For instance so far the app.py file looks like this now:

from werkzeug.middleware.proxy_fix import ProxyFix

from bricktracker.config import load_configuration
from bricktracker.sql import close_all
from bricktracker.socket import new_sock
from bricktracker.views.errors import error_404
from bricktracker.views.index import index_page
from bricktracker.views.set import set_page


if __name__ == '__main__':
    app = Flask(__name__)

    # Load the configuration
    load_configuration(app)

    app.wsgi_app = ProxyFix(
        app.wsgi_app,
        x_for=1,
        x_proto=1,
        x_host=1,
        x_port=1,
        x_prefix=1,
    )

    # Register errors
    app.register_error_handler(404, error_404)

    # Register routes
    app.register_blueprint(index_page)
    app.register_blueprint(set_page)

    # Make sure all connections are closed at the end
    @app.teardown_appcontext
    def close_connections(exception):
        close_all()

    # Create the socket
    s = new_sock(app)

    # Run the application
    s.socket.run(
        app,
        host='0.0.0.0',  # TODO: Parametric
        debug=True,  # TODO: Parametric
        port=3333,  # TODO: Parametric
    )

That's all, and it's not going to grow much besides a couple routes 😁.

Don't worry too much about it, I'll make my own fork whenever and do the proper merge request process. It's going to take a while longer because I'm rewriting everything I see (whenever I can make sense of it). For instance so far the `app.py` file looks like this now: ```from flask import Flask from werkzeug.middleware.proxy_fix import ProxyFix from bricktracker.config import load_configuration from bricktracker.sql import close_all from bricktracker.socket import new_sock from bricktracker.views.errors import error_404 from bricktracker.views.index import index_page from bricktracker.views.set import set_page if __name__ == '__main__': app = Flask(__name__) # Load the configuration load_configuration(app) app.wsgi_app = ProxyFix( app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_port=1, x_prefix=1, ) # Register errors app.register_error_handler(404, error_404) # Register routes app.register_blueprint(index_page) app.register_blueprint(set_page) # Make sure all connections are closed at the end @app.teardown_appcontext def close_connections(exception): close_all() # Create the socket s = new_sock(app) # Run the application s.socket.run( app, host='0.0.0.0', # TODO: Parametric debug=True, # TODO: Parametric port=3333, # TODO: Parametric ) ``` That's all, and it's not going to grow much besides a couple routes 😁.
Collaborator

Got the set parts listing redone, with saving when exiting the input field rather than clicking the save button.
image

This should cover #29.

Also even if it is in an input field, sorting by number of missing pieces is valid for parts now.
image

Got the set parts listing redone, with saving when exiting the input field rather than clicking the save button. ![image](/attachments/9341ff75-5767-4cdb-8d95-d2b82a72ee5c) This should cover #29. Also even if it is in an `input` field, sorting by number of missing pieces is valid for parts now. ![image](/attachments/ee88a98d-0476-4915-83d4-f6f65cd9b290)
Author
Owner

Looks great. Looking forward to seeing the code as I'm sure i have lots to learn. Showed your design to my wife and she said it was so much better than mine.

Looks great. Looking forward to seeing the code as I'm sure i have lots to learn. Showed your design to my wife and she said it was so much better than mine.
Collaborator

Minifigures are in. I can now get to what I was dreading: your create page 😂.

image

Minifigures are in. I can now get to what I was dreading: your create page 😂. ![image](/attachments/f4782c5e-cdaf-4696-8b9a-a5f2b84744ac)
169 KiB
Author
Owner

Basically, most of the create page is just socket.io to send a hacked together status bar back and forth. Honestly it could be skipped.

My rough workflow is like:

  1. Check if the input number contains -. If it does, assume the full set number is what’s in the input field (eg 79000-1 or 71039-7).

  2. If it doesn’t contain -, append -1 to the number.

  3. Check the input number against the downloaded sets.csv from rebrickable. If it isn’t in that file, reject it and say it doesn’t exist.

  4. Use the rebrick library to get set info, then inventory, then minifig info. I think the rebrickable api can return multiple things in one query but I didn’t look into that.

  5. Save to sql and download all images that doesn’t exists already.

  6. Return to start page.

The progress bar checks how many images needs downloaded and makes an estimate of the progress. But it’s very bad and I wish I hadn’t spend so much time on it 😂

Basically, most of the create page is just socket.io to send a hacked together status bar back and forth. Honestly it could be skipped. My rough workflow is like: 1. Check if the input number contains `-`. If it does, assume the full set number is what’s in the input field (eg 79000-1 or 71039-7). 2. If it doesn’t contain `-`, append `-1` to the number. 3. Check the input number against the downloaded sets.csv from rebrickable. If it isn’t in that file, reject it and say it doesn’t exist. 4. Use the rebrick library to get set info, then inventory, then minifig info. I think the rebrickable api can return multiple things in one query but I didn’t look into that. 5. Save to sql and download all images that doesn’t exists already. 6. Return to start page. The progress bar checks how many images needs downloaded and makes an estimate of the progress. But it’s very bad and I wish I hadn’t spend so much time on it 😂

After typing my whole spiel out I could have just started here.

Watching closely, but from my end the more modular (at an html/class level) the better for any CSS tuning. My budibase 'app' looks like this and I'll be trying to emulate it here eventually as I just can't build the features I want

After typing my whole spiel out I could have just started here. Watching closely, but from my end the more modular (at an html/class level) the better for any CSS tuning. My budibase 'app' looks like this and I'll be trying to emulate it here eventually as I just can't build the features I want
Collaborator

Reworked adding a set 🤯:

  • Do not rely on sets.csv to check if a set exists
  • By default displays what it is about to import before importing it (I had some surprises with -2 sets)
  • Retained the "add -1 if no version has been supplied
  • Gives you a link to the page after importing
  • Kept the socket for async progress updates

image
image
image
image
image
image

Reworked adding a set 🤯: - Do not rely on `sets.csv` to check if a set exists - By default displays what it is about to import before importing it (I had some surprises with `-2` sets) - Retained the "add `-1` if no version has been supplied - Gives you a link to the page after importing - Kept the socket for async progress updates ![image](/attachments/4c76804e-972d-46f3-8b57-32eb65bd82ab) ![image](/attachments/a44b5eba-badc-4c88-b399-09df81526d02) ![image](/attachments/d992dcbb-010a-4cde-b503-99c4adc30a7d) ![image](/attachments/eae0c4d0-98cd-4d89-92fb-2931e119db68) ![image](/attachments/062ef95e-abf9-4955-af85-802692308da9) ![image](/attachments/8222f09f-8ab2-4909-9ab9-281ff3127ad4)
Collaborator

Added visual feedback when using the set checkboxes. This is mentioned in #35.

image
image

Added visual feedback when using the set checkboxes. This is mentioned in #35. ![image](/attachments/1d413103-171f-4040-8966-c5253b4d54a2) ![image](/attachments/1a826bf2-b087-49f1-9b68-31f38c0c1c1d)
122 KiB
292 KiB
Collaborator

Reworked the deletion workflow, with a confirmation screen.

image
image
image

Reworked the deletion workflow, with a confirmation screen. ![image](/attachments/5b4a9dcd-de3d-401f-a1c7-36f8f1a598a5) ![image](/attachments/c49f0bf3-dcd2-4651-a627-df85578300f5) ![image](/attachments/64390c0b-c544-4e9f-af4f-54f4c1664981)
Collaborator

Added a visual indicator for the socket. This could help with situations like #33.

image
image

Added a visual indicator for the socket. This could help with situations like #33. ![image](/attachments/4ce84606-941e-4677-a01c-32955d4ab0ab) ![image](/attachments/a3b9d676-3135-47ed-a31a-c9386093ae05)
Author
Owner

Great job so far. I've not started my code cleanup and issue fixing yet, as it would just be double the work. Looking forward to testing your version!

Great job so far. I've not started my code cleanup and issue fixing yet, as it would just be double the work. Looking forward to testing your version!
Collaborator

Reworked gracefully handling app.db not initialized.

image
image
image
image

Reworked gracefully handling `app.db` not initialized. ![image](/attachments/387951fd-6a01-4967-a470-1a87884492d4) ![image](/attachments/5b52dba0-46fc-43ec-be4b-b4e0a6a2a2ea) ![image](/attachments/329c93b9-c429-473d-a6bc-75c5a5a67ef7) ![image](/attachments/78d009fa-3161-40ff-ae5a-91f90e962bac)
Collaborator

Added a lightweight bulk add. It does nothing fancy, and does not resume on error.
It takes in entry a list of sets. On error it put backs into the input whatever was not processed (including the errored set).
This should help with part of #25 🙂.

image
image
image

Added a lightweight bulk add. It does nothing fancy, and does not resume on error. It takes in entry a list of sets. On error it put backs into the input whatever was not processed (including the errored set). This should help with part of #25 🙂. ![image](/attachments/6b7d6709-f85c-47cd-a4ee-426eeae2049e) ![image](/attachments/08ffdbeb-a835-4cc8-bb27-6da8607cabb9) ![image](/attachments/a9fe0b72-53ff-4552-b4e5-78e688bb16dd)
Collaborator

Added a very basic (and optional) read-only/authenticated mode (with a unique password) to be able to share the page read-only.
It blocks:

  • Admin panel
  • Adding/Bulk adding
  • Changing the checkboxes
  • Changing the missing parts
  • Deleting

image
image
image
image
image

Added a **very basic (and optional)** read-only/authenticated mode (with a unique password) to be able to share the page read-only. It blocks: - Admin panel - Adding/Bulk adding - Changing the checkboxes - Changing the missing parts - Deleting ![image](/attachments/ae60e2dc-bb4f-44dd-be61-cccae9b30be5) ![image](/attachments/5ba2e179-8221-45be-8aeb-700f774b9bc7) ![image](/attachments/6f5a40cd-1138-43a3-a11c-65888b6b2324) ![image](/attachments/d1549c5e-2796-406a-b62d-af791d8bfd5b) ![image](/attachments/20b43a43-4af0-4f73-88c1-d3a60b9096bc)
Collaborator

Added part detail (a bit janky because I had a hard time understanding what makes a part unique, I'm no LEGO expert 😅)
It displays which sets are using this part. It's displayed from clicking on a part in the listing

image
image

Added **part** detail (a bit janky because I had a hard time understanding what makes a part unique, I'm no LEGO expert 😅) It displays which sets are using this part. It's displayed from clicking on a part in the listing ![image](/attachments/46e36dc5-51e4-4b9f-9da6-e06ec5c831c6) ![image](/attachments/5843121f-b084-4173-8407-18e64d7390fc)
Author
Owner

Added a very basic (and optional) read-only/authenticated mode (with a unique password) to be able to share the page read-only.

Thinking it's a bit risky, would rather do it using reverse proxy. Implementing proper authentication is hard :)

Added part detail (a bit janky because I had a hard time understanding what makes a part unique, I'm no LEGO expert 😅)

I think i did a mix of element ID and color.

Really loving the design and how you highlight various things!

> Added a **very basic (and optional)** read-only/authenticated mode (with a unique password) to be able to share the page read-only. Thinking it's a bit risky, would rather do it using reverse proxy. Implementing proper authentication is hard :) > Added part detail (a bit janky because I had a hard time understanding what makes a part unique, I'm no LEGO expert 😅) I think i did a mix of element ID and color. Really loving the design and how you highlight various things!
Collaborator

Added a very basic (and optional) read-only/authenticated mode (with a unique password) to be able to share the page read-only.

Thinking it's a bit risky, would rather do it using reverse proxy. Implementing proper authentication is hard :)

Just to reassure you, I didn't produce the authentication code myself. I'm using the existing flask-login package.
I called it basic only because the user database is only one user, and its password is set through an env variable.

But it does use proper sessions, isolation between clients, cryptographically signed sessions and cookies etc... 🙂

        # Setup basic authentication
        app.secret_key = app.config['AUTHENTICATION_KEY']

        manager = login_manager.LoginManager()
        manager.login_view = 'login.login'  # type: ignore
        manager.init_app(app)

        # User loader with only one user
        @manager.user_loader
        def user_loader(*arg):
            return self.User(
                'admin',
                app.config['AUTHENTICATION_PASSWORD']
            )

        # If the password is unset, globally disable
        app.config['LOGIN_DISABLED'] = app.config['AUTHENTICATION_PASSWORD'] == ''  # noqa: E501
> > Added a **very basic (and optional)** read-only/authenticated mode (with a unique password) to be able to share the page read-only. > > Thinking it's a bit risky, would rather do it using reverse proxy. Implementing proper authentication is hard :) > Just to reassure you, I didn't produce the authentication code myself. I'm using the existing `flask-login` package. I called it basic only because the user database is only one user, and its password is set through an env variable. But it does use proper sessions, isolation between clients, cryptographically signed sessions and cookies etc... 🙂 ``` # Setup basic authentication app.secret_key = app.config['AUTHENTICATION_KEY'] manager = login_manager.LoginManager() manager.login_view = 'login.login' # type: ignore manager.init_app(app) # User loader with only one user @manager.user_loader def user_loader(*arg): return self.User( 'admin', app.config['AUTHENTICATION_PASSWORD'] ) # If the password is unset, globally disable app.config['LOGIN_DISABLED'] = app.config['AUTHENTICATION_PASSWORD'] == '' # noqa: E501 ```
Author
Owner

That's great to hear. Thought you had implemented it yourself 👍

That's great to hear. Thought you had implemented it yourself 👍
Collaborator

Added minifigures list and details

image
image
image

Added minifigures list and details ![image](/attachments/372ad95d-d5b2-447d-b184-1d39cb0ba819) ![image](/attachments/ad4c494f-b115-4dbf-89d7-c235d06852e5) ![image](/attachments/40eb0df5-3a7f-425a-bbba-8ead7e214814)
Collaborator

Added "normal" logs when not in DEBUG=true

[2025-01-13 21:55:51,888] INFO - Starting BrickTracker on 0.0.0.0:3333
[2025-01-13 21:56:03,789] INFO - Set 608-2: imported (id: 8f81e80d-1202-40af-a0b5-d45cd6115b28)
[2025-01-13 21:56:10,699] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed minifigures collected status to True
[2025-01-13 21:56:13,633] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set checked status to True
[2025-01-13 21:56:15,256] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set collected status to True
[2025-01-13 21:56:17,442] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set collected status to False
[2025-01-13 21:56:17,849] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set checked status to False
[2025-01-13 21:56:18,299] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed minifigures collected status to False
[2025-01-13 21:56:24,624] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): updated part (558282) missing count to 2
[2025-01-13 21:56:29,278] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): updated part (352475) missing count to 1
[2025-01-13 21:56:37,489] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): updated minifigure (fig-000653) part (4134882) missing count to 1
[2025-01-13 21:56:47,442] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): deleted
[2025-01-13 21:57:12,254] WARNING - Not found: Set with ID 8f81e80d-1202-40af-a0b5-d45cd6115b28 was not found in the database

Including database actions:

[2025-01-13 22:05:03,895] INFO - Starting BrickTracker on 0.0.0.0:3333
[2025-01-13 22:05:39,790] INFO - The database has been initialized
[2025-01-13 22:05:42,866] INFO - The database has been downloaded
[2025-01-13 22:06:06,545] INFO - The database has been dropped
[2025-01-13 22:06:19,737] INFO - The database has been imported using file bullshit.db
[2025-01-13 22:06:19,742] WARNING - An exception occured while loading the admin page: file is not a database
[2025-01-13 22:06:33,569] ERROR - Database error: file is not a database
[2025-01-13 22:06:36,213] WARNING - An exception occured while loading the admin page: file is not a database
[2025-01-13 22:06:41,518] INFO - The database has been deleted
[2025-01-13 22:06:43,235] INFO - The database has been initialized
Added "normal" logs when not in `DEBUG=true` ``` [2025-01-13 21:55:51,888] INFO - Starting BrickTracker on 0.0.0.0:3333 [2025-01-13 21:56:03,789] INFO - Set 608-2: imported (id: 8f81e80d-1202-40af-a0b5-d45cd6115b28) [2025-01-13 21:56:10,699] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed minifigures collected status to True [2025-01-13 21:56:13,633] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set checked status to True [2025-01-13 21:56:15,256] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set collected status to True [2025-01-13 21:56:17,442] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set collected status to False [2025-01-13 21:56:17,849] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed set checked status to False [2025-01-13 21:56:18,299] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): changed minifigures collected status to False [2025-01-13 21:56:24,624] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): updated part (558282) missing count to 2 [2025-01-13 21:56:29,278] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): updated part (352475) missing count to 1 [2025-01-13 21:56:37,489] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): updated minifigure (fig-000653) part (4134882) missing count to 1 [2025-01-13 21:56:47,442] INFO - Set 608-2 (8f81e80d-1202-40af-a0b5-d45cd6115b28): deleted [2025-01-13 21:57:12,254] WARNING - Not found: Set with ID 8f81e80d-1202-40af-a0b5-d45cd6115b28 was not found in the database ``` Including database actions: ``` [2025-01-13 22:05:03,895] INFO - Starting BrickTracker on 0.0.0.0:3333 [2025-01-13 22:05:39,790] INFO - The database has been initialized [2025-01-13 22:05:42,866] INFO - The database has been downloaded [2025-01-13 22:06:06,545] INFO - The database has been dropped [2025-01-13 22:06:19,737] INFO - The database has been imported using file bullshit.db [2025-01-13 22:06:19,742] WARNING - An exception occured while loading the admin page: file is not a database [2025-01-13 22:06:33,569] ERROR - Database error: file is not a database [2025-01-13 22:06:36,213] WARNING - An exception occured while loading the admin page: file is not a database [2025-01-13 22:06:41,518] INFO - The database has been deleted [2025-01-13 22:06:43,235] INFO - The database has been initialized ```
Collaborator

Displaying all info about the configuration variables in the admin panel, including their values (except for secrets), if they are changed from their defaults, and which environment variable controls it

image

Displaying all info about the configuration variables in the admin panel, including their values (except for secrets), if they are changed from their defaults, and which environment variable controls it ![image](/attachments/7c2b5fb4-175c-492e-b99f-58f150713499)
185 KiB
Collaborator

Reworked how instructions are presented (without the dropdown). Made the name matching more lenient and also added some admin view.
Added a global list of files in instructions

image
image
image
image

Reworked how instructions are presented (without the dropdown). Made the name matching more lenient and also added some admin view. Added a global list of files in `instructions` ![image](/attachments/26f5fb12-366f-4c8a-a3d9-cd78fa5e8b5b) ![image](/attachments/1ce6663e-0cb5-421f-ab71-88ef58a18f23) ![image](/attachments/2e420087-4465-4c27-baf9-dbc9a8ed6d5a) ![image](/attachments/e1717031-8b1d-40ca-8f3c-661870b193ab)
Collaborator

Added list of last or random (if using the RANDOM env variable) sets and minifigures on the front page.

image
image

Added list of last or random (if using the `RANDOM` env variable) sets and minifigures on the front page. ![image](/attachments/b29b2872-7e3a-44ac-917d-78af2aa66799) ![image](/attachments/974bb2a2-4a33-417e-ae83-7ab38a1a8082)
293 KiB
438 KiB
Collaborator

Reworked instructions upload, finished lightweight management of instructions.

image
image
image
image
image

Reworked instructions upload, finished lightweight management of instructions. ![image](/attachments/8990b496-5331-492c-a3dd-e26fa1c30df2) ![image](/attachments/b97b6876-98ee-4fac-9781-b231e0a9871c) ![image](/attachments/5ebe4957-8185-4aa9-9d8c-b971bab61711) ![image](/attachments/be8af0d6-1b3b-46d1-b8e8-a7b801ff5e28) ![image](/attachments/d5d6ed01-73af-45f0-8c09-d98afc8f3fee)
Collaborator

Added gracefully handling missing themes.csv and light management of the file

image
image
image
image

Added gracefully handling missing `themes.csv` and light management of the file ![image](/attachments/0a5db495-4982-4b8f-a7bc-6d354836a228) ![image](/attachments/569ccc2d-73bb-4178-8ac0-b1502c0b2c29) ![image](/attachments/7b7a8027-ac40-4ae0-8d60-5b7185356e04) ![image](/attachments/3bf335e9-0c4a-4364-8342-207e3025bb13)

is there an idea of when a version of this planned to be available for bug testing/functionality tests etc?

is there an idea of when a version of this planned to be available for bug testing/functionality tests etc?
Collaborator

is there an idea of when a version of this planned to be available for bug testing/functionality tests etc?

I'm working on this on my free time, but the only task left I have are:

  • The wishlist
  • Build my version into Docker
  • Test drive it with my own data
  • Open a merge request

Hopefully it is accepted because my version is a massive departure from @FrederikBaerentsen original code, even if it should be compatible with existing data, I haven't changed any of the database or filesystem structure. You could say it is a completely different app even if it has the same behavior 🙂

I don't have an ETA but it should be soon™

> is there an idea of when a version of this planned to be available for bug testing/functionality tests etc? I'm working on this on my free time, but the only task left I have are: - The wishlist - Build my version into Docker - Test drive it with my own data - Open a merge request Hopefully it is accepted because my version is a **massive** departure from @FrederikBaerentsen original code, even if it should be compatible with existing data, I haven't changed any of the database or filesystem structure. You could say it is a completely different app even if it has the same behavior 🙂 I don't have an ETA but it should be soon™
Collaborator

Reworked the wishlist and retired sets date

image
image
image
image

Reworked the wishlist and retired sets date ![image](/attachments/8083b889-66c4-4309-9e1d-95384a02821d) ![image](/attachments/86ea96cf-9bd8-4f58-93fa-accf086e2522) ![image](/attachments/f7beff97-04d4-4bd8-aafd-ccd2cc5732b0) ![image](/attachments/ea162652-3fb7-46f4-9359-94d37910b93b)
Author
Owner

is there an idea of when a version of this planned to be available for bug testing/functionality tests etc?

I'm working on this on my free time, but the only task left I have are:

  • The wishlist
  • Build my version into Docker
  • Test drive it with my own data
  • Open a merge request

Hopefully it is accepted because my version is a massive departure from @FrederikBaerentsen original code, even if it should be compatible with existing data, I haven't changed any of the database or filesystem structure. You could say it is a completely different app even if it has the same behavior 🙂

I don't have an ETA but it should be soon™

Sounds great! I am really excited. I am thinking I will retire my code on a separate branch and make your new code the "main code". I'll setup a docker build system, so we don't have to manually build and push this too.

> > is there an idea of when a version of this planned to be available for bug testing/functionality tests etc? > > I'm working on this on my free time, but the only task left I have are: > - The wishlist > - Build my version into Docker > - Test drive it with my own data > - Open a merge request > > Hopefully it is accepted because my version is a **massive** departure from @FrederikBaerentsen original code, even if it should be compatible with existing data, I haven't changed any of the database or filesystem structure. You could say it is a completely different app even if it has the same behavior 🙂 > > I don't have an ETA but it should be soon™ Sounds great! I am really excited. I am thinking I will retire my code on a separate branch and make your new code the "main code". I'll setup a docker build system, so we don't have to manually build and push this too.
Collaborator

is there an idea of when a version of this planned to be available for bug testing/functionality tests etc?

I'm working on this on my free time, but the only task left I have are:

  • The wishlist
  • Build my version into Docker
  • Test drive it with my own data
  • Open a merge request

Hopefully it is accepted because my version is a massive departure from @FrederikBaerentsen original code, even if it should be compatible with existing data, I haven't changed any of the database or filesystem structure. You could say it is a completely different app even if it has the same behavior 🙂

I don't have an ETA but it should be soon™

Sounds great! I am really excited. I am thinking I will retire my code on a separate branch and make your new code the "main code". I'll setup a docker build system, so we don't have to manually build and push this too.

I don't think you need a new branch to keep your old code. You have the git history doing that job 🙂.
My merge request will clean up the old files since I've removed them one by one after re-implementing the code 👍.

> > > is there an idea of when a version of this planned to be available for bug testing/functionality tests etc? > > > > I'm working on this on my free time, but the only task left I have are: > > - The wishlist > > - Build my version into Docker > > - Test drive it with my own data > > - Open a merge request > > > > Hopefully it is accepted because my version is a **massive** departure from @FrederikBaerentsen original code, even if it should be compatible with existing data, I haven't changed any of the database or filesystem structure. You could say it is a completely different app even if it has the same behavior 🙂 > > > > I don't have an ETA but it should be soon™ > > Sounds great! I am really excited. I am thinking I will retire my code on a separate branch and make your new code the "main code". I'll setup a docker build system, so we don't have to manually build and push this too. I don't think you need a new branch to keep your old code. You have the git history doing that job 🙂. My merge request will clean up the old files since I've removed them one by one after re-implementing the code 👍.
Collaborator

So I currently have your version running with a few sets.

image

I've build a new image with my code.

gregoo@docker:~/BrickTracker$ docker build -t localhost/bricktracker:1.0.0 .
[+] Building 59.1s (10/10) FINISHED                                                                                                                                                                                                                     docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                              0.1s
 => => transferring dockerfile: 216B                                                                                                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/python:slim                                                                                                                                                                                                    1.5s
 => [internal] load .dockerignore                                                                                                                                                                                                                                 0.1s
 => => transferring context: 307B                                                                                                                                                                                                                                 0.0s
 => [internal] load build context                                                                                                                                                                                                                                 0.1s
 => => transferring context: 281.85kB                                                                                                                                                                                                                             0.1s
 => [1/5] FROM docker.io/library/python:slim@sha256:23a81be7b258c8f516f7a60e80943cace4350deb8204cf107c7993e343610d47                                                                                                                                             10.1s
 => => resolve docker.io/library/python:slim@sha256:23a81be7b258c8f516f7a60e80943cace4350deb8204cf107c7993e343610d47                                                                                                                                              0.1s
 => => sha256:23a81be7b258c8f516f7a60e80943cace4350deb8204cf107c7993e343610d47 9.12kB / 9.12kB                                                                                                                                                                    0.0s
 => => sha256:f1780cce1ae3da5ccdae9c1ae0259e491bea1d383599062fd7448f30c9e68be2 1.75kB / 1.75kB                                                                                                                                                                    0.0s
 => => sha256:1121ea67ee63d5063e3c990ab8e445d8aef182faedbcbfd75e0e995bb068c7e0 5.03kB / 5.03kB                                                                                                                                                                    0.0s
 => => sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d 28.21MB / 28.21MB                                                                                                                                                                  1.0s
 => => sha256:31fc40b7245b1b5a9c87e53d8a533c3f8bcd41cc4510e92a14a0f41828e77d45 3.51MB / 3.51MB                                                                                                                                                                    0.6s
 => => sha256:b20e3a1a6b0d346f72ea74fcff4ac580894cd2c53b4876161d1f7fea56affce8 12.58MB / 12.58MB                                                                                                                                                                  0.5s
 => => sha256:e81d26d6c5a732f0cff33c82eea14a6cbfa7b497ce0f65b5df2151430b34c33c 249B / 249B                                                                                                                                                                        0.7s
 => => extracting sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d                                                                                                                                                                         5.3s
 => => extracting sha256:31fc40b7245b1b5a9c87e53d8a533c3f8bcd41cc4510e92a14a0f41828e77d45                                                                                                                                                                         0.6s
 => => extracting sha256:b20e3a1a6b0d346f72ea74fcff4ac580894cd2c53b4876161d1f7fea56affce8                                                                                                                                                                         2.4s
 => => extracting sha256:e81d26d6c5a732f0cff33c82eea14a6cbfa7b497ce0f65b5df2151430b34c33c                                                                                                                                                                         0.0s
 => [2/5] WORKDIR /app                                                                                                                                                                                                                                            1.6s
 => [3/5] COPY requirements.txt .                                                                                                                                                                                                                                 0.1s
 => [4/5] RUN pip install -r requirements.txt                                                                                                                                                                                                                    32.6s
 => [5/5] COPY . .                                                                                                                                                                                                                                                0.1s
 => exporting to image                                                                                                                                                                                                                                           12.8s
 => => exporting layers                                                                                                                                                                                                                                          12.8s
 => => writing image sha256:bdb7b261769303c9674c1b60ad5717e11267ebc4173d31f7083de7e4d72304e9                                                                                                                                                                      0.0s
 => => naming to localhost/bricktracker:1.0.0   

Then adjusted my compose for testing

services:
  bricktracker:
    image: localhost/bricktracker:1.0.0
    pull_policy: never

And it seems to work pretty much as a drop-in replacement 🙂

image

The nil and nil_mf files have moved so there a simple fix to apply to put them where they should be

$ wget "https://rebrickable.com/static/img/nil.png"
$ wget "https://rebrickable.com/static/img/nil_mf.jpg"
$ docker container cp "nil.png" "BrickTracker:/app/static/parts/nil.png"
Successfully copied 6.66kB to BrickTracker:/app/static/parts/nil.png
$ docker container cp "nil_mf.jpg" "BrickTracker:/app/static/minifigs/nil_mf.jpg"
Successfully copied 8.7kB to BrickTracker:/app/static/minifigs/nil_mf.jpg

image

I also tortured it to test the Bulk add function

  • The list
    image

  • Midway through
    image

  • Finished
    image

I still have a few quirks and wrong sums to fix and I'll be finished with this first version

So I currently have your version running with a few sets. ![image](/attachments/3f1667aa-0fdc-47c4-b330-1f320922ded7) I've build a new image with my code. ``` gregoo@docker:~/BrickTracker$ docker build -t localhost/bricktracker:1.0.0 . [+] Building 59.1s (10/10) FINISHED docker:default => [internal] load build definition from Dockerfile 0.1s => => transferring dockerfile: 216B 0.0s => [internal] load metadata for docker.io/library/python:slim 1.5s => [internal] load .dockerignore 0.1s => => transferring context: 307B 0.0s => [internal] load build context 0.1s => => transferring context: 281.85kB 0.1s => [1/5] FROM docker.io/library/python:slim@sha256:23a81be7b258c8f516f7a60e80943cace4350deb8204cf107c7993e343610d47 10.1s => => resolve docker.io/library/python:slim@sha256:23a81be7b258c8f516f7a60e80943cace4350deb8204cf107c7993e343610d47 0.1s => => sha256:23a81be7b258c8f516f7a60e80943cace4350deb8204cf107c7993e343610d47 9.12kB / 9.12kB 0.0s => => sha256:f1780cce1ae3da5ccdae9c1ae0259e491bea1d383599062fd7448f30c9e68be2 1.75kB / 1.75kB 0.0s => => sha256:1121ea67ee63d5063e3c990ab8e445d8aef182faedbcbfd75e0e995bb068c7e0 5.03kB / 5.03kB 0.0s => => sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d 28.21MB / 28.21MB 1.0s => => sha256:31fc40b7245b1b5a9c87e53d8a533c3f8bcd41cc4510e92a14a0f41828e77d45 3.51MB / 3.51MB 0.6s => => sha256:b20e3a1a6b0d346f72ea74fcff4ac580894cd2c53b4876161d1f7fea56affce8 12.58MB / 12.58MB 0.5s => => sha256:e81d26d6c5a732f0cff33c82eea14a6cbfa7b497ce0f65b5df2151430b34c33c 249B / 249B 0.7s => => extracting sha256:af302e5c37e9dc1dbe2eadc8f5059d82a914066b541b0d1a6daa91d0cc55057d 5.3s => => extracting sha256:31fc40b7245b1b5a9c87e53d8a533c3f8bcd41cc4510e92a14a0f41828e77d45 0.6s => => extracting sha256:b20e3a1a6b0d346f72ea74fcff4ac580894cd2c53b4876161d1f7fea56affce8 2.4s => => extracting sha256:e81d26d6c5a732f0cff33c82eea14a6cbfa7b497ce0f65b5df2151430b34c33c 0.0s => [2/5] WORKDIR /app 1.6s => [3/5] COPY requirements.txt . 0.1s => [4/5] RUN pip install -r requirements.txt 32.6s => [5/5] COPY . . 0.1s => exporting to image 12.8s => => exporting layers 12.8s => => writing image sha256:bdb7b261769303c9674c1b60ad5717e11267ebc4173d31f7083de7e4d72304e9 0.0s => => naming to localhost/bricktracker:1.0.0 ``` Then adjusted my compose for testing ``` services: bricktracker: image: localhost/bricktracker:1.0.0 pull_policy: never ``` And it seems to work pretty much as a drop-in replacement 🙂 ![image](/attachments/3eebb08f-1612-4f99-8a7c-e847144a5b11) The `nil` and `nil_mf` files have moved so there a simple fix to apply to put them where they should be ``` $ wget "https://rebrickable.com/static/img/nil.png" $ wget "https://rebrickable.com/static/img/nil_mf.jpg" $ docker container cp "nil.png" "BrickTracker:/app/static/parts/nil.png" Successfully copied 6.66kB to BrickTracker:/app/static/parts/nil.png $ docker container cp "nil_mf.jpg" "BrickTracker:/app/static/minifigs/nil_mf.jpg" Successfully copied 8.7kB to BrickTracker:/app/static/minifigs/nil_mf.jpg ``` ![image](/attachments/62ae22bd-ad8e-48e6-bd11-87a8616c6d4b) I also tortured it to test the Bulk add function - The list ![image](/attachments/b0ef1e3a-fcf0-4776-8a59-8ac67bcc88e8) - Midway through ![image](/attachments/6a83184a-c91c-41bf-aa3f-da7212c5ea29) - Finished ![image](/attachments/c51eea89-baa1-4055-92a9-cec5881656ce) I still have a few quirks and wrong sums to fix and I'll be finished with this first version
Collaborator

I detected a bug in your version which inserts 'None' instead of NULL in the missing table for items without an element_id. This leads to "phantom" missing parts. I baked in a web based fix so that users don't have to fiddle with the CLI.

image
image
image

I detected a bug in your version which inserts `'None'` instead of `NULL` in the missing table for items without an `element_id`. This leads to "phantom" missing parts. I baked in a web based fix so that users don't have to fiddle with the CLI. ![image](/attachments/be3ba0f3-7c90-4d2d-9273-672062d5c05a) ![image](/attachments/207a535d-224e-415b-9e26-53819a532dd6) ![image](/attachments/c0ddc1a8-9e1c-4b20-a8b7-e5e4c1464a18)
Collaborator

Also added the default image refresh while I was at it so that no fix is required through the CLI 😂

image
image

Also added the default image refresh while I was at it so that no fix is required through the CLI 😂 ![image](/attachments/6a0beb99-a75c-4ded-b5c8-33c2667f6a6c) ![image](/attachments/de5efad6-a939-48c9-bfad-2baee3297805)
Collaborator

FYI I have submitted merge request #40.
If anyone is eager to test you'll have to clone my fork and build it yourself (if you are brave enough).

FYI I have submitted merge request #40. If anyone is eager to test you'll have to clone my fork and build it yourself (if you are brave enough).
Author
Owner

Tested both local install and using prebuilt image. Made release notes here: https://gitea.baerentsen.space/FrederikBaerentsen/BrickTracker/releases/tag/v1.0.0

All looks good on my test installs and migration is easy. I'll go through the issues this week and close the ones that has been fixed. Great work!

Tested both local install and using prebuilt image. Made release notes here: https://gitea.baerentsen.space/FrederikBaerentsen/BrickTracker/releases/tag/v1.0.0 All looks good on my test installs and migration is easy. I'll go through the issues this week and close the ones that has been fixed. Great work!
Author
Owner

@gregoo Is there a reason why you prefer volumes over local mapping? I always felt like local folders was easier for beginners to use and also to backup.

@gregoo Is there a reason why you prefer volumes over local mapping? I always felt like local folders was easier for beginners to use and also to backup.
Collaborator

My data is stored on a NAS, that's why I'm using volumes + NFS.
Local volumes is simply your data stored in /var/lib/docker, there is nothing fancy about it.

If you have a backup solution, you can simply spawn a container with all the volumes you want to backup mounted, plus the destination where you need to save the data. For the application running inside the container, everything will look local.

On a side note, there is a discrepancy between the tag you applied to your image (v1.0.0) and the one in my compose.yaml file (1.0.0). You should probably drop the v at the beginning if you want to adhere to something like SemVer for the version numbering (https://semver.org/). Otherwise you need to fix the version in the compose file.

My data is stored on a NAS, that's why I'm using volumes + NFS. Local volumes is simply your data stored in `/var/lib/docker`, there is nothing fancy about it. If you have a backup solution, you can simply spawn a container with all the volumes you want to backup mounted, plus the destination where you need to save the data. For the application running inside the container, everything will look local. On a side note, there is a discrepancy between the tag you applied to your image (`v1.0.0`) and the one in my `compose.yaml` file (`1.0.0`). You should probably drop the `v` at the beginning if you want to adhere to something like SemVer for the version numbering (https://semver.org/). Otherwise you need to fix the version in the compose file.
Author
Owner

Alright, I was mainly wondering what would be easiest to have in the sample files. I am thinking of making a Quickstart section of the setup docs with simple copy paste instructions.

Ohh i see. I'll upload a new version without the v asap.

Alright, I was mainly wondering what would be easiest to have in the sample files. I am thinking of making a Quickstart section of the setup docs with simple copy paste instructions. Ohh i see. I'll upload a new version without the `v` asap.
Collaborator

I think we can close this monster thread unless you see something in it that remains to do 🙂

I think we can close this monster thread unless you see something in it that remains to do 🙂
Author
Owner

@gregoo agreed. I was just going through and checking.

@gregoo agreed. I was just going through and checking.
Sign in to join this conversation.
No Milestone
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: FrederikBaerentsen/BrickTracker#8
No description provided.