Merge pull request #395 from aclist/chore/contributing-docs

docs: add contributing docs

CONTRIBUTING.md

@@ -0,0 +1,69 @@
+# Introduction
+
+Thank you for your interest in DZGUI!
+
+This guide goes over development conventions and best practices for contributors.
+If you are a developer, you can skip to the end.
+
+# Requesting help
+
+If you encounter a problem with DZGUI, you can submit tickets on the GitHub
+(issue tracker)[https://github.com/aclist/dztui/issues] under the
+`troubleshooting` tag.
+
+# How can I help the project?
+
+There are several ways to help this project.
+1. Report bugs that you find
+2. Request features that you would like
+3. Contribute features and fixes to the codebase
+4. Contribute documentation to the project
+
+Before making any contribution, please read the `CODE_OF_CONDUCT.md` and act
+accordingly.
+
+## Submitting a ticket
+
+Navigate to the GitHub (issue tracker)[https://github.com/aclist/dztui/issues].
+From there, follow the onscreen prompts. You will be asked questions such as:
+
+- What version are you using?
+- What distribution are you using?
+- What is the issue that you found?
+- How can we reproduce the issue?
+
+You can also attach screenshots, logs, or other data that can help us.
+
+## Requesting a feature
+
+You can also request features via the same issue tracker. It is good practice to
+first search for your idea to see if a similar one has already been posted.
+If not, open a ticket where you describe your feature and its possible benefits.
+
+Please note that this is a community project, so it takes time for us to develop
+features. Putting in a feature request does not mean that it will be implemented,
+but we will do our best to support as many cool ideas as possible.
+
+## Contributing code or documentation
+
+If you would like to assist with an issue on the issue tracker or
+contribute a new change, please follow the guidelines below.
+
+Fork this repository and check out the code up to the `dzgui7` (development) branch.
+
+The following naming conventions apply for PRs:
+- fix/<your-fix> - patch/hotfix branches
+- feat/<your-feature> - feature branches
+- doc/<your-doc-branch> - documentation branches
+- infra/<your-infra-branch> - infrastructure branches
+
+Implement your changes and test them locally. If they work, you may
+open a merge request, and then we review your changes. If everything is OK, it
+will be merged to `dzgui7`. After sufficient changes are consolidated into a new release,
+this branch will be tagged to a certain point in time and a binary release published.
+
+It is recommended to follow
+[Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/), as this
+integrates well with tooling and helps the project review your code.
+
+Please see DEVELOPERS.md for further details.

README.md

@@ -24,13 +24,8 @@ Please refer to the documentation for installation and setup instructions:
 
 Geolocation records from [DB-IP](https://db-ip.com) under [CC 4.0 license](https://creativecommons.org/licenses/by/4.0/)
 
-This tool uses [python-a2s](https://github.com/Yepoleb/python-a2s) and [dayzquery](https://github.com/Yepoleb/dayzquery) as submodules; licenses for these submodules can be found in the LICENSES file
+Executable versions of DZGUI published as release binaries ship with various runtime dependencies and the Python interpreter built in.
-of the project root.
+Users wishing to review the licenses to these components can inspect the `LICENSE` file located in the release tarball.
-
-Both the geolocation records and submodules listed above are not shipped with the source code, but are retrieved and assembled at runtime.
-
-Finally, executable versions of DZGUI shipped as release binaries are thin wrappers around the Python interpreter, and also retrieve and assemble the above dependencies at runtime on the end-user's
-machine, rather than using pre-compiled source code. Users wishing to review these dependencies can inspect the 'pyproject.toml' manifest in the project root.
 
 ## Disclaimer
 

docs/source/dzgui7.rst

@@ -26,7 +26,7 @@ You can install it via the following set of commands:
 .. code:: console
 
    sudo apt update
-   sudo apt install libgirepository-2.0
+   sudo apt install libgirepository-2.0-0
 
 Next steps
 ---------------

dzgui/api/mods.py

@@ -49,6 +49,7 @@ def get_local_mods(workshop_path: Path) -> list[Path]:
     mods = [file for file in workshop_path.iterdir() if file.is_dir()]
     return mods
 
+
 def is_mission(path: Path) -> bool:
     # TODO: parse integrity of other files
     parent = path.parent.name
@@ -57,9 +58,10 @@ def is_mission(path: Path) -> bool:
     file = path / "init.c"
     return file.exists()
 
+
 def tokenize(mod: Path) -> dict[str, Any] | None:
     file = mod.joinpath("meta.cpp")
-    delimiter=r"\s*=\s*"
+    delimiter = r"\s*=\s*"
     modmeta = {}
     try:
         with open(file, "r", encoding="utf-8") as f:
@@ -76,6 +78,7 @@ def tokenize(mod: Path) -> dict[str, Any] | None:
         logger.critical(e)
         return None
 
+
 def get_mod_size(path: Path) -> float:
     s = 0
     for f in path.rglob("*"):
@@ -123,20 +126,27 @@ def _hash(uid: str, use_custom: bool = False) -> str:
     return prefix + md5.hexdigest()[:8]
 
 
-def remove_stale_signatures(config: Path, versions: Path) -> None:
+def remove_stale_signatures(
+    config: Path, versions: Path, ids: list[int] | None = None
+) -> None:
     if versions.is_file() is False:
         logger.warning("Creating new version signatures file")
         versions.touch()
         return
+
     path = lookup(config, Preferences.DEFAULT)
     steam_path = Path(path)
-    ids = get_local_mod_ids(steam_path)
+    if ids is None:
+        local_ids = get_local_mod_ids(steam_path)
+
     with open(versions, "r") as f:
         lines = f.readlines()
-    for line in lines:
+
-        uid = int(line.split(",")[0])
+    if ids is None:
-        if uid not in ids:
+        lines = [line for line in lines if int(line.split(",")[0]) in local_ids]
-            lines.remove(line)
+    else:
+        lines = [line for line in lines if int(line.split(",")[0]) not in ids]
+
     with open(versions, "w") as f:
         for line in lines:
             f.write(line)

dzgui/api/servers.py

@@ -289,13 +289,14 @@ def query_direct(ip: str, qport: int, timeout: float = 3.0) -> dict[str, Any] |
 
 
 def get_details(record: Record) -> Details:
+    # TODO: consolidate try/except statements
     ip = record.ip
     qport = record.qport
     default_str = strings.none_provided
 
     try:
         info = a2s.info((ip, qport))
-    except TimeoutError:
+    except (TimeoutError, a2s.exceptions.BrokenMessageError):
         return Details(None, default_str, False)
     try:
         rules = dayzquery.dayz_rules((ip, qport))

dzgui/api/steam.py

@@ -10,7 +10,6 @@ from shlex import shlex
 from pathlib import Path
 
 from dzgui.init.prereqs import has_steam_client
-from dzgui.api.mods import _hash
 from dzgui.const.constants import (
     APPID_DAYZ,
     APP_NAME,
@@ -58,6 +57,9 @@ def get_steam_paths() -> list[tuple[Path, str]]:
 
 
 def concat_mods(mods: list[str]) -> str:
+    # TODO: circular import workaround
+    from dzgui.api.mods import _hash
+
     hashes = []
     for mod in mods:
         md5sum = _hash(mod)
@@ -170,8 +172,10 @@ def launch_offline(
         "-skipintro",
         f"-name={name}",
         f"-mod={symlinks}",
-        f"-mission={mission}",
     ]
+    if len(mission) > 0:
+        arg = f"-mission={mission}"
+        params.append(arg)
     proc = subprocess.run([*client_args, *params])
     return proc.returncode
 

dzgui/app_init.py

@@ -0,0 +1,159 @@
+import logging
+import os
+import shutil
+from typing import TYPE_CHECKING
+
+
+from dzgui.const.constants import APP_NAME
+from dzgui.const.enum import Preferences
+from dzgui.config.query import lookup
+from dzgui.config.userprefs import UserPrefs
+from dzgui.config.xdg import get_xdg_paths, parse_filepaths
+from dzgui.init.migrate import (
+    has_new_config,
+    copy_state_files,
+    migrate_cols_file,
+)
+from dzgui.init.flock import lock_acquire
+from dzgui.init.prereqs import has_steam_client
+from dzgui.strings import boot
+
+# from dzgui.util.map_count import get_map_count
+from dzgui.util.deck import is_steam_deck, is_game_mode
+from dzgui.util.localize import set_locale
+from dzgui.util.redact import RedactionFilter, REDACTION_PATTERNS
+from dzgui.util.strings import init
+
+from dzgui.views.base import App
+from dzgui.views.dialogs.boot import BootWindow
+from dzgui.views.dialogs.early_alert import EarlyAlertDialog
+from dzgui.views.dialogs.wizard import SetupWizard
+
+if TYPE_CHECKING:
+    from pathlib import Path
+
+logger = logging.getLogger(APP_NAME)
+
+
+# TODO: profile load time
+def make_parents(path: "Path") -> None:
+    path.parent.mkdir(parents=True, exist_ok=True)
+
+
+def setup_logger(log_path: "Path") -> None:
+    # TODO: put in consts?
+    _format = (
+        "%(asctime)s␞%(levelname)s␞%(filename)s::%(funcName)s::%(lineno)s␞%(message)s"
+    )
+
+    fh = logging.FileHandler(log_path)
+    formatter = logging.Formatter(_format)
+    fh.setFormatter(formatter)
+    fh.setLevel(logging.DEBUG)
+
+    _filter = RedactionFilter(patterns=REDACTION_PATTERNS)
+    fh.addFilter(_filter)
+
+    logger.setLevel(logging.DEBUG)
+    logger.addHandler(fh)
+
+
+def copy_bare_configs(config: "Path", resolution: "Path") -> tuple[bool, bool]:
+    # NOTE: temporary workaround for #375
+    conf = "config.json"
+    state = [
+        "dzg.history",
+        "dzg.versions",
+        "dzg.res.json",
+        "dzg.filters.json",
+        "dzg.columns.json",
+        "dzg.notes.json",
+        "ips.csv",
+        ".month",
+    ]
+    config_changed = False
+    state_changed = False
+    if config.parent.exists() is False:
+        new_file = config
+        old_file = config.parent.parent / conf
+        if old_file.is_file():
+            try:
+                make_parents(new_file)
+                shutil.copy(old_file, new_file)
+                config_changed = True
+            except Exception as e:
+                logger.critical(e)
+    if resolution.parent.exists() is False:
+        state_path = resolution.parent
+        for state_file in state:
+            old_file = state_path.parent / state_file
+            if old_file.is_file():
+                try:
+                    new_file = state_path / state_file
+                    make_parents(new_file)
+                    shutil.copy(old_file, new_file)
+                    state_changed = True
+                except Exception as e:
+                    logger.critical(e)
+    return (config_changed, state_changed)
+
+
+def load_gui(version: str, is_debug: bool) -> None:
+    lock = lock_acquire()  # noqa
+
+    set_locale()
+    # NOTE: consider aborting this check if steam deck
+    xdg_paths = get_xdg_paths()
+    XDG = parse_filepaths(xdg_paths)
+
+    copy_bare_configs(XDG.config, XDG.resolution)
+
+    if XDG.resolution.parent.is_dir() is False:
+        make_parents(XDG.resolution)
+
+    if XDG.debug.is_file() is False:
+        make_parents(XDG.debug)
+
+    _is_steam_deck = is_steam_deck()
+    _is_game_mode = is_game_mode() if _is_steam_deck else False
+    if _is_game_mode:
+        # NOTE: this may no longer be necessary on newer versions of SteamOS
+        del os.environ["GTK_IM_MODULE"]
+
+    if has_new_config(XDG.config) is False:
+        migrate_cols_file(XDG.columns)
+        copy_state_files(xdg_paths["XDG_STATE_HOME"])
+        # TODO: add logging inside wizard
+        SetupWizard(_is_steam_deck, XDG.config)
+
+    # NOTE: implies that setup wizard failed or was closed
+    if has_new_config(XDG.config) is False:
+        return
+
+    setup_logger(XDG.debug)
+    with open(XDG.debug, "w") as f:
+        f.truncate(0)
+
+    if _is_steam_deck is False:
+        # TODO: sudo escalation dialog
+        # count = get_map_count()
+        # TODO: move into module
+        if has_steam_client() is False:
+            EarlyAlertDialog(init.requires_steam)
+
+    bootwin = BootWindow(XDG, version)
+    local_coords, latest_release = bootwin.get_results()
+
+    use_miles = lookup(XDG.config, Preferences.DIST)
+    prefs = UserPrefs(
+        is_steam_deck=_is_steam_deck,
+        is_game_mode=_is_game_mode,
+        is_debug=is_debug,
+        coords=local_coords,
+        version=version,
+        paths=XDG,
+        latest_release=latest_release,
+        use_miles=use_miles,
+    )
+    print(boot.all_ok)
+    App(prefs)

dzgui/config/convert.py

@@ -3,6 +3,8 @@ import shlex
 from pathlib import Path
 from typing import Any
 
+from dzgui.const.boilerplate import config_boilerplate
+
 
 """
 Convert legacy dztuirc to config.json
@@ -89,4 +91,8 @@ def rc2json(file: Path) -> str:
     keys["ip_list"] = ips
     keys["use_miles"] = False
     keys["start_tab"] = 0
+    # NOTE: workaround for DZGUI 6 logic (see #375)
+    for key in config_boilerplate.keys():
+        if key not in keys:
+            keys[key] = config_boilerplate[key]
     return json.dumps(keys, indent=2)

dzgui/config/xdg.py

@@ -29,9 +29,9 @@ def is_writeable(path_str: str) -> bool:
     if not path.exists():
         try:
             path.mkdir(parents=True)
+            path.unlink()
         except OSError:
             return False
-        path.unlink()
         return True
 
     try:
@@ -55,11 +55,12 @@ def get_xdg_paths() -> dict:
 
     resolved_paths = {}
     for path in xdg_paths:
-        rp = os.environ.get(path)
+        real_path = os.environ.get(path)
-        if rp is not None and is_writeable(rp):
+        if real_path is not None and is_writeable(real_path):
-            resolved_paths[path] = Path(rp)
+            new_path = Path(real_path)
         else:
-            resolved_paths[path] = xdg_paths[path] / APP_NAME_LOWER
+            new_path = xdg_paths[path]
+        resolved_paths[path] = new_path / APP_NAME_LOWER
     return resolved_paths
 
 

dzgui/const/constants.py

@@ -4,7 +4,8 @@ UDP_PORT = 27016
 VM_FILE = "/proc/sys/vm/max_map_count"
 MIN_COUNT = 1048576
 
-RATE_LIMIT_THRESHOLD = 3
+API_RATE_LIMIT = 1
+CLIENT_RATE_LIMIT = 3
 REQUEST_TIMEOUT = 10
 
 APPNAME_DAYZ = "DayZ"

dzgui/controllers/emitter.py

@@ -81,8 +81,8 @@ class Emitter(GObject.GObject):
     def distcalc_started(self) -> None:
         pass
 
-    @GObject.Signal(flags=GObject.SignalFlags.RUN_LAST, arg_types=(bool,))
+    @GObject.Signal(flags=GObject.SignalFlags.RUN_LAST, arg_types=(bool, int))
-    def mod_page_toggled(self, state: bool) -> None:
+    def mod_page_toggled(self, state: bool, mods: int) -> None:
         pass
 
     @GObject.Signal(flags=GObject.SignalFlags.RUN_LAST, arg_types=(bool,))
@@ -172,3 +172,7 @@ class Emitter(GObject.GObject):
     @GObject.Signal(flags=GObject.SignalFlags.RUN_LAST, arg_types=())
     def invalid_custom_mods(self) -> None:
         pass
+
+    @GObject.Signal(flags=GObject.SignalFlags.RUN_LAST, arg_types=())
+    def all_mods_synched(self) -> None:
+        pass

dzgui/controllers/mc.py

@@ -23,7 +23,7 @@ from dzgui.managers.contextmenu import ContextMenuManager
 from dzgui.managers.notes import NoteManager
 from dzgui.model.servers import ServerModelManager
 from dzgui.util.diag import write_diagnostic
-from dzgui.util.format import format_player_count
+from dzgui.util.format import format_exception, format_player_count
 from dzgui.util.open_links import open_user_workshop, open_workshop_page
 from dzgui.views.dialogs.filepicker import FilePicker
 from dzgui.views.dialogs.generic import ExceptionDialog
@@ -203,7 +203,9 @@ class Controller(GObject.GObject):
                 try:
                     # TODO: where to put config file check
                     self.mediator.grid.notebook.settings.populate_settings()
-                except Exception:
+                except Exception as e:
+                    msg = format_exception(e)
+                    logger.critical(msg)
                     return
             case ButtonType.MODS:
                 self.load_mods()

dzgui/init/libgi.py

@@ -1,11 +1,12 @@
 import platform
 import sys
 
-from pathlib import Path
+from ctypes.util import find_library
 from warnings import deprecated
 
 LIB = "libgirepository-2.0"
 
+
 @deprecated("currently unused")
 def is_debian() -> bool:
     """
@@ -39,12 +40,10 @@ def is_debian() -> bool:
 
 
 def has_libgi() -> bool:
-    parent_dir = Path("/usr/lib64")
+    return bool(find_library("girepository-2.0"))
-    exists = any(LIB in subdir.name for subdir in parent_dir.iterdir())
-    return exists
 
 
-def test_missing_lib() -> bool:
+def test_libgi_missing() -> None:
     msg = f"System is missing the required library '{LIB}'. Install it via your system package manager."
     if has_libgi() is False:
         print(msg)

dzgui/init/proc.py

@@ -1,10 +1,12 @@
 import psutil
 import subprocess
 import shutil
+import logging
 
 from warnings import deprecated
 
 from dzgui.const.constants import (
+    APP_NAME,
     DAYZ_BINARY,
     STEAM_CMD,
     FLATPAK_APPID,
@@ -13,6 +15,10 @@ from dzgui.const.constants import (
     FLATPAK_SANDBOX,
 )
 
+from dzgui.util.format import format_exception
+
+logger = logging.getLogger(APP_NAME)
+
 
 # TODO: move to util.proc
 def is_dayz_running() -> bool:
@@ -22,7 +28,9 @@ def is_dayz_running() -> bool:
     for proc in psutil.process_iter():
         try:
             procs.append(proc.cmdline())
-        except Exception:
+        except Exception as e:
+            msg = format_exception(e)
+            logger.warning(msg)
             continue
     return any(substring in item for sublist in procs for item in sublist)
 

dzgui/main.py

@@ -1,40 +1,11 @@
 import argparse
-import logging
-import os
 import sys
 import warnings
 
-from typing import TYPE_CHECKING
-
 from dzgui.const.constants import APP_NAME
-from dzgui.const.enum import Preferences
+from dzgui.init.libgi import test_libgi_missing
-from dzgui.config.query import lookup
-from dzgui.config.userprefs import UserPrefs
-from dzgui.config.xdg import get_xdg_paths, parse_filepaths
-from dzgui.init.flock import lock_acquire
-from dzgui.init.migrate import (
-    has_new_config,
-    copy_state_files,
-    migrate_cols_file,
-)
 from dzgui.init.prefix import get_version
-from dzgui.init.prereqs import has_steam_client
+from dzgui.util.strings import flags
-from dzgui.strings import boot
-
-# from dzgui.util.map_count import get_map_count
-from dzgui.util.deck import is_steam_deck, is_game_mode
-from dzgui.util.localize import set_locale
-from dzgui.util.strings import init, flags
-
-from dzgui.views.base import App
-from dzgui.views.dialogs.boot import BootWindow
-from dzgui.views.dialogs.early_alert import EarlyAlertDialog
-from dzgui.views.dialogs.wizard import SetupWizard
-
-if TYPE_CHECKING:
-    from pathlib import Path
-
-logger = logging.getLogger(APP_NAME)
 
 parser = argparse.ArgumentParser(description=flags.description)
 parser.add_argument("-v", "--version", action="store_true", help=flags.version)
@@ -43,11 +14,6 @@ parser.add_argument("-d", "--debug", action="store_true", help=flags.debug)
 args = parser.parse_args()
 
 
-# TODO: profile load time
-def make_parents(path: "Path") -> None:
-    path.parent.mkdir(parents=True, exist_ok=True)
-
-
 def uninstall() -> None:
     # TODO: uninstall data files (-u)
     # -u removes state, log, freedesktop
@@ -57,21 +23,7 @@ def uninstall() -> None:
     pass
 
 
-def setup_logger(log_path: "Path") -> None:
-    _format = (
-        "%(asctime)s␞%(levelname)s␞%(filename)s::%(funcName)s::%(lineno)s␞%(message)s"
-    )
-    fh = logging.FileHandler(log_path)
-    formatter = logging.Formatter(_format)
-    fh.setFormatter(formatter)
-    fh.setLevel(logging.DEBUG)
-    logger.setLevel(logging.DEBUG)
-    logger.addHandler(fh)
-
-
 def main() -> None:
-    lock = lock_acquire()  # noqa
-
     if args.version is True:
         print(get_version())
         sys.exit(0)
@@ -82,60 +34,11 @@ def main() -> None:
         warnings.filterwarnings("default", category=DeprecationWarning)
 
     version = get_version()
-    set_locale()
 
     print(f"{APP_NAME} {version}")
+    test_libgi_missing()
 
-    # NOTE: consider aborting this check if steam deck
+    # NOTE: only import ligbirepository modules after the above check
-    xdg_paths = get_xdg_paths()
+    from dzgui.app_init import load_gui
-    XDG = parse_filepaths(xdg_paths)
 
-    if XDG.resolution.parent.is_dir() is False:
+    load_gui(version, args.debug)
-        make_parents(XDG.resolution)
-
-    if XDG.debug.is_file() is False:
-        make_parents(XDG.debug)
-
-    _is_steam_deck = is_steam_deck()
-    _is_game_mode = is_game_mode() if _is_steam_deck else False
-    if _is_game_mode:
-        # NOTE: this may no longer be necessary on newer versions of SteamOS
-        del os.environ["GTK_IM_MODULE"]
-
-    if has_new_config(XDG.config) is False:
-        migrate_cols_file(XDG.columns)
-        copy_state_files(xdg_paths["XDG_STATE_HOME"])
-        # TODO: add logging inside wizard
-        SetupWizard(_is_steam_deck, XDG.config)
-
-    # NOTE: implies that setup wizard failed or was closed
-    if has_new_config(XDG.config) is False:
-        return
-
-    setup_logger(XDG.debug)
-    with open(XDG.debug, "w") as f:
-        f.truncate(0)
-
-    if _is_steam_deck is False:
-        # TODO: sudo escalation dialog
-        # count = get_map_count()
-        # TODO: move into module
-        if has_steam_client() is False:
-            EarlyAlertDialog(init.requires_steam)
-
-    bootwin = BootWindow(XDG, version)
-    local_coords, latest_release = bootwin.get_results()
-
-    use_miles = lookup(XDG.config, Preferences.DIST)
-    prefs = UserPrefs(
-        is_steam_deck=_is_steam_deck,
-        is_game_mode=_is_game_mode,
-        is_debug=args.debug,
-        coords=local_coords,
-        version=version,
-        paths=XDG,
-        latest_release=latest_release,
-        use_miles=use_miles,
-    )
-    print(boot.all_ok)
-    App(prefs)

dzgui/managers/connection.py

@@ -25,17 +25,21 @@ from dzgui.api.mods import (
     update_signatures,
 )
 from dzgui.const.constants import (
+    API_RATE_LIMIT,
     APP_NAME,
     APPID_DAYZ,
     APPID_DAYZ_EXP,
     APPNAME_DAYZ,
     APPNAME_DAYZ_EXP_HUMAN,
-    RATE_LIMIT_THRESHOLD,
 )
 from dzgui.const.enum import NotebookPage, Preferences
 from dzgui.init.proc import is_dayz_running, is_steam_running
 from dzgui.managers.threading import call_on_thread, StoredFunc, ThreadingManager
-from dzgui.strings.dialogs import waiting_for_launch, waiting_for_mods
+from dzgui.strings.dialogs import (
+    waiting_for_launch,
+    waiting_for_mods,
+    waiting_for_directories,
+)
 from dzgui.strings.server_mods import checkmark, resync
 from dzgui.util.format import format_mib
 from dzgui.util.strings import dialog, server_timeout
@@ -144,21 +148,14 @@ class ConnectionManager:
         remote_mods: list[list[str]] = []
         if res.is_modded():
             try:
-                remote_mods = self._query_modlist(record)
+                remote_mods, missing_mods = self._query_modlist(record)
+                self.missing_mods = missing_mods
                 self.remote_mod_ids = [mod[1] for mod in remote_mods]
             except Exception as e:
                 logger.warning(e)
                 self.thread_man.set_cleanup_func(failure_func, destroy_first=True)
                 return
 
-            hashes = get_remote_signatures(self.remote_mod_ids)
-            version_file = prefs.paths.version
-            self.missing_mods = get_needs_update(version_file, hashes)
-
-            for mod in remote_mods:
-                if any(mod[1] in tuple for tuple in self.missing_mods):
-                    mod[2] = resync
-
             if local_version is not None:
                 pefile_path = PeFile.get_pefile_path(steam_path, info.game_id)
                 total, used, free = shutil.disk_usage(pefile_path)
@@ -222,10 +219,13 @@ class ConnectionManager:
             StoredFunc(self._present_details_dialog, details), destroy_first=True
         )
 
-    def _query_modlist(self, record: Servers.Record) -> list[list[str]]:
+    def _query_modlist(
+        self, record: Servers.Record
+    ) -> tuple[list[list[str]], list[tuple[str, str, int, int]]]:
         mods = Servers.get_rules(record)
         steam_path = self.controller.query_config(Preferences.DEFAULT)
         local = get_local_mod_ids(steam_path)
+
         alpha_mods: list[list[str]] = [
             [
                 mod.name,
@@ -235,12 +235,23 @@ class ConnectionManager:
             for mod in mods
         ]
         alpha_mods.sort(key=lambda x: x[0])
-        return alpha_mods
+
+        prefs = self.controller.get_prefs()
+        version_file = prefs.paths.version
+
+        remote_mod_ids = [mod[1] for mod in alpha_mods]
+        hashes = get_remote_signatures(remote_mod_ids)
+        missing_mods = get_needs_update(version_file, hashes)
+        for mod in alpha_mods:
+            if any(mod[1] in tuple for tuple in missing_mods):
+                mod[2] = resync
+
+        return alpha_mods, missing_mods
 
     @call_on_thread(dialog.querying)
     def query_modlist_and_present(self, record: Servers.Record) -> None:
         try:
-            mods = self._query_modlist(record)
+            mods, missing_mods = self._query_modlist(record)
         except Exception:
             self.thread_man.set_cleanup_func(
                 StoredFunc(self._server_timeout), destroy_first=True
@@ -276,7 +287,6 @@ class ConnectionManager:
             self.thread_man.set_cleanup_func(func)
             return
 
-        self.thread_man.show_cancel(False)
         self.thread_man.update_dialog(waiting_for_launch)
         while True:
             # FIXME: cancel should not be visible here per setting above
@@ -303,8 +313,9 @@ class ConnectionManager:
             if self.controller.is_cancel_pending():
                 return
             subscribe(key, int(mod))
-            time.sleep(RATE_LIMIT_THRESHOLD)
+            time.sleep(API_RATE_LIMIT)
 
+        self.thread_man.update_dialog(waiting_for_directories)
         for title, mod, stamp, size in self.missing_mods:
             mod_path = self.workshop / mod
 
@@ -322,8 +333,12 @@ class ConnectionManager:
                 time.sleep(1)
 
         update_signatures(self.missing_mods, prefs.paths.version)
-        # TODO: just push steam path directly
         rebuild_symlinks(prefs.paths.config)
+
+        # NOTE: update table status in main loop
+        self.thread_man.update_emitter("all_mods_synched")
+
+        # TODO: just push steam path directly
         self._connect_steam(menu_only)
 
     @call_on_thread(waiting_for_mods, show_cancel=True)

dzgui/managers/filter.py

@@ -80,6 +80,7 @@ class FilterManager:
         if maps is None:
             return
         if len(maps) < 1:
+            self.reinit_map_store()
             return
 
         self.reinit_map_store()

dzgui/managers/mods.py

@@ -12,10 +12,10 @@ from dzgui.api.mods import (
     remove_stale_signatures,
 )
 from dzgui.const.constants import (
+    API_RATE_LIMIT,
     APP_NAME,
     APPID_DAYZ,
     APPID_DAYZ_EXP,
-    RATE_LIMIT_THRESHOLD,
 )
 from dzgui.const.enum import Preferences
 from dzgui.managers.threading import call_on_thread, StoredFunc, ThreadingManager
@@ -23,6 +23,7 @@ from dzgui.model.model_factory import FastInsertListStore, ModelFactory
 from dzgui.strings import dialogs
 from dzgui.util.format import format_mods
 from dzgui.util.strings import server_timeout
+from dzgui.util.symlink import rebuild_symlinks
 from dzgui.views.dialogs.generic import ExceptionDialog
 
 
@@ -66,9 +67,13 @@ class ModManager:
             self.thread_man.set_cleanup_func(func)
             return
 
+        prefs = self.controller.get_prefs()
+        rebuild_symlinks(prefs.paths.config)
+
         self.store = ModelFactory().make_mod_store()
         self.store.extend(mods)
         func = StoredFunc(self._on_mods_loaded)
+
         self.thread_man.set_cleanup_func(func)
 
     def set_store(self, store: "FastInsertListStore") -> None:
@@ -93,6 +98,7 @@ class ModManager:
                 continue
             mod, _iter = res
             mods.append((mod, _iter))
+        self.thread_man.set_job_count(len(mods))
         self.unsub_all_mods(mods)
 
     def get_mod_from_tree_path(
@@ -109,9 +115,10 @@ class ModManager:
     def unsub_all_mods(self, mods: list[tuple[str, Gtk.TreeIter]]) -> None:
         for mod, _iter in mods:
             self.unsub_atomic_mod(mod)
+            self.thread_man.increment_dialog()
+            time.sleep(API_RATE_LIMIT)
 
-        iters = [_iter for mod, _iter in mods]
+        func = StoredFunc(self._on_mods_unsubbed, mods)
-        func = StoredFunc(self._on_mods_unsubbed, iters)
         self.thread_man.set_cleanup_func(func)
 
     def unsub_atomic_mod(self, mod: str) -> None:
@@ -136,20 +143,23 @@ class ModManager:
             symlink.unlink()
         except PeFile.AppNotInstalledError:
             pass
-        time.sleep(RATE_LIMIT_THRESHOLD)
+        time.sleep(API_RATE_LIMIT)
 
-    def _on_mods_unsubbed(self, iters: list[Gtk.TreeIter]) -> None:
+    def _on_mods_unsubbed(self, mod_iter: list[tuple[str, Gtk.TreeIter]]) -> None:
+        mods = [int(mod) for mod, _iter in mod_iter]
+        iters = [_iter for mod, _iter in mod_iter]
         if self.store is None:
             return
         for _iter in iters:
             self.store.remove(_iter)
-        remove_stale_signatures(self.prefs.paths.config, self.prefs.paths.version)
+        # TODO: process config path in called function
+        remove_stale_signatures(self.prefs.paths.config, self.prefs.paths.version, mods)
         model = self.treeview.get_model()
         if model is None:
             return
-        mods = len(model)
+        total_mods = len(model)
         msg = self.format_mod_statusbar()
-        self.emitter.emit("mods_updated", msg, mods)
+        self.emitter.emit("mods_updated", msg, total_mods)
 
     def uncolorize_mods(self) -> None:
         model = self.treeview.get_model()

dzgui/managers/threading.py

@@ -81,6 +81,11 @@ class ThreadingManager:
     def set_job_count(self, jobs: int) -> None:
         self.jobs = jobs
 
+    def update_emitter(self, signal: str) -> None:
+        if self.controller:
+            emitter = self.controller.get_emitter()
+        GLib.idle_add(lambda: emitter.emit(signal))
+
     def update_dialog(self, msg: str) -> None:
         GLib.idle_add(lambda: self.wait_dialog.update_text(msg))
 

dzgui/model/model_factory.py

@@ -2,7 +2,6 @@ from dataclasses import dataclass
 from typing import Any, Sequence, TYPE_CHECKING
 
 from dzgui.const.enum import HELP_MENU_ROWS
-from dzgui.util.redact import redact_log
 from dzgui.util.strings import delimiter
 
 import gi
@@ -134,9 +133,7 @@ class ModelFactory:
         with open(path, "r") as f:
             lines = [line.split(delimiter) for line in f.read().splitlines()]
             for record in lines:
-                # NOTE: strips PII and API keys
+                store.append(record)
-                clean = redact_log(record)
-                store.append(clean)
         return store
 
     def new_model_from_class(self, cls: type) -> FastInsertListStore:

dzgui/model/servers.py

@@ -395,8 +395,10 @@ class ServerModelManager:
         self.first_iteration = False
 
     def _cleanup_on_success(self) -> None:
-        proxy = self._get_proxy_man().get_proxy_model()
+        proxy = self.proxy_man.get_proxy_model()
         self.tv.set_model(proxy)
+        if self.preserve_on_fail is True:
+            self.proxy_man.wipe_cache()
 
         self.emitter.emit("servers_loaded", self.enum)
         if self.first_iteration:

dzgui/strings/dialogs.py

@@ -1,5 +1,6 @@
 waiting_for_launch = "Waiting for DayZ to launch"
-waiting_for_mods = "Waiting for Steam to update mods"
+waiting_for_mods = "Queueing up mods (step 1/2)"
+waiting_for_directories = "Steam is staging mods (step 2/2)"
 
 fetching_update = "Fetching update"
 failed_to_update = "Failed to update DZGUI executable"

dzgui/strings/preconnect.py

@@ -10,6 +10,7 @@ warnings = "Warnings"
 errors = "Errors"
 mods = "Mods"
 total_mods = "Total mods: "
+up_to_date = "All mods are up to date."
 all_updated = "All mods updated."
 "If you recently installed {build} or moved it to a different drive, "
 "restart Steam to allow these changes to synchronize, then try again."

dzgui/util/diag.py

@@ -1,3 +1,4 @@
+import logging
 import os
 import platform
 
@@ -9,7 +10,9 @@ from dzgui.const.constants import APP_NAME
 from dzgui.const.enum import Preferences
 from dzgui.config.query import lookup
 from dzgui.init.prefix import get_version
-from dzgui.util.redact import redact
+from dzgui.util.redact import redact_home
+
+logger = logging.getLogger(APP_NAME)
 
 
 def get_cpu_model() -> str:
@@ -41,12 +44,16 @@ def write_diagnostic(config: Path, outfile: Path) -> None:
     # TODO: test availability on other distros
     date = datetime.now().isoformat()
 
-    distro = platform.freedesktop_os_release()["ID_LIKE"]
+    try:
+        distro = platform.freedesktop_os_release()["ID_LIKE"]
+    except Exception as e:
+        logger.warn(e)
+        distro = "Unknown"
+
     kernel = os.uname().release
     cpu = get_cpu_model()
     version = get_version()
 
-    branch = lookup(config, Preferences.BRANCH)
     debug = lookup(config, Preferences.DEBUG)
     install = lookup(config, Preferences.INSTALL)
     default = lookup(config, Preferences.DEFAULT)
@@ -54,8 +61,8 @@ def write_diagnostic(config: Path, outfile: Path) -> None:
     steam_path = Path(default)
     workshop_path = get_local_mod_path(steam_path)
 
-    steam_redacted = redact(default)
+    steam_redacted = redact_home(default)
-    workshop_redacted = redact(str(workshop_path))
+    workshop_redacted = redact_home(str(workshop_path))
 
     mods = get_local_mod_ids(steam_path)
     mods_pretty = print_mods(mods)
@@ -65,7 +72,7 @@ def write_diagnostic(config: Path, outfile: Path) -> None:
 
     # FIXME: extraneous newlines in lists of mods
     template = f"""\
-    {APP_NAME} version {version} ({branch})
+    {APP_NAME} version {version}
     Date: {date}
     ===============================
     Distribution: {distro}

dzgui/util/redact.py

@@ -1,20 +1,29 @@
+import logging
 import re
+from typing import Literal
 
-def redact(text: str) -> str:
+api_filter = r"(.*&key=)([^&]*)(.*)"
-    r = r"(/home/)([^/])*"
+home_filter = r"(/home/)([^\s'\/]*)(.*)"
-    cleaned = re.sub(r, r"/home/REDACTED", text)
+REDACTED = r"\1REDACTED\3"
+REDACTION_PATTERNS = [api_filter, home_filter]
+
+
+def redact_home(text: str) -> str:
+    pat = re.compile(home_filter)
+    cleaned = pat.sub(REDACTED, text)
     return cleaned
 
-def redact_log(record: list) -> list[str]:
+
-    """
+class RedactionFilter(logging.Filter):
-    requests library includes Steam API key in URL params
+    def __init__(self, patterns: list[str] | None = None) -> None:
-    """
+        super().__init__()
-    clean = []
+        self._patterns = [re.compile(pat) for pat in (patterns or [])]
-    for item in record:
+
-        if "&key=" in item:
+    def filter(self, record: logging.LogRecord) -> Literal[True]:
-            pat = r"(.*&key=)(\S+)(.*)"
+        for pattern in self._patterns:
-            scrubbed = re.sub(pat, r"\1REDACTED\3", item)
+            try:
-            clean.append(scrubbed)
+                record.msg = pattern.sub(REDACTED, record.msg)
-        else:
+            except TypeError:
-            clean.append(item)
+                exception_text = f"{type(record.msg).__name__}: {record.msg}"
-    return clean
+                record.msg = pattern.sub(REDACTED, exception_text)
+        return True

dzgui/util/strings.py

@@ -130,7 +130,6 @@ servers = {
     "Enter/space/double-click": "connect to server",
     "Right-click/Ctrl-l": "additional context menus",
     "Ctrl-r": "refresh players",
-    "Ctrl-p": "refresh ping",
     "Ctrl-f": "jump to keyword search field",
     "Ctrl-m": "jump to maps field",
     "Ctrl-d": "select default LAN port",
@@ -148,8 +147,8 @@ servers = {
 vim = {
     "j": "Move down a row/scroll up",
     "k": "Move up a row/scroll down",
-    "l": "Jump to main area from sidebar",
+    "h": "Jump to main area from sidebar",
-    "h": "Jump to sidebar from main area",
+    "l": "Jump to sidebar from main area",
     "g": "Jump to first row/top of page",
     "G": "Jump to last row/bottom of page",
 }

dzgui/views/components/buttons.py

@@ -141,6 +141,8 @@ class RefreshButton(IconTextButton):
 
     def _on_refresh_clicked(self, button: Self) -> None:
         """Spawned in a thread"""
+        if self.controller.get_active_treeview().is_loaded() is False:
+            return
         self.is_clicked = True
         self.controller.refresh_tree()
         # TODO: get server tab enum

dzgui/views/components/mod_panel.py

@@ -80,14 +80,18 @@ class ModSelectionPanel(Gtk.Box):
         self.highlight_stale_button.set_sensitive(True)
 
     def _on_mods_updated(self, emitter: "Emitter", msg: str, mods: int) -> None:
-        if mods < 1:
+        self._toggle_panel_sensitivity(mods)
-            self.main_panel.set_sensitive(False)
+
-            self.stale_panel.set_sensitive(False)
+    def _toggle_panel_sensitivity(self, mods: int) -> None:
+        state = bool(mods)
+        for el in self.main_panel, self.stale_panel:
+            el.set_sensitive(state)
 
     def _on_mods_highlighted(self, emitter: "Emitter") -> None:
         self.swap_sensitive(True)
 
-    def _on_mod_page_toggled(self, emitter: "Emitter", state: bool) -> None:
+    def _on_mod_page_toggled(self, emitter: "Emitter", state: bool, mods: int) -> None:
+        self._toggle_panel_sensitivity(mods)
         self.set_visible(state)
 
     def swap_sensitive(self, state: bool) -> None:

dzgui/views/pages/mods.py

@@ -64,10 +64,15 @@ class Mods(Gtk.Box):
         self.controller.open_offline()
 
     def _on_unmap(self, widget: Self) -> None:
-        self.emitter.emit("mod_page_toggled", False)
+        self.emitter.emit("mod_page_toggled", False, 0)
 
     def _on_map(self, widget: Self) -> None:
-        self.emitter.emit("mod_page_toggled", True)
+        model = self.tree.get_model()
+        if model is None:
+            mods = 0
+        else:
+            mods = len(model)
+        self.emitter.emit("mod_page_toggled", True, mods)
         # TODO: delegation
         # NOTE: handles going back from offline mods page
         # more generic cache restoration method

dzgui/views/pages/options.py

@@ -288,6 +288,7 @@ class Options(Gtk.Box):
 
     def _on_api_change_failed(self, emitter: "Emitter") -> None:
         self.old_entry.set_text(self.old_text)
+        # TODO: use popover
         dialog = ExceptionDialog(self.controller, errors.api_validation_error)
         dialog.run()
 
@@ -392,7 +393,7 @@ class Options(Gtk.Box):
         if prefs.paths.config.is_file() is False:
             dialog = ExceptionDialog(self.controller, strings.config_not_found)
             dialog.run()
-            raise Exception
+            raise OSError(f"Config file '{prefs.paths.config}' not found")
 
         config = query.get_config(prefs.paths.config)
 

dzgui/views/pages/preconnect.py

@@ -23,6 +23,7 @@ from gi.repository import Gdk, Gtk  # noqa E402
 if TYPE_CHECKING:
     from dzgui.managers.connection import Prerequisites
     from dzgui.controllers.mc import Controller
+    from dzgui.controllers.emitter import Emitter
 
 
 @dataclass
@@ -191,9 +192,14 @@ class PreConnectionAssistant(Gtk.Box):
         self.add(self.scrolled_box)
         self.add(self.button_box)
 
+        emitter = self.controller.get_emitter()
+        emitter.connect("all_mods_synched", self._on_mods_synched)
         self.connect("key-press-event", self._on_keypress)
         self.connect("map", self._on_map)
 
+    def _on_mods_synched(self, emitter: "Emitter") -> None:
+        self.tree.mark_mods_synched()
+
     def _on_map(self, widget: Self) -> None:
         widgets = (
             self.tree_frame,
@@ -297,16 +303,16 @@ class PreConnectionAssistant(Gtk.Box):
             self._hide_mod_area()
         else:
             self._show_mod_area()
-            msg = "All mods are up to date."
+            msg = preconnect.up_to_date
             self.mod_count.set_text(msg)
 
-        if prereqs.required_space == 0:
+        if prereqs.required_space != 0:
-            self.ok.set_label(preconnect.connect)
-
             pretty = number(prereqs.required_space)
             suffix = f" Need to download {pretty} MiB of mod updates."
             prefix = preconnect.total_mods
             self.mod_count.set_text(f"{prefix}{str(total_mods)}.{suffix}")
+        else:
+            self.ok.set_label(preconnect.connect)
 
         if prereqs.is_last_server:
             self.connect_last.show()

dzgui/views/trees/tree_server_mods.py

@@ -48,6 +48,10 @@ class ServerModTreeView(ContextMixin, TreeView):  # type: ignore
                 case _:
                     pass
 
+    def mark_mods_synched(self) -> None:
+        for row in self.mod_store:
+            row[2] = server_mods.checkmark
+
     # TODO: could be problematic if user downloads mods out of band
     def _on_row_activated(
         self,

pyproject.toml

@@ -4,7 +4,7 @@ description = "DayZ server browser and mod manager for Linux"
 authors = [
     {name = "aclist"}
 ]
-version = "7.0.0b13"
+version = "7.0.0b15"
 license = "GPL-3.0-or-later"
 license-files = ["LICENSE"]
 readme = "README.md"
@@ -107,8 +107,10 @@ config-settings-package = { pygobject-stubs = { config = "Gtk3,Gdk3,Soup2" } }
 [tool.pytest.ini_options]
 markers = [
     "config: config file keys/values",
+    "mods: tests mod metadata/link creation",
     "pefile: validate PE files",
     "post_install: requires a completed installation",
+    "redact: log redaction mechanisms",
     "slow: long-running tests",
     "webtest: checks remote endpoints"
 ]

tests/fixtures/cpp/test.py

@@ -1,3 +0,0 @@
-from dzgui.api.mods import tokenize
-
-print(tokenize("meta4.cpp"))

tests/fixtures/dzg.versions

@@ -0,0 +1,8 @@
+1559212036,1771519119
+1564026768,1770917948
+2545327648,1780174455
+2276010135,1756744083
+1654462998,1780501511
+3410710885,1752864732
+2918418331,1780413260
+3739934289,1780855136

tests/fixtures/dztuirc_3

@@ -0,0 +1,42 @@
+#Path to DayZ installation
+steam_path="/tmp"
+
+#Battlemetrics API key
+api_key=""
+
+#Favorited server IP:PORT array
+ip_list=(
+)
+
+#Favorite server to fast-connect to (limit one)
+fav_server=""
+
+#Favorite server label (human readable)
+fav_label=""
+
+#Custom player name (optional, required by some servers)
+name=""
+
+#Set to 1 to perform dry-run and print launch options
+debug="1"
+
+#Toggle stable/testing branch
+branch="testing"
+
+#Steam API key
+steam_api=""
+
+#Auto-install mods
+auto_install="2"
+
+#Automod staging directory
+staging_dir="/tmp"
+
+#Path to default Steam client
+default_steam_path="/tmp"
+
+#Preferred Steam launch command (for Flatpak support)
+preferred_client="steam"
+
+#DZGUI source path
+src_path=""

tests/test_bare_conf_files.py

@@ -0,0 +1,85 @@
+import pytest
+import tempfile
+import os
+
+from dzgui.app_init import copy_bare_configs
+from dzgui.config.xdg import get_xdg_paths, parse_filepaths
+
+
+CONF_STRING = "DZGUI_CONF\n"
+STATE_STRING = "DZGUI_STATE\n"
+
+
+@pytest.fixture
+def state_files():
+    state = [
+        "dzg.history",
+        "dzg.versions",
+        "dzg.res.json",
+        "dzg.filters.json",
+        "dzg.columns.json",
+        "dzg.notes.json",
+        "ips.csv",
+        ".month",
+    ]
+    return state
+
+
+@pytest.fixture
+def xdg_paths(monkeypatch):
+    routes = {
+        "XDG_CONFIG_HOME": "",
+        "XDG_STATE_HOME": "",
+        "XDG_DATA_HOME": "",
+        "XDG_CACHE_HOME": "",
+    }
+    for route in routes:
+        tmp = tempfile.TemporaryDirectory(delete=False)
+        routes[route] = tmp.name
+    for k, v in routes.items():
+        monkeypatch.setenv(k, v)
+    env = get_xdg_paths()
+    return parse_filepaths(env)
+
+
+@pytest.mark.mods
+def test_config_file_import(xdg_paths, state_files):
+
+    # NOTE: write bare files in root
+    tmp_conf = xdg_paths.config.parent.parent
+    tmp_conf_file = tmp_conf / xdg_paths.config.name
+    tmp_conf_file.write_text(CONF_STRING)
+
+    tmp_state = xdg_paths.resolution.parent.parent
+    for file in state_files:
+        tmp_state.joinpath(file).write_text(STATE_STRING)
+    # NOTE: function should move files into "dzgui" subdirectory
+    config_changed, state_changed = copy_bare_configs(
+        xdg_paths.config, xdg_paths.resolution
+    )
+
+    assert xdg_paths.config.read_text() == CONF_STRING
+    for file in state_files:
+        expected = xdg_paths.resolution.parent.joinpath(file)
+        assert expected.read_text() == STATE_STRING
+
+    assert config_changed is True
+    assert state_changed is True
+
+
+@pytest.mark.mods
+def test_config_file_no_import(xdg_paths, state_files):
+    for subdir in xdg_paths.config.parent, xdg_paths.resolution.parent:
+        subdir.mkdir()
+
+    tmp_conf_file = xdg_paths.config
+    tmp_conf_file.write_text(CONF_STRING)
+
+    for file in state_files:
+        xdg_paths.resolution.parent.joinpath(file).write_text(STATE_STRING)
+
+    config_changed, state_changed = copy_bare_configs(
+        xdg_paths.config, xdg_paths.resolution
+    )
+    assert config_changed is False
+    assert state_changed is False

tests/test_config.py

@@ -1,6 +1,7 @@
 import json
 import pytest
 
+from dzgui.const.boilerplate import config_boilerplate
 from dzgui.config.query import get_config
 from dzgui.config.xdg import get_xdg_paths, parse_filepaths
 from dzgui.config import convert
@@ -11,7 +12,11 @@ pytestmark = pytest.mark.config
 
 @pytest.fixture
 def legacy_config():
-    return fixture_path("dztuirc_one")
+    return fixture_path("dztuirc_1")
+
+@pytest.fixture
+def unset_values():
+    return fixture_path("dztuirc_3")
 
 
 @pytest.fixture
@@ -60,8 +65,8 @@ def test_contains_invalid_values(keys, config):
 @pytest.mark.parametrize(
     "fixture, expect",
     [
-        ("dztuirc_one", (False, True, True)),
+        ("dztuirc_1", (False, True, True)),
-        ("dztuirc_two", (False, False, False)),
+        ("dztuirc_2", (False, False, False)),
     ],
 )
 def test_bool_conversion(fixture, expect):
@@ -86,5 +91,7 @@ def test_key_conversion(legacy_config):
     for key in keys:
         assert key not in j
 
-
+def test_missing_values(unset_values):
-# TODO: test that when a config file is created from scratch, it contains all values
+    j = convert.rc2json(unset_values)
+    j = json.loads(j)
+    assert j.keys() == config_boilerplate.keys()

tests/test_log_redaction.py

@@ -0,0 +1,54 @@
+import logging
+import pytest
+
+from dzgui.util.redact import RedactionFilter, REDACTION_PATTERNS
+
+
+class RecordsListHandler(logging.Handler):
+    def __init__(self) -> None:
+        super().__init__()
+        self.records_list = []
+
+    def emit(self, record: logging.LogRecord) -> None:
+        self.records_list.append(record)
+
+    def pop(self) -> None:
+        return self.records_list[-1].msg
+
+
+@pytest.mark.redact
+@pytest.mark.parametrize(
+    "log_error, expect",
+    [
+        ("/home/SENSITIVE_USERNAME/subdir", "/home/REDACTED/subdir"),
+        (
+            "https://url.com/?api&key=SENSITIVE_KEY&results=10",
+            "https://url.com/?api&key=REDACTED&results=10",
+        ),
+        (
+            "https://url.com/?api&key=SENSITIVE_KEY",
+            "https://url.com/?api&key=REDACTED",
+        ),
+        (
+            "Error in directory: '/home/SENSITIVE_USERNAME/'",
+            "Error in directory: '/home/REDACTED/'",
+        ),
+        (
+            "Error in directory: '/home/SENSITIVE_USERNAME'",
+            "Error in directory: '/home/REDACTED'",
+        ),
+    ],
+)
+def test_log_redaction(log_error: str, expect: str) -> None:
+    logger = logging.getLogger("TEST")
+
+    handler = RecordsListHandler()
+    logger.addHandler(handler)
+    logger.setLevel(logging.DEBUG)
+
+    _filter = RedactionFilter(patterns=REDACTION_PATTERNS)
+    logger.addFilter(_filter)
+
+    logger.critical(log_error)
+    redacted = handler.pop()
+    assert expect == redacted

tests/test_mod_signatures.py

@@ -0,0 +1,52 @@
+import pytest
+import shutil
+import tempfile
+
+from pathlib import Path
+from typing import TYPE_CHECKING
+
+import dzgui.api.mods
+import dzgui.config.query
+
+from tests.fixtures import fixture_path
+
+if TYPE_CHECKING:
+    from dzgui.const.enums import Preferences
+
+pytestmark = pytest.mark.mods
+
+@pytest.fixture
+def versions() -> str:
+    return fixture_path("dzg.versions")
+
+@pytest.fixture
+def tmp() -> str:
+    with tempfile.NamedTemporaryFile(delete=False) as f:
+        tmp = f.name
+    return tmp
+
+def mock_local_ids(path: Path) -> list[int]:
+    return [3410710885, 3739934289]
+
+def mock_lookup(path: Path, prefs: "Preferences") -> str:
+    return ""
+
+def test_signatures(monkeypatch, tmp: str, versions: str) -> None:
+    ids = [1559212036, 1654462998]
+    shutil.copyfile(versions, tmp)
+    path = Path(tmp)
+    monkeypatch.setattr("dzgui.api.mods.lookup", mock_lookup)
+    dzgui.api.mods.remove_stale_signatures(path, path, ids)
+    with open(tmp, "r") as f:
+        lines = f.readlines()
+    assert ids not in lines
+
+def test_signatures_with_no_ids(monkeypatch, tmp: str, versions: str) -> None:
+    shutil.copyfile(versions, tmp)
+    path = Path(tmp)
+    monkeypatch.setattr("dzgui.api.mods.get_local_mod_ids", mock_local_ids)
+    monkeypatch.setattr("dzgui.api.mods.lookup", mock_lookup)
+    dzgui.api.mods.remove_stale_signatures(path, path)
+    with open(tmp, "r") as f:
+        lines = f.readlines()
+    assert lines == ["3410710885,1752864732\n", "3739934289,1780855136\n"]

tests/test_modmeta.py

@@ -1,17 +1,13 @@
 import pytest
+from pathlib import Path
+
 from dzgui.api.mods import tokenize
 from tests.fixtures import fixture_path
 
 @pytest.mark.mods
-@pytest.mark.parametrize("fixture", [
+@pytest.mark.parametrize("i", range(1, 7))
-    fixture_path("cpp/meta1.cpp"),
+def test_modmeta(i: int) -> None:
-    fixture_path("cpp/meta2.cpp"),
+    fixture = fixture_path(f"cpp/mod{i}")
-    fixture_path("cpp/meta3.cpp"),
+    path = Path(fixture)
-    fixture_path("cpp/meta4.cpp"),
+    meta = tokenize(path)
-    fixture_path("cpp/meta5.cpp"),
-    fixture_path("cpp/meta6.cpp"),
-    ]
-)
-def test_modmeta(fixture: str) -> None:
-    meta = tokenize(fixture)
     assert meta["name"] == "ModName"

tests/test_remote_api.py

@@ -20,7 +20,6 @@ def test_steam(config):
     key = config["steam_api"]
     assert probe.test_steam_api(key)
 
-
 def test_bm(config):
     key = config["bm_api"]
     assert probe.test_bm_api(key)