Merge branch 'release-4.2.0'

ChangeLog.md

@@ -1,66 +1,21 @@
 This is the master changelog for Elite Dangerous Market Connector.  Entries are in reverse chronological order (latest first).
 ---
 
-Pre-Release 4.2.0-beta3
+Release 4.2.0
 ===
 
-* Allow `--force-localserver-for-auth` to actually work. See [#891
+*This release increases the Minor version due to the major change in how
-  `--force-localserver-for-auth` fails due to JounalLock being imported from
+multiple-instance checking is done.*
-  monitor.py](https://github.com/EDCD/EDMarketConnector/issues/891)
 
-Pre-Release 4.2.0-beta2
+* Adds Steam and Epic to the list of "audiences" in the Frontier Auth callout
-===
+  so that you can authorise using those accounts, rather than their associated
-
+  Frontier Account details.
-This release actually includes the commits to enable Steam and Epic
-authentication.
-
-**NB: The correct form for the runas command is as follows:**
-
-`runas /user:<USER> "\"c:\Program Files (x86)\EDMarketConnector\EDMarketConnector.exe\" --force-localserver-for-auth"`
-
-If anything has messed with the backslash characters there then know that you
-need to have " (double-quote) around the entire command (path to program .exe
-*and* any extra arguments), and as a result need to place a backslash before
-any double-quote characters in the command (such as around the space-including
-path to the program).
-
-I've verified it renders correctly [on GitHub](https://github.com/EDCD/EDMarketConnector/blob/Release/4.2.0-beta2/ChangeLog.md).
-
-Pre-Release 4.2.0-beta1
-===
-
-*NB: This contains further work on top of 4.1.7-rc1.  Due to the major change
-in how multiple-instance checking is done we felt the need to bump the minor
-version.*
-
-There is a major change in this release with respect to how the main
-application checks if there is already another instance running.
-
-For most users things will operate no differently, although note that the
-multiple instance check does now apply to platforms other than Windows.
-
-For anyone wanting to run multiple instances of the program this is now
-possible via:
-
-`runas /user:OTHERUSER <path to>EDMarketConnector.exe --force-localserver-for-auth`
-
-The old check was based solely on there being a window present with the title
-we expect.  This prevented using `runas /user:SOMEUSER ...` to run a second
-copy of the application, as the resulting window would still be within the
-same desktop environment and thus be found in the check.
-
-The new method does assume that the Journals directory is writable by the
-user we're running as.  This might not be true in the case of sharing the
-file system to another host in a read-only manner.  If we fail to open the
-lock file read-write then the application aborts the checks and will simply
-continue running as normal.
-
-Note that any single instance of EDMarketConnector.exe will still only monitor
-and act upon the *latest* Journal file in the configured location.  If you run
-Elite Dangerous for another Commander then the application will want to start
-monitoring that separate Commander.  See [wiki:Troubleshooting#i-run-two-instances-of-ed-simultaneously-but-i-cant-run-two-instances-of-edmc](https://github.com/EDCD/EDMarketConnector/wiki/Troubleshooting#i-run-two-instances-of-ed-simultaneously-but-i-cant-run-two-instances-of-edmc>)
-which will be updated when this change is in a full release.
 
+* New status message "CAPI: No commander data returned" if a `/profile`
+  request has no commander in the returned data.  This can happen if you
+  literally haven't yet created a Commander on the account.  Previously you'd
+  get a confusing `'commander'` message shown.
+  
 * Changes the "is there another process already running?" check to be based on
   a lockfile in the configured Journals directory.  The name of this file is
   `edmc-journal-lock.txt` and upon successful locking it will contain text
@@ -73,17 +28,46 @@ which will be updated when this change is in a full release.
   The lock will be released and applied to the new directory if you change it
   via Settings > Configuration.  If the new location is already locked you'll
   get a 'Retry/Ignore?' pop-up.
-  
+
+  For most users things will operate no differently, although note that the
+  multiple instance check does now apply to platforms other than Windows.
+    
+  For anyone wanting to run multiple instances of the program this is now
+  possible via:
+    
+  `runas /user:<USER> "\"c:\Program Files (x86)\EDMarketConnector\EDMarketConnector.exe\" --force-localserver-for-auth"`
+    
+  If anything has messed with the backslash characters there then know that you
+  need to have " (double-quote) around the entire command (path to program .exe
+  *and* any extra arguments), and as a result need to place a backslash before
+  any double-quote characters in the command (such as around the space-including
+  path to the program).
+    
+  I've verified it renders correctly [on GitHub](https://github.com/EDCD/EDMarketConnector/blob/Release/4.2.0/ChangeLog.md).
+    
+  The old check was based solely on there being a window present with the title
+  we expect.  This prevented using `runas /user:SOMEUSER ...` to run a second
+  copy of the application, as the resulting window would still be within the
+  same desktop environment and thus be found in the check.
+    
+  The new method does assume that the Journals directory is writable by the
+  user we're running as.  This might not be true in the case of sharing the
+  file system to another host in a read-only manner.  If we fail to open the
+  lock file read-write then the application aborts the checks and will simply
+  continue running as normal.
+    
+  Note that any single instance of EDMarketConnector.exe will still only monitor
+  and act upon the *latest* Journal file in the configured location.  If you run
+  Elite Dangerous for another Commander then the application will want to start
+  monitoring that separate Commander.  See [wiki:Troubleshooting#i-run-two-instances-of-ed-simultaneously-but-i-cant-run-two-instances-of-edmc](https://github.com/EDCD/EDMarketConnector/wiki/Troubleshooting#i-run-two-instances-of-ed-simultaneously-but-i-cant-run-two-instances-of-edmc>)
+  which will be updated when this change is in a full release.
+
 * Adds the command-line argument `--force-localserver-for-auth`. This forces 
   using a local webserver for the Frontier Auth callback.  This should be used
   when running multiple instances of the application **for all instances** 
   else there's no guarantee of the `edmc://` protocol callback reaching the
   correct process and Frontier Auth will fail.
   
-* Adds Steam and Epic to the list of "audiences" in the Frontier Auth callout
-  so that you can authorise using those accounts, rather than their associated
-  Frontier Account details.
-  
 * Adds the command-line argument `--suppress-dupe-process-popup` to exit
   without showing the warning popup in the case that EDMarketConnector found
   another process already running.
@@ -92,6 +76,7 @@ which will be updated when this change is in a full release.
   batch file or similar.
 
 
+
 Release 4.1.6
 ===
 

config.py

@@ -13,7 +13,7 @@ appcmdname = 'EDMC'
 # appversion **MUST** follow Semantic Versioning rules:
 # <https://semver.org/#semantic-versioning-specification-semver>
 # Major.Minor.Patch(-prerelease)(+buildmetadata)
-appversion = '4.2.0-beta3'  #-rc1+a872b5f'
+appversion = '4.2.0'  #-rc1+a872b5f'
 # For some things we want appversion without (possible) +build metadata
 appversion_nobuild = str(semantic_version.Version(appversion).truncate('prerelease'))
 copyright = u'© 2015-2019 Jonathan Harris, 2020 EDCD'

journal_lock.py

@@ -0,0 +1,231 @@
+"""Implements locking of Journal directory."""
+
+import pathlib
+import tkinter as tk
+from os import getpid as os_getpid
+from sys import platform
+from tkinter import ttk
+from typing import TYPE_CHECKING, Callable, Optional
+
+from config import config
+from EDMCLogging import get_main_logger
+
+logger = get_main_logger()
+
+if TYPE_CHECKING:
+    def _(x: str) -> str:
+        return x
+
+
+class JournalLock:
+    """Handle locking of journal directory."""
+
+    def __init__(self) -> None:
+        """Initialise where the journal directory and lock file are."""
+        self.journal_dir: str = config.get('journaldir') or config.default_journal_dir
+        self.journal_dir_path = pathlib.Path(self.journal_dir)
+        self.journal_dir_lockfile_name: Optional[pathlib.Path] = None
+        # We never test truthiness of this, so let it be defined when first assigned.  Avoids type hint issues.
+        # self.journal_dir_lockfile: Optional[IO] = None
+
+    def obtain_lock(self) -> bool:
+        """
+        Attempt to obtain a lock on the journal directory.
+
+        :return: bool - True if we successfully obtained the lock
+        """
+        self.journal_dir_lockfile_name = self.journal_dir_path / 'edmc-journal-lock.txt'
+        logger.trace(f'journal_dir_lockfile_name = {self.journal_dir_lockfile_name!r}')
+        try:
+            self.journal_dir_lockfile = open(self.journal_dir_lockfile_name, mode='w+', encoding='utf-8')
+
+        # Linux CIFS read-only mount throws: OSError(30, 'Read-only file system')
+        # Linux no-write-perm directory throws: PermissionError(13, 'Permission denied')
+        except Exception as e:  # For remote FS this could be any of a wide range of exceptions
+            logger.warning(f"Couldn't open \"{self.journal_dir_lockfile_name}\" for \"w+\""
+                           f" Aborting duplicate process checks: {e!r}")
+            return False
+
+        if platform == 'win32':
+            logger.trace('win32, using msvcrt')
+            # win32 doesn't have fcntl, so we have to use msvcrt
+            import msvcrt
+
+            try:
+                msvcrt.locking(self.journal_dir_lockfile.fileno(), msvcrt.LK_NBLCK, 4096)
+
+            except Exception as e:
+                logger.info(f"Exception: Couldn't lock journal directory \"{self.journal_dir}\""
+                            f", assuming another process running: {e!r}")
+                return False
+
+        else:
+            logger.trace('NOT win32, using fcntl')
+            try:
+                import fcntl
+
+            except ImportError:
+                logger.warning("Not on win32 and we have no fcntl, can't use a file lock!"
+                               "Allowing multiple instances!")
+                return True  # Lie about being locked
+
+            try:
+                fcntl.flock(self.journal_dir_lockfile, fcntl.LOCK_EX | fcntl.LOCK_NB)
+
+            except Exception as e:
+                logger.info(f"Exception: Couldn't lock journal directory \"{self.journal_dir}\", "
+                            f"assuming another process running: {e!r}")
+                return False
+
+        self.journal_dir_lockfile.write(f"Path: {self.journal_dir}\nPID: {os_getpid()}\n")
+        self.journal_dir_lockfile.flush()
+
+        logger.trace('Done')
+        return True
+
+    def release_lock(self) -> bool:
+        """
+        Release lock on journal directory.
+
+        :return: bool - Success of unlocking operation.
+        """
+        unlocked = False
+        if platform == 'win32':
+            logger.trace('win32, using msvcrt')
+            # win32 doesn't have fcntl, so we have to use msvcrt
+            import msvcrt
+
+            try:
+                # Need to seek to the start first, as lock range is relative to
+                # current position
+                self.journal_dir_lockfile.seek(0)
+                msvcrt.locking(self.journal_dir_lockfile.fileno(), msvcrt.LK_UNLCK, 4096)
+
+            except Exception as e:
+                logger.info(f"Exception: Couldn't unlock journal directory \"{self.journal_dir}\": {e!r}")
+
+            else:
+                unlocked = True
+
+        else:
+            logger.trace('NOT win32, using fcntl')
+            try:
+                import fcntl
+
+            except ImportError:
+                logger.warning("Not on win32 and we have no fcntl, can't use a file lock!")
+                return True  # Lie about being unlocked
+
+            try:
+                fcntl.flock(self.journal_dir_lockfile, fcntl.LOCK_UN)
+
+            except Exception as e:
+                logger.info(f"Exception: Couldn't unlock journal directory \"{self.journal_dir}\": {e!r}")
+
+            else:
+                unlocked = True
+
+        # Close the file whether or not the unlocking succeeded.
+        self.journal_dir_lockfile.close()
+
+        self.journal_dir_lockfile_name = None
+        # Avoids type hint issues, see 'declaration' in JournalLock.__init__()
+        # self.journal_dir_lockfile = None
+
+        return unlocked
+
+    class JournalAlreadyLocked(tk.Toplevel):
+        """Pop-up for when Journal directory already locked."""
+
+        def __init__(self, parent: tk.Tk, callback: Callable) -> None:
+            """
+            Init the user choice popup.
+
+            :param parent: - The tkinter parent window.
+            :param callback: - The function to be called when the user makes their choice.
+            """
+            tk.Toplevel.__init__(self, parent)
+
+            self.parent = parent
+            self.callback = callback
+            self.title(_('Journal directory already locked'))
+
+            # remove decoration
+            if platform == 'win32':
+                self.attributes('-toolwindow', tk.TRUE)
+
+            elif platform == 'darwin':
+                # http://wiki.tcl.tk/13428
+                parent.call('tk::unsupported::MacWindowStyle', 'style', self, 'utility')
+
+            self.resizable(tk.FALSE, tk.FALSE)
+
+            frame = ttk.Frame(self)
+            frame.grid(sticky=tk.NSEW)
+
+            self.blurb = tk.Label(frame)
+            self.blurb['text'] = _("The new Journal Directory location is already locked.{CR}"
+                                   "You can either attempt to resolve this and then Retry, or choose to Ignore this.")
+            self.blurb.grid(row=1, column=0, columnspan=2, sticky=tk.NSEW)
+
+            self.retry_button = ttk.Button(frame, text=_('Retry'), command=self.retry)
+            self.retry_button.grid(row=2, column=0, sticky=tk.EW)
+
+            self.ignore_button = ttk.Button(frame, text=_('Ignore'), command=self.ignore)
+            self.ignore_button.grid(row=2, column=1, sticky=tk.EW)
+            self.protocol("WM_DELETE_WINDOW", self._destroy)
+
+        def retry(self) -> None:
+            """Handle user electing to Retry obtaining the lock."""
+            logger.trace('User selected: Retry')
+            self.destroy()
+            self.callback(True, self.parent)
+
+        def ignore(self) -> None:
+            """Handle user electing to Ignore failure to obtain the lock."""
+            logger.trace('User selected: Ignore')
+            self.destroy()
+            self.callback(False, self.parent)
+
+        def _destroy(self) -> None:
+            """Destroy the Retry/Ignore popup."""
+            logger.trace('User force-closed popup, treating as Ignore')
+            self.ignore()
+
+    def update_lock(self, parent: tk.Tk) -> None:
+        """
+        Update journal directory lock to new location if possible.
+
+        :param parent: - The parent tkinter window.
+        """
+        current_journaldir = config.get('journaldir') or config.default_journal_dir
+
+        if current_journaldir == self.journal_dir:
+            return  # Still the same
+
+        self.release_lock()
+
+        self.journal_dir = current_journaldir
+        self.journal_dir_path = pathlib.Path(self.journal_dir)
+        if not self.obtain_lock():
+            # Pop-up message asking for Retry or Ignore
+            self.retry_popup = self.JournalAlreadyLocked(parent, self.retry_lock)
+
+    def retry_lock(self, retry: bool, parent: tk.Tk) -> None:
+        """
+        Try again to obtain a lock on the Journal Directory.
+
+        :param retry: - does the user want to retry?  Comes from the dialogue choice.
+        :param parent: - The parent tkinter window.
+        """
+        logger.trace(f'We should retry: {retry}')
+
+        if not retry:
+            return
+
+        current_journaldir = config.get('journaldir') or config.default_journal_dir
+        self.journal_dir = current_journaldir
+        self.journal_dir_path = pathlib.Path(self.journal_dir)
+        if not self.obtain_lock():
+            # Pop-up message asking for Retry or Ignore
+            self.retry_popup = self.JournalAlreadyLocked(parent, self.retry_lock)