navidrome/plugins/manifest_permissions_test.go
Deluan Quintão f1fc2cd9b9
feat(plugins): experimental support for plugins (#3998)
* feat(plugins): add minimal test agent plugin with API definitions

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add plugin manager with auto-registration and unique agent names

Introduced a plugin manager that scans the plugins folder for subdirectories containing plugin.wasm files and auto-registers them as agents using the directory name as the unique agent name. Updated the configuration to support plugins with enabled/folder options, and ensured the plugin manager is started as a concurrent task during server startup. The wasmAgent now returns the plugin directory name for AgentName, ensuring each plugin agent is uniquely identifiable. This enables dynamic plugin discovery and integration with the agents orchestrator.

* test: add Ginkgo suite and test for plugin manager auto-registration

Added a Ginkgo v2 suite bootstrap (plugins_suite_test.go) for the plugins package and a test (manager_test.go) to verify that plugins in the testdata folder are auto-registered and can be loaded as agents. The test uses a mock DataStore and asserts that the agent is registered and its AgentName matches the plugin directory. Updated go.mod and go.sum for wazero dependency required by plugin WASM support.

* test(plugins): ensure test WASM plugin is always freshly built before running suite; add real-plugin Ginkgo tests. Add BeforeSuite to plugins suite to build plugins/testdata/agent/plugin.wasm using Go WASI build command, matching README instructions. Remove plugin.wasm before build to guarantee a clean build. Add full real-plugin Ginkgo/Gomega tests for wasmAgent, covering all methods and error cases. Fix manager_test.go to use pointer to Manager. This ensures plugin tests are always run against a freshly compiled WASM binary, increasing reliability and reproducibility.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement persistent compilation cache for WASM agent plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement instance pooling for wasmAgent to improve resource management

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): enhance logging for wasmAgent and plugin manager operations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement HttpService for handling HTTP requests in WASM plugins

Also add a sample Wikimedia plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): standardize error handling in wasmAgent and MinimalAgent

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: clean up wikimedia plugin code

Standardized error creation using 'errors.New' where formatting was not needed. Introduced a constant for HTTP request timeouts. Removed commented-out log statement. Improved code comments for clarity and accuracy.

* refactor: use unified SPARQLResult struct and parser for SPARQL responses

Introduced a single SPARQLResult struct to represent all possible SPARQL response fields (sitelink, wiki, comment, img). Added a parseSPARQLResult helper to unmarshal and check for empty results, simplifying all fetch functions and improving type safety and maintainability.

* feat(plugins): improve error handling in HTTP request processing

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: background plugin compilation, logging, and race safety

Implemented background WASM plugin compilation with concurrency limits, proper closure capture, and global compilation cache to avoid data races. Added debug and warning logs for plugin compilation results, including elapsed time. Ensured plugin registration is correct and all tests pass.

* perf: implement true lazy loading for agents

Changed agent instantiation to be fully lazy. The Agents struct now stores agent names in order and only instantiates each agent on first use, caching the result. This preserves agent call order, improves server startup time, and ensures thread safety. Updated all agent methods and tests to use the new pattern. No changes to agent registration or interface. All tests pass.

* fix: ensure wasm plugin instances are closed via runtime.AddCleanup

Introduced runtime.AddCleanup to guarantee that the Close method of WASM plugin instances is called, even if they are garbage collected from the sync.Pool. Modified the sync.Pool.New function in manager.go to register a cleanup function for each loaded instance that implements Close. Updated agent.go to handle the pooledInstance wrapper containing the instance and its cleanup handle. Ensured cleanup.Stop() is called before explicitly closing an instance (on error or agent shutdown) to prevent double closing. This fixes a potential resource leak where instances could be GC'd from the pool without proper cleanup.

* refactor: break down long functions in plugin manager and agent

Refactored plugins/manager.go and plugins/agent.go to improve readability and reduce function length. Extracted pool initialization logic into newPluginPool and background compilation/agent factory logic into precompilePlugin/createAgentFactory in manager.go. Extracted pool retrieval/validation and cleanup function creation into getValidPooledInstance/createPoolCleanupFunc in agent.go.

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename wasmAgent to wasmArtistAgent

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): add AlbumMetadataService with AlbumInfo and AlbumImages requests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugin): rename MinimalAgent for artist metadata service

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): implement wasmAlbumAgent for album metadata service with GetAlbumInfo and GetAlbumImages methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): simplify wasmAlbumAgent and wasmArtistAgent by using wasmBasePlugin

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add support for ArtistMetadataService and AlbumMetadataService in plugin manager

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): enhance plugin pool creation with custom runtime and precompilation support

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): implement generic plugin pool and agent factory for improved service handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): reorganize plugin management

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): improve function signatures for clarity and consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement background precompilation for plugins and agent factory creation

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): include instanceID in logging for better traceability

Signed-off-by: Deluan <deluan@navidrome.org>

* test(plugins): add tests for plugin pre-compilation and agent factory synchronization

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add minimal album test agent plugin for AlbumMetadataService

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): rename fake artist and album test agent plugins for metadata services

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(makefile): add Makefile for building plugin WASM binaries

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add FakeMultiAgent plugin implementing Artist and Album metadata services

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): remove log statements from FakeArtistAgent and FakeMultiAgent methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: split AlbumInfoRetriever and AlbumImageRetriever, update all usages

Split the AlbumInfoRetriever interface into two: AlbumInfoRetriever (for album metadata) and AlbumImageRetriever (for album images), to better separate concerns and simplify implementations. Updated all agents, providers, plugins, and tests to use the new interfaces and methods. Removed the now-unnecessary mockAlbumAgents in favor of the shared mockAgents. Fixed a missing images slice declaration in lastfm agent. All tests pass except for known ignored persistence tests. This change reduces code duplication, improves clarity, and keeps the codebase clean and organized.

* feat(plugins): add Cover Art Archive AlbumMetadataService plugin for album cover images

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove wasm module pooling

it was causing issues with the GC and the Close methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename metadata service files to adapter naming convention

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: unify album and artist method calls by introducing callMethod function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: unify album and artist method calls by introducing callMethod function

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle nil values in data redaction process

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: add timeout for plugin compilation to prevent indefinite blocking

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement ScrobblerService plugin with authorization and scrobbling capabilities

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: simplify generalization

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance plugin management by improving scanning and loading mechanisms

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update plugin creation functions to return specific interfaces for better type safety

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance wasmBasePlugin to support specific plugin types for improved type safety

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement MediaMetadataService with combined artist and album methods

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: improve MediaMetadataService plugin implementation and testing structure

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: add tests for Adapter Media Agent and improve plugin documentation

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: add README for Navidrome Plugin System with detailed architecture and usage guidelines

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance agent management with plugin loading and caching

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: update agent discovery logic to include only local agent when no config is specified

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: encapsulate agent caching logic in agentCache struct\n\nReplaced direct map/mutex usage for agent caching in Agents with a dedicated agentCache struct. This improves readability, maintainability, and testability by centralizing TTL and concurrency logic. Cleaned up comments and ensured all linter and test requirements are met.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: correct file extension filter in goimports command

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: use defer to unlock the mutex

Signed-off-by: Deluan <deluan@navidrome.org>

* chore: move Cover Art Archive AlbumMetadataService plugins to an example folder

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle errors when creating media metadata and scrobbler service plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: increase compilation timeout to one minute

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add configurable plugin compilation timeout

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement plugin scrobbler support in PlayTracker

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add context management and Stop method to buffered scrobbler

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add username field to scrobbler requests and update logging

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: data race in test

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename http proto files to host and update references

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove unused plugin registration methods from manager

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: extend plugin manifests and implement plugin management commands

Signed-off-by: Deluan <deluan@navidrome.org>

* Update utils/files.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* fix for code scanning alert no. 43: Arbitrary file access during archive extraction ("Zip Slip")

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

* feat: add plugin dev workflow support

Added new CLI commands to improve plugin development workflow: 'plugin dev' to create symlinks from development directories to plugins folder, 'plugin refresh' to reload plugins without restarting Navidrome, enhanced 'plugin remove' to handle symlinked development plugins correctly, and updated 'plugin list' to display development plugins with '(dev)' indicator. These changes make the plugin development workflow more efficient by allowing developers to work on plugins in their own directories, link them to Navidrome without copying files, refresh plugins after changes without restart, and clean up safely.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement timer service with register and cancel functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: lint errors

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(README): update documentation to include TimerCallbackService and its functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add InitService with OnInit method and initialization tracking - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add tests for InitService and plugin initialization tracking

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): expand documentation on plugin system implementation and architecture

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: panic

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): redirect plugins' stderr to logs

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add safe accessor methods for TimerService

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add plugin-specific configuration support in InitRequest and documentation

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add TimerCallbackService plugin adapter and integration

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename services for consistency and clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add mutex for configuration access and clone plugin config

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(tests): remove configtest dependency to prevent data races in integration tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): remove PluginName method from WASM plugin implementations and update LoadPlugin to accept service type

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement instance pooling for wasmBasePlugin to improve performance - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add wasmInstancePool for managing WASM plugin instances with TTL and max size

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(plugins): correctly pass error to done function in wasmBasePlugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename service types to capabilities for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): simplify instance management in wasmBasePlugin by removing error handling in closure

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): update wasmBasePlugin and wasmInstancePool to return errors for better error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename InitService to LifecycleManagement for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): fix instance ID logging in wasmBasePlugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): extract instance ID logging to a separate function in wasmBasePlugin, to avoid vet error

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): make timers be isolated per plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): make timers be isolated per plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugins): rename HttpServiceImpl to httpServiceImpl for consistency and improve logging

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add config service for plugin-specific configuration management

Signed-off-by: Deluan <deluan@navidrome.org>

* Update plugins/manager.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* Update plugins/manager.go

Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>

* feat(crontab): implement crontab service for scheduling and canceling jobs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(singleton): fix deadlock issue when a constructor calls GetSingleton again

Signed-off-by: Deluan <deluan@navidrome.org> (+1 squashed commit)
Squashed commits:
[325a96ea2] fix(singleton): fix deadlock issue when a constructor calls GetSingleton again

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): implement Scheduler for one-time and recurring job scheduling, merging CrontabService and TimerService

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scheduler): race condition in the scheduleOneTime and scheduleRecurring methods when replacing jobs with the same ID

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(scheduler): consolidate job scheduling logic into a single helper function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(plugin): rename GetInstance method to Instantiate for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add WebSocket service for handling connections and messages

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(crypto-ticker): add WebSocket plugin for real-time cryptocurrency price tracking

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(websocket): enhance connection management and callback handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manager): only create one adapter instance for each adapter/capability pair

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(websocket): ensure proper resource management by closing response body and use defer to unlocking mutexes

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: flaky test

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugin): refactor WebSocket service integration and improve error logging

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugin): add SchedulerCallback support and improve reconnection logic

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: test panic

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: add crypto-ticker plugin example to README

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manager): add LoadAllPlugins and LoadAllMediaAgents methods with slice.Map integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(api): add Timestamp field to ScrobblerNowPlayingRequest and update related methods

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(websocket): add error field to response messages for better error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(cache): implement CacheService with string, int, float, and byte operations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(tests): update buffered scrobbler tests for improved scrobble verification and use RWMutex in mock repo

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(cache): simplify cache service implementation and remove unnecessary synchronization

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(tests): add build step for test plugins in the test suite

Signed-off-by: Deluan <deluan@navidrome.org>

* wip

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scheduler): implement named scheduler callbacks and enhance Discord plugin integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(rpc): enhance activity image processing and improve error handling in Discord integration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(discord): enhance activity state with artist list and add large text asset

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(artwork): implement ArtworkService for retrieving artwork URLs

Signed-off-by: Deluan <deluan@navidrome.org>

* Add playback position to scrobble NowPlaying (#4089)

* test(playtracker): cover playback position

* address review comment

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* fix merge

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: remove unnecessary check for empty slice in Map function

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: update reflex.conf to include .wasm file extension

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): normalize attribute strings and add edge case tests for PID calculation

Relates to https://github.com/navidrome/navidrome/issues/4183#issuecomment-2952729458

Signed-off-by: Deluan <deluan@navidrome.org>

* test(ui): fix warnings (#4187)

* fix(ui): address test warnings

* ignore lint error in test

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(server): optimize top songs lookup (#4189)

* optimize top songs lookup

* Optimize title matching queries

* refactor: simplify top songs matching

* improve error handling and logging in track loading functions

Signed-off-by: Deluan <deluan@navidrome.org>

* test: add cases for fallback to title matching and combined MBID/title matching

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): playlist details overflow in spotify-based themes (#4184)

* test: ensure playlist details width

* fix(test): simplify expectation for minWidth in NDPlaylistDetails

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(test): test all themes

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): update TagLib to version 2.1 (#4185)

* chore: update cross-taglib

* fix(taglib): add logging for TagLib version

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* test: verify agents fallback (#4191)

* build(docker): downgrade Alpine version from 3.21 to 3.19, oldest supported version.

This is to reduce the image size, as we don't really need the latest.

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(runtime): implement pooled WASM runtime and module for better instance management

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(discord-plugin): adjust timer delay calculation for track completion

Signed-off-by: Deluan <deluan@navidrome.org>

* resolve PR comments

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): implement cache cleanup by size functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(manager): return error from getCompilationCache and handle it in ScanPlugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix possible rce condition

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(docs): update README to include Cache and Artwork services

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manager): add permissions support for host services in custom runtime - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manifest): add permissions field to plugin manifests - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* test(permissions): implement permission validation and testing for plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(plugins): add unauthorized_plugin to test permission enforcement - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(docs): add Plugin Permission System section to README - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(manifest): add detailed reasons for permissions in plugin manifests - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(permissions): implement granular HTTP permissions for plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(permissions): implement HTTP and WebSocket permissions for plugins - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: unexport all plugins package private symbols

Signed-off-by: Deluan <deluan@navidrome.org>

* update docs

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename plugin_lifecycle_manager

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: add discord-rich-presence plugin example to README

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add support for PATCH, HEAD, and OPTIONS HTTP methods

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: use folder names as unique identifiers for plugins

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: read config just once, to avoid data race in tests

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename pluginName to pluginID for consistency across services

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: use symlink name instead of folder name for plugin registration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update plugin output format to include ID and enhance README with symlink usage

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement shared plugin discovery function to streamline plugin scanning and error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: show plugin permissions in `plugin info`

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add JSON schema for Navidrome Plugin manifest and generate corresponding Go types - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement typed permissions for plugins to enhance permission handling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refactor plugin permissions to use typed schema and improve validation - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update HTTP permissions handling to use typed schema for allowed URLs - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: remove unused JSON schema validation for plugin manifests

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: remove unused fields from PluginPackage struct in package.go

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: update file permissions in tests and remove unused permission parsing function

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: refactor test plugin creation to use typed permissions and remove legacy helper

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add website field to plugin manifests and update test cases

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: permission schema to use basePermission structure for consistency

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance host service management by adding permission checks for each service

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: reorganize code files

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: simplify custom runtime creation by removing compilation cache parameter

Signed-off-by: Deluan <deluan@navidrome.org>

* doc: add WebSocketService and update ConfigService for plugin-specific configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: implement WASM loading optimization to enhance plugin instance creation speed

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename custom runtime functions and update related tests for clarity

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance plugin structure with compilation handling and error reporting

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: improve logging and context tracing in runtime and wasm base plugin

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance runtime management with scoped runtime and caching improvements

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement EnsureCompiled method for improved plugin compilation handling

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: implement cached module management with TTL for improved performance

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: replace map with sync.Map

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: adjust time tolerance in scrobble buffer repository tests to avoid flakiness

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: enhance image processing with fallback mechanism for improved error handling

Signed-off-by: Deluan <deluan@navidrome.org>

* docs: review test plugins readme

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: set default timeout for HTTP client to 10 seconds

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: enhance wasm instance pool with concurrency limits and timeout settings

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(discordrp): implement caching for processed image URLs with configurable TTL

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: ellipsis-dev[bot] <65095814+ellipsis-dev[bot]@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2025-06-22 20:45:38 -04:00

526 lines
18 KiB
Go

package plugins
import (
"context"
"encoding/json"
"os"
"path/filepath"
"github.com/navidrome/navidrome/conf"
"github.com/navidrome/navidrome/conf/configtest"
"github.com/navidrome/navidrome/plugins/schema"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
// Helper function to create test plugins with typed permissions
func createTestPlugin(tempDir, name string, permissions schema.PluginManifestPermissions) string {
pluginDir := filepath.Join(tempDir, name)
Expect(os.MkdirAll(pluginDir, 0755)).To(Succeed())
// Use the generated PluginManifest type directly - it handles JSON marshaling automatically
manifest := schema.PluginManifest{
Name: name,
Author: "Test Author",
Version: "1.0.0",
Description: "Test plugin for permissions",
Website: "https://test.navidrome.org/" + name,
Capabilities: []schema.PluginManifestCapabilitiesElem{
schema.PluginManifestCapabilitiesElemMetadataAgent,
},
Permissions: permissions,
}
// Marshal the typed manifest directly - gets all validation for free
manifestData, err := json.Marshal(manifest)
Expect(err).NotTo(HaveOccurred())
manifestPath := filepath.Join(pluginDir, "manifest.json")
Expect(os.WriteFile(manifestPath, manifestData, 0600)).To(Succeed())
// Create fake WASM file (since plugin discovery checks for it)
wasmPath := filepath.Join(pluginDir, "plugin.wasm")
Expect(os.WriteFile(wasmPath, []byte("fake wasm content"), 0600)).To(Succeed())
return pluginDir
}
var _ = Describe("Plugin Permissions", func() {
var (
mgr *Manager
tempDir string
ctx context.Context
)
BeforeEach(func() {
DeferCleanup(configtest.SetupConfig())
ctx = context.Background()
mgr = createManager()
tempDir = GinkgoT().TempDir()
})
Describe("Permission Enforcement in createRuntime", func() {
It("should only load services specified in permissions", func() {
// Test with limited permissions using typed structs
permissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration settings",
},
}
runtimeFunc := mgr.createRuntime("test-plugin", permissions)
// Create runtime to test service availability
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
// The runtime was created successfully with the specified permissions
Expect(runtime).NotTo(BeNil())
// Note: The actual verification of which specific host functions are available
// would require introspecting the WASM runtime, which is complex.
// The key test is that the runtime creation succeeds with valid permissions.
})
It("should create runtime with empty permissions", func() {
permissions := schema.PluginManifestPermissions{}
runtimeFunc := mgr.createRuntime("empty-permissions-plugin", permissions)
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
// Should succeed but with no host services available
Expect(runtime).NotTo(BeNil())
})
It("should handle all available permissions", func() {
// Test with all possible permissions using typed structs
permissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration settings",
},
Scheduler: &schema.PluginManifestPermissionsScheduler{
Reason: "To schedule periodic tasks",
},
Websocket: &schema.PluginManifestPermissionsWebsocket{
Reason: "To handle real-time communication",
AllowedUrls: []string{"wss://api.example.com"},
AllowLocalNetwork: false,
},
Cache: &schema.PluginManifestPermissionsCache{
Reason: "To cache data and reduce API calls",
},
Artwork: &schema.PluginManifestPermissionsArtwork{
Reason: "To generate artwork URLs",
},
}
runtimeFunc := mgr.createRuntime("full-permissions-plugin", permissions)
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
Expect(runtime).NotTo(BeNil())
})
})
Describe("Plugin Discovery with Permissions", func() {
BeforeEach(func() {
conf.Server.Plugins.Folder = tempDir
})
It("should discover plugin with valid permissions manifest", func() {
// Create plugin with http permission using typed structs
permissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch metadata from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
},
}
createTestPlugin(tempDir, "valid-plugin", permissions)
// Scan for plugins
mgr.ScanPlugins()
// Verify plugin was discovered (even without valid WASM)
pluginNames := mgr.PluginNames("MetadataAgent")
Expect(pluginNames).To(ContainElement("valid-plugin"))
})
It("should discover plugin with no permissions", func() {
// Create plugin with empty permissions using typed structs
permissions := schema.PluginManifestPermissions{}
createTestPlugin(tempDir, "no-perms-plugin", permissions)
mgr.ScanPlugins()
pluginNames := mgr.PluginNames("MetadataAgent")
Expect(pluginNames).To(ContainElement("no-perms-plugin"))
})
It("should discover plugin with multiple permissions", func() {
// Create plugin with multiple permissions using typed structs
permissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch metadata from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
},
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read plugin configuration settings",
},
Scheduler: &schema.PluginManifestPermissionsScheduler{
Reason: "To schedule periodic data updates",
},
}
createTestPlugin(tempDir, "multi-perms-plugin", permissions)
mgr.ScanPlugins()
pluginNames := mgr.PluginNames("MetadataAgent")
Expect(pluginNames).To(ContainElement("multi-perms-plugin"))
})
})
Describe("Existing Plugin Permissions", func() {
BeforeEach(func() {
// Use the testdata directory with updated plugins
conf.Server.Plugins.Folder = testDataDir
mgr.ScanPlugins()
})
It("should discover fake_scrobbler with empty permissions", func() {
scrobblerNames := mgr.PluginNames(CapabilityScrobbler)
Expect(scrobblerNames).To(ContainElement("fake_scrobbler"))
})
It("should discover multi_plugin with scheduler permissions", func() {
agentNames := mgr.PluginNames(CapabilityMetadataAgent)
Expect(agentNames).To(ContainElement("multi_plugin"))
})
It("should discover all test plugins successfully", func() {
// All test plugins should be discovered with their updated permissions
testPlugins := []struct {
name string
capability string
}{
{"fake_album_agent", CapabilityMetadataAgent},
{"fake_artist_agent", CapabilityMetadataAgent},
{"fake_scrobbler", CapabilityScrobbler},
{"multi_plugin", CapabilityMetadataAgent},
{"fake_init_service", CapabilityLifecycleManagement},
}
for _, testPlugin := range testPlugins {
pluginNames := mgr.PluginNames(testPlugin.capability)
Expect(pluginNames).To(ContainElement(testPlugin.name), "Plugin %s should be discovered", testPlugin.name)
}
})
})
Describe("Permission Validation", func() {
It("should enforce permissions are required in manifest", func() {
// Create a manifest JSON string without the permissions field
manifestContent := `{
"name": "test-plugin",
"author": "Test Author",
"version": "1.0.0",
"description": "A test plugin",
"website": "https://test.navidrome.org/test-plugin",
"capabilities": ["MetadataAgent"]
}`
manifestPath := filepath.Join(tempDir, "manifest.json")
err := os.WriteFile(manifestPath, []byte(manifestContent), 0600)
Expect(err).NotTo(HaveOccurred())
_, err = LoadManifest(tempDir)
Expect(err).To(HaveOccurred())
Expect(err.Error()).To(ContainSubstring("field permissions in PluginManifest: required"))
})
It("should allow unknown permission keys", func() {
// Create manifest with both known and unknown permission types
pluginDir := filepath.Join(tempDir, "unknown-perms")
Expect(os.MkdirAll(pluginDir, 0755)).To(Succeed())
manifestContent := `{
"name": "unknown-perms",
"author": "Test Author",
"version": "1.0.0",
"description": "Manifest with unknown permissions",
"website": "https://test.navidrome.org/unknown-perms",
"capabilities": ["MetadataAgent"],
"permissions": {
"http": {
"reason": "To fetch data from external APIs",
"allowedUrls": {
"*": ["*"]
}
},
"unknown": {
"customField": "customValue"
}
}
}`
Expect(os.WriteFile(filepath.Join(pluginDir, "manifest.json"), []byte(manifestContent), 0600)).To(Succeed())
// Test manifest loading directly - should succeed even with unknown permissions
loadedManifest, err := LoadManifest(pluginDir)
Expect(err).NotTo(HaveOccurred())
Expect(loadedManifest).NotTo(BeNil())
// With typed permissions, we check the specific fields
Expect(loadedManifest.Permissions.Http).NotTo(BeNil())
Expect(loadedManifest.Permissions.Http.Reason).To(Equal("To fetch data from external APIs"))
// The key point is that the manifest loads successfully despite unknown permissions
// The actual handling of AdditionalProperties depends on the JSON schema implementation
})
})
Describe("Runtime Pool with Permissions", func() {
It("should create separate runtimes for different permission sets", func() {
// Create two different permission sets using typed structs
permissions1 := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
}
permissions2 := schema.PluginManifestPermissions{
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration settings",
},
}
runtimeFunc1 := mgr.createRuntime("plugin1", permissions1)
runtimeFunc2 := mgr.createRuntime("plugin2", permissions2)
runtime1, err1 := runtimeFunc1(ctx)
Expect(err1).NotTo(HaveOccurred())
defer runtime1.Close(ctx)
runtime2, err2 := runtimeFunc2(ctx)
Expect(err2).NotTo(HaveOccurred())
defer runtime2.Close(ctx)
// Should be different runtime instances
Expect(runtime1).NotTo(BeIdenticalTo(runtime2))
})
})
Describe("Permission System Integration", func() {
It("should successfully validate manifests with permissions", func() {
// Create a valid manifest with permissions
pluginDir := filepath.Join(tempDir, "valid-manifest")
Expect(os.MkdirAll(pluginDir, 0755)).To(Succeed())
manifestContent := `{
"name": "valid-manifest",
"author": "Test Author",
"version": "1.0.0",
"description": "Valid manifest with permissions",
"website": "https://test.navidrome.org/valid-manifest",
"capabilities": ["MetadataAgent"],
"permissions": {
"http": {
"reason": "To fetch metadata from external APIs",
"allowedUrls": {
"*": ["*"]
}
},
"config": {
"reason": "To read plugin configuration settings"
}
}
}`
Expect(os.WriteFile(filepath.Join(pluginDir, "manifest.json"), []byte(manifestContent), 0600)).To(Succeed())
// Load the manifest - should succeed
manifest, err := LoadManifest(pluginDir)
Expect(err).NotTo(HaveOccurred())
Expect(manifest).NotTo(BeNil())
// With typed permissions, check the specific permission fields
Expect(manifest.Permissions.Http).NotTo(BeNil())
Expect(manifest.Permissions.Http.Reason).To(Equal("To fetch metadata from external APIs"))
Expect(manifest.Permissions.Config).NotTo(BeNil())
Expect(manifest.Permissions.Config.Reason).To(Equal("To read plugin configuration settings"))
})
It("should track which services are requested per plugin", func() {
// Test that different plugins can have different permission sets
permissions1 := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration settings",
},
}
permissions2 := schema.PluginManifestPermissions{
Scheduler: &schema.PluginManifestPermissionsScheduler{
Reason: "To schedule periodic tasks",
},
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration for scheduler",
},
}
permissions3 := schema.PluginManifestPermissions{} // Empty permissions
createTestPlugin(tempDir, "plugin-with-http", permissions1)
createTestPlugin(tempDir, "plugin-with-scheduler", permissions2)
createTestPlugin(tempDir, "plugin-with-none", permissions3)
conf.Server.Plugins.Folder = tempDir
mgr.ScanPlugins()
// All should be discovered
pluginNames := mgr.PluginNames(CapabilityMetadataAgent)
Expect(pluginNames).To(ContainElement("plugin-with-http"))
Expect(pluginNames).To(ContainElement("plugin-with-scheduler"))
Expect(pluginNames).To(ContainElement("plugin-with-none"))
})
})
Describe("Runtime Service Access Control", func() {
It("should successfully create runtime with permitted services", func() {
// Create runtime with HTTP permission using typed struct
permissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
}
runtimeFunc := mgr.createRuntime("http-only-plugin", permissions)
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
// Runtime should be created successfully - host functions are loaded during runtime creation
Expect(runtime).NotTo(BeNil())
})
It("should successfully create runtime with multiple permitted services", func() {
// Create runtime with multiple permissions using typed structs
permissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration settings",
},
Scheduler: &schema.PluginManifestPermissionsScheduler{
Reason: "To schedule periodic tasks",
},
}
runtimeFunc := mgr.createRuntime("multi-service-plugin", permissions)
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
// Runtime should be created successfully
Expect(runtime).NotTo(BeNil())
})
It("should create runtime with no services when no permissions granted", func() {
// Create runtime with empty permissions using typed struct
emptyPermissions := schema.PluginManifestPermissions{}
runtimeFunc := mgr.createRuntime("no-service-plugin", emptyPermissions)
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
// Runtime should still be created, but with no host services
Expect(runtime).NotTo(BeNil())
})
It("should demonstrate secure-by-default behavior", func() {
// Test that default (empty permissions) provides no services
defaultPermissions := schema.PluginManifestPermissions{}
runtimeFunc := mgr.createRuntime("default-plugin", defaultPermissions)
runtime, err := runtimeFunc(ctx)
Expect(err).NotTo(HaveOccurred())
defer runtime.Close(ctx)
// Runtime should be created but with no host services
Expect(runtime).NotTo(BeNil())
})
It("should test permission enforcement by simulating unauthorized service access", func() {
// This test demonstrates that plugins would fail at runtime when trying to call
// host functions they don't have permission for, since those functions are simply
// not loaded into the WASM runtime environment.
// Create two different runtimes with different permissions using typed structs
httpOnlyPermissions := schema.PluginManifestPermissions{
Http: &schema.PluginManifestPermissionsHttp{
Reason: "To fetch data from external APIs",
AllowedUrls: map[string][]schema.PluginManifestPermissionsHttpAllowedUrlsValueElem{
"*": {schema.PluginManifestPermissionsHttpAllowedUrlsValueElemWildcard},
},
AllowLocalNetwork: false,
},
}
configOnlyPermissions := schema.PluginManifestPermissions{
Config: &schema.PluginManifestPermissionsConfig{
Reason: "To read configuration settings",
},
}
httpRuntime, err := mgr.createRuntime("http-only", httpOnlyPermissions)(ctx)
Expect(err).NotTo(HaveOccurred())
defer httpRuntime.Close(ctx)
configRuntime, err := mgr.createRuntime("config-only", configOnlyPermissions)(ctx)
Expect(err).NotTo(HaveOccurred())
defer configRuntime.Close(ctx)
// Both runtimes should be created successfully, but they will have different
// sets of host functions available. A plugin trying to call unauthorized
// functions would get "function not found" errors during instantiation or execution.
Expect(httpRuntime).NotTo(BeNil())
Expect(configRuntime).NotTo(BeNil())
})
})
})