From 626e5a7bb0240b56438004fc18d62cbf4c403df4 Mon Sep 17 00:00:00 2001 From: Deluan Date: Sun, 13 Apr 2025 00:10:23 -0400 Subject: [PATCH] docs: Add plugin directory structure info and implementation plan - Add section 5.6 describing the plugin directory structure\n- Add new implementation plan document with phased approach\n- Include progress tracking with checkboxes --- docs/hld-plugins.md | 36 ++++ docs/plugins-implementation-plan.md | 280 ++++++++++++++++++++++++++++ 2 files changed, 316 insertions(+) create mode 100644 docs/plugins-implementation-plan.md diff --git a/docs/hld-plugins.md b/docs/hld-plugins.md index 562e18be6..3ff50e9f5 100644 --- a/docs/hld-plugins.md +++ b/docs/hld-plugins.md @@ -892,4 +892,40 @@ A typical development workflow: 5. Repeat steps 3-4 until implementation is complete 6. Package as `.ndp` file for distribution +### 5.6 Plugin Directory Structure + +Plugins are stored in a dedicated plugins directory, which by default is a subdirectory of Navidrome's data folder: + +``` +/plugins/ +├── lastfm/ # Each plugin has its own subdirectory +│ ├── plugin.wasm # The WebAssembly binary +│ ├── manifest.json # The plugin manifest +│ └── README.md # Optional documentation +├── spotify/ +│ ├── plugin.wasm +│ ├── manifest.json +│ └── README.md +└── other-plugin/ + ├── plugin.wasm + ├── manifest.json + └── README.md +``` + +The plugins directory location can be configured in `navidrome.toml`: + +```toml +[Plugins] +Enabled = true +Directory = "${DataFolder}/plugins" # Default, can be overridden +``` + +When Navidrome starts, it scans this directory for subdirectories containing WASM files and manifests, loads the plugins, and registers them with the appropriate subsystems based on their declared capabilities. + +For development purposes, the `plugin dev` command can create a symlink to a development directory outside of the standard plugins directory, allowing developers to work on plugin files without having to manually copy them after each change. + ## 6. Implementation Plan + +``` + +``` diff --git a/docs/plugins-implementation-plan.md b/docs/plugins-implementation-plan.md new file mode 100644 index 000000000..5a5062ded --- /dev/null +++ b/docs/plugins-implementation-plan.md @@ -0,0 +1,280 @@ +# Navidrome Plugin System Implementation Plan + +## Progress Tracking + +### Phase 1: Foundational Infrastructure + +- [ ] 1.1: Plugin Manifest and Configuration +- [ ] 1.2: Basic WebAssembly Runtime Integration +- [ ] 1.3: Permission Management System +- [ ] 1.4: Project Structure and CLI Commands + +### Phase 2: Protocol Definition and Host Functions + +- [ ] 2.1: Protocol Buffer Definitions +- [ ] 2.2: Host Function Implementation +- [ ] 2.3: Plugin Context Management + +### Phase 3: Plugin Loading and Execution + +- [ ] 3.1: WebAssembly Runtime Configuration +- [ ] 3.2: Testing Infrastructure +- [ ] 3.3: Plugin Developer Tools + +### Phase 4: Agent Plugin Integration + +- [ ] 4.1: Agent Plugin Adapter Implementation +- [ ] 4.2: Plugin Registration with Agent System +- [ ] 4.3: Last.fm Agent Plugin Implementation +- [ ] 4.4: Integration Testing + +### Phase 5: Enhanced Management and User Experience + +- [ ] 5.1: Enhanced CLI Management +- [ ] 5.2: Plugin Package Format +- [ ] 5.3: Runtime Monitoring +- [ ] 5.4: Administrative UI (Optional) + +### Phase 6: Documentation and Release + +- [ ] 6.1: User Documentation +- [ ] 6.2: Developer Documentation +- [ ] 6.3: Example Plugin Templates +- [ ] 6.4: Final Testing and Feature Flags + +## Phase 1: Foundational Infrastructure + +**Goal:** Establish the core plugin infrastructure without affecting existing functionality. + +### 1.1: Plugin Manifest and Configuration + +- Create plugin manifest schema and validation functions +- Add plugin-related configuration to `conf` package: + - Global plugin settings: enabled, directory, default limits + - Per-plugin settings: enabled, limits, configuration +- Add tests for manifest validation and configuration parsing + +### 1.2: Basic WebAssembly Runtime Integration + +- Add `knqyf263/go-plugin` dependency +- Create initial plugin loader that can: + - Discover plugin files in configured directory + - Read and validate manifests + - Basic security validation (no plugin execution yet) +- Add unit tests for plugin discovery and manifest loading + +### 1.3: Permission Management System + +- Implement the `PermissionManager` component: + - URL allowlist validation + - Host function allowlist validation + - Internal network access prevention + - Configuration access control +- Add comprehensive security tests for all permission rules + +### 1.4: Project Structure and CLI Commands + +- Create plugin-related directory structure: + ``` + plugins/ + ├── proto/ # Protocol Buffer definitions + ├── manager.go # Plugin Manager implementation + ├── host.go # Host function implementations + ├── permission.go # Permission manager + └── adapters/ # Adapters for different plugin types + ``` +- Implement basic CLI commands for plugin management: + - `navidrome plugin list` + - `navidrome plugin info [name]` + +**Deliverable:** Foundation layer that discovers plugins and validates permissions without executing any plugin code. + +## Phase 2: Protocol Definition and Host Functions + +**Goal:** Define the communication protocol between Navidrome and plugins. + +### 2.1: Protocol Buffer Definitions + +- Define Protocol Buffer specifications for: + - Agent plugin interface + - Host functions interface + - Common request/response structures +- Generate Go code from Protocol Buffers +- Create test stubs for interface implementations + +### 2.2: Host Function Implementation + +- Implement core host functions: + - `GetConfig` for configuration access + - `Log` for plugin logging + - `HttpDo` for controlled HTTP access +- Add comprehensive tests for each host function +- Implement permission checks for all host functions + +### 2.3: Plugin Context Management + +- Create plugin context structure to track: + - Current plugin name + - Permission scope + - Runtime state +- Implement proper isolation between plugin calls + +**Deliverable:** Complete protocol definition and host function implementations without executing actual plugins. + +## Phase 3: Plugin Loading and Execution (Minimal) + +**Goal:** Enable basic plugin loading and execution in isolation from the rest of the system. + +### 3.1: WebAssembly Runtime Configuration + +- Configure WebAssembly runtime with appropriate security settings +- Implement plugin initialization with configuration passing +- Add proper error handling for plugin loading failures + +### 3.2: Testing Infrastructure + +- Create test harness for plugin execution +- Implement simple test plugins for validation +- Add integration tests for plugin loading and execution + +### 3.3: Plugin Developer Tools + +- Implement development commands: + - `navidrome plugin dev [folder_path]` + - `navidrome plugin refresh [name]` +- Create basic development documentation + +**Deliverable:** Working plugin loading and execution system that can be tested in isolation. + +## Phase 4: Agent Plugin Integration + +**Goal:** Connect the plugin system to the existing agent architecture. + +### 4.1: Agent Plugin Adapter Implementation + +- Create adapter that implements all agent interfaces: + - Convert between Protobuf and agent interfaces + - Implement proper error handling and timeouts + - Add trace logging for debugging +- Add unit tests for all adapter methods + +### 4.2: Plugin Registration with Agent System + +- Implement plugin registration with the existing agent system +- Extend configuration to support plugin agent ordering +- Make plugin agents respect the same priority system as built-in agents + +### 4.3: Last.fm Agent Plugin Implementation + +- Implement prototype Last.fm plugin as proof of concept +- Create plugin manifest with necessary permissions +- Add tests comparing plugin behavior to built-in agent + +### 4.4: Integration Testing + +- Add comprehensive integration tests for: + - Plugin discovery and loading + - Agent API functionality + - Error handling and recovery + - Configuration changes + +**Deliverable:** Working plugin system with Last.fm plugin implementation that can be toggled via configuration without breaking existing functionality. + +## Phase 5: Enhanced Management and User Experience + +**Goal:** Improve plugin management and user experience. + +### 5.1: Enhanced CLI Management + +- Complete remaining CLI commands: + - `navidrome plugin install [file]` + - `navidrome plugin remove [name]` + - `navidrome plugin config-template [name]` +- Add command validation and error handling + +### 5.2: Plugin Package Format + +- Implement `.ndp` package format: + - Package creation + - Validation + - Installation +- Add tests for package integrity checking + +### 5.3: Runtime Monitoring + +- Add runtime statistics: + - Plugin execution time + - Resource usage + - Error tracking +- Implement health checks and recovery mechanisms + +### 5.4: Administrative UI (Optional) + +- Create basic admin UI for plugin management: + - View installed plugins + - Enable/disable plugins + - View permissions + - Configure plugins + +**Deliverable:** Complete plugin management tooling with good user experience. + +## Phase 6: Documentation and Release + +**Goal:** Prepare the plugin system for production use and developer adoption. + +### 6.1: User Documentation + +- Create comprehensive user documentation: + - Plugin installation and management + - Configuration options + - Security considerations + - Troubleshooting + +### 6.2: Developer Documentation + +- Create plugin development guide: + - API reference + - Development workflow + - Best practices + - Examples + +### 6.3: Example Plugin Templates + +- Create starter templates for common plugin types: + - Basic agent plugin + - Custom service plugin +- Include CI/CD configurations + +### 6.4: Final Testing and Feature Flags + +- Add feature flag to enable/disable plugin system +- Perform comprehensive integration testing +- Address any final security concerns + +**Deliverable:** Production-ready plugin system with documentation and examples. + +## Risk Assessment and Mitigation + +1. **Security Risks** + + - **Risk**: Plugin execution could compromise system security + - **Mitigation**: Strict permission model, WebAssembly sandbox, URL validation + +2. **Performance Impact** + + - **Risk**: WebAssembly execution might be slower than native code + - **Mitigation**: Benchmarking, caching mechanisms, performance monitoring + +3. **Backward Compatibility** + + - **Risk**: Changes might break existing functionality + - **Mitigation**: Feature flags, phased integration, comprehensive testing + +4. **User Experience** + + - **Risk**: Plugin management could be complex for users + - **Mitigation**: Clear documentation, intuitive CLI, potential UI integration + +5. **Developer Adoption** + - **Risk**: Plugin development might be too complex + - **Mitigation**: Clear documentation, example templates, developer tooling