navidrome/utils/str/str_test.go
Deluan Quintão c795bcfcf7
feat(bfr): Big Refactor: new scanner, lots of new fields and tags, improvements and DB schema changes (#2709)
* fix(server): more race conditions when updating artist/album from external sources

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): add .gitignore syntax to .ndignore. Resolves #1394

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): null

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): pass configfile option to child process

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): resume interrupted fullScans

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): remove old scanner code

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): rename old metadata package

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): move old metadata package

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: tests

Signed-off-by: Deluan <deluan@navidrome.org>

* chore(deps): update Go to 1.23.4

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(test):

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: log level

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add config for scanner watcher

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: children playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: replace `interface{}` with `any`

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: smart playlists with genres

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow any tags in smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: artist names in playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: smart playlist's sort by tags

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): use generic JSONArray for OS arrays

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): use https in test

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add releaseTypes to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add recordLabels to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic): rename JSONArray to Array

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to AlbumID3

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to Child

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(scanner): do not pre-populate smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): implement a simplified version of ArtistID3.

See https://github.com/opensubsonic/open-subsonic-api/discussions/120

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add artists to album child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add contributors to mediafile Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add albumArtists to mediafile Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add displayArtist and displayAlbumArtist

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add displayComposer to Child

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add roles to ArtistID3

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): use " • " separator for displayComposer

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor:

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic):

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(subsonic): respect `PreferSortTags` config option

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor(subsonic):

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize purging non-unused tags

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: don't run 'refresh artist stats' concurrently with other transactions

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor:

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: log message

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: add Scanner.ScanOnStartup config option, default true

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: better json parsing error msg when importing NSPs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't update album's imported_time when updating external_metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: handle interrupted scans and full scans after migrations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: run `analyze` when migration requires a full rescan

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: run `PRAGMA optimize` at the end of the scan

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't update artist's updated_at when updating external_metadata

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: handle multiple artists and roles in smart playlists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): dim missing tracks

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album missing logic

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: error encoding in gob

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: separate warnings from errors

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: mark albums as missing if they were contained in a deleted folder

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: add participant names to media_file and album tables

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: use participations in criteria, instead of m2m relationship

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: rename participations to participants

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add moods to album child

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: albumartist role case

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(scanner): run scanner as an external process by default

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): show albumArtist names

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): dim out missing albums

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: flaky test

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(server): scrobble buffer mapping. fix #3583

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: more participations renaming

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: listenbrainz scrobbling

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: send release_group_mbid to listenbrainz

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): implement OpenSubsonic explicitStatus field (#3597)

* feat: implement OpenSubsonic explicitStatus field

* fix(subsonic): fix failing snapshot tests

* refactor: create helper for setting explicitStatus

* fix: store smaller values for explicit-status on database

* test: ToAlbum explicitStatus

* refactor: rename explicitStatus helper function

---------

Co-authored-by: Deluan Quintão <deluan@navidrome.org>

* fix: handle album and track tags in the DB based on the mappings.yaml file

Signed-off-by: Deluan <deluan@navidrome.org>

* save similar artists as JSONB

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: getAlbumList byGenre

Signed-off-by: Deluan <deluan@navidrome.org>

* detect changes in PID configuration

Signed-off-by: Deluan <deluan@navidrome.org>

* set default album PID to legacy_pid

Signed-off-by: Deluan <deluan@navidrome.org>

* fix tests

Signed-off-by: Deluan <deluan@navidrome.org>

* fix SIGSEGV

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't lose album stars/ratings when migrating

Signed-off-by: Deluan <deluan@navidrome.org>

* store full PID conf in properties

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: keep album annotations when changing PID.Album config

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: reassign album annotations

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: use (display) albumArtist and add links to each artist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: not showing albums by albumartist

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: error msgs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: hide PID from Native API

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album cover art resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: trim participant names

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: reduce watcher log spam

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: panic when initializing the watcher

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: various artists

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't store empty lyrics in the DB

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unused methods

Signed-off-by: Deluan <deluan@navidrome.org>

* drop full_text indexes, as they are not being used by SQLite

Signed-off-by: Deluan <deluan@navidrome.org>

* keep album created_at when upgrading

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): null pointer

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album artwork cache

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't expose missing files in Subsonic API

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: searchable interface

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: filter out missing items from subsonic search

* fix: filter out missing items from playlists

* fix: filter out missing items from shares

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): add filter by artist role

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): only return albumartists in getIndexes and getArtists endpoints

Signed-off-by: Deluan <deluan@navidrome.org>

* sort roles alphabetically

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: artist playcounts

Signed-off-by: Deluan <deluan@navidrome.org>

* change default Album PID conf

Signed-off-by: Deluan <deluan@navidrome.org>

* fix albumartist link when it does not match any albumartists values

Signed-off-by: Deluan <deluan@navidrome.org>

* fix `Ignoring filter not whitelisted` (role) message

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: trim any names/titles being imported

Signed-off-by: Deluan <deluan@navidrome.org>

* remove unused genre code

Signed-off-by: Deluan <deluan@navidrome.org>

* serialize calls to Last.fm's getArtist

Signed-off-by: Deluan <deluan@navidrome.org>

xxx

Signed-off-by: Deluan <deluan@navidrome.org>

* add counters to genres

Signed-off-by: Deluan <deluan@navidrome.org>

* nit: fix migration `notice` message

Signed-off-by: Deluan <deluan@navidrome.org>

* optimize similar artists query

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: last.fm.getInfo when mbid does not exist

Signed-off-by: Deluan <deluan@navidrome.org>

* ui only show missing items for admins

Signed-off-by: Deluan <deluan@navidrome.org>

* don't allow interaction with missing items

Signed-off-by: Deluan <deluan@navidrome.org>

* Add Missing Files view (WIP)

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: merged tag_counts into tag table

Signed-off-by: Deluan <deluan@navidrome.org>

* add option to completely disable automatic scanner

Signed-off-by: Deluan <deluan@navidrome.org>

* add delete missing files functionality

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: playlists not showing for regular users

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce updateLastAccess frequency to once every minute

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce update player frequency to once every minute

Signed-off-by: Deluan <deluan@navidrome.org>

* add timeout when updating player

Signed-off-by: Deluan <deluan@navidrome.org>

* remove dead code

Signed-off-by: Deluan <deluan@navidrome.org>

* fix duplicated roles in stats

Signed-off-by: Deluan <deluan@navidrome.org>

* add `; ` to artist splitters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix stats query

Signed-off-by: Deluan <deluan@navidrome.org>

* more logs

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: support legacy clients (DSub) by removing OpenSubsonic extra fields - WIP

Signed-off-by: Deluan <deluan@navidrome.org>

* add record label filter

Signed-off-by: Deluan <deluan@navidrome.org>

* add release type filter

Signed-off-by: Deluan <deluan@navidrome.org>

* fix purgeUnused tags

Signed-off-by: Deluan <deluan@navidrome.org>

* add grouping filter to albums

Signed-off-by: Deluan <deluan@navidrome.org>

* allow any album tags to be used in as filters in the API

Signed-off-by: Deluan <deluan@navidrome.org>

* remove empty tags from album info

Signed-off-by: Deluan <deluan@navidrome.org>

* comments in the migration

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: Cannot read properties of undefined

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: listenbrainz scrobbling (#3640)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: remove duplicated tag values

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: don't ignore the taglib folder!

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: show track subtitle tag

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: show artists stats based on selected role

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: inspect

Signed-off-by: Deluan <deluan@navidrome.org>

* add media type to album info/filters

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: change format of subtitle in the UI

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: subtitle in Subsonic API and search

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: subtitle in UI's player

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: split strings should be case-insensitive

Signed-off-by: Deluan <deluan@navidrome.org>

* disable ScanSchedule

Signed-off-by: Deluan <deluan@navidrome.org>

* increase default sessiontimeout

Signed-off-by: Deluan <deluan@navidrome.org>

* add sqlite command line tool to docker image

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: resources override

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: album PID conf

Signed-off-by: Deluan <deluan@navidrome.org>

* change migration to mark current artists as albumArtists

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): Allow filtering on multiple genres (#3679)

* feat(ui): Allow filtering on multiple genres

Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Signed-off-by: Deluan <deluan@navidrome.org>

* add multi-genre filter in Album list

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Henrik Nordvik <henrikno@gmail.com>

* add more multi-valued tag filters to Album and Song views

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): unselect missing files after removing

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): song filter

Signed-off-by: Deluan <deluan@navidrome.org>

* fix sharing tracks. fix #3687

Signed-off-by: Deluan <deluan@navidrome.org>

* use rowids when using search for sync (ex: Symfonium)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Report Real Paths" option for subsonic clients

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Report Real Paths" option for subsonic clients for search

Signed-off-by: Deluan <deluan@navidrome.org>

* add libraryPath to Native API /songs endpoint

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(subsonic): add album version

Signed-off-by: Deluan <deluan@navidrome.org>

* made all tags lowercase as they are case-insensitive anyways.

Signed-off-by: Deluan <deluan@navidrome.org>

* feat(ui): Show full paths, extended properties for album/song (#3691)

* feat(ui): Show full paths, extended properties for album/song

- uses library path + os separator + path
- show participants (album/song) and tags (song)
- make album/participant clickable in show info

* add source to path

* fix pathSeparator in UI

Signed-off-by: Deluan <deluan@navidrome.org>

* fix local artist artwork (#3695)

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: parse vorbis performers

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: clean function into smaller functions

Signed-off-by: Deluan <deluan@navidrome.org>

* fix translations for en and pt

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace log to show annotations reassignment

Signed-off-by: Deluan <deluan@navidrome.org>

* add trace log to show annotations reassignment

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: allow performers without instrument/subrole

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: metadata clean function again

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: optimize split function

Signed-off-by: Deluan <deluan@navidrome.org>

* refactor: split function is now a method of TagConf

Signed-off-by: Deluan <deluan@navidrome.org>

* fix: humanize Artist total size

Signed-off-by: Deluan <deluan@navidrome.org>

* add album version to album details

Signed-off-by: Deluan <deluan@navidrome.org>

* don't display album-level tags in SongInfo

Signed-off-by: Deluan <deluan@navidrome.org>

* fix genre clicking in Album Page

Signed-off-by: Deluan <deluan@navidrome.org>

* don't use mbids in Last.fm api calls.

From https://discord.com/channels/671335427726114836/704303730660737113/1337574018143879248:

With MBID:
```
GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&mbid=a41ac10f-0a56-4672-9161-b83f9b223559&method=artist.getInfo

{
artist: {
name: "Bee Gees",
mbid: "bf0f7e29-dfe1-416c-b5c6-f9ebc19ea810",
url: "https://www.last.fm/music/Bee+Gees",
}
```

Without MBID:
```
GET https://ws.audioscrobbler.com/2.0/?api_key=XXXX&artist=Van+Morrison&format=json&lang=en&method=artist.getInfo

{
artist: {
name: "Van Morrison",
mbid: "a41ac10f-0a56-4672-9161-b83f9b223559",
url: "https://www.last.fm/music/Van+Morrison",
}
```

Signed-off-by: Deluan <deluan@navidrome.org>

* better logging for when the artist folder is not found

Signed-off-by: Deluan <deluan@navidrome.org>

* fix various issues with artist image resolution

Signed-off-by: Deluan <deluan@navidrome.org>

* hide "Additional Tags" header if there are none.

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify tag rendering

Signed-off-by: Deluan <deluan@navidrome.org>

* enhance logging for artist folder detection

Signed-off-by: Deluan <deluan@navidrome.org>

* make folderID consistent for relative and absolute folderPaths

Signed-off-by: Deluan <deluan@navidrome.org>

* handle more folder paths scenarios

Signed-off-by: Deluan <deluan@navidrome.org>

* filter out other roles when SubsonicArtistParticipations = true

Signed-off-by: Deluan <deluan@navidrome.org>

* fix "Cannot read properties of undefined"

Signed-off-by: Deluan <deluan@navidrome.org>

* fix lyrics and comments being truncated (#3701)

* fix lyrics and comments being truncated

* specifically test for lyrics and comment length

* reorder assertions

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>

* fix(server): Expose library_path for playlist (#3705)

Allows showing absolute path for UI, and makes "report real path" work for playlists (Subsonic)

* fix BFR on Windows (#3704)

* fix potential reflected cross-site scripting vulnerability

Signed-off-by: Deluan <deluan@navidrome.org>

* hack to make it work on Windows

* ignore windows executables

* try fixing the pipeline

Signed-off-by: Deluan <deluan@navidrome.org>

* allow MusicFolder in other drives

* move windows local drive logic to local storage implementation

---------

Signed-off-by: Deluan <deluan@navidrome.org>

* increase pagination sizes for missing files

Signed-off-by: Deluan <deluan@navidrome.org>

* reduce level of "already scanning" watcher log message

Signed-off-by: Deluan <deluan@navidrome.org>

* only count folders with audio files in it

See https://github.com/navidrome/navidrome/discussions/3676#discussioncomment-11990930

Signed-off-by: Deluan <deluan@navidrome.org>

* add album version and catalog number to search

Signed-off-by: Deluan <deluan@navidrome.org>

* add `organization` alias for `recordlabel`

Signed-off-by: Deluan <deluan@navidrome.org>

* remove mbid from Last.fm agent

Signed-off-by: Deluan <deluan@navidrome.org>

* feat: support inspect in ui (#3726)

* inspect in ui

* address round 1

* add catalogNum to AlbumInfo

Signed-off-by: Deluan <deluan@navidrome.org>

* remove dependency on metadata_old (deprecated) package

Signed-off-by: Deluan <deluan@navidrome.org>

* add `RawTags` to model

Signed-off-by: Deluan <deluan@navidrome.org>

* support parsing MBIDs for roles (from the https://github.com/kgarner7/picard-all-mbids plugin) (#3698)


* parse standard roles, vorbis/m4a work for now

* fix djmixer

* working roles, use DJ-mix

* add performers to file

* map mbids

* add a few more tests

* add test

Signed-off-by: Deluan <deluan@navidrome.org>

* try to simplify the performers logic

Signed-off-by: Deluan <deluan@navidrome.org>

* stylistic changes

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Co-authored-by: Deluan <deluan@navidrome.org>

* remove param mutation

Signed-off-by: Deluan <deluan@navidrome.org>

* run automated SQLite optimizations

Signed-off-by: Deluan <deluan@navidrome.org>

* fix playlists import/export on Windows

* fix import playlists

* fix export playlists

* better handling of Windows volumes

Signed-off-by: Deluan <deluan@navidrome.org>

* handle more album ID reassignments

Signed-off-by: Deluan <deluan@navidrome.org>

* allow adding/overriding tags in the config file

Signed-off-by: Deluan <deluan@navidrome.org>

* fix(ui): Fix playlist track id, handle missing tracks better (#3734)

- Use `mediaFileId` instead of `id` for playlist tracks
- Only fetch if the file is not missing
- If extractor fails to get the file, also error (rather than panic)

* optimize DB after each scan.

Signed-off-by: Deluan <deluan@navidrome.org>

* remove sortable from AlbumSongs columns

Signed-off-by: Deluan <deluan@navidrome.org>

* simplify query to get missing tracks

Signed-off-by: Deluan <deluan@navidrome.org>

* mark Scanner.Extractor as deprecated

Signed-off-by: Deluan <deluan@navidrome.org>

---------

Signed-off-by: Deluan <deluan@navidrome.org>
Signed-off-by: Henrik Nordvik <henrikno@gmail.com>
Co-authored-by: Caio Cotts <caio@cotts.com.br>
Co-authored-by: Henrik Nordvik <henrikno@gmail.com>
Co-authored-by: Kendall Garner <17521368+kgarner7@users.noreply.github.com>
2025-02-19 20:35:17 -05:00

149 lines
12 KiB
Go
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

package str_test
import (
"github.com/navidrome/navidrome/utils/str"
. "github.com/onsi/ginkgo/v2"
. "github.com/onsi/gomega"
)
var _ = Describe("String Utils", func() {
Describe("Clear", func() {
DescribeTable("replaces some Unicode chars with their equivalent ASCII",
func(input, expected string) {
Expect(str.Clear(input)).To(Equal(expected))
},
Entry("k-os", "kos", "k-os"),
Entry("kos", "kos", "k-os"),
Entry(`"Weird" Al Yankovic`, "“Weird” Al Yankovic", `"Weird" Al Yankovic`),
Entry("Single quotes", "Single quotes", "'Single' quotes"),
)
})
Describe("LongestCommonPrefix", func() {
It("finds the longest common prefix", func() {
Expect(str.LongestCommonPrefix(testPaths)).To(Equal("/Music/iTunes 1/iTunes Media/Music/"))
})
It("does NOT handle partial prefixes", func() {
albums := []string{
"/artist/albumOne",
"/artist/albumTwo",
}
Expect(str.LongestCommonPrefix(albums)).To(Equal("/artist/album"))
})
})
})
var testPaths = []string{
"/Music/iTunes 1/iTunes Media/Music/ABBA/Gold_ Greatest Hits/Dancing Queen.m4a",
"/Music/iTunes 1/iTunes Media/Music/ABBA/Gold_ Greatest Hits/Mamma Mia.m4a",
"/Music/iTunes 1/iTunes Media/Music/Art Blakey/A Night At Birdland, Vol. 1/01 Annoucement By Pee Wee Marquette.m4a",
"/Music/iTunes 1/iTunes Media/Music/Art Blakey/A Night At Birdland, Vol. 1/02 Split Kick.m4a",
"/Music/iTunes 1/iTunes Media/Music/As Frenéticas/As Frenéticas/Perigosa.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bachman-Turner Overdrive/Gold/Down Down.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bachman-Turner Overdrive/Gold/Hey You.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bachman-Turner Overdrive/Gold/Hold Back The Water.m4a",
"/Music/iTunes 1/iTunes Media/Music/Belle And Sebastian/Write About Love/01 I Didn't See It Coming.m4a",
"/Music/iTunes 1/iTunes Media/Music/Belle And Sebastian/Write About Love/02 Come On Sister.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Elephunk/03 Let's Get Retarded.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Elephunk/04 Hey Mama.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Monkey Business/10 They Don't Want Music (Feat. James Brown).m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/The E.N.D/1-01 Boom Boom Pow.m4a",
"/Music/iTunes 1/iTunes Media/Music/Black Eyed Peas/Timeless/01 Mas Que Nada.m4a",
"/Music/iTunes 1/iTunes Media/Music/Blondie/Heart Of Glass/Heart Of Glass.m4a",
"/Music/iTunes 1/iTunes Media/Music/Bob Dylan/Nashville Skyline/06 Lay Lady Lay.m4a",
"/Music/iTunes 1/iTunes Media/Music/Botany/Feeling Today - EP/03 Waterparker.m4a",
"/Music/iTunes 1/iTunes Media/Music/Céu/CéU/06 10 Contados.m4a",
"/Music/iTunes 1/iTunes Media/Music/Chance/Six Through Ten/03 Forgive+Forget.m4a",
"/Music/iTunes 1/iTunes Media/Music/Clive Tanaka Y Su Orquesta/Jet Set Siempre 1°/03 Neu Chicago (Side A) [For Dance].m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Absolute Rock Classics/1-02 Smoke on the water.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Almost Famous Soundtrack/10 Simple Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Audio News - Rock'n' Roll Forever/01 Rock Around The Clock.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Austin Powers_ International Man Of Mystery/01 The Magic Piper (Of Love).m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Austin Powers_ The Spy Who Shagged Me/04 American Woman.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Back To Dance/03 Long Cool Woman In A Black Dress.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Back To The 70's - O Album Da Década/03 American Pie.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Bambolê/09 In The Mood.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Bambolê - Volume II/03 Blue Moon.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Big Brother Brasil 2004/04 I Will Survive.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Collateral Soundtrack/03 Hands Of Time.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Forrest Gump - The Soundtrack/1-12 California Dreamin'.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Forrest Gump - The Soundtrack/1-16 Mrs. Robinson.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Ghost World - Original Motion Picture Soundtrack/01 Jaan Pechechaan Ho.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Grease [Original Soundtrack]/01 Grease.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/La Bamba/09 Summertime Blues.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Pretty Woman/10 Oh Pretty Woman.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents African Groove/01 Saye Mogo Bana.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Arabic Groove/02 Galbi.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Asian Groove/03 Remember Tomorrow.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/01 Midnight Dream.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/03 Banal Reality.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/04 Parchman Blues.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Blues Lounge/06 Run On.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Brazilian Groove/01 Maria Moita.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Brazilian Lounge/08 E Depois....m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Brazilian Lounge/11 Os Grilos.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/01 Un Simple Histoire.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/02 Limbe.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/05 Sempre Di Domenica.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents Euro Lounge/12 Voulez-Vous_.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents World Lounge/03 Santa Maria.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ A New Groove/02 Dirty Laundry.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ Blues Around the World/02 Canceriano Sem Lar (Clinica Tobias Blues).m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ Euro Groove/03 Check In.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Putumayo Presents_ World Groove/01 Attention.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Saturday Night Fever/01 Stayin' Alive.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/Saturday Night Fever/03 Night Fever.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/The Best Air Guitar Album In The World... Ever!/2-06 Johnny B. Goode.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/The Full Monty - Soundtrack/02 You Sexy Thing.m4a",
"/Music/iTunes 1/iTunes Media/Music/Compilations/The Full Monty - Soundtrack/11 We Are Family.m4a",
"/Music/iTunes 1/iTunes Media/Music/Cut Copy/Zonoscope (Bonus Version)/10 Corner of the Sky.m4a",
"/Music/iTunes 1/iTunes Media/Music/David Bowie/Changesbowie/07 Diamond Dogs.m4a",
"/Music/iTunes 1/iTunes Media/Music/Douster & Savage Skulls/Get Rich or High Tryin' - EP/01 Bad Gal.m4a",
"/Music/iTunes 1/iTunes Media/Music/Elton John/Greatest Hits 1970-2002/1-04 Rocket Man (I Think It's Going to Be a Long, Long Time).m4a",
"/Music/iTunes 1/iTunes Media/Music/Elvis Presley/ELV1S 30 #1 Hits/02 Don't Be Cruel.m4a",
"/Music/iTunes 1/iTunes Media/Music/Eric Clapton/The Cream Of Clapton/03 I Feel Free.m4a",
"/Music/iTunes 1/iTunes Media/Music/Fleetwood Mac/The Very Best Of Fleetwood Mac/02 Don't Stop.m4a",
"/Music/iTunes 1/iTunes Media/Music/Françoise Hardy/Comment te dire adieu/Comment te dire adieu.m4a",
"/Music/iTunes 1/iTunes Media/Music/Games/That We Can Play - EP/01 Strawberry Skies.m4a",
"/Music/iTunes 1/iTunes Media/Music/Grand Funk Railroad/Collectors Series/The Loco-Motion.m4a",
"/Music/iTunes 1/iTunes Media/Music/Henry Mancini/The Pink Panther (Music from the Film Score)/The Pink Panther Theme.m4a",
"/Music/iTunes 1/iTunes Media/Music/Holy Ghost!/Do It Again - Single/01 Do It Again.m4a",
"/Music/iTunes 1/iTunes Media/Music/K.C. & The Sunshine Band/The Best of/03 I'm Your Boogie Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/K.C. & The Sunshine Band/Unknown Album/Megamix (Thats The Way, Shake Your Booty, Get Down Tonight, Give It Up).m4a",
"/Music/iTunes 1/iTunes Media/Music/Kim Ann Foxman & Andy Butler/Creature - EP/01 Creature.m4a",
"/Music/iTunes 1/iTunes Media/Music/Nico/Chelsea Girl/01 The Fairest Of The Seasons.m4a",
"/Music/iTunes 1/iTunes Media/Music/oOoOO/oOoOO - EP/02 Burnout Eyess.m4a",
"/Music/iTunes 1/iTunes Media/Music/Peter Frampton/The Very Best of Peter Frampton/Baby, I Love Your Way.m4a",
"/Music/iTunes 1/iTunes Media/Music/Peter Frampton/The Very Best of Peter Frampton/Show Me The Way.m4a",
"/Music/iTunes 1/iTunes Media/Music/Raul Seixas/A Arte De Raul Seixas/03 Metamorfose Ambulante.m4a",
"/Music/iTunes 1/iTunes Media/Music/Raul Seixas/A Arte De Raul Seixas/18 Eu Nasci há 10 Mil Anos Atrás.m4a",
"/Music/iTunes 1/iTunes Media/Music/Rick James/Street Songs/Super Freak.m4a",
"/Music/iTunes 1/iTunes Media/Music/Rita Lee/Fruto Proibido/Agora Só Falta Você.m4a",
"/Music/iTunes 1/iTunes Media/Music/Rita Lee/Fruto Proibido/Esse Tal De Roque Enrow.m4a",
"/Music/iTunes 1/iTunes Media/Music/Roberto Carlos/Roberto Carlos 1966/05 Negro Gato.m4a",
"/Music/iTunes 1/iTunes Media/Music/SOHO/Goddess/02 Hippychick.m4a",
"/Music/iTunes 1/iTunes Media/Music/Stan Getz/Getz_Gilberto/05 Corcovado (Quiet Nights of Quiet Stars).m4a",
"/Music/iTunes 1/iTunes Media/Music/Steely Dan/Pretzel Logic/Rikki Don't Loose That Number.m4a",
"/Music/iTunes 1/iTunes Media/Music/Stevie Wonder/For Once In My Life/I Don't Know Why.m4a",
"/Music/iTunes 1/iTunes Media/Music/Teebs/Ardour/While You Doooo.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Magical Mystery Tour/08 Strawberry Fields Forever.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Past Masters, Vol. 1/10 Long Tall Sally.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Please Please Me/14 Twist And Shout.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Beatles/Sgt. Pepper's Lonely Hearts Club Band/03 Lucy In The Sky With Diamonds.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/Amorica/09 Wiser Time.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/By Your Side/05 Only A Fool.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/Shake Your Money Maker/04 Could I''ve Been So Blind.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/The Southern Harmony And Musical Companion/01 Sting Me.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Black Crowes/Three Snakes And One Charm/02 Good Friday.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Doors/Strange Days (40th Anniversary Mixes)/01 Strange Days.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Rolling Stones/Forty Licks/1-03 (I Can't Get No) Satisfaction.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Velvet Underground/The Velvet Underground & Nico/02 I'm Waiting For The Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Velvet Underground/The Velvet Underground & Nico/03 Femme Fatale.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Velvet Underground/White Light_White Heat/04 Here She Comes Now.m4a",
"/Music/iTunes 1/iTunes Media/Music/The Who/Sings My Generation/My Generation.m4a",
"/Music/iTunes 1/iTunes Media/Music/Village People/The Very Best Of Village People/Macho Man.m4a",
"/Music/iTunes 1/iTunes Media/Music/Vondelpark/Sauna - EP/01 California Analog Dream.m4a",
"/Music/iTunes 1/iTunes Media/Music/War/Why Can't We Be Friends/Low Rider.m4a",
"/Music/iTunes 1/iTunes Media/Music/Yes/Fragile/01 Roundabout.m4a",
}