discogs: Parse IDs from new release URL format

[jdheyburn]

Problem

• With some files located in directory at with no tags
• With discogs plugin enabled

Running this command in verbose (-vv) mode:

$ beet -vv import at

Then entering i to enter release ID, for example: https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy

Led to this problem:

Enter release ID: https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Tagging  - 
Searching for album ID: https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Requesting MusicBrainz release https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Invalid MBID (https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy).
discogs: Searching for release https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy
Evaluating 0 candidates.
Sending event: before_choose_candidate
No matching release found for 4 tracks.
For help, see: https://beets.readthedocs.org/en/latest/faq.html#nomatch

Here's a link to the music files that trigger the bug (if relevant): N/A - any music files will do

Here is some verbose logging while beets was starting up:

 0.87
Sending event: albuminfo_received
Candidate: Skrewdriver - Boots And Braces (1654191)
Computing track assignment...
...done.
Success. Distance: 0.85
Sending event: albuminfo_received
Candidate: Saint Etienne - Continental (1778268)
Computing track assignment...
...done.
Success. Distance: 0.88
Sending event: albuminfo_received
Candidate: X-102 - Titan EP (1393615)
Computing track assignment...
...done.
Success. Distance: 0.83
Evaluating 5 candidates.

Note that the search option worked.

Enter search, enter Id, aBort, eDit, edit Candidates? e
Artist: Aphex Twin
Album: Come to daddy
Tagging  - 
No album ID found.
Search terms: Aphex Twin - Come to daddy
Album might be VA: False
Searching for MusicBrainz releases with: {'release': 'come to daddy', 'artist': 'aphex twin', 'tracks': '4'}
Requesting MusicBrainz release f7a09aff-6a8c-49c1-b8fb-4d2e93bebd6b
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: single
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (f7a09aff-6a8c-49c1-b8fb-4d2e93bebd6b)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 9197ad55-4c0a-3cb3-b190-e1f9062b867c
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: single
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (9197ad55-4c0a-3cb3-b190-e1f9062b867c)
Computing track assignment...
...done.
Success. Distance: 0.76
Requesting MusicBrainz release 32ad4a8c-cd44-3637-ac39-3479d7be8fb2
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: ep
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (32ad4a8c-cd44-3637-ac39-3479d7be8fb2)
Computing track assignment...
...done.
Success. Distance: 0.81
Requesting MusicBrainz release c8d680de-3bbd-3546-bf0f-ac151e079c1b
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: ep
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (c8d680de-3bbd-3546-bf0f-ac151e079c1b)
Computing track assignment...
...done.
Success. Distance: 0.81
Requesting MusicBrainz release 0931f506-5138-3c27-b561-fc95f0d813a9
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
Sending event: mb_track_extract
primary MB release type: ep
Sending event: mb_album_extract
Sending event: albuminfo_received
Candidate: Aphex Twin - Come to Daddy (0931f506-5138-3c27-b561-fc95f0d813a9)
Computing track assignment...
...done.
Success. Distance: 0.81
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 1.8842348098754882 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 2.157538557052612 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 2.085971975326538 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 1.992177629470825 seconds
discogs: Searching for master release 27457
discogs: hit rate limit, waiting for 2.0841585636138915 seconds
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (3662)
Computing track assignment...
...done.
Success. Distance: 0.69
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (3903368)
Computing track assignment...
...done.
Success. Distance: 0.69
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (6157)
Computing track assignment...
...done.
Success. Distance: 0.78
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (52731)
Computing track assignment...
...done.
Success. Distance: 0.78
Sending event: albuminfo_received
Candidate: Aphex Twin - Come To Daddy (10818482)
Computing track assignment...
...done.
Success. Distance: 0.69
Evaluating 10 candidates.
Sending event: before_choose_candidate
Finding tags for album " - ".
Candidates:
1. Aphex Twin - Come To Daddy (31.2%) (tracks, album, artist, ...) (Discogs, Vinyl, 1997, UK, Warp Records, WAP94)
2. Aphex Twin - Come To Daddy (31.2%) (tracks, album, artist, ...) (Discogs, Vinyl, 2012, UK, Warp Records, WAP94)
3. Aphex Twin - Come To Daddy (31.2%) (tracks, album, artist, ...) (Discogs, Vinyl, 2017, UK, Warp Records, WAP94)
4. Aphex Twin - Come to Daddy (23.8%) (tracks, album, artist) (CD, 1997, GB, Warp, WAP94CD)
5. Aphex Twin - Come to Daddy (23.8%) (tracks, album, artist) (12" Vinyl, 1997, GB, Warp, WAP94)
6. Aphex Twin - Come To Daddy (22.1%) (tracks, missing tracks, album, ...) (Discogs, CD, 1997, UK, Warp Records, WAP94CDX)
7. Aphex Twin - Come To Daddy (22.1%) (tracks, missing tracks, album, ...) (Discogs, CD, 1997, US, Sire, 31001)
8. Aphex Twin - Come to Daddy (18.9%) (tracks, missing tracks, album, ...) (CD, 1997, DE, Rough Trade, RTD126.3383.2)
9. Aphex Twin - Come to Daddy (18.9%) (tracks, missing tracks, album, ...) (CD, 1997, JP, Sire Records, WPCR-1603)
10. Aphex Twin - Come to Daddy (18.9%) (tracks, missing tracks, album, ...) (CD, 1997, US, Sire Records, 31001)

Setup

• OS: PopOS
• Python version: 3.9.5
• beets version: 1.5.0
• Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:

plugins: edit discogs inline info badfiles missing embedart fetchart
directory: /mnt/nfs/Backup/media/vinyl
library: /mnt/nfs/Backup/media/beets-db/beets-vinyl.db

import:
    move: no
    copy: yes
    timid: yes

paths:
    default: $albumartist/$album%aunique{}/%if{$multidisc,CD$disc/}$track $artist - $title
    comp: Compilations/$album%aunique{}/%if{$multidisc,CD$disc/}$track $artist - $title
    singleton: Singles/$artist - $title
item_fields:
    multidisc: 1 if disctotal > 1 and media == 'CD' else 0
original_date: yes
threaded: yes
fetchart:
    cover_names: cover
    minwidth: 500
    store_source: yes
    auto: yes
    maxwidth: 0
    quality: 0
    max_filesize: 0
    enforce_ratio: no
    cautious: no
    sources:
    - filesystem
    - coverart
    - itunes
    - amazon
    - albumart
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    fanarttv_key: REDACTED
    lastfm_key: REDACTED
    high_resolution: no
embedart:
    maxwidth: 0
    auto: yes
    compare_threshold: 0
    ifempty: no
    remove_art_file: no
    quality: 0
pathfields: {}
album_fields: {}
missing:
    count: no
    total: no
    album: no
edit:
    albumfields: album albumartist
    itemfields: track title artist album
    ignore_fields: id path
discogs:
    apikey: REDACTED
    apisecret: REDACTED
    tokenfile: discogs_token.json
    source_weight: 0.5
    user_token: REDACTED
    separator: ', '
    index_tracks: no

I just tried setting my own discogs.apikey and discogs.apisecret and I received the same hit rate limit output as above.

Might be worth mentioning that this was working 2 days ago, I hadn't changed anything in my setup between then. My requirements.txt file is:

audioread==2.1.9
beautifulsoup4==4.9.3
beetcamp==0.8.0
beets==1.5.0
cached-property==1.5.2
certifi==2020.12.5
chardet==4.0.0
confuse==1.4.0
discogs-client==2.3.0
idna==2.10
isodate==0.6.0
jellyfish==0.8.2
mediafile==0.6.0
munkres==1.1.4
musicbrainzngs==0.7.1
mutagen==1.45.1
oauthlib==3.1.0
packaging==20.9
Pillow==8.1.0
pyacoustid==1.2.0
pycountry==20.7.3
pyparsing==2.4.7
PyYAML==5.4.1
requests==2.25.1
requests-oauthlib==1.3.0
six==1.15.0
soupsieve==2.2
Unidecode==1.2.0
urllib3==1.26.3

[sampsyo]

It looks like discogs may have changed the way they format their URLs? As you can see, we only expect a number after the / in the URL, possibly followed by a bracket:

beets/beetsplug/discogs.py

Lines 212 to 213 in a187739

	match = re.search(r'(^|\[*r|discogs\.com/.+/release/)(\d+)($|\])',
	album_id)

What happens, for example, if you try pasting just https://www.discogs.com/release/18377836?

[jdheyburn]

That URL didn't work either, but you had me go to regex101 and check the regex being used vs what Discogs uses now [1] versus before [2] and you can see there is definitely a change. Both URLs still work so I guess the regex will need to be updated to account for both?

[1]
https://www.discogs.com/release/18377836-Aphex-Twin-Come-To-Daddy

[2]
https://www.discogs.com/Roy-Davis-Jr-Tha-Black-Light-Exposure-EP/release/19766479

[jdheyburn]

Just did a switcheroo of

https://www.discogs.com/release/15963324-Larry-Heard-Sceneries-Not-Songs-Volume-One

to

https://www.discogs.com/Larry-Heard-Sceneries-Not-Songs-Volume-One/release/15963324

and it was able to pick it up

Enter release ID: https://www.discogs.com/Larry-Heard-Sceneries-Not-Songs-Volume-One/release/15963324
Correcting tags from:
    (unknown album)
To:
    Larry Heard - Sceneries Not Songs, Volume One
URL:
    https://www.discogs.com/release/15963324-Larry-Heard-Sceneries-Not-Songs-Volume-One

[sampsyo]

Ah, very tricky! Looks like we should make our regex flexible enough to work for both styles of URLs… or else just write two different regexes.