Maven 3.9.12 - parallel PUT

[slawekjaranowski]

Affected version

3.9.12

Bug description

Due to improvements in parallel PUT method - used for deploy artifacts - in resolver 1.9.25,
bugs of concurrency processing in some repository managers can be disclosed.

Known problems:

• Nexus 2.x - can create multiple staging repositories for one deployment

Workaround

use user property: aether.connector.basic.parallelPut=false

[slachiewicz]

How should the Repository Manager know that uploaded in parallel files are part of the same staging release?
If Nexus has issues, how is it resolved for example in Artifactory?

[cstamas]

Both, NX and AF has/had issues, reported by users immediately when parallel PUT feature went out (3.9.1). Fx @bmarwell had a tweet about AF (I am not on it, and it does not allows me searching so I don't have link for tweet).

In case of Nx2 staging suite things are a bit different: there, the deploy PUT requests are sent against "staging" endpoint and not against some repository. The feature was locally tested by me against Nx2 OSS instance (deploying to repository), and parallel PUT to repository works just fine. The Nx2 bug is limited to "staging feature" of Nx2 PRO ONLY.

And about to "how should it know": again, this is a special endpoint of Staging plugin. It receives requests, ties them logically, and, if needed, creates backing (staging, Nx managed) repository on the fly. The data points based on which are requests connected logically uses several properties of request:

• user (as this endpoint requires authentication)
• client IP address
• client User Agent (will be same for all reqs -- ie Maven does not change UA between requests)
• maybe some more, I don't remember

Also, seems this is a known issue of Nx2 staging, here is a gradle discussion about same issue:
https://discuss.gradle.org/t/solving-multiple-staging-repositories-being-built-when-publishing-to-nexus/44834

[cstamas]

Also, to be clear, the concurrency issue (IMO) exists only on "backward compatibility endpoint" (for maven-deploy-plugin), the well known https://repository.apache.org/service/local/staging/deploy/maven2 as Staging Plugin has own REST API that is used among others by Nx staging maven plugin. Via REST you explicitly control the flow, you "start" staging (Nx creates repo for you, returns you the repo ID), then with received repo ID you perform direct deploy into newly created repository, close it, etc....

It is only the /staging/deploy endpoint that does this "magic" (grouping requests, selecting staging profile based on GAV, creating staging repository on the fly -- if needed, etc). This was also the reason why we had to (via WebUI) manually close and release (after successful vote) or drop the staging repo.... All these actions can be triggered via REST as well...

Also, Maven since 3.9.1 was also uploading in parallel already, but, it had a workaround (for totally unrelated reason: auth cache bug in Resolver Apache transport) to send first PUT alone, wait for it to be finished/processed (where Nx2 performed all the magic), and then let loose in parallel the rest of requests... (but at that point staging repository already existed).

[bmarwell]

Hi, I can confirm I have set parallel put to false, using NX3. Might work in the meantime, but I never bothered to remove this setting. 🤷🏻‍♂️

[cstamas]

Isn't this fixed in 3.9.13?

[kwin]

Fixed with 18a5264.