[General] generator fails with oas 3 that includes relative local file ref that includes remote file ref

[fuj1g0n]

Description

Generator fails when it reads openapi.yaml that includes relative local file ref that includes remote file ref
(i.e. root -> local ref -> remote ref).

I use "jaxrs-jersey" language for this sample, but I debugged and found that the problem is not limtied in jaxrs-jersey or java. All languages are affected, because the parser has the root problem.

openapi-generator version

current master(e605882 )

OpenAPI declaration file content or url

https://gist.github.com/fujigon/e2e1594a011022245e66fa6c21954ea9

Command line used for generation

# git clone, etc
mvn package
java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -i openapi.yaml -g jaxrs-jersey

Steps to reproduce

When I execute the generator with the gisted openapi.yaml file, I get

$ java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -i openapi.yaml -g jaxrs-jersey
java.nio.file.InvalidPathException: Illegal char <:> at index 7: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
        at sun.nio.fs.WindowsPathParser.normalize(Unknown Source)
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source)
        at sun.nio.fs.WindowsPath.parse(Unknown Source)
        at sun.nio.fs.WindowsFileSystem.getPath(Unknown Source)
        at sun.nio.fs.AbstractPath.resolve(Unknown Source)
        at io.swagger.v3.parser.util.RefUtils.readExternalRef(RefUtils.java:148)
        at io.swagger.v3.parser.ResolverCache.loadRef(ResolverCache.java:116)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:47)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefSchema(ExternalRefProcessor.java:675)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processProperties(ExternalRefProcessor.java:180)
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:142)
        at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema(SchemaProcessor.java:201)
        at io.swagger.v3.parser.processors.SchemaProcessor.processSchema(SchemaProcessor.java:36)
        at io.swagger.v3.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:48)
        at io.swagger.v3.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:63)
        at io.swagger.v3.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:83)
        at io.swagger.v3.parser.OpenAPIResolver.resolve(OpenAPIResolver.java:49)
        at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:53)
        at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:19)
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:524)
        at org.openapitools.codegen.cmd.Generate.run(Generate.java:341)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:59)
[main] WARN  io.swagger.v3.parser.OpenAPIV3Parser - Exception while reading:
java.lang.RuntimeException: Unable to load RELATIVE ref: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
        at io.swagger.v3.parser.util.RefUtils.readExternalRef(RefUtils.java:159) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.ResolverCache.loadRef(ResolverCache.java:116) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:47) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefSchema(ExternalRefProcessor.java:675) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processProperties(ExternalRefProcessor.java:180) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ExternalRefProcessor.processRefToExternalSchema(ExternalRefProcessor.java:142) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.SchemaProcessor.processReferenceSchema(SchemaProcessor.java:201) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.SchemaProcessor.processSchema(SchemaProcessor.java:36) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.ResponseProcessor.processResponse(ResponseProcessor.java:48) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.OperationProcessor.processOperation(OperationProcessor.java:63) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.processors.PathsProcessor.processPaths(PathsProcessor.java:83) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.OpenAPIResolver.resolve(OpenAPIResolver.java:49) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.v3.parser.OpenAPIV3Parser.readLocation(OpenAPIV3Parser.java:53) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at io.swagger.parser.OpenAPIParser.readLocation(OpenAPIParser.java:19) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:524) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at org.openapitools.codegen.cmd.Generate.run(Generate.java:341) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:59) [openapi-generator-cli.jar:3.2.2-SNAPSHOT]
Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 7: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
        at sun.nio.fs.WindowsPathParser.normalize(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsPathParser.parse(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsPath.parse(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.WindowsFileSystem.getPath(Unknown Source) ~[na:1.8.0_161]
        at sun.nio.fs.AbstractPath.resolve(Unknown Source) ~[na:1.8.0_161]
        at io.swagger.v3.parser.util.RefUtils.readExternalRef(RefUtils.java:148) ~[openapi-generator-cli.jar:3.2.2-SNAPSHOT]
        ... 16 common frames omitted
[main] WARN  o.o.codegen.utils.ModelUtils - Failed to get the schema name: ./components/schemas/SchemaIncludingExternalRef.yaml#/components/schemas/SchemaIncludingExternalRef
[main] WARN  o.o.codegen.utils.ModelUtils - Failed to get the schema name: ./components/schemas/SchemaIncludingExternalRef.yaml#/components/schemas/SchemaIncludingExternalRef
Exception in thread "main" org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
 | Error count: 1, Warning count: 1
Errors:
        -Unable to load RELATIVE ref: ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml
Warnings:
        -Unused model: SchemaIncludingExternalRef

        at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:541)
        at org.openapitools.codegen.cmd.Generate.run(Generate.java:341)
        at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:59)

Related issues/PRs

N/A

Suggest a fix/enhancement

The parser has problem with handling remote ref while resolving the ref.
The ref is converted to ./https://raw.githubusercontent.com/swagger-api/swagger-parser/v2.0.2/modules/swagger-parser-v3/src/test/resources/relative/globals.yaml (./ is attached to the uri) while parsing.

https://github.com/fujigon/swagger-parser/commits/feature/relative-ref-with-http-ref-bugfix branch will fix this problem.

[jmini]

Please file a PR against https://github.com/swagger-api/swagger-parser in order to get this issue fixed (the best case would be to open a separated issue first https://github.com/swagger-api/swagger-parser/issues).

Please post the link to this PR here so that I can keep track of it.

Updating Swagger-Parser is something we can consider for 3.3.0 (due date by the end of this month).

[fuj1g0n]

Thanks for quick reply.

• I created an issue in parser project.
  problem with oas 3 that includes relative local file ref that includes remote file ref swagger-api/swagger-parser#805
• I sent a PR to parser project, too
  fix relative ref with http ref bug swagger-api/swagger-parser#806

Updating Swagger-Parser is something we can consider for 3.3.0 (due date by the end of this month).

Great. thanks for letting me know the schedule.

[jmini]

For the time beeing, I have merged OpenAPITools/swagger-parser#15, meaning that the version 2.0.3-SNAPSHOT of our fork of swagger-parser will contain the change.

    <dependency>
      <groupId>org.openapitools.swagger.parser</groupId>
      <artifactId>swagger-parser</artifactId>
      <version>2.0.3-SNAPSHOT</version>
    </dependency>

Artifact is published in the sonatype snapshots repository:

https://oss.sonatype.org/content/repositories/snapshots/

---

To test it in OpenAPI-Generator, the swagger-parser-version value in the main pom:

openapi-generator/pom.xml

Line 1136 in 3d06900

<swagger-parser-version>2.0.2-OpenAPITools.org-1</swagger-parser-version>

can be changed to 2.0.3-SNAPSHOT (for the moment we are already using the org.openapitools.swagger.parser value as groupId for the swagger-parser dependency)

But this can not be commited to our master or 3.3.x branches, because we do not want to use SNAPSHOT versions in our builds.

---

More background:

The idea (see #68) is to release a 2.0.3-OpenAPITools.org-1 if swagger-parser release 2.0.3 does not comes in time for our OpenAPI-Generator 3.3.0 release.

[jmini]

Swagger-Parser update containing the fix #951

[jmini]

Can you give 3.3.0-SNAPSHOT a try?

[fuj1g0n]

I confirmed that the problem has been fixed.
thanks!

btw, I found another problem while trying.

I'll file another issue.