SmallRye OpenAPI outputs Chinese garbled characters in Json

[MorganMaohong]

Description

I downloaded a template project from the official website, added the SmallRey OpenAPI extension, added the @tag annotation to the Rest class, and used Chinese. When I accessed /q/openapi, the response was in garbled format. I guessed that the project was in UTF-8 format. Finally, I tried all the places where I could set the encoding format to UTF-8, but it still didn't match the Chinese display.
Implementation ideas
If you have any implementation ideas, they can go here, however please note that all design change proposals should be posted to the Quarkus developer mailing list (or the corresponding Google Group; see the decisions process document for more information. However, it is normal for the REST request response result to be displayed in Chinese.

@Path("/hello")
@Tag(name = "测试模块")
public class ExampleResource {
    private static final Logger log = LoggerFactory.getLogger(ExampleResource.class);
    @Inject
    EntityManager entityManager;

    @POST
    @Path("/1")
    @Produces(MediaType.TEXT_PLAIN)
    public String hello() {
        String s = new String("".getBytes(StandardCharsets.UTF_8));
        List<User> result = entityManager.createQuery("select e from User e", User.class).getResultList();
        log.info("{}", result);
        return "success哈哈哈哈";
    }
}

Implementation ideas

No response

[quarkus-bot]

/cc @EricWittmann (openapi), @Ladicek (smallrye), @MikeEdgar (openapi), @jmartisk (smallrye), @phillip-kruger (openapi,smallrye), @radcortez (smallrye)

[MorganMaohong]

Under normal circumstances, red Chinese characters will be displayed. If it is in utf-8 encoding format, this should not happen.

[gsmet]

Can you have a look at the Content-Type HTTP header of your response?

[yuhaibohotmail]

@MorganMaohong There is no problem in my environment.
@Tag(name = "CategoryService服务", description = "分类服务")

[MorganMaohong]

您能看一下Content-Type你的响应的 HTTP 标头吗？

Can you have a look at the Content-Type HTTP header of your response?

I see that the character set of content-type is utf-8, but there are still garbled Chinese characters

[MorganMaohong]

@MorganMaohong There is no problem in my environment. @Tag(name = "CategoryService服务", description = "分类服务")

Thank you for your reply. Can you provide me with a template project with smallrye-openApi extension so that I can test it in my local environment? Thank you very much!

[yuhaibohotmail]

code-with-quarkus.zip

@MorganMaohong Here is a simple.

[MorganMaohong]

@gsmet @yuhaibohotmail
My god, after I downloaded your project and ran it, I also reset the Maven environment, but Chinese characters were still displayed in garbled form. Finally, I changed the Windows system encoding format to UTF-8, and after restarting the computer and running the project, the display was correct!

[MorganMaohong]

@gsmet This extension is very useful. I think it would be perfect if these configurations could be exposed and modified through configuration files.

[geoand]

Which extension are you referring to @MorganMaohong ?

[MorganMaohong]

@geoand In this extension, when I use the @Tag and @Operation annotations, the content returned by /q/openapi is garbled in Chinese.

        <dependency>
            <groupId>io.quarkus</groupId>
            <artifactId>quarkus-smallrye-openapi</artifactId>
        </dependency>

[geoand]

But it sounds like it's an issue with your environment, no?

[MorganMaohong]

@geoand Sorry, it is indeed my environment problem. The extension obtains the system encoding format. I would like to suggest a way to provide a configuration file to avoid this problem. It would be better if the extension uses utf-8 encoding by default.

[geoand]

Let's see what @MikeEdgar thinks about that

[MorganMaohong]

Thank you so much

[MikeEdgar]

@MorganMaohong , I'm curious if you run mvn package and check the resulting application jar file at META-INF/quarkus-generated-openapi-doc.json (or the YAML version) whether the characters are also garbled there. That will help pinpoint the step where the characters are incorrectly interpreted.

[MorganMaohong]

@MikeEdgar,Thank you for reminding me. I think I understand why this happens. I checked the file in your way. The default UTF-8 encoding format in vs code is normal. When it is switched to Chinese GBK encoding format, it shows the same garbled characters as the /q/openapi interface. I downloaded the file in the dev interface and found that there was no suffix. After adding the suffix YAML or JSON, it was displayed as garbled characters. I guess that when the IO stream is written without setting the encoding format or the file without the suffix, the system default encoding format will be used. My computer is in the Chinese operating system environment, so it is GBK encoding format. So when I switched to Unicode encoding format, Chinese garbled characters will not appear.

The file found after mvn package

quarkus-generated-openapi-doc.JSON

OpenAPI file downloaded from the dev interface,GitHub cannot upload .yaml files, I changed it to .json

openapi.json

These are my speculations, please forgive me if there are any mistakes.

[MikeEdgar]

I downloaded the file in the dev interface and found that there was no suffix. After adding the suffix YAML or JSON, it was displayed as garbled characters. I guess that when the IO stream is written without setting the encoding format or the file without the suffix, the system default encoding format will be used.

The response will use UTF-8 in the Content-type regardless of how the OpenAPI doc is requested (suffix or not, Accept header or not). I'm curious if you use curl or some other tool besides the browser to fetch /q/openapi whether you will see the correct data being written to disk or the terminal.

quarkus/extensions/smallrye-openapi/runtime/src/main/java/io/quarkus/smallrye/openapi/runtime/OpenApiHandler.java

Line 59 in 8df922d

resp.headers().set("Content-Type", format.getMimeType() + ";charset=UTF-8");

[MorganMaohong]

curl or some other tool besides the browser to fetch /q/openapi whether you will see the correct data being written to disk or the terminal.

@MikeEdgar,I switched the Windows language setting back to the Chinese environment, and then used curl and postman to do a test, but garbled characters still appeared. If I switched to unicode, no garbled characters would appear no matter what method I used.

using curl

Using Postman

Windows Unicode beta testing feature

[MikeEdgar]

@MorganMaohong does it happen with other endpoints in the application or just the OpenAPI response? For example, if you create a simple REST endpoint that returnes a byte[] that contains the same UTF-8 encoded strings, do they appear correct to your HTTP clients?

[MorganMaohong]

@MikeEdgar According to what you said, I created a test REST request and it was normal. In addition, I used various REST tools to test and it was also normal.
There are no problems with other REST requests in the application, regardless of Get, Post or other request methods.

[MikeEdgar]

@MorganMaohong here's one more thing to try to understand where the problem is occurring. Create a META-INF/openapi.json that contains some properly-formatted Chinese characters. After running mvn package, look again at META-INF/quarkus-generated-openapi-doc.json. You previously confirmed that generated JSON was correct when using only annotations, but now we'll confirm if both annotations + other static file result in the problem. If so, I have an idea where the problem is.

[MorganMaohong]

@MikeEdgar I set up to load the static openapi.json file, and I found that they merged, the key is that they are characters are normal

Then mvn package merged the quarkus-generated-openapi-doc.json file /hello REST request was added after the merge, and the openapi version was changed from 3.0.3 to 3.0.1, which shows that the merge was correct
quarkus-generated-openapi-doc.JSON

[MikeEdgar]

And just to confirm, you did that using the Chinese Windows environment, correct?

[MorganMaohong]

@MikeEdgar Yes, I'm sure, I checked,GBK encoding is the default encoding for Chinese Windows systems