[gwt-dev:2.9] Dev-Mode not working after Upgrade 2.8.2 to 2.9 if Webapp contains ASM lib

[gruenewa]

GWT version: 2.9
Browser (with version):
Operating System:

---

Description

We are trying to upgrade our GWT 2.8.2 Webapp to GWT 2.9. We are using IntelliJ IDEA 2020.1 with GWT Support. Our Webapp currently has an asm:5.0.3 library dependency in the backend. Using GWT 2.8.2 there are no issues using Super-Dev Mode in IntelliJ IDEA. Unfortunately the Super Dev Mode does not work with our Webapp when upgrading to GWT 2.9.

The issue seems to be that with gwt-dev:2.9 the ASM dependency was upgraded to asm:7.2. At the same time gwt-dev:2.9 uses an old Jetty dependency 9.2.x. This old Jetty dependency does only implement ClassVisitors compatible to asm:5.x. Since gwt-dev:2.9 uses asm:7.2 the Jetty dependency is overwritten and also uses asm:7.2 when doing a classpath scan. Now it happens that when there is a dependency in the classpath that contains a Java 9 module-info.class it gives an exception, since the ClassVisitor of the jetty (which only supports asm:5.x) does not recognize this language feature.

If we use an old ASM dependency (like 5.x) in our Webapp the GWT 2.9 Dev-Mode is not working. It seems to require a newer version of ASM. If we choose a new ASM dependency (like 6.x, 7.x, 8.x) the Dev-Mode is also not working, since the Jetty's classpath scan finds the ASM dependency in our Webapp, which (since of ASM6) happens to contain a module-info.class, which is not supported by Jetty 9.2.x.

Steps to reproduce

Known workarounds

Links to further discussions

[niloc132]

Thanks for the report. The embedded jetty wiring is first and foremost meant to be used for compiling the application and serving the content - there are a variety of cases where forcing the jetty version to a specific build can cause conflicts.

There are two main options to overcome this.

The first is the general best practice: only use the internal jetty server for demo-sized applications, once you scale past some certain point (of required features and flexibility, not necessarily code size) you will find that the embedded jetty no longer makes sense. Instead, we strongly encourage you to use -noserver to tell GWT not to run the *:8888 webserver, and point -war/-launchDir at the directory that a normal jetty instance will serve from. This will free you up to run Jetty as you would like, using your IDE or build tooling, any version you want, any classpath etc. I should also point out that your classpaths should be kept separate as well - gwt-user and gwt-dev have no business being on your server's classpath, and likewise keep your server classpath out of your client project whenever possible.

The second option is to update jetty in a custom build of GWT. This has been done before, see #9606, which tracks generally updating GWT to a newer Jetty. Even once this happens in mainline GWT though, it may eventually fall behind again, so we strongly encourage keeping client separate from server.

--

Finally, I should point out that in the tooling being built for j2cl/gwt3, we are not including a built-in server at all for "dev mode" work, and instead directing teams to use their preferred server's own dev tooling, to guarantee that these classpaths are kept separate and each side can upgrade at its own pace.

I'll leave this open for a bit more discussion, then will close in favor of #9606.

[Elisabitao]

did you try old browsers that are compatible with Superdev mode it dosntcwork on new chrome or Firefox there is only one download for the old browsers. You can find it but it's tough. Let me know if thats it sounds like it.

…

On Thu, Jun 4, 2020, 3:44 PM Colin Alworth ***@***.***> wrote: Thanks for the report. The embedded jetty wiring is first and foremost meant to be used for compiling the application and serving the content - there are a variety of cases where forcing the jetty version to a specific build can cause conflicts. There are two main options to overcome this. The first is the general best practice: only use the internal jetty server for demo-sized applications, once you scale past some certain point (of required features and flexibility, not necessarily code size) you will find that the embedded jetty no longer makes sense. Instead, we strongly encourage you to use -noserver to tell GWT not to run the *:8888 webserver, and point -war/-launchDir at the directory that a normal jetty instance will serve from. This will free you up to run Jetty as you would like, using your IDE or build tooling, any version you want, any classpath etc. I should also point out that your classpaths should be kept separate as well - gwt-user and gwt-dev have no business being on your server's classpath, and likewise keep your server classpath out of your client project whenever possible. The second option is to update jetty in a custom build of GWT. This has been done before, see #9606 <#9606>, which tracks generally updating GWT to a newer Jetty. Even once this happens in mainline GWT though, it may eventually fall behind again, so we strongly encourage keeping client separate from server. -- Finally, I should point out that in the tooling being built for j2cl/gwt3, we are not including a built-in server at all for "dev mode" work, and instead directing teams to use their preferred server's own dev tooling, to guarantee that these classpaths are kept separate and each side can upgrade at its own pace. I'll leave this open for a bit more discussion, then will close in favor of #9606 <#9606>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#9693 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABU2LXU5VHTGGMQTE55FGATRU72QVANCNFSM4NS3QONA> .

[eliasbalasis]

I am sorry to say but it all sounds very awkward and totally inconvenient.

It sounds like the recommended practice is not to use Jetty.

What is the point of supporting Jetty in GWT tools if support for it is broken in GWT core itself?
Shouldn't the two efforts be aligned?

Of course I will choose to run a custom server for production environments or other but during development with GWT tools it is preferable and much more convenient to use Jetty instead.

Please correct me if I have misunderstood.

[niloc132]

The recommended practice is to use Jetty, or Tomcat, or whatever you like - but not from within GWT. Jetty is not broken inside GWT, but it is necessarily limited to the exact version that GWT was built with. If you want more flexibility than that, the correct answer is to run your own server as you see fit, and tell GWT where to deploy content so that your server will provide it to the browser.

All of the "gwt3" tooling being built around J2CL makes the same assumption - the server code is outside of the turn-java-into-js code, so that this isn't an issue in the future. With the server and client decoupled entirely, there won't be this confusion.

[eliasbalasis]

J2CL is a very welcome direction.

In the meantime however, the problem still remains with running SuperDevMode due to Jetty transitive dependencies baseline misalignment it seems.

It is definitely inconvenient having to run a custom server during development instead of the embedded Jetty and at least for GWT 2.9, unlike 2.8.2, I haven't been able to figure out a custom combination of Jetty and other dependencies that works.

I am practically stuck with GWT 2.8.2

[lofidewanto]

There are two main options to overcome this.

The first is the general best practice: only use the internal jetty server for demo-sized applications, once you scale past some certain point (of required features and flexibility, not necessarily code size) you will find that the embedded jetty no longer makes sense. Instead, we strongly encourage you to use -noserver to tell GWT not to run the *:8888 webserver, and point -war/-launchDir at the directory that a normal jetty instance will serve from. This will free you up to run Jetty as you would like, using your IDE or build tooling, any version you want, any classpath etc. I should also point out that your classpaths should be kept separate as well - gwt-user and gwt-dev have no business being on your server's classpath, and likewise keep your server classpath out of your client project whenever possible.

The second option is to update jetty in a custom build of GWT. This has been done before, see #9606, which tracks generally updating GWT to a newer Jetty. Even once this happens in mainline GWT though, it may eventually fall behind again, so we strongly encourage keeping client separate from server.

The third option, which I use all the time: just use GWT internal Jetty for development (normal). If you want to deploy you just "copy" the JavaScript files with Maven to your real server web app. Remember the result is always JS so it will work everywhere the same as long as you use the same web browsers for testing ;-)

Here is my example with Spring Boot: https://github.com/gwtboot/domino-rest-enum-date

Hope this helps.

[eliasbalasis]

Well said, I cannot agree more with Jetty limitations in terms of features and flexibility.

Yet, UI applications are generally expected not to be monolithic, performing basic functions like CRUD which is expected not to exceed Jetty capabilities.
It is implied that any heavy work should be delegated to other applications.

This is the practice my team has been following quite successfully.
Keeping everything as simple as possible and making use of DevMode has helped increase our productivity.
We run everything in the IDE.

Of course, DevMode classpath issues due to misalignment with Jetty baseline have inevitably become a bottleneck for us and may keep being a pain for people wanting to keep using DevMode during development, unless the dependency baselines are aligned (see #9720)

The approach you follow is interesting though.
I had similar thoughts in the past to keep the GWT app classpath clean and support splitting of front-end and back-end work across different teams.

[lofidewanto]

The third option, which I use all the time: just use GWT internal Jetty for development (normal). If you want to deploy you just "copy" the JavaScript files with Maven to your real server web app. Remember the result is always JS so it will work everywhere the same as long as you use the same web browsers for testing ;-)

Here is my example with Spring Boot: https://github.com/gwtboot/domino-rest-enum-date

With the approach above, you can work very fast and have a very fast turnaround time...

I also "mock" every server "services" with Java in my client "UI" and just exchange them with different "modules":

• one for development with mock
• one for deployment on CI/CD, with real server "services"

So you can work with your client UI stand-alone without any server component. To work with UI you need to be fast and you should be able to look at your result without waiting on the server part, IMO.

Maybe I'll make an example for this ;-)

[eliasbalasis]

Well said @lofidewanto, we can try this separately,
I am keen to apply this structure in one of my team's future projects.

However, the original problem remains.
In my view, DevMode classpath issues due to misalignment with Jetty dependencies baseline have inevitably become a bottleneck and may keep being a pain for people still having to use DevMode during development, even if #9606 is implemented, unless the dependency baselines are aligned (see #9720)

[tbroyer]

May I ask why you "have to" use DevMode?

DevMode initially made a (very bad IMO) decision in mixing the DevMode classpath with the webapp classloader. This has caused a lot of issues over the years for anyone doing more than a "demo size" project: Spring seeing its XML files twice, issues with logging libraries, classloader issues ("cannot cast class p.C to p.C"), etc.
I proposed several times that we stop doing that and just serve the webapp with a standard servlet classloader, but some people took advantage of it and it would break their workflow.
So the decision had been that we tell people that they shouldn't use DevMode's embedded server, and use a separate servlet container; without removing that "broken feature" from GWT to not break those who already used it successfully.

My TL;DR would be: DevMode is deprecated, use CodeServer instead.

[eliasbalasis]

Allow me to say and please don't take this the wrong way, but why do we have to change our recipes?

Well said, It was decided that DevMode should be preserved.
However DevMode does increase my team's productivity and it can keep doing so as long as we keep the UI apps very simple or even if we start following structures similar to the one @lofidewanto proposed above.

Regardless, the classpath issues will still be an obstacle and I would like something to be done about it, if possible
(see #9606 and #9720)

Meanwhile, after having studied the source code deeper I believe I have found the source of the problem (see #9720)

[tbroyer]

Back to the originally reported issue: I can reproduce the error with GWT 2.8.2, so at least this is not a regression, and rather a request for enhancement to support "module JARs" in DevMode's embedded Jetty.

@gruenewa if you have a MCVE that works with 2.8.2 and fails with 2.9.0 and could be investigated, that would be welcome; but it looks like you're suffering from dependency resolution choices where your app will use whichever version GWT comes with and/or you (or IntelliJ) put the server dependencies into the DevMode classpath (in addition to the WEB-INF/lib), possibly shadowing GWT's own versions (preventing you from using ASM 5). If that's the case, then you need to split your project to separate the server dependencies (that you'll put into the WEB-INF/lib, inclusing ASM 5, so it won't break the Jetty class scanner) and the client dependencies (without ASM 5 then, so it won't shadow GWT's own ASM 7).

As said above, there are 2 possibilities here:

• update Jetty, this is already covered by Update jetty-9.2.14.v20151106 in gwt-dev to more recent version 9.4+ #9606
• stop using DevMode to serve the webapp and deploy it into a separate server (and on GWT's side, possibly remove that possibility entirely; or at a minimum add a bold warning to the documentation that this should only be used for "demo-size projects" and could easily break as you add external dependencies to your project, at which point you should just stop using it)

There's a workaround too: https://wiki.eclipse.org/Jetty/Howto/Avoid_slow_deployment; put the XML snippet (adapted to match the JARs that need to be scanned, possibly none) into a WEB-INF/jetty-web.xml file.

[eliasbalasis]

I have finally found a workaround.

study https://github.com/eliasbalasis/eclipse-gwt-recipe

Import this Maven project into Eclipse
and
use "GWT Eclipse Plugin" with "Super Development Mode" or any other DevMode combination.

This should reproduce the problem.

Enable exclusions section around <artifactId>jetty-annotations</artifactId> at modules\gwt-web-eclipse\pom.xml to bypass the problem (read explanation in comments section).

I still strongly believe that the GWT classpath must be aligned with that of the associated Jetty though.

[tbroyer]

I still strongly believe that the GWT classpath must be aligned with that of the associated Jetty though.

Speaking of ASM specifically; ASM has a backward-compatibility contract since version 4; that means you can use ASM 9 at runtime with bytecode that was compiled against ASM 4 and it will Just Work™. This is what GWT does: uses ASM 7, with a version of Jetty that has been compiled against ASM 5; Jetty's code compiled against ASM 5 will work just as well with ASM 7 at runtime. You can read more about the ASM backward compatibility contract in https://asm.ow2.io/asm4-guide.pdf

The problem here is that:

• Jetty scans all JARs in the webapp (in WEB-INF/lib because that's required by the Servlets 2.5 spec), or more accurately, all JARs that match the org.eclipse.jetty.server.webapp.WebInfIncludeJarPattern attribute, which defaults to matching everything.
• ASM 5 was incompatible with module-info.class, so any such class in the WEB-INF/lib will generate an error (the one we're concerned about it). This remains true even if ASM 7 is used at runtime, per ASM's backward compatibility contract.
• the actual problem: the project's setup is such that it puts one such JAR in WEB-INF/lib, namely here ASM 7. This is not GWT's fault, it's either the project's needs, or the tooling around GWT (Maven? GWT Eclipse Plugin? I don't really know).

If ASM > 6 comes from the project's needs, then it has to be changed to no longer use the "embedded server" from DevMode, because that one doesn't support JARs that include module-info.class files (classes compiled for recent JDKs could cause errors too).
Updating GWT to a more recent Jetty version, compiled against a more recent ASM version and/or with specific support for module-info.class files, would fix the problem too indeed; the question is whether we want to do it and continue to support this feature, meaning updating Jetty regularly (and it's not that easy, as Jetty is known to break its APIs even in bugfix releases), updating ASM regularly, etc. and of course cutting a new release only for ensuring that this specific feature works‼ (a feature that we think people shouldn't be using for anything more than "demo-size projects")

If the project needs ASM but could run with ASM 5, and it gets upgraded to ASM 7 due to dependency resolution (because server dependencies are mixed with client dependencies); then the project needs to be changed to clearly separate the server and client dependencies. That way, ASM 5 can be put in WEB-INF/lib and GWT can run with ASM 7 in its classpath.

And of course if the project doesn't need ASM, then it means the tooling that puts ASM (coming from GWT dependencies then) to WEB-INF/lib has to be fixed, or the project needs to be configured differently such that the tooling doesn't put the JAR there; but this is outside the scope of GWT anyway.

Finally, if the project doesn't need ASM but is downgrading to an earlier version due to dependency resolution, then the project needs to be configured differently to stop downgrading ASM. This was apparently your problem.

[eliasbalasis]

Well said.

Under https://github.com/eliasbalasis/eclipse-gwt-recipe, I am doing exactly what you are describing, manipulating the classpath to exclude the JARs that Jetty and DevMode conflict with, primarily letting "Jetty" resolve the otherwise conflicting dependencies through the parent classloader instead of "WEB-INF/lib", and it worked.

Updating GWT to a more recent Jetty version, compiled against a more recent ASM version and/or with specific support for module-info.class files, would fix the problem too indeed; the question is whether we want to do it and continue to support this feature,

I agree, this is what I have been trying to explain.
Upgrading Jetty makes a lot of sense from my point of view, but, I am not sure either whether or not we should continue upgrading it going forward or for how long we would be able to do so.
I have confidence that we can keep upgrading Jetty without any side effects.

And of course if the project doesn't need ASM, then it means the tooling that puts ASM (coming from GWT dependencies then) to WEB-INF/lib has to be fixed, or the project needs to be configured differently such that the tooling doesn't put the JAR there; but this is outside the scope of GWT anyway. This was apparently your problem.

Assuming the application introduces the same or a later version of ASM to the one GWT and Jety were built against, it can be resolved with the same classpath manipulation technique.

[niloc132]

GWT 2.10 will be released in the next few days, marking this as closed.