GeoNetwork 5 user interface #8451
Comments
|
"Offer a continuity of functionality with the current" and "Abandon once and for all AngularJS" sounds like a good plan but in a number of presentations or blog posts, "geonetwork-ui" is presented as a "revolution". It has also been discussed a couple of times that "geonetwork-ui" would probably not replace the current UI as a whole because the "audience is too broad" and "one-size-fits-all" approach as you pointed out. Revolution and continuity is probably something counter-intuitive to put together and that we should rephrase to what is the overall goal of GeoNetwork if both should occur? From existing applications built with https://github.com/geonetwork/geonetwork-ui, it seems that they've been focusing on a particular type of metadata with a custom metadata model (dataset only - with discussion about services), which seems restrictive compared to what GeoNetwork was designed initially (ie. schema plugins). What is the impact of this proposal toward managing a large variety of types of resources with fine grained models? If we fix the model and switch from an almost "schema-less" model (Dublin core, ISO, ISO plugins, ...) to a fixed one, would it make sense to fix it server side first (and simplifying database storage and everything) and then expose this model in the API and then use it on client apps? What is the benefit of having schema plugins if the client app provides a fixed model? Beyond UI/UX and technologies, concepts used in "geonetwork-ui" apps are significantly different from GeoNetwork (eg. a metadata may be named "dataset", a draft is something else). Before adopting "geonetwork-ui" for the future GeoNetwork core user interface, it would be relevant to agree on the concepts and how to use them. When looking at the current GeoNetwork core user interface as a whole, a variety of components are required. "geonetwork-ui" is based on https://material.angular.io/ but a number of "classic" UI components have been created from scratch (eg. https://geonetwork.github.io/geonetwork-ui/main/storybook/demo/?path=/story/layout-carouselcomponent--primary). The scope of the actual UI is quite well known for search/map/editor/admin apps, using a more complete component library (eg. https://primeng.org/carousel, https://ng-bootstrap.github.io/#/components/carousel/examples) would probably facilitate the overall work (eg. for some advanced components https://primeng.org/picklist), providing more flexibility, lowering the cost of maintenance when updating Angular and lowering the knowledge required to have a team concentrating on the catalogue user stories and related "high-level" components. Adding "implement missing functionalities" in "geonetwork-ui" seems something that should be proposed to the "geonetwork-ui" project. This sounds quite important because as seen previously concepts are not all inlines, most of the initial work on the datahub was supported by the geOrchestra community and it is the geOrchestra home page app, "geonetwork-ui" also contains components related to other project server side packages (eg. datafeeder). So there is maybe also a question of representativity here. As a GeoNetwork PSC member, I don't feel legitimate for the "geonetwork-ui" project to choose who can become a committer or vote for a proposal... So this is maybe something to clarify. About repository, indeed having a dedicated frontend repository has a number of advantages but we also discussed recently the need of trying to be able to plug into the application modules (eg. harvester) composed of both server side and client side code. Take the datafeeder for example, client side code in "geonetwork-ui" repo, server side code in geOrchestra. Is a mono frontend repo for lib and apps a proper approach for building such a modular system? Although what has been done in the "geonetwork-ui" repository is of high level, I feel that it may not be the best match for a GeoNetwork evolution in continuity, if continuity represents users, peoples, companies and organisations actually contributing to core, but it is already and definitely a good source of inspiration. So it may be important to agree first on core concepts (among others: schema plugins, all-in-one or modules, type of resources, ...) and if any revolution has to occur or not and then we may be able to "establish a strategy". |
|
Hey @fxprunayre
I'm not sure about the use of the word "revolution" to be honest, I can't really remember. There were many discussions already about using GN-UI to replace all of the current GN4 interface. GN-UI was always intended to be a replacement to the AngularJS UI eventually, the reasons why it hasn't happened yet are multiple and maybe not that relevant here.
GN-UI can show whatever records are present in the ElasticSearch index. There is no technical limitation here, only the lack of intent so far.
This is a wider consideration than goes beyond this proposal. GN-UI simply renders whatever it is given either as XML records or documents from ElasticSearch (similar to the GN4 interface). That said, one obvious benefit of schema plugins is the ability to index metadata records of many different forms and shapes under a common index document model.
The way things are shown and named in GN-UI is usage-driven. If usage dictates that metadata should be called metadata, then so be it!
Using PrimeNG is absolutely something that can be considered, although I would call this an implementation detail. A lot of effort happened on the presentation layer (UI components) in GN-UI because the added value of this project has always been a very good user experience. That includes sticking precisely to a design, which is not always practical with component libraries.
Datafeeder is indeed mostly a geOrchestra app and its fate is still unclear for now. Please keep in mind that the work on this app also helped push the GN-UI project forward. This being said, most of the recent funding comes from non-geOrchestra customers, e.g. Swisstopo in CH and IGN in FR.
I'm sorry but I'm not sure I understood this part. Are you saying that the GN-UI project shouldn't have its own group of committers?
Grouping code by domain (e.g. harvesting, editing, etc.) instead of by architecture (e.g. frontend, backend) would be a completely different paradigm. But so far I haven't seen something like that clearly expressed for GN5. The planned architecture of GN5 seems to be a monolithic server application similar to GN4. There will always be the possibility to add modules/plugins in the backend and write a custom frontend app to interact with their own API. If a plugin lives in a separate repository, then this repository could contain both the backend and frontend code.
Let me know if my answer helped clarify things, and hopefully we can agree on those topics sooner than later. |
|
Thanks for the proposal @jahow! I subscribe to the high level concept of building separate Apps for clearly defined audiences. This should provide developers the ability to develop new Apps for things we currently don't even know we need them. So what I would like to see is a repository of Apps and a separate repository with a component library that can be used to build an App. But not intertwined in one geonetwork-ui repository. The Apps provided can be a full-fledged editor that is controlled by the metadata schemas as we currently know in GN4, or one that is build as you currently do in gn-ui. Same for the search front end. Datahub is an option, as well as a front end that has a different perspective on a search interface and can be oriented to a different audience. The same is true for every other type of App developed. |
|
Thanks also for the contributions @fxprunayre! I think my response above also aligns with that. @jahow, my idea is that we actually work towards a model where the back end may be "monolitic" but extensible, while the front end could consist of multiple isolated applications that target specific use cases. Those then use the same back end services, authentication and authorisation mechanisms. |
In my opinion this is really hard to do in practice. Any feature implemented in an application will require in 99% of cases to modify both the app code and the components it uses. There is simply no way to come up with a components library that won't evolve over time to meet the apps needs. This means that having both app and component code in separate repositories will be a huge hurdle as time goes on, because development will be done on both in parallel. The monorepo paradigm is essentially a response to this, as it allows to gather all development efforts in one repository while offering tools to run tasks only on affected parts of the apps-libs graph. Please also keep in mind that presentation components are only a fraction of what has been done in GN-UI. A big part of the efforts went into creating a search system, map-related functionalities, data visualization, querying the API for keywords, thesauri, translations, etc. And these can be used with any presentation component. |
|
Just to be clear, I would definitely understand if this proposal doesn't go through because of all the work done in https://github.com/GeoCat/geonetwork/tree/features/frontend-exp. GN-UI can stay as it currently is, but I just feel like having a clear and sound strategy for the UI of GeoNetwork would be beneficial for the community as a whole. |
|
Thank you @jahow for this great and detailed proposal. First of all, I think that we should not confuse geonetwork-ui and the datahub Our proposal is
geonetwork-ui has benefited from hundreds of days of development, supervised by frontend architects and experts.
Honestly, I wouldn't really understand why the GeoNetwork project wouldn't build on top of this solid foundations, and would rather rewrite it all from scratch. As @jahow said, presentation is one thing, but the major value of geonetwork-ui is not the UI components, but what we called the smart components (+ all the architecture of the repository).
You could build any application, with any UI requirement, with any UI components on top of this logic, but it would just be a waste to rewrite all the core logic. Now, let's try to answer some of your feedback
I don't see the point here. Why not bringing revolution in GeoNetwork 5 ? Why would it cut with continuity ?
There is no restriction in geonetwork-ui about using dataset only. Again, it's something about the datahub, which is about to change (services, maps and most of all resource types will be handled).
I agree, we called this a
You are right. When choosing a way to implement a UI component, we have to think about the tradeoffs :
We've built some components, we use some ng-material ones (which is the official UI library from Angular), we use external UI components from external libraries... It's not always easy to choose the correct solution. We use ng-prime in another project and I wanted to use the Note that this is a presentation detail, and that we could use other UI libraries within geonetwork-ui repository, or within new applications. This is not a limitation in geonetwork-ui.
As already mentioned, the datafeeder should be removed from geonetwork-ui.
I don't know if it's really "high level", what I know is that it's very important emphase on the design of the architecture, to build clean and strong foundations so that it's easy
From my experience, a one man system growing and growing won't result in a code base which will be easy to maintain, develop, and onboard people.
An application in geonetwork-ui could not rely at all on the library of components of the repo. It would be a waste not to use the logic, but it would be perfectly fine not to use the UI components. @ticheler could you please develop maybe a bit your vision about your repo (which would not be a monorepo), but which would contain several applications ? How do you see this technically, one Angular application per folder, or an Angular space with several application but no library ? Having a monorepo ensures that all the applications are on the same code base (at least the same version of Angular), and that the reused code is shared between the applications. I don't really see the benefit of building several applications completely independent from one to the others .. like one app for the search, one for the admin and one for the editor ? It's not clear to me. That's all for my feedback. |
The prototyping (which was not that much work) done this summer in https://github.com/GeoCat/geonetwork/tree/features/frontend-exp is mainly about analyzing how to do a progressive move from AngularJS to something else in continuity with some of the actual base concepts in use by existing users and define a way for an (client and server) evolution rather than a (client facade only) revolution. This is not an easy exercise due to the complexity of the project and a strategy need to be defined to progressively move from one camp to another and another ... eg.
GN-UI definitely set up quite some logic (with new UI configuration model, a new themes mechanism, a new metadata model, a new search model, a new editor config, "forget about metadata schema"...) but with limited consideration about the (maybe wrong, outdated, whatever...) principles of the current app. Again, having some inline concepts or minimum convergence between server and client side and with the past approaches would facilitate adoption.
I still see a couple of names on the commit history https://github.com/geonetwork/core-geonetwork/commits/main/ but that's definitely a challenge for an old project like GeoNetwork. Again, coding a client facade with the latest framework will not help the project as a whole. GeoNetwork 5 proposal is all about defining strategy to cut complexity, improve performances, redefine business model taking into consideration our history and nowadays needs, ... In 2022, around Bolsena's dinner table, a company explained that the level of the project was not anymore good enough for their standards and in a rather solmn manner announced to abandon maintenance of the core project. This doesn't sound the best way to collaborate on the long term and onboard people. Java may be scary and XSLT even worse, and Angular ..., but the project should be considered as a whole with its active and past contributors, users and history (thanks to all of them).
Maybe, maybe not, but I think the point raised by Jeroen is also about how to build an extension/app composed of server and client side components and how to organize it in repository(ies). The data feeder is an app and it probably makes sense to have server side and client side code base in the same repo even if it depends on this library? Datahub is client side only, an editor app, an harvester app would also be client/server side. How the various types of extensions could be properly organized and how to make them pluggable into the system?
I agree |
|
Hey 🙂 I feel like we're getting sidetracked here; the proposal lays out a potential strategy for moving forward on this topic, maybe we can get back to vote/discuss it? The essential points of the proposal are:
|
|
I am finding this proposal verbose, with lots of background, but unclear to me what the structure ends up being proposed? With GeoNetwork 5 we end up with a little constellation of spring-boot apps: ---
title: GeoNetwork 5 Deployment
---
classDiagram
note "Forgive abuse of class diagram"
GeoNetwork5 <|-- GeoNetwork4
GeoNetwork5 <|-- Database
GeoNetwork4 <|-- Database
GeoNetwork5 <|-- Elasticsearch
GeoNetwork4 <|-- Elasticsearch
class GeoNetwork5 {
Spring Boot 6
Java 21
}
class GeoNetwork4 {
Spring Framework 5.3
Java 11
}
class Database {
metadata
}
class Elasticsearch {
index
}
above: GeoNetwork 5 is operating as a gateway routing requests to GeoNetwork 4 during the transition period. Running these distinct Apps (Java 11 and Java 21) assume docker compose file, or Kubernetes environment. So my question is what the GeoNetwork 5 user interface proposal covers? I assume it is proposing another App in the above deployment, but I am confused about how it is phrased in the proposal.
Priority is REST API which is best served by having several clients: geonetwork-ui search front ends already in use would be excellent to keep API stable. I am fond of GeoCat Bridge REST API also. Direct usability of REST API would be wonderful to consider also. I really want to see a stable foundation that we can all trust to develop against. I am personally looking forward to clean break with GeoNework 4 AngularJS user interface, so I am confused by "continuity with GeoNetwork 4" phase above. It would be great if geonetwork-ui project could maintain an active fork (and whatever test coverage is available) against a geonetwork 5 branch as it is developed. There is not really any good way to understand what REST API functionality is actively used, and how changes to Elasticsearch index structure affects different UI components...
Because of the Java 11 / Java 21 split we are forced into doing an App approach. I expect any bundling would be via published docker compose or Kubernetes graph.
I would like to see reuse. For me it would be okay if we have several apps in src/apps that depend on artifacts published by different repositories. I guess I would like to see a comparison with other cloud native projects to understand best practice.
|
|
Hey @jodygarnett,
No, the proposal says that a frontend application (built by GN-UI) will be served by the GN5 backend. This does not add any actual apps or deployments in the mix above.
100% agreed, but I don't think this is in the scope of this proposal. GN-UI can and will adapt to any API it needs to.
This means that functionalities currently available in the GN4 ui should be present in GN5 ui in a more or less similar form, unless the functionality is deemed obsolete. This has been consistently asked by the PSC regarding the GN-UI project.
This could actually be done right away without the need for a fork. I'll see if we can find some budget to do that work. Thanks for the idea!
I'm sure we can find a way to "embed" a frontend app in the GN5 java app and have it serve it as well. See above, this would not require any additional deployment.
Yes, that sounds good I guess. Again, I'm not 100% I understand the "reuse" term here, do you mean that other apps should reuse the same components? If so, please see the discussions above, esp. the points raised by @ticheler.
In my understanding GN5 will not be a cloud native project at heart. It will be a monolithic java web app with the ability to be extended by plugins/modules, but to me it makes sense that such an app comes with a built-in frontend app (just like Geoserver, GeoNetwork 4, etc.). Also GS-cloud comes with a web-ui module that is optional but doesn't really have equivalents. |
|
@jahow I am still seeking to assist on this activity prior to the now delayed PSC meeting. |
|
Thank you but I'm still not sure what to do here. I won't be there for the PSC meeting. |
|
Lots of ideas from PSC meeting and chat about GN5:
So a great goal for this activity would be a public facing geonetwork-ui only deployment. This would allow an organization to have a geonetwork system that does not contain any AngularJS. As GN5 becomes fully capable the GN4 delegate can be removed for publication of data. In terms of project stability having such an app with integration tests would aid development and allow us to know when GN5 and accomplished such a goal. ---
title: Transition Public Facing App Deployment
---
classDiagram
note for GeoNetwork5 "geonetwork-ui and minimal GN5 modules"
note for GeoNetwork4 "minimal REST API and no AngularJS"
GeoNetwork5 <|-- GeoNetwork4
GeoNetwork5 <|-- Database
GeoNetwork4 <|-- Database
GeoNetwork5 <|-- Elasticsearch
GeoNetwork4 <|-- Elasticsearch
class GeoNetwork5 {
geonetwork-ui
gn-searching
gn-domain
gn-index-model
Spring Boot 6
Java 21
}
class GeoNetwork4 {
gn-core
gn-main
gn-services
Spring Framework 5.3
Java 11
}
class Database {
metadata
}
class Elasticsearch {
index
}
Notes:
I would love to see a minimal deployment to only needing GN5 App and Elasticsearch (no database) for a scalable minimal public deployment with no AngularJS.
---
title: Scalable Public Facing App Deployment
---
classDiagram
note for GeoNetwork5 "geonetwork-ui and minimal GN5 modules"
GeoNetwork5 <|-- Elastic search
class GeoNetwork5 {
geonetwork-ui
gn-searching
gn-domain
gn-index-model
Spring Boot 6
Java 21
}
class Elasticsearch {
index
}
Counterpoint I would love to see a database only approach using PostgreSQL (and not have the user interface tightly bound to Elasticsearch) as part of a small minimal approach for municipalities or similar. ---
title: Database GN5 Small App
---
classDiagram
note for GeoNetwork5 "database use with full-text search"
GeoNetwork5 <|-- Databbase
class GeoNetwork5 {
geonetwork-frontend-ui
gn-searching
gn-domain
gn-index-model
Spring Boot 6
Java 21
}
class Databbase {
index
record
}
|
|
Thank you @jody for your commitment to this proposal; it took me a while but I'll hopefully answer some of your questions.
I 100% agree, I've started trying to set up a working GN5 docker composition to run GN-UI against it.
There is an abstraction between most library code of GN-UI and the use of ElasticSearch API. GN-UI has a
This is a fair point. Camptocamp has so far been busy without interruption on GN-UI since 2+ years. The team working on GN-UI has also grown from ~2 to ~4-5 depending on phases and contracts. There's also a non negligible backlog. This leads me to believe that Camptocamp has no interest to cease contribution in the near future. Another point is that we have been trying to make the GN-UI codebase as accessible as possible. It is a complex project so we try to continuously improve the technical documentation. We have successfully onboarded several developers in the past and learn from this experience every time. I hope this clarify things! |
Introduction
This proposal is meant as a complement to the GeoNetwork 5 proposal.
The goal of this proposal is to establish a strategy for the user-facing interface (frontend) of GeoNetwork 5.
More specifically:
Proposal
The proposed solution is as follows:
Build on the GeoNetwork-UI project
The GeoNetwork-UI project has been around for several years now. Because so far it has lived side-by-side with the GeoNetwork 4 built-in UI, its scope does not currently cover the complete feature set that a long-time GeoNetwork user might expect.
Things that are currently not available are:
Although this list may seem intimidating, two things need to be taken into account here: first, these features will have to be rewritten one way or another, as they haven't existed yet in anything else than AngularJS.
Second, the GeoNetwork-UI project already has very solid foundations such as:
Lastly, one of the most remarkable aspects of the project is that it is "battle tested": the Datahub app has been in production for almost two years without any major issue and its adoption has grown organically to many geonetwork platforms. The Metadata Editor app is on the other hand going through an alpha phase right now and will be soon ready to use in production.
This proposal thus articulates itself essentially around the GeoNetwork-UI project:
Bundle default UI application in GeoNetwork 5
As was demonstrated in this previous proposal it is straightforward to include a GN-UI application in a Java backend. The frontend application will use the authentication mechanism offered by the backend.
GN-UI offers several applications targeted to different use cases, so this is what this proposal includes:
Keep the UI code in a separate repository
This has been the choice made for GeoNetwork-UI from day one and it has brought significant advantages:
As a result, this proposal simply advises to keep the GeoNetwork-UI code as it is currently: a separate repository organized in a "monorepo" fashion.
"Monorepo" is a term used to describe the fact that the project contains several libraries and multiple applications using them. This means that several applications can be built using the same set of components and services, thus allowing to build applications tailored to specific use-cases.
This is important as GeoNetwork-UI was designed with the intention to avoid having a unique "one-size-fits-all" application as was the case with the built-in GeoNetwork 4 UI. This approach proved very difficult to maintain in the long run, and also turned into a nearly impossible challenge in terms of UX as the intended audience was simply too broad.
The monorepo architecture of GeoNetwork-UI has stood the test of time and it is now clear that it can scale up to include new applications in the future.
Final words
This proposal is not intended to lead to implementation right away. If approved, separate proposals for each of the points above will be submitted with mockups and more detailed information.
One of the challenges of this proposal is related to the fact that many GeoNetwork contributors have not yet started working on GeoNetwork-UI. To address this, Camptocamp (main GN-UI contributor) will further expand the technical documentation of the project to make it easier for newcomers to understand the code base.
Voting
PSC Support:
Community Support:
The text was updated successfully, but these errors were encountered: