From 7f75adbf0b0f6ffdbfc497e13fc561e760f704b2 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Wed, 3 Apr 2024 23:58:24 +0100 Subject: [PATCH 01/11] doc'ing --- docs/features/feature-list.md | 208 ++++++++++++++++++++++++++++++++++ 1 file changed, 208 insertions(+) create mode 100644 docs/features/feature-list.md diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md new file mode 100644 index 0000000000..b2d624a4c6 --- /dev/null +++ b/docs/features/feature-list.md @@ -0,0 +1,208 @@ +Original file from Wells Fargo. They want an open source view of Waltz features and capabilities + + +Capability / Feature | Waltz +-- | -- +Enterprise Architecture Management | +Standards | +Patterns | +Authoritative Source | Ability to disposition for a given capability - what is the authoritative source for the data; can accommodate for heritage scenarios. +Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) +Manage Architectures | +Models | +Assessments | +Portfolio Management | +Application Rationalization | +Roadmaps (application, products) | +Functional commissions/de-commissions | Ability to show when capabilities start/stop within a portfolio +Lifecycle Mappings | +Data Lineage | Logical flows captured (point to point with data-type metadata via integration method) with visualization/mapping abilities. Significant maturity planned. +Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. +Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. +Cost allocations - Taxonomies | Ability to demarcate the percentage of costs (shredding) for the purposes of planning/budgeting of a given function. Application may be performing multiple capabilities and needs to have appropriate allocations for each functional usage +Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability. +Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. +Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. +Analytics / Ad-hoc | +Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). +Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) + | + | +Data | +Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations + | +User Interface | +American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; +Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users +Data Sensitivity | Database level encryption; planned for finer-grained entitlements for sensitive data +System integrations | +CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) +iGrafx / BizzDesign / any other notable vendors | +API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) +Stack | Database - Postgresql ; JooQ (transformation layer/ DSL); Java (v8+>>11/17); JooQ; immutables library; Spark API DSL >> javelin >> springboot. Liquidbase +Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 +Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed +How are modifications to the application made (business logic and/or data structures)? | +Cost / Licensing | + + +## Applications + +One of the primary entities in Waltz is the _Application_. +Applications have some basic properties such _name_, _type_, and _description_. +Additional properties include _lifecycle_ indicators (status, dates) and _organizational ownership_ information. + +### Extensions + +As with other entities in Waltz, applications can be enriched with additional data points: + +- _Involvements_ which link _people_ to apps via a formal role (e.g. Architect). + - Involvements can be used for fine-grained permissioning. +- _Assessments_ are used to associate arbitrary enumerated data points with an application. +These can be grouped and are either imported from other systems as read-only values or captured and managed directly in Waltz. +Assessments are versatile and have been used for a wide range of purposes including: + - Cloud migration tracking + - Legal Holds + - Information Classification and Criticality + - SDLC compliance indicators + - Recovery time objectives +- Categorized _Bookmarks_ allow for a standardized way of collecting a common set of links to additional information resources about the application +- _Taxonomy Mappings_, which are further discussed below + +## Taxonomies and Taxonomy Mappings + +Organizations can create multiple _Taxonomies_ in Waltz. +Commonly these will be similar to: + +- Business Function +- Capability +- Process +- Regulation +- SDLC Compliance +- Records + +- Portfolio Management +- App rationalisation +- Functional decomms and roadmaps +- Extensions (e.g. assessments) +- Costs and Allocations + +-------- + +## Data Management Features + +### Data Types + +Waltz supports a hierarchical taxonomy of data types. +Each type may be declared as `concrete` indicating it is sufficiently granular to be used when documenting flows. + +### Data Flows + +Logical flows (generic) and physical flows (specific descriptions of flows) can be captured and linked to data types. + +Each logical flow provides information about a data flow between a source and target system. +These systems are typically applications but may be _Actors_ which are used to represent internal and external entities such as _Exchanges_ and _Risk Officers_. +Taken together, the flows provide a directed graph which shows the raw topology of your organization. + +The Waltz UI allows flows to be maintained and has functionality to do more advanced operations such as _merging_ duplicate flows. + +Flows and their data types can be attested by either a named set of users or users specifically associated (via a specific role) with the source or target applications. + +### Lineage + +Waltz has rudimentary support for describing lineage using _Flow Diagrams_, these diagrams are created using a _constrained editor_ which only allows registered flows to be used. + +Each diagram is stored as a _Bill-of-Materials_ allowing aggregated information to be calculated, _Aggregate views_ are discussed later. +The diagrams allow for overlaying and filtering, for example you could overlay an indicator showing which of the application in the diagram perform _Trade Capture_. + +For more detailed and comprehensive support for lineage we commonly use the data in Waltz with systems such as _Solidatus_. +Alternatively, the information can be exported into a graph database for further analysis using custom tooling. + +### Flow Classifications + +Rules can be created to capture concepts such as _Authority Statements_. +Each rule defines a source, data type, consumer set, and outcome. +For example: '_BookMgr_ (source) is an _Authoritative Source_ (outcome) of _Book Data_ (data type) for the _Investment Bank_ (consumer set)'. + +Defining these rules allows them to be captured and applied to the data flow model to measure 'right sourcing' within an organisation. + +### Data Assessments and other extensions + +Flows, data types, and related entities can use the standard Waltz enrichment mechanisms. +Particularly useful are assessments, bookmarks, and links to entities such as people (e.g. _Data Owners_) and groups. + +-------- + +## Surveys + +-------- + +## Aggregate views + +### Vantage points + +- By taxonomy +- By people +- By data type +- By org unit +- By diagram +- By arbitrary group + +### Views + +- Flows +- Attestations +- Taxonimies +- Report grids + + +#### Bank on a Page + +Waltz has the ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. + + + +-------- + +## Integration + +Waltz ships with **no** _out of the box_ integrations. +Integration is typically done via periodic batch jobs which load data into the Waltz database. + +Users have integrated with systems such as Collibra, Solidatus, Service Now, Apptio, Sparx Enterprise Architect and more. +We are hoping to offer standardised intgerations in the future, with Collibra (Cloud) being the likely first offering. + +--------- + +## Technical Specs + +### Frontend + +Major libraries / frameworks being used: + +- Svelte, a lightweight performant framework being used in Waltz to replace the legacy AngularJS code +- D3 for data visualization +- Lodash for data manipulation +- Slickgrid for large tables, being used to replace UI-Grid (AngularJS based grid library) +- Sass, CSS pre-processor +- Webpack, build framework +- Mocha, unit testing + +### Server + +- Java, currently supporting 8, but shortly moving to JDK 11 or 17 (tbc) +- Spring framework for auto-wiring components +- Spark Java, a lightweight framework for creating REST endpoints +- jOOQ, a java DSL for writing and executing database queries +- Immutables for domain objects + +### Database + +- Production databases: MSSQL, Postgres +- Test databases: H2 + +### Deployment options + +- Webcontainer, e.g. Tomcat +- Standalone JAR +- Docker image From eed90872ea5e09093a9968b8f41106896e1d97a4 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 17:31:20 +0100 Subject: [PATCH 02/11] doc'ing --- docs/features/feature-list.md | 41 +++++++++++++++++++++++++---------- 1 file changed, 29 insertions(+), 12 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index b2d624a4c6..9bcced060d 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -72,20 +72,37 @@ Assessments are versatile and have been used for a wide range of purposes includ ## Taxonomies and Taxonomy Mappings Organizations can create multiple _Taxonomies_ in Waltz. -Commonly these will be similar to: - -- Business Function -- Capability -- Process -- Regulation -- SDLC Compliance +Each taxonomy is a hierarchical collection of related elements. +Examples of these taxonomies are: + +- Business Functions +- Capabilities +- Processes +- Regulations +- Agile Tribes - Records -- Portfolio Management -- App rationalisation -- Functional decomms and roadmaps -- Extensions (e.g. assessments) -- Costs and Allocations +### Portfolio Management +Organizations can describe and document their application portfolio using _mappings_ to these taxonomies. +Each mapping has an associated _rating_, typically a buy-sell-hold indicator, but the _rating scheme_ can be customised. + +The mappings may also be flagged as _primary_ for an application, which allows for more contextual information to be presented when lists of apps are shown elsewhere in Waltz. + +### Roadmaps +Decommission and commission information can be associated with each mapping, allowing detailed roadmaps, shwoing how the portfolio will change over time, to be sourced from the captured data. + +### Allocations +The mappings may have allocation percentages associated to them, this is commonly used to allow a breakdown of costs per capability for applications (and then aggregated across collections of applications). + +### Rating Assessments +Finally, we recently added the ability to add assessments to ratings. +This supports a more nuanced view of the applications, for example we could add product and location assessments to capability ratings indicating what types of financial products and in what location are being handled by a specific application capability. + +### Application Rationalisation +Once the application portfolio has been mapped against a consistent set of taxonomies it becomes simple to see high-level opportunities for app consolidation/rationalisation by viewing all applications which are associated to a taxonomy item. +These views are aggregated if higher level items in the taxomomy are selected. + +The rating assessments, mentioned above, will provide increased -------- From a569800eb8bae1fba6e74c0c3d7cc55c225474a1 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 20:43:10 +0100 Subject: [PATCH 03/11] wip #6948 --- docs/features/feature-list.md | 143 ++++++++++++++++++++++++++-------- 1 file changed, 110 insertions(+), 33 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index 9bcced060d..cdad25b2f3 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -6,16 +6,16 @@ Capability / Feature | Waltz Enterprise Architecture Management | Standards | Patterns | -Authoritative Source | Ability to disposition for a given capability - what is the authoritative source for the data; can accommodate for heritage scenarios. +Authoritative Source | [Auth sources, including heritage flows](#data-classifications) Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) -Manage Architectures | +Manage Architectures | x Models | -Assessments | -Portfolio Management | -Application Rationalization | -Roadmaps (application, products) | +Assessments | x +Portfolio Management | x +Application Rationalization | x +Roadmaps (application, products) | x Functional commissions/de-commissions | Ability to show when capabilities start/stop within a portfolio -Lifecycle Mappings | +Lifecycle Mappings | x Data Lineage | Logical flows captured (point to point with data-type metadata via integration method) with visualization/mapping abilities. Significant maturity planned. Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. @@ -23,7 +23,7 @@ Cost allocations - Taxonomies | Ability to demarcate the percentage of costs ( Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability. Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. -Analytics / Ad-hoc | +Analytics / Ad-hoc | x Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) | @@ -46,13 +46,13 @@ How are modifications to the application made (business logic and/or data st Cost / Licensing | -## Applications +## Applications One of the primary entities in Waltz is the _Application_. Applications have some basic properties such _name_, _type_, and _description_. Additional properties include _lifecycle_ indicators (status, dates) and _organizational ownership_ information. -### Extensions +### Extensions As with other entities in Waltz, applications can be enriched with additional data points: @@ -69,7 +69,9 @@ Assessments are versatile and have been used for a wide range of purposes includ - Categorized _Bookmarks_ allow for a standardized way of collecting a common set of links to additional information resources about the application - _Taxonomy Mappings_, which are further discussed below -## Taxonomies and Taxonomy Mappings +-------- + +## Taxonomies and Taxonomy Mappings Organizations can create multiple _Taxonomies_ in Waltz. Each taxonomy is a hierarchical collection of related elements. @@ -79,41 +81,40 @@ Examples of these taxonomies are: - Capabilities - Processes - Regulations -- Agile Tribes +- Agile Domains and Tribes - Records -### Portfolio Management +### Portfolio Management + Organizations can describe and document their application portfolio using _mappings_ to these taxonomies. Each mapping has an associated _rating_, typically a buy-sell-hold indicator, but the _rating scheme_ can be customised. The mappings may also be flagged as _primary_ for an application, which allows for more contextual information to be presented when lists of apps are shown elsewhere in Waltz. -### Roadmaps -Decommission and commission information can be associated with each mapping, allowing detailed roadmaps, shwoing how the portfolio will change over time, to be sourced from the captured data. +### Roadmaps +Decommission and commission information can be associated with each mapping, allowing detailed roadmaps, showing how the portfolio will change over time, to be sourced from the captured data. -### Allocations +### Allocations The mappings may have allocation percentages associated to them, this is commonly used to allow a breakdown of costs per capability for applications (and then aggregated across collections of applications). -### Rating Assessments +### Rating Assessments Finally, we recently added the ability to add assessments to ratings. This supports a more nuanced view of the applications, for example we could add product and location assessments to capability ratings indicating what types of financial products and in what location are being handled by a specific application capability. -### Application Rationalisation +### Application Rationalisation Once the application portfolio has been mapped against a consistent set of taxonomies it becomes simple to see high-level opportunities for app consolidation/rationalisation by viewing all applications which are associated to a taxonomy item. -These views are aggregated if higher level items in the taxomomy are selected. -The rating assessments, mentioned above, will provide increased -------- -## Data Management Features +## Data Management Features -### Data Types +### Data Types Waltz supports a hierarchical taxonomy of data types. Each type may be declared as `concrete` indicating it is sufficiently granular to be used when documenting flows. -### Data Flows +### Data Flows Logical flows (generic) and physical flows (specific descriptions of flows) can be captured and linked to data types. @@ -125,7 +126,11 @@ The Waltz UI allows flows to be maintained and has functionality to do more adva Flows and their data types can be attested by either a named set of users or users specifically associated (via a specific role) with the source or target applications. -### Lineage +Physical flows provide more detail about specific data flows. +They can be linked to a specification which provides field level detail with optional linkage to a data dictionary. +Specifications can be shared and are often used to represent standardised data schemas within an organisation. + +### Lineage Waltz has rudimentary support for describing lineage using _Flow Diagrams_, these diagrams are created using a _constrained editor_ which only allows registered flows to be used. @@ -135,27 +140,69 @@ The diagrams allow for overlaying and filtering, for example you could overlay a For more detailed and comprehensive support for lineage we commonly use the data in Waltz with systems such as _Solidatus_. Alternatively, the information can be exported into a graph database for further analysis using custom tooling. -### Flow Classifications +### Flow Classifications Rules can be created to capture concepts such as _Authority Statements_. Each rule defines a source, data type, consumer set, and outcome. -For example: '_BookMgr_ (source) is an _Authoritative Source_ (outcome) of _Book Data_ (data type) for the _Investment Bank_ (consumer set)'. +For example: '_BookMgr_ (source) is an _Authoritative Source_ (outcome) of _Book Data_ (data type) for the _Investment Bank_ (consumer set)'. Defining these rules allows them to be captured and applied to the data flow model to measure 'right sourcing' within an organisation. -### Data Assessments and other extensions +By defining tight (sometimes singular apps) as the consumer set Waltz can allow for concepts such as '_Heritage_' flows - those that are non-authoritative but allowed for legacy reasons. + +### Data Assessments and other extensions Flows, data types, and related entities can use the standard Waltz enrichment mechanisms. Particularly useful are assessments, bookmarks, and links to entities such as people (e.g. _Data Owners_) and groups. -------- -## Surveys +## Surveys + +Waltz provides a survey feature which is deeply integrated into the Waltz data model. +Surveys can be designed and issued against applications (or change initiatives) matching a set of criteria and assigned to users via their involvement to the application. + +For example a survey could be issued to all applications which perform any payment related capability and be assigned to the business analysts and business owners of the relevant apps. + +### Survey Lifecycle + +Surveys have a basic lifecycle support with an optional approval step once the survey is completed. +It is common for custom code to perform additional tasks through the survey lifecycle. +Via custom code survey creation may be triggered via an external event. +It is also common for an assessment to be set upon survey approval, for example a survey pertaining to '_Cloud Onboarding_' may result in a '_Cloud Migration Recommendation_' assessment to be assigned. + +### Survey templates and predicates + +The surveys themselves can be quite sophisticated and reference Waltz model elements (taxonomies, people etc). +The questions can be shown/hidden by predicates which are either based on other answers in the survey, or they may be predicated on environmental attributes related to the application (e.g. only show a question if the application has a '_High Criticality Rating_'). + + +-------- + +## Change Initiatives + +Change Initiatives are used to capture change programmes and projects in an organization. +They can be associated with applications, people and other Waltz entities. +It is common to use change initiatives, surveys and assessments in conjunction to perform '_Architectural governance_' over the organizations change programmes. + + +-------- + +## OSS + +### Library Usage tracking - OSS Risk + + + +### OSS Licence tracking - Licence Risk + -------- ## Aggregate views +These views are aggregated if higher level items in the taxonomy are selected. + ### Vantage points - By taxonomy @@ -169,7 +216,7 @@ Particularly useful are assessments, bookmarks, and links to entities such as pe - Flows - Attestations -- Taxonimies +- Taxonomies - Report grids @@ -178,7 +225,6 @@ Particularly useful are assessments, bookmarks, and links to entities such as pe Waltz has the ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. - -------- ## Integration @@ -187,7 +233,21 @@ Waltz ships with **no** _out of the box_ integrations. Integration is typically done via periodic batch jobs which load data into the Waltz database. Users have integrated with systems such as Collibra, Solidatus, Service Now, Apptio, Sparx Enterprise Architect and more. -We are hoping to offer standardised intgerations in the future, with Collibra (Cloud) being the likely first offering. +We are hoping to offer standardised integrations in the future, with Collibra (Cloud) being the likely first offering. + +### Software Development Lifecycle (SDLC) + +_Integration with service now to provide controls around dq and enforce regular attestations_ + + +--------- + + +## NFR + +### Security + +### Standards --------- @@ -207,7 +267,7 @@ Major libraries / frameworks being used: ### Server -- Java, currently supporting 8, but shortly moving to JDK 11 or 17 (tbc) +- Java, currently supporting 8, but shortly moving to JDK 11 and then 17 - Spring framework for auto-wiring components - Spark Java, a lightweight framework for creating REST endpoints - jOOQ, a java DSL for writing and executing database queries @@ -220,6 +280,23 @@ Major libraries / frameworks being used: ### Deployment options -- Webcontainer, e.g. Tomcat +- Web container, e.g. Tomcat - Standalone JAR - Docker image + +--------- + +# Current Development Themes + +## 2024 + +Currently, the main focus for 2024 is to improve support around data flows and lineage. +This will include extending flow classification rules to include 'authoritative consumers', adding a request based model to flow modifications and registration, and improved support for linking EUDAs to flows. + +In addition to flows, we expect to look at tools to boost data quality, specifically by allowing organizations to set up correlation rules between taxonomies. +An evaluator would evaluate the rules and raise potential issues where data does not '_look right_'. +For instance a rule could be set up to say that the '_KYC_' capability expects to see '_Person_' related data and be involved in one or more of a set of processes. + +Another theme is around the Bank on a Page feature set. +This may involve closer integration with other reporting tools (Tableau, Superset) or we may pursue more custom visualisations. +We also intend to investigate how to integrate more closely with tools such as Archi and other open source diagramming tools. \ No newline at end of file From cd02800e808f96f3a741406fc96383f45aa70fad Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 21:15:26 +0100 Subject: [PATCH 04/11] wip #6948 --- docs/features/feature-list.md | 121 ++++++++++++++++++++-------------- 1 file changed, 71 insertions(+), 50 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index cdad25b2f3..aaad32d0a9 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -1,47 +1,47 @@ Original file from Wells Fargo. They want an open source view of Waltz features and capabilities -Capability / Feature | Waltz --- | -- +Capability / Feature | Waltz | More +-- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------- Enterprise Architecture Management | Standards | Patterns | -Authoritative Source | [Auth sources, including heritage flows](#data-classifications) -Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) -Manage Architectures | x +Authoritative Source | Ability to disposition for a given data type - what is the authoritative source for the data; can accommodate for heritage scenarios | [Auth sources, including heritage flows](#data-classifications) +Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) +Manage Architectures | x Models | -Assessments | x -Portfolio Management | x -Application Rationalization | x -Roadmaps (application, products) | x -Functional commissions/de-commissions | Ability to show when capabilities start/stop within a portfolio -Lifecycle Mappings | x -Data Lineage | Logical flows captured (point to point with data-type metadata via integration method) with visualization/mapping abilities. Significant maturity planned. -Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. -Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. -Cost allocations - Taxonomies | Ability to demarcate the percentage of costs (shredding) for the purposes of planning/budgeting of a given function. Application may be performing multiple capabilities and needs to have appropriate allocations for each functional usage -Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability. -Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. -Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. -Analytics / Ad-hoc | x -Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). -Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) +Assessments | Assessements can be associated to most entities in Waltz | | +Portfolio Management | x +Application Rationalization | x +Roadmaps (application, products) | x +Functional commissions/de-commissions | Ability to show when capabilities start/stop within a portfolio +Lifecycle Mappings | x +Data Lineage | Logical flows captured (point to point with data-type metadata via integration method) with visualization/mapping abilities. Significant maturity planned. +Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. +Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. +Cost allocations - Taxonomies | Ability to demarcate the percentage of costs (shredding) for the purposes of planning/budgeting of a given function. Application may be performing multiple capabilities and needs to have appropriate allocations for each functional usage +Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability. +Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. +Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. +Analytics / Ad-hoc | x +Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). +Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) | | Data | -Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations +Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations | User Interface | -American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; -Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users -Data Sensitivity | Database level encryption; planned for finer-grained entitlements for sensitive data +American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; +Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users +Data Sensitivity | Database level encryption; planned for finer-grained entitlements for sensitive data System integrations | -CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) +CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) iGrafx / BizzDesign / any other notable vendors | -API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) -Stack | Database - Postgresql ; JooQ (transformation layer/ DSL); Java (v8+>>11/17); JooQ; immutables library; Spark API DSL >> javelin >> springboot. Liquidbase -Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 -Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed +API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) +Stack | Database - Postgresql ; JooQ (transformation layer/ DSL); Java (v8+>>11/17); JooQ; immutables library; Spark API DSL >> javelin >> springboot. Liquidbase +Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 +Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed How are modifications to the application made (business logic and/or data structures)? | Cost / Licensing | @@ -188,14 +188,21 @@ It is common to use change initiatives, surveys and assessments in conjunction t -------- -## OSS +## OSS -### Library Usage tracking - OSS Risk +Waltz is used to track consumption of open source in applications across an organisation. +### Library Usage tracking - OSS Risk +You can associate applications with their dependencies (down to the version level) we can quickly cross-reference with any of the other Waltz data points to see how libraries are used. This association is done via external processes, such as custom integration with the [_XRay_](https://jfrog.com/solution-sheet/jfrog-xray/) tool from [_JFrog_](https://jfrog.com/). -### OSS Licence tracking - Licence Risk +### OSS Licence tracking - Licence Risk + +Waltz also allows for OSS licences to be imported (there is a sample loader and an admin ui to support ad-hoc additions). +OSS licences are linked to the software licences mentioned above and are therefore transitively associated to applications. + +Typically several assessments will be attached to licences indicating their suitability for use under specific conditions (internal, public-web, client side installation, etc). -------- @@ -249,40 +256,54 @@ _Integration with service now to provide controls around dq and enforce regular ### Standards +### Capacity + +Waltz is happily serving several hundred users per day, out of a total user base of tens of thousands. +The Waltz instance has information on over 5K applications and a similar number of change initiatives. There are many more dataflows, surveys, and assessments managed by Waltz. +Several of the taxonomies stretch to thousands of nodes with many more related mappings. + +This is all served by a load balanced pair of Tomcat servers with on a shared JVM of just 2GB. +The (SQLServer) database has 8GB of memory. + --------- -## Technical Specs +## Technical Details ### Frontend Major libraries / frameworks being used: -- Svelte, a lightweight performant framework being used in Waltz to replace the legacy AngularJS code -- D3 for data visualization -- Lodash for data manipulation -- Slickgrid for large tables, being used to replace UI-Grid (AngularJS based grid library) -- Sass, CSS pre-processor -- Webpack, build framework -- Mocha, unit testing +- [Svelte](https://svelte.dev/), a lightweight performant framework being used in Waltz to replace the legacy [AngularJS](https://angularjs.org/) code +- [D3](https://d3js.org/) for data visualization +- [Lodash](https://lodash.com/) for data manipulation +- [Slickgrid](https://slickgrid.net/) for large tables, being used to replace [UI-Grid](http://ui-grid.info/) (AngularJS based grid library) +- [Sass](https://sass-lang.com/), CSS pre-processor +- [Webpack](https://webpack.js.org/), build framework +- [Mocha](https://mochajs.org/), unit testing ### Server -- Java, currently supporting 8, but shortly moving to JDK 11 and then 17 -- Spring framework for auto-wiring components -- Spark Java, a lightweight framework for creating REST endpoints -- jOOQ, a java DSL for writing and executing database queries -- Immutables for domain objects +- [Java](https://www.java.com/en/), currently supporting 8, but shortly moving to JDK 11 and then 17 +- [Spring framework](https://spring.io/projects/spring-framework) for auto-wiring components +- [Spark Java](http://sparkjava.com/), a lightweight framework for creating REST endpoints +- [jOOQ](https://www.jooq.org/), a java DSL for writing and executing database queries +- [Immutables](https://immutables.github.io/) for domain objects ### Database -- Production databases: MSSQL, Postgres -- Test databases: H2 +- Production databases: [MSSQL](https://www.microsoft.com/en-gb/sql-server/), [Postgres](https://www.postgresql.org/) + - If MSSQL is chosen and you wish to actively work on the Waltz code base a commercial jOOQ licence is required. jOOQ is free for free databases, but not for commercial databases +- Test databases: [H2](https://www.h2database.com/html/main.html) ### Deployment options -- Web container, e.g. Tomcat +- Web container, e.g. [Tomcat](https://tomcat.apache.org/) - Standalone JAR -- Docker image +- [Docker](https://www.docker.com/) image + +### Notes + + --------- From 6dfe2858224c6d2013afcbb2fa8f3e8c1bb2e2c3 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 22:08:47 +0100 Subject: [PATCH 05/11] wip #6948 --- docs/features/feature-list.md | 73 ++++++++++++++++++++++++++--------- 1 file changed, 55 insertions(+), 18 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index aaad32d0a9..c39571a20a 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -1,4 +1,3 @@ -Original file from Wells Fargo. They want an open source view of Waltz features and capabilities Capability / Feature | Waltz | More @@ -67,7 +66,8 @@ Assessments are versatile and have been used for a wide range of purposes includ - SDLC compliance indicators - Recovery time objectives - Categorized _Bookmarks_ allow for a standardized way of collecting a common set of links to additional information resources about the application -- _Taxonomy Mappings_, which are further discussed below +- _Taxonomy Mappings_, which are further [discussed below](#taxonomies) +- _Technology_: servers, and databases can be associated to apps. In addition OSS library components can be linked ([discussed below](#oss-usage)) -------- @@ -206,25 +206,52 @@ Typically several assessments will be attached to licences indicating their suit -------- -## Aggregate views +## Aggregate views -These views are aggregated if higher level items in the taxonomy are selected. +The features we have discussed so far are mostly concerned with single entities (apps, change initiatives). +Waltz has comprehensive support for viewing the organizations portfolio by many different _vantage points_ - simply a term we use to represent any element which can be used to group the portfolio. +Vantage points are often hierarchical and Waltz will accordingly show an aggregated view. -### Vantage points +### Views + +Waltz supports many aggregated views. +Some of the most common and useful views are listed below: + +- Taxonomies + - From a vantage point Waltz can aggregate across all taxonomies. This can be used to show how apps that are linked ato a _Payments_ capability (the vantage point) may be related to the process taxonomy (the view) +- Flows + - The applications in a vantage point can be used to produce an aggregate view of the related data flows. This view can either be _intra_ flows (only flows between nodes in the vantage point), _outbound_ (flows 'leaving' the vantage point), _inbound_ (flows coming into the vantage point), or _all_ + - The flows are scored against the flow classification rules giving a quick and easy way to see how well the given vantage point is aligning to 'right sourcing' rules +- Attestations + - Waltz supports attestation of data sets (e.g. taxonomy mappings or flows). Vantage points allow users to see what portion of the included applications have a recent or overdue attestations +- Costs + - For a given vantage point users can quickly see which applications are the most costly according to whatever cost data has been associated with apps (e.g. Support costs, Infra costs) + +#### Report grids + +Report grids are a very powerful mechanism which allow any user to simply build self-service reports from Waltz. +A large proportion of the Waltz data set can be included into a report grid (e.g. taxonomies, data types, assessments, people, costs, survey responses). +Derived columns can be set up which compute additional data points. + +Once defined, a grid can be used across all vantage points and shared with other users or made public to all users. + +Report grids have been widely used for both ad-hoc and systematic data analysis, supporting several key processes in organisations. +The contents of a report grid can be exported via APIs and a custom (though currently not open-sourced) wrapper was developed to expose grids via the [OData](https://www.odata.org/) protocol so they could be wired into Tableau reports. + +### Typical vantage points - By taxonomy + - Any of the taxonomies can be used and the vantage point is constructed of all applications with any mappings to the taxonomy item (or any of it's children) - By people -- By data type + - Using the person involvements to applications and change initiaitives allows Waltz to show a custom view for that person, which included all of their direct and indirect reports. This allows is especially useful for quick comparisons across divisional leads. - By org unit + - A very common aggregate view is to use the organisational units to view details on the apps which belong to various departments +- By data type + - The data type hierarchy can be used by aggregating apps based on data flows which reference the data types - By diagram + - Diagrams in Waltz are often broken down into a '_Bill-of-Materials (BoM)_'. Waltz can use this BoM as the source for the aggregate views. - By arbitrary group - -### Views - -- Flows -- Attestations -- Taxonomies -- Report grids + - A very popular feature is to allow users (or automated processes) to create custom groups of apps (and change initiatives). These groups are non-hierarchical but are very flexible. We periodically review 'structured' sets of groups to see if there is an underlying taxonomy that needs to be surfaced. #### Bank on a Page @@ -239,14 +266,21 @@ Waltz has the ability to visualize capability/taxonomy for a specific function ( Waltz ships with **no** _out of the box_ integrations. Integration is typically done via periodic batch jobs which load data into the Waltz database. -Users have integrated with systems such as Collibra, Solidatus, Service Now, Apptio, Sparx Enterprise Architect and more. +Users have integrated with systems such as [Collibra](https://www.collibra.com/us/en), [Solidatus](https://www.solidatus.com/), [Service Now](https://www.servicenow.com/uk/), [Apptio](https://www.apptio.com/), [Sparx Enterprise Architect](https://sparxsystems.com/products/ea/) and more. We are hoping to offer standardised integrations in the future, with Collibra (Cloud) being the likely first offering. +Specific examples are: +- using Collibra for providing the Data classes and authority statements. +- using Sparx for the process taxonomy +- using Service now for CMDB information + + ### Software Development Lifecycle (SDLC) _Integration with service now to provide controls around dq and enforce regular attestations_ + --------- @@ -265,6 +299,11 @@ Several of the taxonomies stretch to thousands of nodes with many more related m This is all served by a load balanced pair of Tomcat servers with on a shared JVM of just 2GB. The (SQLServer) database has 8GB of memory. + +### Releases + +Waltz typically has a major release, requiring a Liquibase scheme migration, every month or two. Patch releases are more frequent and produced as required. All releases are available via the [Waltz GitHub releases page](https://github.com/finos/waltz/releases) + --------- ## Technical Details @@ -291,8 +330,9 @@ Major libraries / frameworks being used: ### Database +- Schema management: Waltz uses [Liquibase](https://www.liquibase.com/) to ensure accurate and easy scheme migrations - Production databases: [MSSQL](https://www.microsoft.com/en-gb/sql-server/), [Postgres](https://www.postgresql.org/) - - If MSSQL is chosen and you wish to actively work on the Waltz code base a commercial jOOQ licence is required. jOOQ is free for free databases, but not for commercial databases + - If MSSQL is chosen, and you wish to actively work on the Waltz code base a commercial jOOQ licence is required. jOOQ is free for free databases, but not for commercial databases - Test databases: [H2](https://www.h2database.com/html/main.html) ### Deployment options @@ -301,9 +341,6 @@ Major libraries / frameworks being used: - Standalone JAR - [Docker](https://www.docker.com/) image -### Notes - - --------- From 8e1794e9894e1a9a590b6b9148559c4d4096b9ad Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 22:45:27 +0100 Subject: [PATCH 06/11] wip #6948 --- docs/features/feature-list.md | 135 +++++++++++++++++++++------------- 1 file changed, 82 insertions(+), 53 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index c39571a20a..abcfcdc0bc 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -1,51 +1,13 @@ +# Waltz Features +## History -Capability / Feature | Waltz | More --- |----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------- -Enterprise Architecture Management | -Standards | -Patterns | -Authoritative Source | Ability to disposition for a given data type - what is the authoritative source for the data; can accommodate for heritage scenarios | [Auth sources, including heritage flows](#data-classifications) -Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) -Manage Architectures | x -Models | -Assessments | Assessements can be associated to most entities in Waltz | | -Portfolio Management | x -Application Rationalization | x -Roadmaps (application, products) | x -Functional commissions/de-commissions | Ability to show when capabilities start/stop within a portfolio -Lifecycle Mappings | x -Data Lineage | Logical flows captured (point to point with data-type metadata via integration method) with visualization/mapping abilities. Significant maturity planned. -Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. -Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. -Cost allocations - Taxonomies | Ability to demarcate the percentage of costs (shredding) for the purposes of planning/budgeting of a given function. Application may be performing multiple capabilities and needs to have appropriate allocations for each functional usage -Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability. -Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. -Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. -Analytics / Ad-hoc | x -Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). -Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) - | - | -Data | -Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations - | -User Interface | -American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; -Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users -Data Sensitivity | Database level encryption; planned for finer-grained entitlements for sensitive data -System integrations | -CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) -iGrafx / BizzDesign / any other notable vendors | -API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) -Stack | Database - Postgresql ; JooQ (transformation layer/ DSL); Java (v8+>>11/17); JooQ; immutables library; Spark API DSL >> javelin >> springboot. Liquidbase -Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 -Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed -How are modifications to the application made (business logic and/or data structures)? | -Cost / Licensing | - - -## Applications +Waltz has been under continuous development since 2016. +It was contributed to the FINOS foundation in 2020. + +--------- + +## Application Metadata One of the primary entities in Waltz is the _Application_. Applications have some basic properties such _name_, _type_, and _description_. @@ -83,6 +45,10 @@ Examples of these taxonomies are: - Regulations - Agile Domains and Tribes - Records +- Technology products + - usage of infra such as messaging systems, databases, workflow tools etc. +- Contracts + - a taxonomy opf contracts linked to products can be transitively applied to applications based on their technology product usage ### Portfolio Management @@ -107,7 +73,7 @@ Once the application portfolio has been mapped against a consistent set of taxon -------- -## Data Management Features +## Data Management ### Data Types @@ -288,10 +254,20 @@ _Integration with service now to provide controls around dq and enforce regular ### Security +SSL for browser/server comms. +SSO or OAuth (community developed) integration for authentication. +Basic role permissions and/or a granular permission framework based on person involvements to entities (e.g. restrict application editing to only the _domain architect_ and _IT owner_). +Database encryption as per jdbc driver support. + ### Standards +Application uses standard web technology. +Accessibility is a focus for future development and is part of the reason for moving to Svelte where the framework helps to ensure accessibility is addressed. -### Capacity +### Testing +A mixture of unit testing (junit and mocha) and integration testing (also junit, with playwright) is used. +Increasing coverage is a focus for upcoming releases. +### Capacity Waltz is happily serving several hundred users per day, out of a total user base of tens of thousands. The Waltz instance has information on over 5K applications and a similar number of change initiatives. There are many more dataflows, surveys, and assessments managed by Waltz. Several of the taxonomies stretch to thousands of nodes with many more related mappings. @@ -301,9 +277,11 @@ The (SQLServer) database has 8GB of memory. ### Releases - Waltz typically has a major release, requiring a Liquibase scheme migration, every month or two. Patch releases are more frequent and produced as required. All releases are available via the [Waltz GitHub releases page](https://github.com/finos/waltz/releases) +### Licence +Waltz is released under the [Apache 2.0 Licence](https://github.com/finos/waltz?tab=Apache-2.0-1-ov-file#readme) and is part of the FINOS foundation. + --------- ## Technical Details @@ -344,7 +322,7 @@ Major libraries / frameworks being used: --------- -# Current Development Themes +# Current Development Themes ## 2024 @@ -355,6 +333,57 @@ In addition to flows, we expect to look at tools to boost data quality, specific An evaluator would evaluate the rules and raise potential issues where data does not '_look right_'. For instance a rule could be set up to say that the '_KYC_' capability expects to see '_Person_' related data and be involved in one or more of a set of processes. -Another theme is around the Bank on a Page feature set. -This may involve closer integration with other reporting tools (Tableau, Superset) or we may pursue more custom visualisations. -We also intend to investigate how to integrate more closely with tools such as Archi and other open source diagramming tools. \ No newline at end of file +Another theme is around the _Bank on a Page_ feature set. +This may involve closer integration with other reporting tools ([Tableau](https://www.tableau.com/en-gb), [Superset](https://superset.apache.org/)) or we may pursue more custom visualisations. +We also intend to investigate how to integrate more closely with tools such as [Archi](https://www.archimatetool.com/) and other open source diagramming tools ([plantuml](https://plantuml.com/), [d2](https://d2lang.com/), [c4](https://c4model.com/) etc). + + + + +------- + +# Original Tabular feature list + +## Enriched with answers and links to above + +**Note:** this section is still WIP, updating with links to the above feature list and moving some of the questions/topics up to the above document. + +Capability / Feature | Waltz +-- | -- +**Enterprise Architecture Management** | +Standards | Waltz promotes standards through assessments, consistent taxonomies and data quality checks. Surveys are used to perform more detail standard compliance checks. +Patterns | No direct support, though assessments are often used to indicate which patterns are related to an application (e.g. cloud migration r-types) +Authoritative Source | Ability to disposition for a given capability - what is the authoritative source for clearthe data; can accommodate for heritage scenarios. +Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) +**Manage Architectures** | +Models | basic diagramming support. Interest in developing this out further through tool integration +Assessments | Waltz supports assessments on many entities including: [apps](#app-extensions), [taxonomy rating](#taxonomy-rating-assessments), and [data-flows](#data-assessments) +Portfolio Management | Support for basic [Protfolio management](#portfolio-management) and [roadmaps](#roadmaps) +Application Rationalization | [taxonomy mappings](#taxonomies) allow users to perform [app rationalization](#app-rationalisation) +Functional commissions/de-commissions | Ability to show when capabilities [start/stop](#roadmaps)(#portfolio-management) within a portfolio +Data Lineage | [Logical flows](#data-flows) captured (point to point with data-type metadata via integration method) with [visualization/mapping abilities](#aggregate-views). Significant maturity [planned](#plans). +Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. +Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. +Cost allocations - Taxonomies | Ability to demarcate the percentage of costs ([shredding](#allocations)) for the purposes of planning/budgeting of a given function. Application may be performing multiple capabilities and needs to have appropriate allocations for each functional usage +Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability (see [taxonomy rating assessments](#taxonomy-rating-assessments)). +Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. +Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. +**Analytics / Ad-hoc** | +Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). +Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) +**Data** | +Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations + | +**User Interface** | +American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; +Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users +Data Sensitivity | Database level encryption; planned for finer-grained entitlements for sensitive data +System integrations | +CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) +iGrafx / BizzDesign / any other notable vendors | +API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) +Stack | Database - Postgresql ; JooQ (transformation layer/ DSL); Java (v8+>>11/17); JooQ; immutables library; Spark API DSL >> javelin >> springboot. Liquidbase +Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 +Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed +How are modifications to the application made (business logic and/or data structures)? | +Cost / Licensing | \ No newline at end of file From 65aeee584be446eb18fa3ef851173f2ef6b86921 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 22:46:13 +0100 Subject: [PATCH 07/11] wip #6948 --- docs/features/feature-list.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index abcfcdc0bc..2302b0c09c 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -373,7 +373,6 @@ Diagramming | Flow-diagrams based on underlying metadata (constrained diagram Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) **Data** | Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations - | **User Interface** | American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users From 9730cb16e51762fe3e38aec34f8033fa547db0c1 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 22:50:41 +0100 Subject: [PATCH 08/11] wip #6948 --- docs/features/feature-list.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index 2302b0c09c..ea80f836ee 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -252,22 +252,24 @@ _Integration with service now to provide controls around dq and enforce regular ## NFR -### Security +### Security SSL for browser/server comms. SSO or OAuth (community developed) integration for authentication. Basic role permissions and/or a granular permission framework based on person involvements to entities (e.g. restrict application editing to only the _domain architect_ and _IT owner_). Database encryption as per jdbc driver support. -### Standards +### Standards Application uses standard web technology. Accessibility is a focus for future development and is part of the reason for moving to Svelte where the framework helps to ensure accessibility is addressed. -### Testing +### Testing A mixture of unit testing (junit and mocha) and integration testing (also junit, with playwright) is used. +Architecture tests (layering etc) are performed using [ArchUnit](https://www.archunit.org/). + Increasing coverage is a focus for upcoming releases. -### Capacity +### Capacity Waltz is happily serving several hundred users per day, out of a total user base of tens of thousands. The Waltz instance has information on over 5K applications and a similar number of change initiatives. There are many more dataflows, surveys, and assessments managed by Waltz. Several of the taxonomies stretch to thousands of nodes with many more related mappings. @@ -276,10 +278,10 @@ This is all served by a load balanced pair of Tomcat servers with on a shared JV The (SQLServer) database has 8GB of memory. -### Releases +### Releases Waltz typically has a major release, requiring a Liquibase scheme migration, every month or two. Patch releases are more frequent and produced as required. All releases are available via the [Waltz GitHub releases page](https://github.com/finos/waltz/releases) -### Licence +### Licence Waltz is released under the [Apache 2.0 Licence](https://github.com/finos/waltz?tab=Apache-2.0-1-ov-file#readme) and is part of the FINOS foundation. --------- @@ -374,10 +376,10 @@ Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various d **Data** | Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations **User Interface** | -American Disability Act | Angular (v1.8); migration to Svelte; modular sections managed by Javascript; -Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | SSO enabled; Auth enabled; open-source contributions from community of users -Data Sensitivity | Database level encryption; planned for finer-grained entitlements for sensitive data -System integrations | +American Disability Act | see [standards](#nfr-standards) +Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | see [security](#nfr-security) +Data Sensitivity | Database level encryption; planned for increased finer-grained entitlements for sensitive data +**System integrations** | CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) iGrafx / BizzDesign / any other notable vendors | API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) From 8601a718ffe2386d80842ba22cea8d38e32ca6e7 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 22:53:49 +0100 Subject: [PATCH 09/11] wip #6948 --- docs/features/feature-list.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index ea80f836ee..a4028e8d52 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -286,7 +286,7 @@ Waltz is released under the [Apache 2.0 Licence](https://github.com/finos/waltz? --------- -## Technical Details +## Technical Details ### Frontend @@ -381,9 +381,9 @@ Security Model (Ability to leverage Channel Secure / AD-ENT authentication / Data Sensitivity | Database level encryption; planned for increased finer-grained entitlements for sensitive data **System integrations** | CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) -iGrafx / BizzDesign / any other notable vendors | +iGrafx / BizzDesign / any other notable vendors | No integrations planned API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) -Stack | Database - Postgresql ; JooQ (transformation layer/ DSL); Java (v8+>>11/17); JooQ; immutables library; Spark API DSL >> javelin >> springboot. Liquidbase +Stack | see (#stack) Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed How are modifications to the application made (business logic and/or data structures)? | From a9ef8e4797f661656084ea52d85e58d8d3016b11 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Sun, 7 Apr 2024 22:54:49 +0100 Subject: [PATCH 10/11] wip #6948 --- docs/features/feature-list.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index a4028e8d52..1b2229ca97 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -383,7 +383,7 @@ Data Sensitivity | Database level encryption; planned for increased finer-grai CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) iGrafx / BizzDesign / any other notable vendors | No integrations planned API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) -Stack | see (#stack) +Stack | see [tech stack](#stack) Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed How are modifications to the application made (business logic and/or data structures)? | From 193633c74dbb34c3152b6a97520c5ec77a33b704 Mon Sep 17 00:00:00 2001 From: David Watkins Date: Thu, 30 May 2024 08:52:42 +0100 Subject: [PATCH 11/11] Update feature-list.md --- docs/features/feature-list.md | 47 ----------------------------------- 1 file changed, 47 deletions(-) diff --git a/docs/features/feature-list.md b/docs/features/feature-list.md index 1b2229ca97..480218582e 100644 --- a/docs/features/feature-list.md +++ b/docs/features/feature-list.md @@ -341,50 +341,3 @@ We also intend to investigate how to integrate more closely with tools such as [ - -------- - -# Original Tabular feature list - -## Enriched with answers and links to above - -**Note:** this section is still WIP, updating with links to the above feature list and moving some of the questions/topics up to the above document. - -Capability / Feature | Waltz --- | -- -**Enterprise Architecture Management** | -Standards | Waltz promotes standards through assessments, consistent taxonomies and data quality checks. Surveys are used to perform more detail standard compliance checks. -Patterns | No direct support, though assessments are often used to indicate which patterns are related to an application (e.g. cloud migration r-types) -Authoritative Source | Ability to disposition for a given capability - what is the authoritative source for clearthe data; can accommodate for heritage scenarios. -Authoritative Consumers | Ability to identify who should be the authoritative consumers for a given dataset (bi-directional relationships for data consumption) -**Manage Architectures** | -Models | basic diagramming support. Interest in developing this out further through tool integration -Assessments | Waltz supports assessments on many entities including: [apps](#app-extensions), [taxonomy rating](#taxonomy-rating-assessments), and [data-flows](#data-assessments) -Portfolio Management | Support for basic [Protfolio management](#portfolio-management) and [roadmaps](#roadmaps) -Application Rationalization | [taxonomy mappings](#taxonomies) allow users to perform [app rationalization](#app-rationalisation) -Functional commissions/de-commissions | Ability to show when capabilities [start/stop](#roadmaps)(#portfolio-management) within a portfolio -Data Lineage | [Logical flows](#data-flows) captured (point to point with data-type metadata via integration method) with [visualization/mapping abilities](#aggregate-views). Significant maturity [planned](#plans). -Bank / Architecture on a Page | Ability to visualize capability/taxonomy for a specific function (e.g., operations, CIO-specific views) - and overlay various metadata (costs, regions, locations, etc. ) with roadmap/timeline to show volatilities/projections. Dynamically generated vantage points. -Data Products | Data-based reference view of underlying datamodels for application portfolio for consumption and usage purposes. -Cost allocations - Taxonomies | Ability to demarcate the percentage of costs ([shredding](#allocations)) for the purposes of planning/budgeting of a given function. Application may be performing multiple capabilities and needs to have appropriate allocations for each functional usage -Application Ratings based on Capability | Application may have a legal-hold; supports a given capability and assigning a rating. Providing a more nuanced/detailed view of what an application is doing specifically within the capability (see [taxonomy rating assessments](#taxonomy-rating-assessments)). -Supply Chain / Contract Management | Taxonomy of contracts; associate application with capability/service and with a contract - providing pivot views of a vendor-relationship what are all the applications linked to it. -Investment Planning | Built-in to the ratings function/service/capability. Provide a ratings-item which details the strategic / attribute alignment. Providing driving alignment views for a given application. -**Analytics / Ad-hoc** | -Diagramming | Flow-diagrams based on underlying metadata (constrained diagram - upstream/downstream graphs). Plans to link with Archie - leverage reference data with Archie. Ability to incorporate Archie / Sparx / BPM diagrams into Waltz and leveraged for overlay purposes (Bank/Architecture on a Page). -Analytics / Reporting | ReportGrid (API-enabled) ; ability to select various data dimensions /cross-reference (meta-data) and develop derived columns that can be leveraged to showcase a portfolio's dispositions. Ability to regex for terms to enhance dispositions based on keyword look-ups. Plans to be leveraged by other product reporting tools (e.g., Tableau, Apache Superset, etc) -**Data** | -Data Model | OLTP model; Flows of data-types; hierarchical model; leveraging Collibra model; can accommodate for migrations -**User Interface** | -American Disability Act | see [standards](#nfr-standards) -Security Model (Ability to leverage Channel Secure / AD-ENT authentication / entitlement services) | see [security](#nfr-security) -Data Sensitivity | Database level encryption; planned for increased finer-grained entitlements for sensitive data -**System integrations** | -CMDB | ServiceNow (planned bi-directional) ; Collibra Cloud (planned) -iGrafx / BizzDesign / any other notable vendors | No integrations planned -API (What standard APIs are available for the package?) | ServiceNow (planned bi-directional; integrated into checkpoints to trigger notifications of an application / architecture sunrise gate) ; Collibra Cloud (planned) -Stack | see [tech stack](#stack) -Open Source / Non-proprietary technologies | OpenSource since inception (2016); Apache public license 2 -Which components of the system change as a result of any business functionality changes? | 2 types of releases; major version release monthly and coincide with database change; minor patch releases as needed -How are modifications to the application made (business logic and/or data structures)? | -Cost / Licensing | \ No newline at end of file