Diff screen child relations, faster parallelization on macOS, HTML2PDF fixes, performance improvements

This release several enhancements, bug fixes, and performance improvements:

• The Diff screen now supports child relations. Special thanks to @mplum for contributing the core patch.
• On macOS, the parallelization algorithm now uses "fork" instead of "spawn", leading to a good performance boost.
• The source file traceability HTML is now incrementally generated, re-rendering only when the source file or its related requirements are modified.
• The HTML2PDF feature excludes included documents when creating a bundled document. The assumption is that a user is only interested in seeing the final including documents, not the included standalone documents.
• Asset search optimization: Asset-finding code now filters out irrelevant directories early, avoiding repeated and unnecessary searches, improving efficiency.

What's Changed

• Bump version to 0.4.0 by @stanislaw in #2038
• html2pdf: do not export included documents by @stanislaw in #2040
• tests/integration: move Excel and ReqIF tests to features/ by @stanislaw in #2042
• Performance: Use 'fork' method for creating processes on macOS by @stanislaw in #2044
• document_finder: improve the performance of finding assets by @stanislaw in #2043
• Code climate: source file generator: encapsulate writing to file better by @stanislaw in #2045
• traceability_index: many_to_many_set: add link edges support by @stanislaw in #2046
• tests/end2end: helpers: ensure more reliable scrolling when moving grammar fields by @stanislaw in #2047
• traceability_index: remove legacy SDocNodeConnections by @stanislaw in #2048
• traceability_index: improvements in the incremental generation by @stanislaw in #2049
• traceability_index: incremental generation of source files by @stanislaw in #2050
• traceability_index: incremental generation that does not modify file dates by @stanislaw in #2051
• Added 'child-relation' to diff view for enhanced change impact visualization by @mplum in #2052
• HTML2PDF: update to include latest fixes, exclude included documents from bundle document by @stanislaw in #2053
• Diff: Recognize when child relations are modified by @stanislaw in #2054
• Bump version to 0.5.0 by @stanislaw in #2055

Full Changelog: 0.4.0...0.5.0

Language-aware C++ parsing, interface with Doxygen, automatic version/date document meta information

This release introduces language-aware tracing between requirements and C++ source code.

Currently, only C++ functions can be traced using a marker like @relation(REQ-1, scope=function). The ability to add markers to classes has not yet been implemented.

A new export format has been added to generate a Doxygen tagfile for a documentation tree. When the tagfile is registered in the Doxygen configuration, requirements can be referenced using Doxygen keywords. For example: \relation{REQ-1, scope=function}. Special thanks to @johanenglund for contributing the idea and helping to implement the solution.

The @-macros @GIT_VERSION, @GIT_BRANCH, @GIT_COMMIT_DATE, and @GIT_COMMIT_DATETIME were introduced to simplify the automatic insertion of document version information and publication dates into published documents.

An edge case for reading source files has been addressed. StrictDoc will now issue a warning if it encounters a binary file instead of a source code file, preventing a crash. [Contributed by @haxtibal]

A user has contributed an example demonstrating how requirements can be generated into an Excel questionnaire for users to fill out, enabling requirements compliance tracking. The example is located at tests/integration/scripting_examples/questionnaires/. [Contributed by @BenGardiner]

Additionally, a minor HTML2PDF formatting issue has been resolved. Previously, the RST admonitions in StrictDoc's own documentation were not rendered correctly when spanning across two pages.

What's Changed

• Feature: Export Doxygen TAGFILE with a map from requirements to StrictDoc documentation by @stanislaw in #2011
• bugfix: file_traceability_index: relax the assert on a function definition by @stanislaw in #2015
• doxygen: Change compound type and file suffix by @johanenglund in #2016
• marker_parser: Add support for curly braces by @johanenglund in #2017
• backend/sdoc_source_code: recognize C++ class declarations/definitions by @stanislaw in #2018
• backend/sdoc_source_code: C++: disambiguate overloaded constructors by @stanislaw in #2019
• export/html: add .admonition-title as data-no-hanging-selectors for PDF by @stanislaw in #2020
• docs: update release notes, roadmap, document Doxygen tagfile by @stanislaw in #2021
• docs: deploy strictdoc's own docs to Read the Docs by @stanislaw in #2023
• docs: roadmap diagram: update backlog by @stanislaw in #2024
• backend/sdoc_source_code: C++: recognize functions returning references by @stanislaw in #2027
• Bugfix: project_statistics: fix the "Sections without any text" metric by @stanislaw in #2028
• docs: update FAQ by @stanislaw in #2029
• Code climate: tests/integration: reorganize the test folders around features/ by @stanislaw in #2032
• export/html: document meta information: recognize @GIT_VERSION and @GIT_BRANCH markers by @stanislaw in #2035
• Skip binaries in source tree rather than failing by @haxtibal in #2033
• test/integration: add custom script for exporting questionnaires by @BenGardiner in #2034
• export/html: document meta information: recognize @GIT_COMMIT_DATE and @GIT_COMMIT_DATETIME markers by @stanislaw in #2036

New Contributors

• @johanenglund made their first contribution in #2016

Full Changelog: 0.3.0...0.4.0

Requirements and C source files: Resolve definition markers from declarations markers

This release includes an enhancement to the feature for tracing requirements to C source code.

From now on, if a @relation marker is specified in a C function declaration, StrictDoc will perform its "magic" and automatically connect the referenced requirement to the corresponding C function definition. This allows a @relation marker to be placed in the main documentation block, typically located above C function declarations, while ensuring the requirement is also linked to the function definition.

What's Changed

• Bump version to 0.2.1 by @stanislaw in #1999
• backend/sdoc_source_code: recognize function declarations in .c and h files by @stanislaw in #2000
• backend/sdoc_source_code: make @relation backwards-compatible without the language parsing by @stanislaw in #2002
• backend/sdoc_source_code: C language: resolve definition markers from declarations markers by @stanislaw in #2003
• Code climate: backend/sdoc_source_code: rename parts -> functions by @stanislaw in #2005
• tests/end2end: disable HTML2PDF test on macOS for now by @stanislaw in #2006
• Code climate: remove unused pylint rule disabler by @stanislaw in #2004
• tests/integration: make the cache folder local to a test folder by @stanislaw in #2008
• Bump version to 0.3.0 by @stanislaw in #2010

Full Changelog: 0.2.1...0.3.0

LINKing to documents and Diff/Changelog autoescaping fix

This release includes a new feature, a bug fix, and some documentation updates.

• The [LINK: ...] feature now supports linking to documents by UID. [@haxtibal]
• HTML escaping was fixed on the Diff/Changelog screen. [@haxtibal]

The documentation now includes two new pages:

• The "Feature Map" document provides a high-level overview of the major StrictDoc features from the user's perspective.
• The Troubleshooting document offers advice on clearing the user cache, one of the common solutions to user issues. We plan to expand this section with more tips over time.

What's Changed

• Bump version to 0.2.0 by @stanislaw in #1984
• Regenerate CHANGELOG by @stanislaw in #1985
• docs: migrate several links to new @relation syntax by @stanislaw in #1986
• Track UID_TO_NODE for SDocDocument to resolve LINK to DOCUMENT by @haxtibal in #1987
• docs: caching, troubleshooting, roadmap by @stanislaw in #1989
• Update issue templates by @stanislaw in #1990
• .github: add/edit the issue templates by @stanislaw in #1991
• .github: bug_report: use the same markup as for the feature by @stanislaw in #1992
• docs: user guide: enable MID by @stanislaw in #1995
• diff/changelog: remove an outdated MID component by @stanislaw in #1994
• docs: enable MID for all remaining documents by @stanislaw in #1996
• Mark diff colors safe and escape raw diff input by @haxtibal in #1993
• docs: feature map by @stanislaw in #1997
• Code climate: Diff screen view objects: remove unused code by @stanislaw in #1998

Full Changelog: 0.2.0...0.2.1

Improved tracing in C, Python, and general code parsers

This release introduces several enhancements to the source code processing introduced in release 0.1.0.

The backend now supports improved function tracing in C, Python, and general parsing code:

• Forward ranges to C functions now include the top comment.
• Each range type is displayed in the output HTML.
• The C source code reader has been updated to support file-level scoping.
• The source code reader now caches parsed objects to disk, improving reading performance.
• Proper handling of functions inside nested Python classes has been implemented, allowing syntax like Foo.Bar.do_baz. Thanks to @haxtibal for contributing this enhancement.

Additionally, caching has been centralized, and the cache directory is now configurable. The project configuration file now supports a cache_dir option, which can be set to values such as ./output/build. This setting can help make caching artifacts visible alongside documentation artifacts.

What's Changed

• Forward functions: Search all parent classes by @haxtibal in #1974
• backend/sdoc_source_code: enable caching of source files by @stanislaw in #1975
• export/html2pdf: use the cache dir from project config by @stanislaw in #1976
• export/html: html_templates: use the cache dir from project config by @stanislaw in #1977
• git_client: use the cache dir from project config by @stanislaw in #1978
• tasks: server: set --output-path to a local cache folder by @stanislaw in #1979
• backend/sdoc_source_code: general source code reader: support scope=file by @stanislaw in #1980
• backend/sdoc_source_code: each range is completed with a type by @stanislaw in #1981
• backend/sdoc_source_code: forward C function relations: include the top comment to the range by @stanislaw in #1982
• backend/sdoc_source_code: C reader: recognize scope=file by @stanislaw in #1983

Full Changelog: 0.1.0...0.2.0

Language-specific parsers for Python and C. Tracing requirements to functions and classes.

This backward-compatible release introduces several new features for tracing requirements to source files:

• StrictDoc now integrates with Tree-sitter, enabling it to parse multiple programming languages. Using AST information, it achieves more precise tracing of requirements to source code.

• Language-specific parsers for Python and C have been added, allowing functions (in C and Python) or classes (in Python) to be linked to requirements.

• Both forward linking of requirements to source files and backward linking of source files to requirements are supported. These features can be used independently or together within the same project.

With this release, we are also transitioning to a more semantic versioning-oriented release scheme. From now on, the MAJOR.MINOR.PATCH version components will be maintained according to the recommendations of the semantic versioning specification.

What's Changed

• Bump version to 0.0.60 by @stanislaw in #1962
• Feature: Parsing source code functions into requirements graph by @stanislaw in #1956
• backend/sdoc_source_code: calculate coverage for forward range and function markers by @stanislaw in #1964
• file_traceability_index: deduplicate requirement links based on functions by @stanislaw in #1965
• backend/sdoc_source_code: adjust the marker parser to the new @relation syntax by @stanislaw in #1966
• backend/sdoc_source_code: make the default parser recognize @relation by @stanislaw in #1967
• backend/sdoc_source_code: support "Class.function" syntax for forward links to Python code by @stanislaw in #1968
• backend/sdoc_source_code: support scope=class by @stanislaw in #1969
• html2pdf: update to the newer Chrome web driver API by @stanislaw in #1971
• backend/sdoc_source_code: enable scope=file marker for Python by @stanislaw in #1970

Full Changelog: 0.0.60...0.1.0

Bugfix release: Project Statistics queries autoescaping fix, query failures on the search screen, small step/bugfix towards supporting TAGs in Web UI

What's Changed

• pyproject.toml: remove a some unneeded artifacts from the release tar by @stanislaw in #1948
• Regenerate CHANGELOG by @stanislaw in #1949
• docs: migrate all FREETEXT to TEXT by @stanislaw in #1950
• pyproject.toml: remove unneeded lxml dependency by @stanislaw in #1952
• Fix generated search URLs for project statistics by @haxtibal in #1954
• Code climate: sdoc_source_code: rename all occurrences: pragma -> marker by @stanislaw in #1955
• Prevent query failure when node is missing requested field by @mplum in #1958
• Add 'TAG' as a supported fieldtype for requirement nodes by @mplum in #1959

New Contributors

• @mplum made their first contribution in #1958

Full Changelog: 0.0.59...0.0.60

Automatic escaping of Jinja templates, ReqIF improvements, HTML2PDF improvements, passthrough command deprecated

This release includes several important improvements. Thanks to @haxtibal for implementing and testing many of the implemented changes.

• Fixed multiple issues related to the handling of the UID field's format and accepted characters, both on its own and when used as part of LINK and ANCHOR tags. [@haxtibal]

• Implemented a more consistent and unified approach to escaping Jinja templates. With this change, the Jinja templates that StrictDoc uses for rendering content are now automatically escaped, except for a few known edge cases that have been handled separately. For more details, see the "HTML Escaping" section of the Design Document, which describes the rationale and approach taken. [@haxtibal]

• Extended the HTML2PDF feature to support the --generate-bundle-document option, which allows generating a single PDF document from all documents in an SDoc documentation tree.

• Additionally, a project configuration option was added to specify a custom path to the ChromeDriver used by the HTML2PDF printer. [@haxtibal]

• Enhanced the SDoc-ReqIF-SDoc roundtrip to include relation roles such as Refines, Implements, etc. These roles are now recognized in both directions. This feature was requested by a single user and requires further testing.

• Deprecated the passthrough command. All passthrough functionality has been moved to the export command under the option --formats=sdoc. Users are encouraged to switch to the export command, as using passthrough will now trigger a deprecation warning. [@haxtibal]

• Fixed a bug related to editing custom (non-requirement) nodes and adding links between them, based on a report by @elfman2.

• The Python 3.7 support was removed. The lowest Python baseline will now be 3.8 for some time.

What's Changed

• Extend UID regex and use it in SDocNode by @haxtibal in #1902
• backend/sdoc: SDocNode: Make MID a normal SDocField by @stanislaw in #1903
• Add parentheses to UID regex by @haxtibal in #1904
• LINKs: Fix link to anchor in other sdoc by @haxtibal in #1906
• rst: Fix link followed by non-ws/punctuation by @haxtibal in #1908
• rst: Render links and anchors in node fields by @haxtibal in #1909
• views: Support custom views for passthrough by @haxtibal in #1910
• links: Consistent anchor id whitespace replacement by @haxtibal in #1917
• export: HTML2PDF: render bundle document by @stanislaw in #1915
• Code climate: get_included_document -> get_including_document by @stanislaw in #1919
• Code climate: fix several issues related to new Ruff checks by @stanislaw in #1922
• Code climate: document_iterator: add SDocAnyNode type hint by @stanislaw in #1923
• Code climate: remove unused render_free_text() by @stanislaw in #1924
• Code climate: markup_renderer: remove unused code by @stanislaw in #1926
• Code climate: markup_renderer: fix all mypy issues by @stanislaw in #1927
• Use Jinja2 autoescaping by @haxtibal in #1921
• Fix html escaping of source line marks by @haxtibal in #1929
• backend/reqif: relation roles export/import roundtrip by @stanislaw in #1930
• Bump version to 0.0.59a1 by @stanislaw in #1931
• Fix e2e test by using new chrome headless mode by @haxtibal in #1934
• HTML2PDF: System chromedriver for PDF export by @haxtibal in #1932
• Mark more special HTML in templates as safe by @haxtibal in #1936
• Move passthrough commands to export --formats sdoc by @haxtibal in #1937
• UI: edit node: fix a case when editing a custom node by @stanislaw in #1939
• docs: update the release notes by @stanislaw in #1940
• Code climate: SDocNode: requirement_type -> node_type by @stanislaw in #1941
• CI: drop Python 3.7 and 3.8 by @stanislaw in #1944
• tests/integration: HTML escaping: add assertions for project index and traceability matrix by @stanislaw in #1943
• tasks: remove assert on the path to chromedriver by @stanislaw in #1945
• Drop Python 3.7 support by @stanislaw in #1946
• Bump version to 0.0.59, regenerate Read the Docs by @stanislaw in #1947

Full Changelog: 0.0.58...0.0.59

0.0.58

This is a release with a single fix and a minor documentation update.

The manage auto-uid command is now compatible with grammars that define the UID field as REQUIRED: True. Previously, StrictDoc would raise a validation message if the auto-uid command was run against a document with UID defined as a required field but containing nodes without UID. The new behavior allows the auto-uid command to operate without validation and correctly creates a new UID for the node. Thanks to @simhein for reporting this issue.

@haxtibal contributed a patch to the User Guide that explains the StrictDoc convention of how the single-line (meta information) fields should be used compared to the multiline fields, such as STATEMENT, RATIONALE, COMMENT, etc. Quoting the key part:

Each grammar element must have exactly one content field named STATEMENT, DESCRIPTION or CONTENT. The content field plays a key role in the HTML user interface as well as other export formats.

All fields before the content field are considered meta information. Meta information fields are assumed to be single-line. The content field and all following fields accept single-line and multiline strings.

See the updated User Guide for more details.

What's Changed

• Bump version to 0.0.57, update documentation by @stanislaw in #1895
• User Guide: Explain conventions related to content field by @haxtibal in #1898
• backend/sdoc: relax the validation of UID when running Manage/AutoUID by @stanislaw in #1899
• Bump version to 0.0.58 by @stanislaw in #1901

Full Changelog: 0.0.57...0.0.58

FREETEXT-TEXT migration, adding LINKs to arbitrary nodes, improved validation messages

This release contains a significant, non-breaking change that affects the entire StrictDoc codebase and the SDoc data model: the FREETEXT-TEXT migration.

The description of the migration and the migration paths are described in the User Guide, https://strictdoc.readthedocs.io/en/stable/strictdoc_01_user_guide.html#freetext-text-migration-june-2024.

/cc @adamgreig @BenGardiner @bernhard-tuvsud @dahbar @elfman2 @fniu @gregshue @haxtibal @i-interprete-on-the-fly @kestewart @lochsh @nicpappler @richardbarlow @RobertoBagnara @simhein @vik378

---

Other changes in this release:

• The validation messages for the uniqueness of MID fields were improved for the Document, Section and Requirement/Text nodes. Thanks to @bernhard-tuvsud for the improvement suggestion.

• Due to the FREETEXT-TEXT migration, it is now possible to add LINKs to arbitrary nodes, such as REQUIREMENT and TEXT. Previously, LINKs could be added only to SECTION nodes. Thanks to @haxtibal for contributing the initial implementation.

• The UI validations have been added for SingleChoice and MultipleChoice fields. Thanks to @haxtibal for contributing the multiple choice field validation.

• The experimental Graphviz/Dot traceability graph generator has been removed because this feature didn't show much value, mainly due to the static and non-programmable nature of PDF and SVG outputs produced by Graphviz.

---

What's Changed

• UI: Form validation: SingleChoice and REQUIRED fields by @stanislaw in #1829
• backend/sdoc: validate grammar from file like a normal grammar by @stanislaw in #1831
• backend/sdoc: instantiate doc/grammar meta models at start time by @stanislaw in #1832
• backend/sdoc: grammar: teach SDocNodeField to parse FreeText parts by @stanislaw in #1833
• backend/reqif: --reqif-multiline-is-html: extend the integration test by @stanislaw in #1834
• tests/integration: add an RST code block test just in case by @stanislaw in #1838
• backend/sdoc: introduce [TEXT] node by @stanislaw in #1837
• Feature: LINK to requirements and custom nodes (static export only for now) by @haxtibal in #1835
• Code climate: remove useless init.py files by @stanislaw in #1840
• Code climate: Introduce THIS_TEST_FOLDER to increase the portability of most itests by @stanislaw in #1842
• Feature: [LINK] from all fields of Requirement or other nodes by @stanislaw in #1843
• export/html, LINK: referenced node shows an anchor link if referenced with [LINK] by @stanislaw in #1844
• UI: requirement_form_object: improve handling of multiline field boundary by @stanislaw in #1845
• Code climate: tests/end2end: anchors -> LINK_AND_ANCHOR by @stanislaw in #1846
• docs: release notes for 0.0.56 by @stanislaw in #1847
• export/html and UI: Rendering [TEXT] node, basic end2end tests by @stanislaw in #1848
• UI: Delete section/node: Add validations to the confirmation window by @stanislaw in #1849
• backend/reqif: when --reqif-multiline-is-xhtml, generate FREETEXT spec type attribute as XHTML by @stanislaw in #1851
• HTML2PDF: Fail early if feature is not enabled by @haxtibal in #1850
• UI: Create node: parse and validate LINK/ANCHOR by @stanislaw in #1852
• UI: Create node: parse, validate, and render LINK/ANCHOR by @stanislaw in #1854
• html: Rename "parent relations" to "parents" and "child relations" to "children" by @haxtibal in #1855
• UI: Delete node: incoming links to anchors validation by @stanislaw in #1859
• UI: Create/Update/Delete node: Implement all existing cases related to LINK/ANCHOR by @stanislaw in #1862
• UI: HTML markup changes and adjust remaining tests, removing dependencies on FREETEXT by @stanislaw in #1866
• UI: Diff screen: Ensure escaping of requirement fields by @stanislaw in #1867
• backend/sdoc: Document: remove .free_texts by @stanislaw in #1868
• Node machine identifiers (MID): improve uniqueness validation message by @stanislaw in #1869
• Bump version to 0.0.57a2 by @stanislaw in #1870
• Remove Graphviz and DocumentDotGenerator by @stanislaw in #1873
• UI: Edit grammar element relations form: TEXT node can have no relations by @stanislaw in #1871
• UI: Ensure that documents with HTML markup get rendered correctly when edited by @stanislaw in #1872
• tasks: remove pylint and flake8, create aliases for most common tasks by @stanislaw in #1874
• docs: prepare the release notes for FREETEXT-TEXT migration by @stanislaw in #1875
• UI: Node form validation: allow empty SingleChoice fields when REQUIRED is False by @stanislaw in #1877
• UI: Support MultipleChoice in requirement form by @haxtibal in #1878
• UI: Do not auto-generate UID for TEXT nodes for now by @stanislaw in #1879
• docs: update release notes by @stanislaw in #1880
• docs: User Guide: implement FREETEXT-TEXT changes by @stanislaw in #1881
• cli: passthrough: --free-text-to-text option to migrate FREETEXT->TEXT by @stanislaw in #1882
• docs: User Guide: describe FREETEXT-TEXT migration path by @stanislaw in #1883
• backend/sdoc: writer: --free-text-to-text for SECTIONs by @stanislaw in #1886
• export/html: update hints on node control by @mettta in #1885
• UI: TRM: Exclude text nodes from the list by @stanislaw in #1893

New Contributors

• @haxtibal made their first contribution in #1835

Full Changelog: 0.0.56...0.0.57