tcWebHooksPlugin 2.0.1

Improvements and bugfixes in WebHook editing and internal cache management

As part of the threading improvements in tcWebHooks 2.0.0, the minimum supported version of TeamCity is 2019.1
Also, 2024.03 upgraded the framework on which the REST API is developed. There are now 3 files. Only two need to be installed:

1. tcWebHooks-2.0.1-build.502.zip - the plugin that sends webhook events.
   Compatible with TeamCity 2019.1 and newer
2. tcWebHooks-REST-API-2.0.1-build.502.zip - REST API for editing WebHook configurations and templates.
   Compatible with TeamCity 2024.03 and newer
3. tcWebHooks-REST-API-2.0.1-build.502-legacy.zip - REST API for editing WebHook configurations and templates.
   Compatible with TeamCity 2019.1 through to 2023.11.x

Fixes in this release

• Added support for Project Copy Events
  Copying a project with webhooks configured will copy the webhooks and refresh the internal cache correctly.
  Duplicated webhooks are allocated a new unique ID and loaded into the cache.
  It will also attempt to remap the internal build Ids referenced by the webhook of the Build Configurations from the old bt number to the corresponding bt number in the newly created builds.
  Fixes #238
• Added support for Build Configuration Delete events
  Fixes #242
• Implemented handling of edited projects outside of TeamCity
  For example, versioned settings, modification of the TeamCity project configuration, etc will be reloaded and re-cached.
  Fixes #232, #229 and possibly #194.

Many thanks to @matt-richardson for all the feedback and testing on this release

Full Changelog: v2.0.0...v2.0.1

tcWebHooksPlugin 2.0.0

Support for TeamCity 2019.1 to latest

As part of the threading improvements (below), the minimum supported version of TeamCity is 2019.1
Also, 2024.03 upgraded the framework on which the REST API is developed. There are now 3 files. Only two need to be installed:

1. tcWebHooks-2.0.0-build.495.zip - the plugin that sends webhook events.
   Compatible with TeamCity 2019.1 and newer
2. tcWebHooks-REST-API-2.0.0-build.495.zip - REST API for editing WebHook configurations and templates.
   Compatible with TeamCity 2024.03 and newer
3. tcWebHooks-REST-API-2.0.0-build.495-legacy.zip - REST API for editing WebHook configurations and templates.
   Compatible with TeamCity 2019.1 through to 2023.11.x

Please download the REST API plugin that is compatible with the version of TeamCity that you're running.
If you require support for TeamCity versions before 2019, then please use a 1.2.x version of tcWebHooks.

Threading Improvements on large TeamCity instances

On large systems with thousands of build agents, and many webhooks, the TeamCity thread pool can be filled up processing webhook executions.
This prevents certain aspects of TeamCity from functioning. In the case reported, this was preventing agents from registering.
This release has the following changes from 1.2.5.

Minimum support TeamCity version is 2019.1

This is the minimum version of TeamCity that supports creating a dedicated threadpool with a configurable size. To be able to request TeamCity to create a threadpool, we needed to update to this minimum version of TeamCity.

Removed the tcWebHooks agent plugin.

The agent plugin was forcing all agents to connect, download the agent plugins, restart and then re-connect. On large systems this process puts additional load on the default TeamCity thread pool.

Don't queue tasks for webhooks that won't execute.

Previously, every configured webhook would be submitted to the queue for processing for every build event. Most of these would then be skipped. This is because the majority of webhooks are only configured to run on the build finished event. With this change, only webhooks that are configured to actually run for that build event are queued for processing.
This has the consequence that the skipped count of webhooks will be significantly reduced, because webhooks are no longer assessed for every build event. Previously, most events were "701 :: Webhook not enabled for build state 'event'". These are now not logged and don't appear in the statistics.

tcWebHooks runs webhooks in its own ThreadPool (ExecutorService).

tcWebHooks now has a dedicated threadpool from which webhooks are executed. This leaves the default threadpool available for other TeamCity activities.

The threadpool defaults to minimum of 1 worker thread, a maximum of 50 worker threads and a queue of 3000 items.
The pool size determines the number of webhooks that can be executed simultaneously. The pool will automatically scale up from min up to the max size when items are added. The queue size determines the number of webhooks that can be queued for execution once the pool is already processing the maximum number of webhooks.

The new ThreadPool is configurable by adding a dedicatedThreadPool element in the webhooks section of main-config.xml. For example:

<webhooks>
  <dedicatedThreadPool enabled="true" minPoolSize="10" maxPoolSize="100" queueSize="5000" />
</webhooks>

Note: Changes to the size of pool and queue will not take effect until TeamCity is restarted.

These settings have the following effects:

1. enabled="true" - This enables the use of a separate thread pool. If set to false, tcWebHooks will revert the behaviour and use the default thread pool like tcWebHooks version 1.2.2 did.
2. minPoolSize="10" - Sets the minimum pool size to 10. Without this value, the default is 1.
3. maxPoolSize="100" - Sets the maximum webhooks workers to 100 at a time. Without this value, the default is 50.
4. queueSize="5000" - Set the maximum number of webhooks that can be queued waiting to be executed. Without this value, the default is 3000.

If the pool and queue both fill up, webhooks will error with code 802 :: WebHook Executor Queue full. Exception was: Unable to put new task into the queue because maximum capacity is exceeded. Current queue size: 1000. Executor info: Thread factory "WebHookThread-" {corePoolSize=10; maxPoolSize=50; activeCount=50}

This can been seen in the error list on the WebHooks Administation tab.

These errors are also logged to the teamcity-server.log

[2023-03-27 08:33:04,891] ERROR - jetbrains.buildServer.SERVER - WebHookThreadingExecutorImpl :: trackingId: 68c207ae-829b-4bee-9a33-d6cb91b6e78b :: projectId: TestTopProject :: webhookId: id_968610231 :: templateId: json-redux, errorCode: 802, errorMessage: Unable to put new task into the queue because maximum capacity is exceeded. Current queue size: 1000. Executor info: Thread factory "WebHookThread-" {corePoolSize=10; maxPoolSize=50; activeCount=50}

v2.0.0-rc.5

Bugfix release for TeamCity 2024.03 - NOT COMPATIBLE with older TeamCity Versions

If you're upgrading from v1.2.5, please also read the release notes for previous 2.0.0-rc versions.

Fixes

• Fixes ability to Preview and Execute a webhook while editing a WebHook template.

Compatibility

• tcWebHook plugin is expected to work with TeamCity 2019 and newer because there are no changes since v2.0.0-rc3.
• tcWebHooks-REST plugin is not expected to work with TeamCity version 2023.x or older. It's only compatible with 2024.03 and newer.

Notes

• The team at Jetbrains recently updated the version of Jersey (the REST API framework) on which the TeamCity REST API is built. This update was released with TeamCity 2024.03
• tcWebhooks REST API extends the TeamCity REST API to enable REST Configuration of webhooks, parameters and templates.

The recent changes to Jersey also rely on a recent TeamCity version. Therefore the work done in 2023 for tcWebHooks 2.0.0 was the base on which to build the updated tcWebHooks REST version. No attempt has been made to back-port the Jersey changes to tcWebHooks 1.2.x

v2.0.0-rc.4

Bugfix release for TeamCity 2024.03 - NOT COMPATIBLE with older TeamCity Versions

Addresses #236. If you're upgrading from v1.2.5, please also read the release notes for previous 2.0.0-rc versions.

Compatibility

• tcWebHook plugin is expected to work with TeamCity 2019 and newer because there are no changes since v2.0.0-rc3.
• tcWebHooks-REST plugin is not expected to work with TeamCity version 2023.x or older. It's only compatible with 2024.03 and newer.

Notes

• The team at Jetbrains recently updated the version of Jersey (the REST API framework) on which the TeamCity REST API is built. This update was released with TeamCity 2024.03
• tcWebhooks REST API extends the TeamCity REST API to enable REST Configuration of webhooks, parameters and templates.

The recent changes to Jersey also rely on a recent TeamCity version. Therefore the work done in 2023 for tcWebHooks 2.0.0 was the base on which to build the updated tcWebHooks REST version. No attempt has been made to back-port the Jersey changes to tcWebHooks 1.2.x

Known bugs:

• The endpoint for previewing and testing a template is not fixed yet. However, editing Templates and WebHooks works correctly from the testing I have done.
• All automated unit and integration tests for the REST plugin have been disabled because they don't work with Jersey2

v2.0.0-rc.3

Bugfix for TC 2023.05.4, remove Build Agent component, and Add $dateTool variable available for velocity templates

Bugfix for TC 2023.05.4 throwing 500 error on webhook save

A method not found error was being thrown when editing a webhook via the tcWebHooks REST API.
The most recent version of TeamCity removes a function from their REST API which allows conditionally including an object.
This fix removes reliance on this method, and uses a different method to determine whether to populate the parameters object in a WebHook configuration.

Removed the tcWebHooks agent plugin.

The agent plugin was forcing all agents to connect, download the agent plugins, restart and then re-connect. On large systems this process puts additional load on the default TeamCity thread pool.
Fixed a regression where this component was still available in v2.0.0-rc-2

Add $dateTool variable available for velocity templates

This allows us to get the current date, and other functions as described by Velocity DateTool

Note: in those examples, they refer to $date. In our implementation it is $dateTool, because $date is already in use for a formatted date string.

Full Changelog: v2.0.0-rc2...v2.0.0-rc.3

tcWebHooks Plugin 1.2.5 bugfix

Fix for 500 error when editing a webhook via the REST API on TeamCity 2023.05.4

A method not found error was being thrown when editing a webhook via the tcWebHooks REST API.
The most recent version of TeamCity removes a function from their REST API which allows conditionally including an object.
This fix removes reliance on this method, and uses a different method to determine whether to populate the parameters object in a WebHook configuration.

Full Changelog: v1.2.4...v1.2.5

tcWebHooks Plugin 1.2.4 bugfix

Bugfix to make the $dateTool variable available for velocity templates

This allows us to get the current date, and other functions as described by Velocity DateTool

Note: in those examples, they refer to $date. In our implementation it is $dateTool, because $date is already in use for a formatted date string.

Full Changelog: v1.2.3...v1.2.4

v2.0.0-rc2

Fix for Tests Muted and other library update

This is a rebase of the issue_212-threading-issues branch on top of version 1.2.3.
It only contains the following two fixes when compared to v2.0.0-rc1

• Fix support for muted tests event in #220 Thanks to @matt-richardson
• Bump guava from 29.0-jre to 32.0.0-jre in /tcwebhooks-core by @dependabot in #216

v1.2.3

Fix for Tests Muted and other library update

• Fix support for muted tests event in #220 Thanks to @matt-richardson
• Bump guava from 29.0-jre to 32.0.0-jre in /tcwebhooks-core by @dependabot in #216

Full Changelog: v1.2.2...v1.2.3

tcWebHooksPlugin 2.0.0-rc.1

Bugfix Threading Issues on large TeamCity instances

On large systems with thousands of build agents, and many webhooks, the TeamCity thread pool can be filled up processing webhook executions.
This prevents certain aspects of TeamCity from functioning. In the case reported, this was preventing agents from registering.
This release has the following changes from 1.2.2.

Minimum support TeamCity version is 2019.1

This is the minimum version of TeamCity that supports creating a dedicated threadpool with a configurable size. To be able to request TeamCity to create a threadpool, we needed to update to this minimum version of TeamCity.

Removed the tcWebHooks agent plugin.

The agent plugin was forcing all agents to connect, download the agent plugins, restart and then re-connect. On large systems this process puts additional load on the default TeamCity thread pool.

Don't queue tasks for webhooks that won't execute.

Previously, every configured webhook would be submitted to the queue for processing for every build event. Most of these would then be skipped. This is because the majority of webhooks are only configured to run on the build finished event. With this change, only webhooks that are configured to actually run for that build event are queued for processing.
This has the consequence that the skipped count of webhooks will be significantly reduced, because webhooks are no longer assessed for every build event. Previously, most events were "701 :: Webhook not enabled for build state 'event'". These are now not logged and don't appear in the statistics.

tcWebHooks runs webhooks in its own ThreadPool (ExecutorService).

tcWebHooks now has a dedicated threadpool from which webhooks are executed. This leaves the default threadpool available for other TeamCity activities.

The threadpool defaults to mininum of 1 worker thread, maximum of 50 worker threads and a queue of 3000 items.
The pool size determines the number of webhooks that can be executed simultaneously. The pool will automatically scale up from min up to the max size when items are added. The queue size determines the number of webhooks that can be queued for execution once the pool is already processing the maximum number of webhooks.

The new ThreadPool is configurable by adding a dedicatedThreadPool element in the webhooks section of main-config.xml. For example:

<webhooks>
  <dedicatedThreadPool enabled="true" minPoolSize="10" maxPoolSize="100" queueSize="5000" />
</webhooks>

Note: Changes to the size of pool and queue will not take effect until TeamCity is restarted.

These settings have the following effects:

1. enabled="true" - This enables the use of a separate thread pool. If set to false, tcWebHooks will revert the behaviour and use the default thread pool like tcWebHooks version 1.2.2 did.
2. minPoolSize="10" - Sets the minimum pool size to 10. Without this value, the default is 1.
3. maxPoolSize="100" - Sets the maximum webhooks workers to 100 at a time. Without this value, the default is 50.
4. queueSize="5000" - Set the maximum number of webhooks that can be queued waiting to be executed. Without this value, the default is 3000.

If the pool and queue both fill up, webhooks will error with code 802 :: WebHook Executor Queue full. Exception was: Unable to put new task into the queue because maximum capacity is exceeded. Current queue size: 1000. Executor info: Thread factory "WebHookThread-" {corePoolSize=10; maxPoolSize=50; activeCount=50}

This can been seen in the error list on the WebHooks Administation tab.

These errors are also logged to the teamcity-server.log

[2023-03-27 08:33:04,891] ERROR - jetbrains.buildServer.SERVER - WebHookThreadingExecutorImpl :: trackingId: 68c207ae-829b-4bee-9a33-d6cb91b6e78b :: projectId: TestTopProject :: webhookId: id_968610231 :: templateId: json-redux, errorCode: 802, errorMessage: Unable to put new task into the queue because maximum capacity is exceeded. Current queue size: 1000. Executor info: Thread factory "WebHookThread-" {corePoolSize=10; maxPoolSize=50; activeCount=50}