Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dependency caching: small output improvements #2597

Merged
merged 2 commits into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions lib/caching-utils.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/caching-utils.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 18 additions & 3 deletions lib/dependency-caching.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/dependency-caching.js.map

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions lib/util.js

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion lib/util.js.map

Large diffs are not rendered by default.

4 changes: 3 additions & 1 deletion src/caching-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,16 @@ import { isHostedRunner, tryGetFolderBytes } from "./util";
* Returns the total size of all the specified paths.
* @param paths The paths for which to calculate the total size.
* @param logger A logger to record some informational messages to.
* @param quiet A value indicating whether to suppress logging warnings (default: false).
* @returns The total size of all specified paths.
*/
export async function getTotalCacheSize(
paths: string[],
logger: Logger,
quiet: boolean = false,
): Promise<number> {
const sizes = await Promise.all(
paths.map((cacheDir) => tryGetFolderBytes(cacheDir, logger)),
paths.map((cacheDir) => tryGetFolderBytes(cacheDir, logger, quiet)),
);
return sizes.map((a) => a || 0).reduce((a, b) => a + b, 0);
}
Expand Down
21 changes: 18 additions & 3 deletions src/dependency-caching.ts
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,7 @@ export async function uploadDependencyCaches(config: Config, logger: Logger) {
// use the cache quota that we compete with. In that case, we do not wish to use up all of the quota
// with the dependency caches. For this, we could use the Cache API to check whether other workflows
// are using the quota and how full it is.
const size = await getTotalCacheSize(cacheConfig.paths, logger);
const size = await getTotalCacheSize(cacheConfig.paths, logger, true);

// Skip uploading an empty cache.
if (size === 0) {
Expand All @@ -181,10 +181,25 @@ export async function uploadDependencyCaches(config: Config, logger: Logger) {
const key = await cacheKey(language, cacheConfig);

logger.info(
`Uploading cache of size ${size} for ${language} with key ${key}`,
`Uploading cache of size ${size} for ${language} with key ${key}...`,
);

await actionsCache.saveCache(cacheConfig.paths, key);
try {
await actionsCache.saveCache(cacheConfig.paths, key);
} catch (error) {
// `ReserveCacheError` indicates that the cache key is already in use, which means that a
// cache with that key already exists or is in the process of being uploaded by another
// workflow. We can ignore this.
if (error instanceof actionsCache.ReserveCacheError) {
logger.info(
`Not uploading cache for ${language}, because ${key} is already in use.`,
);
logger.debug(error.message);
} else {
// Propagate other errors upwards.
throw error;
}
}
}
}

Expand Down
9 changes: 8 additions & 1 deletion src/util.ts
Original file line number Diff line number Diff line change
Expand Up @@ -785,16 +785,23 @@ export function listFolder(dir: string): string[] {
*
* @param cacheDir A directory to get the size of.
* @param logger A logger to log any errors to.
* @param quiet A value indicating whether to suppress warnings for errors (default: false).
* Ignored if the log level is `debug`.
* @returns The size in bytes of the folder, or undefined if errors occurred.
*/
export async function tryGetFolderBytes(
cacheDir: string,
logger: Logger,
quiet: boolean = false,
): Promise<number | undefined> {
try {
return await promisify<string, number>(getFolderSize)(cacheDir);
} catch (e) {
logger.warning(`Encountered an error while getting size of folder: ${e}`);
if (!quiet || logger.isDebug()) {
logger.warning(
`Encountered an error while getting size of '${cacheDir}': ${e}`,
);
}
return undefined;
}
}
Expand Down
Loading