VSCode npm support

[Kawanaao]

Adaptation of the VSCode ebuild for npm, which VSCode switched to in newer versions. See issue #377

• npm cache for offline (tar) building
• VSCode builds in offline mode
• VSCode builds in online mode
• VSCode works as intended

[Kawanaao]

It seems that the workflow can't use a sandbox, which is why there is no issue with loading header files.

[Kawanaao]

Ah, I deleted the Yarn configuration where the nodedir was set and didn't replace it with npm. Now it makes sense.

[Kawanaao]

I'm not quite sure, reh requires the vscode-ripgrep binary release, but should it be replaced with the system one?

[Kawanaao]

It seems that reh cannot be installed offline for now.

[PF4Public]

It seems that the workflow can't use a sandbox, which is why there is no issue with loading header files.

Sandbox there is a little bit crippled, yes.

I'm not quite sure, reh requires the vscode-ripgrep binary release, but should it be replaced with the system one?

Would it work with the system one?

It seems that reh cannot be installed offline for now.

What makes you think that way? Maybe we miss some additional dependencies?

[Kawanaao]

Would it work with the system one?

System ripgrep works great, use the new version of VSCode myself.

What makes you think that way? Maybe we miss some additional dependencies?

reh requires loading node-20. As I saw in the ebuild, there was a TODO to implement support for the system node, but for now, it still requires loading node.

https://github.com/microsoft/vscode/blob/main/build/gulpfile.reh.js

BUILD_TARGETS.forEach(({ platform, arch }) => {
	gulp.task(task.define(`node-${platform}-${arch}`, () => {
		const nodePath = path.join('.build', 'node', `v${nodeVersion}`, `${platform}-${arch}`);

		if (!fs.existsSync(nodePath)) {
			util.rimraf(nodePath);

			return nodejs(platform, arch)
				.pipe(vfs.dest(nodePath));
		}

		return Promise.resolve(null);
	}));
});
...
['reh', 'reh-web'].forEach(type => {
	const bundleTask = task.define(`bundle-vscode-${type}`, task.series(
		util.rimraf(`out-vscode-${type}`),
		optimize.bundleTask(
			{
				out: `out-vscode-${type}`,
				esm: {
					src: 'out-build',
					entryPoints: [
						...(type === 'reh' ? serverEntryPoints : serverWithWebEntryPoints),
						...bootstrapEntryPoints
					],
					resources: type === 'reh' ? serverResources : serverWithWebResources,
					fileContentMapper: createVSCodeWebFileContentMapper('.build/extensions', type === 'reh-web' ? tweakProductForServerWeb(product) : product)
				}
			}
		)
	));

	const minifyTask = task.define(`minify-vscode-${type}`, task.series(
		bundleTask,
		util.rimraf(`out-vscode-${type}-min`),
		optimize.minifyTask(`out-vscode-${type}`, `https://main.vscode-cdn.net/sourcemaps/${commit}/core`)
	));
	gulp.task(minifyTask);

	BUILD_TARGETS.forEach(buildTarget => {
		const dashed = (str) => (str ? `-${str}` : ``);
		const platform = buildTarget.platform;
		const arch = buildTarget.arch;

		['', 'min'].forEach(minified => {
			const sourceFolderName = `out-vscode-${type}${dashed(minified)}`;
			const destinationFolderName = `vscode-${type}${dashed(platform)}${dashed(arch)}`;

			const serverTaskCI = task.define(`vscode-${type}${dashed(platform)}${dashed(arch)}${dashed(minified)}-ci`, task.series(
---->			gulp.task(`node-${platform}-${arch}`),
				util.rimraf(path.join(BUILD_ROOT, destinationFolderName)),
				packageTask(type, platform, arch, sourceFolderName, destinationFolderName)
			));
			gulp.task(serverTaskCI);

			const serverTask = task.define(`vscode-${type}${dashed(platform)}${dashed(arch)}${dashed(minified)}`, task.series(
				compileBuildTask,
				compileExtensionsBuildTask,
				compileExtensionMediaBuildTask,
				minified ? minifyTask : bundleTask,
				serverTaskCI
			));
			gulp.task(serverTask);
		});
	});
});

[Kawanaao]

Well... I think that using npm cache add is a mistake, this mechanism is too closed off, npm is very focused on cache proper functioning. Any modifications lead to issues with incorrect resolution of dependent packages, even when installing tgz files. I think it is worth returning to a local mirror

[Kawanaao]

Project Shrinkpack looks quite interesting, it replacing online dependencies with their offline tar versions, which should be helpful. I think it would be enough to make a few adjustments to support tar.gz in a specific folder

[Kawanaao]

No thanks, it's not tasty at all

TAR_BAD_ARCHIVE: Truncated input
npm warn tar zlib: Array buffer allocation failed
npm silly tar ZlibError: zlib: Array buffer allocation failed
npm silly tar     at Unzip.write (/usr/lib64/node_modules/npm/node_modules/minizlib/index.js:154:22)
npm silly tar     at Unpack.write (/usr/lib64/node_modules/npm/node_modules/tar/lib/parse.js:417:57)
npm silly tar     at [emitData] (/usr/lib64/node_modules/npm/node_modules/minipass/dist/commonjs/index.js:799:24)
npm silly tar     at Minipass.emit (/usr/lib64/node_modules/npm/node_modules/minipass/dist/commonjs/index.js:759:37)
npm silly tar     at Minipass.write (/usr/lib64/node_modules/npm/node_modules/minipass/dist/commonjs/index.js:376:18)
npm silly tar     at [emitData] (/usr/lib64/node_modules/npm/node_modules/minipass/dist/commonjs/index.js:799:24)
npm silly tar     at ReadStream.emit (/usr/lib64/node_modules/npm/node_modules/minipass/dist/commonjs/index.js:759:37)
npm silly tar     at ReadStream.emit (/usr/lib64/node_modules/npm/node_modules/fs-minipass/lib/index.js:173:22)
npm silly tar     at ReadStream.write (/usr/lib64/node_modules/npm/node_modules/minipass/dist/commonjs/index.js:376:18)
npm silly tar     at [_handleChunk] (/usr/lib64/node_modules/npm/node_modules/fs-minipass/lib/index.js:141:19)

Npm it just doesn't want to digest tgz, it's simply not capable of decompressing them in such quantities, but in order to build a dependency tree, you need to load all tgz, although the memory usage is low, but for some reason it completely ignores the memory and some errors constantly occur inside node like

' at Proxy.<anonymous> (/usr/lib64/node_modules/npm/bin/npm-cli.js:12:28)\n' +
' at /usr/lib64/node_modules/npm/ lib/cli/exit-handler.js:169:21\n' +
' at afterWrite (node:internal/streams/writable:708:5)\n' +
' at afterWriteTick (node:internal/streams/writable:694 :10)\n' +
' at process.processTicksAndRejections (node:internal/process/task_queues:81:21)',

and it seems like this signals correct reading.. but.. tar is damaged, tgz may be too , But and it doesn't want to use memory either.. and when reading correctly node returns 1 which terminates npm without any error message

[Kawanaao]

It seems that, moreover, VSCode requires the presence of a .git folder. While using SSH-remote, I noticed that validation fails because, instead of the commit hash, VSCode uses the version from package.json as a fallback

[PF4Public]

VSCode requires the presence of a .git folder. […] validation fails because, instead of the commit hash, VSCode uses the version from package.json as a fallback

I think I've solved that via

gentoo-overlay/app-editors/vscode/vscode-9999.ebuild

Lines 308 to 310 in ae9677d

	einfo "Editing build/lib/getVersion.js"
	sed -i '/.*\!version.*/{s++if \(false\)\{+;h};${x;/./{x;q0};x;q1}' \
	build/lib/getVersion.js || die

and

gentoo-overlay/app-editors/vscode/vscode-9999.ebuild

Line 330 in ae9677d

export BUILD_SOURCEVERSION="${COMMIT_ID}"