Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream requestbody to file and convert from file #12

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Empty file added src/bun.lockb
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See previous comment

Empty file.
39 changes: 34 additions & 5 deletions src/html-to-pdf-client.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,39 @@
import { spawn, which } from 'bun';
import { spawn, which, file } from 'bun';
import { PinoLogger } from './logger.ts';

export type HtmlToPdfClient = (req: Request, outputPath: string) => Promise<void>;
export const htmlToPdfClient: HtmlToPdfClient = async (req, outputPath) => {
export type HtmlToPdfClient = (inputPath: string, outputPath: string, logger: PinoLogger, timeout?: number) => Promise<void>;
export const htmlToPdfClient: HtmlToPdfClient = async (inputPath, outputPath, logger, timeout = 10_000) => {
const bin = which('wkhtmltopdf');
if (bin === null) {
throw new Error('Missing HTML to PDF binary');
}

const inputFile = Bun.file(inputPath);
if (!await inputFile.exists()) {
throw new Error(`Html for conversion ${inputPath} does not exist or is not readable`)
}

const htmlSize = inputFile.size;
if (htmlSize < 1) {
throw new Error(`Html file-size for conversion ${inputPath} is smaller than 1 byte`)
}

logger.info(`Starting conversion of HTML to PDF from ${inputPath}, size: ${htmlSize}`);

const proc = spawn(
['wkhtmltopdf', '--quiet', '--print-media-type', '--no-outline', '-', outputPath],
{stdin: req, stderr: 'pipe'},
['wkhtmltopdf', '--log-level', 'warn', '--print-media-type', '--disable-javascript', '--no-outline', inputPath, outputPath],
{stderr: 'pipe'},
);

const timer = setTimeout(() => {
proc.kill(129);
throw new Error(`Timing out after ${timeout} ms while calling wkhtmltopdf, killing the process manually`);
}, timeout);

const exitCode = await proc.exited;

clearTimeout(timer);

const errors: string = await Bun.readableStreamToText(proc.stderr);
if (errors) {
throw new Error(errors);
Expand All @@ -21,4 +43,11 @@ export const htmlToPdfClient: HtmlToPdfClient = async (req, outputPath) => {
if (exitCode !== 0) {
throw new Error(`Failed to convert HTML to PDF, the process exited with code ${exitCode}`);
}

const outputFile = file(outputPath);
const pdfSize = outputFile.size;
if (pdfSize < 1) {
throw new Error(`PDF file-size for conversion ${outputPath} is smaller than 1 byte`)
}
logger.info(`created PDF output with size: ${pdfSize}`);
};
2 changes: 1 addition & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import { createServer } from './server.ts';
import { trapShutdown } from './shutdown.ts';

const server = createServer();
const server = await createServer();

trapShutdown(async () => server.stop());
5 changes: 3 additions & 2 deletions src/logger.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
import pino, { type Logger as PinoLogger } from 'pino';
import pino from 'pino';

export const loggerUsingPino = () => pino({
name: 'html-pdf-export',
});

export type Logger = () => PinoLogger;
export type PinoLogger = typeof pino;
export type LoggerFactory = () => PinoLogger;
Empty file added src/package.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and bun.lockb are both unnecessary, probably an accidental bun install in the wrong directory

Empty file.
68 changes: 49 additions & 19 deletions src/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,58 @@ import { file } from 'bun';
import { nanoid } from 'nanoid';
import { mkdir, unlink } from 'node:fs/promises';
import { tmpdir } from 'os';
import { ReadableStream } from "stream/web";
import { htmlToPdfClient, type HtmlToPdfClient } from './html-to-pdf-client.ts';
import { type Logger, loggerUsingPino } from './logger.ts';
import { type LoggerFactory, loggerUsingPino, PinoLogger } from './logger.ts';

export interface CreateServerOptions {
port?: number;
logger?: Logger;
logger?: LoggerFactory;
htmlToPdfClient?: HtmlToPdfClient;
}

export const createServer = (options?: CreateServerOptions) => {
const convertHtmlToPdf = async(
requestBody: ReadableStream,
requestId: string,
tmpDir: string,
client: HtmlToPdfClient,
logger: PinoLogger
) => {
const outputPath = `${tmpDir}/${requestId}.pdf`;
const inputPath = `${tmpDir}/${requestId}.html`;

logger.info(`Writing request body to file: ${inputPath}`);
await Bun.write(inputPath, await Bun.readableStreamToBlob(requestBody));

const startTime = process.hrtime();

await client(inputPath, outputPath, logger);

const duration = process.hrtime(startTime);
logger.info(`Done converting HTML to PDF in ${duration}`);

const pdfOutput = file(outputPath);
logger.info(`created output size: ${pdfOutput.size}`);

pdfOutput.stream().getReader().closed.then(async() => {
await unlink(outputPath);
await unlink(inputPath);
});

return pdfOutput;
};

export const createServer = async(options?: CreateServerOptions) => {
const port = options?.port ?? 8000;
const logger = options?.logger?.() ?? loggerUsingPino();
const client = options?.htmlToPdfClient ?? htmlToPdfClient;

const tmpDir = process.env.HTML_PDF_EXPORT_TMPDIR ?? tmpdir();
if (!(await file(tmpDir).exists())) {
logger.info('Temporary file directory not found, creating a new directory');
await mkdir(tmpDir, {recursive: true});
}

logger.info(`Listening on port ${port}...`);

return Bun.serve({
Expand Down Expand Up @@ -44,23 +82,15 @@ export const createServer = (options?: CreateServerOptions) => {
return new Response(null, {status: 400});
}

const tmpDir = process.env.HTML_PDF_EXPORT_TMPDIR ?? tmpdir();
if (!(await file(tmpDir).exists())) {
logger.info('Temporary file directory not found, creating a new directory');
await mkdir(tmpDir, {recursive: true});
}
const pdfOutput = await convertHtmlToPdf(
req.body,
requestId,
tmpDir,
client,
logger,
);

const outputPath = `${tmpDir}/${requestId}.pdf`;
const contentLength = req.headers.get('content-length');
logger.info('Starting conversion of HTML to PDF', {contentLength});
const startTime = process.hrtime();
await client(req, outputPath);
const duration = process.hrtime(startTime);
logger.info('Done converting HTML to PDF', {contentLength, duration});

const output = file(outputPath);
output.stream().getReader().closed.then(() => unlink(outputPath));
return new Response(output, {status: 200, headers: {'content-type': 'application/pdf'}});
return new Response(pdfOutput, {status: 200, headers: {'content-type': 'application/pdf'}});
},
error(err) {
logger.error(err);
Expand Down
5 changes: 2 additions & 3 deletions src/shutdown.ts
Original file line number Diff line number Diff line change
@@ -1,13 +1,12 @@
import { sleep } from 'bun';
import { type Logger as PinoLogger } from 'pino';
import { type Logger, loggerUsingPino } from './logger.ts';
import { type LoggerFactory, loggerUsingPino, PinoLogger } from './logger.ts';

class ShutdownTimedOutError extends Error {
}

interface ShutdownOptions {
timeout?: number;
logger?: Logger;
logger?: LoggerFactory;
}

export function trapShutdown(callback: () => Promise<void>, options?: ShutdownOptions) {
Expand Down