Cannot get final name for export 'toBase64' @smithy/[email protected] in NextJS #6686

MinhOmega · 2024-11-22T03:11:35Z

Checkboxes for prior research

I've gone through Developer Guide and API reference
I've checked AWS Forums and StackOverflow.
I've searched for previous similar issues and didn't find any solution.

Describe the bug

I am facing an issue when integrating media uploads to S3 using presigned URLs. I have installed @aws-sdk/[email protected] and @aws-sdk/[email protected], and I am using Next.js version 14.2.3. It works fine in development mode, but when I build the application, both locally and on Vercel, it throws the following error:

./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-stream/dist-es/index.js + 29 modules
Cannot get final name for export 'toBase64' of ./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-base64/dist-es/index.js

To resolve this, I tried downgrading to version 3.485.0, and it is now working fine.

Regression Issue

Select this option if this issue appears to be a regression.

SDK version number

@aws-sdk/[email protected], @aws-sdk/[email protected]

Which JavaScript Runtime is this issue in?

Node.js

Details of the browser/Node.js/ReactNative version

v20.9.0

Reproduction Steps

I have used server action to create the presigned URL then put the image with that response, here is sample code:

media.ts:

'use server'

import { getNextAuthServerSession } from '@/lib/auth'
import env from '@/lib/env'
import { PutObjectCommand, S3Client } from '@aws-sdk/client-s3'
import { getSignedUrl } from '@aws-sdk/s3-request-presigner'
import { nanoid } from 'nanoid'

// Initialize S3 client
const s3Client = new S3Client({
  region: env.AWS_REGION,
  credentials: {
    accessKeyId: env.AWS_ACCESS_KEY_ID,
    secretAccessKey: env.AWS_SECRET_ACCESS_KEY,
  },
})

export interface PresignedUrlResponse {
  uploadUrl: string
  fileKey: string
  error?: string
}

const ALLOWED_FILE_TYPES = ['image/jpeg', 'image/png', 'image/webp', 'image/gif']
const MAX_FILE_SIZE = 5 * 1024 * 1024 // 5MB

export const getPresignedUrl = async (
  fileName: string,
  fileType: string,
  directory: string = 'uploads'
): Promise<PresignedUrlResponse> => {
  try {
    // Check authentication (can ignore that and hard code for example)
    const session = await getNextAuthServerSession()
    if (!session?.user?.id) {
      return { uploadUrl: '', fileKey: '', error: 'Unauthorized' }
    }

    // Validate file type
    if (!ALLOWED_FILE_TYPES.includes(fileType)) {
      return {
        uploadUrl: '',
        fileKey: '',
        error: `Invalid file type. Allowed types: ${ALLOWED_FILE_TYPES.join(', ')}`,
      }
    }

    // Clean the filename and get extension
    const cleanFileName = fileName.replace(/[^a-zA-Z0-9.-]/g, '-').toLowerCase()
    const fileExtension = cleanFileName.split('.').pop()

    if (!fileExtension) {
      return { uploadUrl: '', fileKey: '', error: 'Invalid file extension' }
    }

    // Generate a unique file key
    const uniqueId = nanoid()
    const fileKey = `${directory}/${uniqueId}-${cleanFileName}`

    // Create the presigned URL
    const putObjectCommand = new PutObjectCommand({
      Bucket: env.AWS_BUCKET_NAME,
      Key: fileKey,
      ContentType: fileType,
      // Add additional metadata
      Metadata: {
        uploadedBy: session.user.id,
        originalName: fileName,
      },
    })

    const uploadUrl = await getSignedUrl(s3Client, putObjectCommand, {
      expiresIn: 60 * 5, // URL expires in 5 minutes
      signableHeaders: new Set(['content-type']), // Only sign necessary headers
    })

    return {
      uploadUrl,
      fileKey,
    }
  } catch (error) {
    console.error('Error generating presigned URL:', error)
    return {
      uploadUrl: '',
      fileKey: '',
      error: 'Failed to generate upload URL',
    }
  }
}

export const getPublicUrl = async (fileKey: string): Promise<string> => {
  if (!fileKey) return ''

  // Clean the file key to prevent directory traversal
  const cleanFileKey = fileKey.replace(/^\/+/, '').replace(/\.{2,}/g, '.')
  return `${env.AWS_CLOUDFRONT_URL}/${cleanFileKey}`
}

// Helper function to validate file size
export const validateFileSize = async (size: number): Promise<boolean> => {
  return size <= MAX_FILE_SIZE
}

// Helper function to validate file type
export const validateFileType = async (type: string): Promise<boolean> => {
  return ALLOWED_FILE_TYPES.includes(type)
}

in the upload in client side:

// Get presigned URL for the upload
  const { uploadUrl, fileKey, error } = await getPresignedUrl(file.name, file.type, 'slider-images')
  
  if (error || !uploadUrl) {
    console.error('Failed to get upload URL')
    continue
  }
  
  // Upload the file directly to S3
  const uploadResponse = await fetch(uploadUrl, {
    method: 'PUT',
    body: file,
    headers: {
      'Content-Type': file.type,
    },
    mode: 'cors',
  })

Observed Behavior

./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-stream/dist-es/index.js + 29 modules
Cannot get final name for export 'toBase64' of ./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-base64/dist-es/index.js

Expected Behavior

Build the project success without any error same as version 3.485.0

Possible Solution

No response

Additional Information/Context

No response

The text was updated successfully, but these errors were encountered:

zshzbh · 2024-11-22T16:28:11Z

Hey @MinhOmega ,

When you see the error locally but not in the dev environment, it's typically due to differences in module resolution and environment configurations.

Could you please try to clear your local cache and try again?

# Clear caches
rm -rf .next
rm -rf node_modules/.cache

# Remove node_modules
rm -rf node_modules

# Clear package manager cache
npm cache clean --force
# or
yarn cache clean
# or
pnpm store prune

# Reinstall dependencies
npm install

Let us know if you are still facing this issue.

Thanks!
Maggie

MinhOmega · 2024-11-24T08:29:27Z

Hi @zshzbh ,
After do step by step above, I still got the error in both locally and in Vercel deployment.

pnpm build ; pnpm start

> [email protected] build /Users/minhvo/Desktop/Web/tinnha
> next build

  ▲ Next.js 14.2.3
  - Environments: .env
  - Experiments (use with caution):
    · scrollRestoration
    · instrumentationHook

   Creating an optimized production build ...
<w> [webpack.cache.PackFileCacheStrategy] Serializing big strings (107kiB) impacts deserialization performance (consider using Buffer instead and decode when needed)
Failed to compile.

./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-stream/dist-es/index.js + 29 modules
Cannot get final name for export 'toBase64' of ./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-base64/dist-es/index.js


> Build failed because of webpack errors
 ELIFECYCLE  Command failed with exit code 1.

and in Vercel:

[00:28:20.296] Running build in Washington, D.C., USA (East) – iad1
[00:28:20.380] Cloning github.com/MinhOmega/tinnha-nextjs (Branch: feat/implement-top-header-and-switch-language, Commit: 04cf8ca)
[00:28:21.101] Cloning completed: 720.237ms
[00:28:38.411] Restored build cache from previous deployment (DBxpriKPwMkfBY9mJ5aBXV5Bukup)
[00:28:38.495] Running "vercel build"
[00:28:39.788] Vercel CLI 39.1.0
[00:28:40.288] Detected `pnpm-lock.yaml` version 9 generated by [email protected]
[00:28:40.316] Installing dependencies...
[00:28:40.970] Lockfile is up to date, resolution step is skipped
[00:28:41.089] Progress: resolved 1, reused 0, downloaded 0, added 0
[00:28:41.297] Packages: +100
[00:28:41.297] ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
[00:28:42.090] Progress: resolved 100, reused 0, downloaded 7, added 1
[00:28:43.091] Progress: resolved 100, reused 0, downloaded 11, added 1
[00:28:44.092] Progress: resolved 100, reused 0, downloaded 17, added 1
[00:28:45.093] Progress: resolved 100, reused 0, downloaded 20, added 2
[00:28:45.804] Progress: resolved 100, reused 0, downloaded 100, added 100, done
[00:28:46.664] 
[00:28:46.664] dependencies:
[00:28:46.664] + @aws-sdk/client-s3 3.697.0
[00:28:46.664] + @aws-sdk/s3-request-presigner 3.697.0
[00:28:46.664] 
[00:28:46.709] 
[00:28:46.709] > [email protected] postinstall /vercel/path0
[00:28:46.709] > prisma generate
[00:28:46.709] 
[00:28:49.849] Done in 9.5s
[00:28:49.872] Detected Next.js version: 14.2.3
[00:28:49.897] Running "pnpm run build"
[00:28:50.396] 
[00:28:50.396] > [email protected] build /vercel/path0
[00:28:50.396] > next build
[00:28:50.396] 
[00:28:51.640]   ▲ Next.js 14.2.3
[00:28:51.641]   - Experiments (use with caution):
[00:28:51.641]     · scrollRestoration
[00:28:51.641]     · instrumentationHook
[00:28:51.641] 
[00:28:51.718]    Creating an optimized production build ...
[00:29:57.165] <w> [webpack.cache.PackFileCacheStrategy] Serializing big strings (107kiB) impacts deserialization performance (consider using Buffer instead and decode when needed)
[00:29:57.322] Failed to compile.
[00:29:57.322] 
[00:29:57.322] ./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-stream/dist-es/index.js + 29 modules
[00:29:57.322] Cannot get final name for export 'toBase64' of ./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-base64/dist-es/index.js
[00:29:57.322] 
[00:29:57.323] 
[00:29:57.323] > Build failed because of webpack errors
[00:29:57.438]  ELIFECYCLE  Command failed with exit code 1.
[00:29:57.455] Error: Command "pnpm run build" exited with 1
[00:29:58.015]

inmanenz · 2024-11-26T01:42:38Z

I'm experiencing this same issue. I've also had to downgrade to 3.485.0 in order for my application to build

ringge · 2024-11-26T02:56:55Z

I'm experiencing same issue, hope to have a fix soon, thanks

trung1051996 · 2024-11-26T03:28:32Z

Hi @MinhOmega @zshzbh ,
I have the same problem, do you have any other solution?

MinhOmega · 2024-11-26T07:36:23Z

Hi @trung1051996 ,

Currently, we still need the maintainer of this library to find the root cause and fix it. As a temporary solution, I think we can downgrade to version 3.485.0, which is working fine on my side. You can try it on your side as well.

Thank you!

zshzbh · 2024-11-26T18:40:46Z

Using ▲ Next.js 14.2.18 with

   "@aws-sdk/client-s3": "^3.700.0",
    "@aws-sdk/s3-request-presigner": "^3.700.0",

and I don't see the error. I don't think this error comes from AWS SDK as it has a webpack error before it.

<w> [webpack.cache.PackFileCacheStrategy] Serializing big strings (107kiB) impacts deserialization performance (consider using Buffer instead and decode when needed)
Failed to compile.

The error above could cause the error

./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-stream/dist-es/index.js + 29 modules
Cannot get final name for export 'toBase64' of ./node_modules/.pnpm/@[email protected]/node_modules/@smithy/util-base64/dist-es/index.js

This is the log after running npm run build in the next.js app


 my-upload-app git:(main) ✗ npm run build

> [email protected] build
> next build

  ▲ Next.js 14.2.18
  - Environments: .env.local

   Creating an optimized production build ...
 ✓ Compiled successfully
 ✓ Linting and checking validity of types    
 ✓ Collecting page data    
 ✓ Generating static pages (5/5)
 ✓ Collecting build traces    
 ✓ Finalizing page optimization    

Route (app)                              Size     First Load JS
┌ ○ /                                    979 B          88.1 kB
└ ○ /_not-found                          875 B            88 kB
+ First Load JS shared by all            87.1 kB
  ├ chunks/117-5bf057607a392738.js       31.6 kB
  ├ chunks/fd9d1056-7d9b70e9456442b8.js  53.7 kB
  └ other shared chunks (total)          1.85 kB


○  (Static)  prerendered as static content

zshzbh · 2024-11-26T19:28:38Z

This is my minimal code reproduction GH repo - https://github.com/zshzbh/my-upload-app

The error is possibly related to Turbopack which doesn't use dist-cjs variant, as per error log. I will dive deep into this issue and discuss this with the team.

zshzbh · 2024-11-27T23:07:19Z

Hey @MinhOmega , @trung1051996, @inmanenz @ringge

The provided minimal repro that MinhOmega posted has too many dependencies. I created a basic application with JavaScript using Next.js 14.2.18 without any config, only with the necessary config

The application can be viewed on GitHub at https://github.com/zshzbh/my-upload-app

Since I don't have this issue on my repro, can you use my repro and remove dependencies from your minimal reproduction to check what might be adding to the repro project I have?

iamhimanshuraikwar · 2024-12-18T08:32:39Z

Hi @zshzbh

I’ve tried everything, but it still doesn’t work. Is there any other way I can approach this issue?

Yuripetusko · 2024-12-18T09:42:51Z

Adding this to next.config.mjs worked for us

 experimental: {
    serverComponentsExternalPackages: ['@smithy', 'util-stream'],
  },

iamhimanshuraikwar · 2024-12-18T12:36:26Z

@Yuripetusko Bro You saved us, thank you so much.

MinhOmega added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Nov 22, 2024

github-actions bot added the potential-regression Marking this issue as a potential regression to be checked by team member label Nov 22, 2024

zshzbh self-assigned this Nov 22, 2024

zshzbh added p2 This is a standard priority issue response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. and removed needs-triage This issue or PR still needs to be triaged. labels Nov 22, 2024

github-actions bot removed the response-requested Waiting on additional info and feedback. Will move to \"closing-soon\" in 7 days. label Nov 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cannot get final name for export 'toBase64' @smithy/[email protected] in NextJS #6686

Cannot get final name for export 'toBase64' @smithy/[email protected] in NextJS #6686

MinhOmega commented Nov 22, 2024

zshzbh commented Nov 22, 2024

MinhOmega commented Nov 24, 2024

inmanenz commented Nov 26, 2024

ringge commented Nov 26, 2024

trung1051996 commented Nov 26, 2024

MinhOmega commented Nov 26, 2024

zshzbh commented Nov 26, 2024

zshzbh commented Nov 26, 2024

zshzbh commented Nov 27, 2024

iamhimanshuraikwar commented Dec 18, 2024

Yuripetusko commented Dec 18, 2024

iamhimanshuraikwar commented Dec 18, 2024

Cannot get final name for export 'toBase64' @smithy/[email protected] in NextJS #6686

Cannot get final name for export 'toBase64' @smithy/[email protected] in NextJS #6686

Comments

MinhOmega commented Nov 22, 2024

Checkboxes for prior research

Describe the bug

Regression Issue

SDK version number

Which JavaScript Runtime is this issue in?

Details of the browser/Node.js/ReactNative version

Reproduction Steps

Observed Behavior

Expected Behavior

Possible Solution

Additional Information/Context

zshzbh commented Nov 22, 2024

MinhOmega commented Nov 24, 2024

inmanenz commented Nov 26, 2024

ringge commented Nov 26, 2024

trung1051996 commented Nov 26, 2024

MinhOmega commented Nov 26, 2024

zshzbh commented Nov 26, 2024

zshzbh commented Nov 26, 2024

zshzbh commented Nov 27, 2024

iamhimanshuraikwar commented Dec 18, 2024

Yuripetusko commented Dec 18, 2024

iamhimanshuraikwar commented Dec 18, 2024