Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I cannot find in IPFS gateway / online (using CID) the content I uploaded through Helia #138

Closed
Eikix opened this issue May 27, 2023 · 5 comments
Labels
P2 Medium: Good to have, but can wait until someone steps up

Comments

@Eikix
Copy link

Eikix commented May 27, 2023

Hi all, my name's Elias,

I'm new to Helia, very cool project.
I'm currently writing a nodeJS script to add JSON files to IPFS using Helia:

import { createHelia } from 'helia';
import { strings } from '@helia/strings';
import { content } from './uri.mjs';
import fs from 'fs/promises';
import { createLibp2p } from 'libp2p';
import { tcp } from '@libp2p/tcp';
import { noise } from '@chainsafe/libp2p-noise';
import { yamux } from '@chainsafe/libp2p-yamux';
import { MemoryBlockstore } from 'blockstore-core';
import { MemoryDatastore } from 'datastore-core';

const blockstore = new MemoryBlockstore();
const datastore = new MemoryDatastore();

const libp2p = await createLibp2p({
  addresses: {
    listen: ['/ip4/127.0.0.1/tcp/8080'],
  },
  transports: [tcp()],
  connectionEncryption: [noise()],
  streamMuxers: [yamux()],
  datastore: datastore,
  identify: {
    host: {
      agentVersion: 'helia/0.0.0',
    },
  },
});

const helia = await createHelia({ libp2p, blockstore, datastore });
const s = strings(helia);

const addedCIDs = [];

async function uploadMetadata(content) {
  const metadata = {
    name: content.name,
    description: content.description,
    external_url: content.external_url,
    image: `ipfs://${content.image}`,
    attributes: content.attributes,
  };

  try {
    const cid = await s.add(JSON.stringify(metadata));
    addedCIDs.push(cid.toString());

    // Test that the CID can be retrieved
    const myString = await s.get(cid);
    console.log(JSON.parse(myString));

    // Parse CID
    const cidString = cid.toString();

    console.log(`Uploaded metadata for ${waifu.name}. CID info: ${cidString}`);
  } catch (err) {
    console.error(`Failed to upload metadata for ${waifu.name}. Error: ${err}`);
  }
}

async function main() {
  for (const item of content) {
    await uploadMetadata(item);
  }

  console.log(addedCIDs);


  await fs.writeFile('./addedCIDs.json', JSON.stringify(addedCIDs));

  console.log('All CIDs have been saved to addedCIDs.json');
}

main()
  .then(() => process.exit(0))
  .catch(console.error)
  .finally(() => process.exit(1));

The script successfully adds creates the CIDs in my addedCIDs.json file:

[
  "bafkreicbkmd3ba2y35co7v7n2kjnqfh2rrrw72j3etm5fmoumz5r65pa54",
  "bafkreihaparff6nojjm5qvnt55xnrnv4g4gxkmhexfbm45yozhqupko6am",
  "bafkreic3qaase5us7jkdd55mokytbed7fe4seujknmm52pff756t5nyn4m",
  "bafkreifhg3jgse6g2khv4npzg35buiqjn36xuoz7mu5aoobeh5ckqapspm",
  "bafkreigb5nozyw232voxad2vllafwgmwfhsla46uggzpsrrxrhkiszlk2q",
  "bafkreicwjokn4jhnpmesnhtj6vmktgl3vl2b6ebzocn5jl6fdq74ocyslu",
  "bafkreihptw6rz43otvvgx3ntf6rlxh5knfiod6l64nlaieo3t2inw4x5qm",
  "bafkreiaoekhluctmyehexg3loeb7jspx4pkkw6k2aixhhnypl7lofeujsm",
  "bafkreib5ilj4vm3yk35nyzvgup6kcybz2g7mzey4k4qssz733ftynybq5q",
  "bafkreiaa4zvkv563usfbbljci56jdkf7ven5lt5vbcxt6ki7sipcbezbfq",
  "bafkreieju7xdev3z4sabaeejm5f2r2mnaqeltm4adzklk6yeeyancuccle",
  "bafkreig7r5duyx2fcxqocx5q7opaqq4oqiahgguodc65zu45jj2sdimq2i",
  "bafkreibs6k2gbncsel2apz6m2oighgiy5zftfaeeq6l2swmss2h22liayq",
  "bafkreicytf3eapch4ghte2ybyjgz2lbkiw5xgywsmo4w7e3l3j7n445gg4",
  "bafkreibmsb6z3fd44bua3grtep3ah3cpcrpmy2lgooj3sjhdy5mba3zk5u",
  "bafkreigc3waovtz6zgqvfscqdmphwarmaie6vrf7q6sf7j35qcip6kkeze",
  "bafkreibhm37epellasuaqvc44geqv32cpittak2zcoqevkxreb7qp4uyhu"
]

But when I look them up on IPFS, I cannot seem to find them.

Is the problem linked with pinning? Or something I missed?

@Eikix
Copy link
Author

Eikix commented May 27, 2023

Terminal output:

image

Thanks for your help!

@Eikix
Copy link
Author

Eikix commented May 27, 2023

I also tried to use @helia/json to add the content to IPFS. Interestingly, the resulting CID looks like bagaaieraifjqpmedldpuj36x5xjjfwau7kggg37jhmsntuvr2rthwh3v4dxq and is no longer prefixed by "bafk" + "rei"

@Eikix Eikix changed the title How to pin content to my local IPFS node I cannot find in IPFS gateways using CID the content I uploaded through Helia May 30, 2023
@Eikix Eikix changed the title I cannot find in IPFS gateways using CID the content I uploaded through Helia I cannot find in IPFS gateway / online (using CID) the content I uploaded through Helia May 30, 2023
@BigLep
Copy link
Contributor

BigLep commented Jun 7, 2023

2023-06-07 maintainer conversation:

  1. We should point to an example that demonstrates this case clearly of making content retrievable from the gateway
  2. Give notification to user here about making a post in discuss.ipfs.tech

@BigLep BigLep added the P2 Medium: Good to have, but can wait until someone steps up label Jun 7, 2023
@maxdesalle
Copy link

Getting the same issue on my end, even with basic strings! Tried multiple IPFS explorers, none of them seem to index the uploaded data.

@SgtPooki
Copy link
Member

@Eikix @maxdesalle @zveroboy @AlexCherrypi

This is related to #159, ipfs-examples/helia-examples#43, ipfs-examples/helia-examples#38, and the discuss.ipfs.tech post located at https://discuss.ipfs.tech/t/how-to-retrieve-content-uploaded-via-helia-using-the-ipfs-gateway/16582.

I answered at https://discuss.ipfs.tech/t/how-to-retrieve-content-uploaded-via-helia-using-the-ipfs-gateway/16582/4?u=sgtpooki, but let's continue there as this is not a Helia bug and more of a learning opportunity.

I also tried to use @helia/json to add the content to IPFS. Interestingly, the resulting CID looks like bagaaieraifjqpmedldpuj36x5xjjfwau7kggg37jhmsntuvr2rthwh3v4dxq and is no longer prefixed by "bafk" + "rei"

This is how IPFS works. Please go through https://proto.school/anatomy-of-a-cid/ if you haven't yet and it should explain why using a different codec changes what the final CID is.

If you're running a non-browser node, the support provided by @achingbrain in ipfs-examples/helia-examples#43 may also be of assistance to you.

We should point to an example that demonstrates this case clearly of making content retrievable from the gateway

For an issue tracking the creation of an example where we host content on a Helia node and access that content from an IPFS gateway, please follow ipfs-examples/helia-examples#39

@github-project-automation github-project-automation bot moved this from 🥞 Todo to 🎉 Done in Helia Working Group (inactive) Jun 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P2 Medium: Good to have, but can wait until someone steps up
Projects
No open projects
Development

No branches or pull requests

4 participants