windows drive letter not working as store path

[littleskunk]

My config:

  "storage": {
    "C:\\Storj": {
      "use_folder_tree": false, 
      "limit": "1G"
    }
  }, 

I also tried:

  "storage": {
    "C:/Storj": {
      "use_folder_tree": false, 
      "limit": "1G"
    }
  }, 

Both configs don't work. Files are stored in %userprofile%/Storj in both cases.

[littleskunk]

I set the limit to 100G for a better logoutput. Both configs give me the following output:

2016-02-22 22:28:11,845 WARNING storjnode.storage.manager 62: Invalid storage limit for C:\Users\jensh_000\Storj: 107374182400 > available 85214018904. Using available 85214018904!
2016-02-22 22:28:11,848 WARNING storjnode.storage.manager 62: Invalid storage limit for C:\Users\jensh_000\Users\jensh_000\Storj: 85214018904 > available 1517244416. Using available 1517244416!

[F483]

This is the code that gets the path. Why it doesn't use this util function I don't remember, there may be a reason, or it may just be an oversight on my part.

I moved many things from storjnode to storjlib, and in the future storjnode will need to be changed to use storjlib for that code (this will slim down storjnode a lot).

This is one of the functions I moved and changed here. I will backport the changes, but will need someone with a windows machine to confirm it works on windows as well.

[F483]

It seems my above analysis is incorrect. I already did this a while ago here and the function is being used correctly here.

@robertsdotpm @littleskunk could one of you debug what’s wrong with this on windows? I currently do not have access to (or am used to using) windows.

[littleskunk]

def full_path(path):
    """Resolves, sym links, rel paths, variables, and tilds to abs paths."""
    print path
    prefix = "/" if path.startswith("/") else ""
    print prefix
    normalized = prefix + os.path.join(*re.findall(r"[^\\|^/]+", path))
    print normalized
    print os.path.abspath(os.path.expandvars(os.path.expanduser(normalized)))
    return os.path.abspath(os.path.expandvars(os.path.expanduser(normalized)))

Output:

path = C:\Python27\Scripts\Storj
prefix = 
normalized = C:Python27\Scripts\Storj
return = C:\Python27\Scripts\Python27\Scripts\Storj

[F483]

So the full_path function is correct on windows? If so it must have something to do with how the config is been given.

What were the exact arguments given to storjnode and the exact config used?

[littleskunk]

I see 2 problems. At the moment the config returns a wrong path. That is not the fault of full_path. We should find and fix that bug first but it looks like the return value of full_path as a second bug.

[littleskunk]

My config:

  "storage": {
    "C:/Storj": {
      "use_folder_tree": false, 
      "limit": "100G"
    }
  },

[F483]

Out of curiosity what does full_path return for "C:/Storj" on windows? Maybe there is some case I didn't see.

[littleskunk]

I am running python for windows (C:\Python27\Scripts): python storjnode --wallet=[...] --debug farm
return = C:\Python27\Scripts\Python27\Scripts\Storj

[F483]

That means it should be loading from the default config path and I assume the config you posted above is that.

@littleskunk what output do you get for storjnode --wallet=[...] --debug cfg_get_current?

[littleskunk]

Server down at the moment. I will debug some more tomorrow starting with https://github.com/Storj/storjnode/blob/develop/storjnode/api.py#L67

cfg_get_current returns my config file %userprofile%.storj\cfg.json.

[littleskunk]

C:\Python27\Scripts>python storjnode --wallet xprv9s21ZrQH143K3m4QEqqF2hkQvTHczjeQbebVprErgQLF684MzDmbpMRDN4YdaTC9fxAFmw8sSKh6FqDXQYVZukmRgLhuYSbcTFca16Ut3EV cfg_get_current
2016-03-06 16:55:13,167 INFO storjnode.network.node 136: Started storjnode on port 30333 with address 1Gw99SmF9C1J8jLHHUQum4MMXeuB9kz78V
{
  "cold_storage": [],
  "version": 1,
  "storage": {
    "C:/Storj": {
      "use_folder_tree": false,
      "limit": "100G"
    }
  },
  "network": {
    "monitor": {
      "crawler_interval": 3600,
      "crawler_limit": 20,
      "enable_responses": true,
      "enable_crawler": true
    },
    "disable_data_transfer": false,
    "bootstrap_nodes": [],
    "bandwidth_limits": {
      "sec": {
        "downstream": 0,
        "upstream": 0
      },
      "month": {
        "downstream": "5G",
        "upstream": "5G"
      }
    },
    "refresh_neighbours_interval": 0,
    "port": "random"
  }
}

[littleskunk]

Found the bug.
https://github.com/Storj/storjnode/blame/develop/storjnode/util.py#L118

Setup is called 2 times. The first call returns the wrong path that is the input for the second call.

2016-03-06 17:30:12,319 INFO storjnode.network.monitor 379: MONITOR POST CRAWL STATISTICS: {
  "successfully_processed": 0,
  "testing_bandwidth": false,
  "waiting_for_bandwidth_test": 0,
  "scanning_info_and_peers": 0
}
path = C:/Storj
prefix =
normalized = C:Storj
return = C:\Python27\Scripts\Storj
2016-03-06 17:30:12,385 WARNING storjnode.storage.manager 62: Invalid storage limit for C:\Python27\Scripts\Storj: 107374182400 > available 73353321997. Using available 73353321997!
path = C:\Python27\Scripts\Storj
prefix =
normalized = C:Python27\Scripts\Storj
return = C:\Python27\Scripts\Python27\Scripts\Storj

[littleskunk]

Output of that return statement
os.path.expanduser(normalized) = C:Storj
os.path.expandvars(os.path.expanduser(normalized)) = C:Storj
os.path.abspath(os.path.expandvars(os.path.expanduser(normalized))) = C:\Python27\Scripts\Storj