ERROR: MethodError: Cannot convert an object of type NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}} to an object of type String

[ctroupin]

The parameter auxdata_files cannot be written to the parameter file (netCDF).

Error message

1-element ExceptionStack:
LoadError: MethodError: Cannot `convert` an object of type NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}} to an object of type String

Closest candidates are:
  convert(::Type{T}, ::PyObject) where T<:AbstractString
   @ PyCall ~/.julia/packages/PyCall/ilqDX/src/conversions.jl:92
  convert(::Type{T}, ::Union{InitialValues.SpecificInitialValue{typeof(*)}, InitialValues.SpecificInitialValue{typeof(Base.mul_prod)}}) where T<:Union{AbstractString, Number}
   @ InitialValues ~/.julia/packages/InitialValues/OWP8V/src/InitialValues.jl:258
  convert(::Type{S}, ::CategoricalArrays.CategoricalValue) where S<:Union{AbstractChar, AbstractString, Number}
   @ CategoricalArrays ~/.julia/packages/CategoricalArrays/0yLZN/src/value.jl:92
  ...

Stacktrace:
  [1] setindex!(A::Vector{String}, x::NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}, i1::Int64)
    @ Base ./array.jl:969
  [2] _unsafe_copyto!(dest::Vector{String}, doffs::Int64, src::Vector{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}}, soffs::Int64, n::Int64)
    @ Base ./array.jl:250
  [3] unsafe_copyto!
    @ ./array.jl:304 [inlined]
  [4] _copyto_impl!
    @ ./array.jl:327 [inlined]
  [5] copyto!
    @ ./array.jl:314 [inlined]
  [6] copyto!
    @ ./array.jl:339 [inlined]
  [7] copyto_axcheck!
    @ ./abstractarray.jl:1182 [inlined]
  [8] Vector{String}(x::Vector{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}})
    @ Base ./array.jl:621
  [9] (::DINCAE.var"#123#124"{Int64, Int64, Bool, Vector{Int64}, UnitRange{Int64}, Float64, Int64, Int64, StepRange{Int64, Int64}, Symbol, Float64, Tuple{Float32, Float32}, Int64, Float64, Float64, Float64, Tuple{Float64}, Vector{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}}, Bool, Int64, Int64, Vector{Float64}})(ds_::NCDataset{Nothing})
    @ DINCAE ~/.julia/packages/DINCAE/SH3PW/src/points.jl:714
 [10] NCDataset(::DINCAE.var"#123#124"{Int64, Int64, Bool, Vector{Int64}, UnitRange{Int64}, Float64, Int64, Int64, StepRange{Int64, Int64}, Symbol, Float64, Tuple{Float32, Float32}, Int64, Float64, Float64, Float64, Tuple{Float64}, Vector{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}}, Bool, Int64, Int64, Vector{Float64}}, ::String, ::Vararg{String}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ NCDatasets ~/.julia/packages/NCDatasets/st9Jz/src/dataset.jl:255
 [11] NCDataset
    @ ~/.julia/packages/NCDatasets/st9Jz/src/dataset.jl:252 [inlined]
 [12] reconstruct_points(T::Type, Atype::Type, filename::String, varname::String, grid::Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}, StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}}, fnames_rec::Vector{String}; epochs::Int64, batch_size::Int64, truth_uncertain::Bool, enc_nfilter_internal::Vector{Int64}, skipconnections::UnitRange{Int64}, clip_grad::Float64, regularization_L1_beta::Int64, regularization_L2_beta::Int64, save_epochs::StepRange{Int64, Int64}, upsampling_method::Symbol, probability_skip_for_training::Float64, jitter_std_pos::Tuple{Float32, Float32}, ntime_win::Int64, learning_rate::Float64, learning_rate_decay_epoch::Float64, min_std_err::Float64, loss_weights_refine::Tuple{Float64}, auxdata_files::Vector{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}}, paramfile::String, savesnapshot::Bool, laplacian_penalty::Int64, laplacian_error_penalty::Int64)
    @ DINCAE ~/.julia/packages/DINCAE/SH3PW/src/points.jl:695
 [13] top-level scope
    @ ~/Projects/CPR-DINCAE/src/run_DINCAE_testparams.jl:114
 [14] include(fname::String)
    @ Base.MainInclude ./client.jl:478
 [15] top-level scope
    @ REPL[10]:1
 [16] top-level scope
    @ ~/.julia/packages/CUDA/35NC6/src/initialization.jl:190
in expression starting at /home/ctroupin/Projects/CPR-DINCAE/src/run_DINCAE_testparams.jl:100

Where the code fails

My bad: I misunderstood how the conversion from the list to a string should be performed.

DINCAE.jl/src/points.jl

Line 714 in 64d402e

ds_.attrib["auxdata_files"] = Vector{String}(collect(auxdata_files))

and

julia> typeof(auxdata_files)
Vector{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}} (alias for Array{NamedTuple{(:filename, :varname, :errvarname), Tuple{String, String, …}}, 1})
``

[Alexander-Barth]

maybe this ?

ds_.attrib["auxdata_filenames"] =  getindex.(auxdata_files,:filename)
ds_.attrib["auxdata_varnames"] =  getindex.(auxdata_files,:varname)
...

[ctroupin]

thanks, that seems to solve the issue in an elegant way. I'll put the changes on my branch and do the PR.

[ctroupin]

I forgot to come back here...
With

NCDataset(paramfile,"c") do ds_
  ...
  ds_.attrib["auxdata_filenames"] = getindex.(auxdata_files,:filename)
  ds_.attrib["auxdata_varnames"] = getindex.(auxdata_files,:varname)
  ds_.attrib["auxdata_errvarname"] = getindex.(auxdata_files,:errvarname)
  ...
end

we can indeed have the correct parameter values written in a netCDF file but when there is no auxiliary files then it fails because the default value of the auxdata_files is the empty list:

DINCAE.jl/src/points.jl

Line 549 in 5289964

auxdata_files = [],

I was trying to change it to

auxdata_files = [(filename = "", varname = "", errvarname = "")],

but then length(auxdata_files) > 0.

Solution?

We keep the default value auxdata_files=[] and then when writing the parameter file (netCDF) we check is there are auxdata_files:

NCDataset(paramfile,"c") do ds_
  ...
  if length(auxdata_files) > 0
    ds_.attrib["auxdata_filenames"] = getindex.(auxdata_files,:filename)
    ds_.attrib["auxdata_varnames"] = getindex.(auxdata_files,:varname)
    ds_.attrib["auxdata_errvarname"] = getindex.(auxdata_files,:errvarname)
  else
   @debug("No auxiliary data file")
  end
  ...
end

or do you see something more direct?

[Alexander-Barth]

Thanks, testing with if length(auxdata_files) is a good solution.