Integrate JsonProvider with Deedle

[tpetricek]

It should be possible to use Frame.expandAllCols on JsonValue to flatten json values into columns. Something like this (but this does not quite work):

Frame.CustomExpanders.Add(typeof<FSharp.Data.Runtime.JsonDocument>, fun o -> seq {
  let doc = o :?> FSharp.Data.Runtime.JsonDocument
  match doc.JsonValue with 
  | JsonValue.Object(map) ->
      for KeyValue(k,v) in map do yield k,typeof<JsonValue>,box v 
  | _ -> () })
Frame.CustomExpanders.Add(typeof<JsonValue>, fun o -> seq {
  let doc = o :?> JsonValue
  match doc with 
  | JsonValue.Object(map) ->
      for KeyValue(k,v) in map do yield k,typeof<JsonValue>,box v 
  | _ -> () })

[ for l in weatherHistory.List -> series [ "It" => l ] ]
|> Frame.ofRowsOrdinal
|> Frame.expandAllCols 10

[ovatsus]

👍

[jrwdexter]

This almost works, but forces full expansion of the JSON record (due to recursion):

let rec expander key value =
    seq {
        match value with
        | JsonValue.String  (s) -> yield key,typeof<string>,box s
        | JsonValue.Boolean (b) -> yield key,typeof<bool>,box b
        | JsonValue.Float   (f) -> yield key,typeof<float>,box f
        | JsonValue.Null    (_) -> yield key,typeof<obj>,box ()
        | JsonValue.Number  (n) -> yield key,typeof<decimal>,box n
        | JsonValue.Record  (r) -> yield! r |> Seq.collect ((<||)expander)
        | JsonValue.Array   (a) ->
            yield! a
            |> Seq.collect (expander "arrayItem")
    }

Frame.CustomExpanders.Add(typeof<JsonDocument>, fun o -> (o :?> JsonDocument).JsonValue |> expander "root")
Frame.CustomExpanders.Add(typeof<JsonValue>, fun o ->  o :?> JsonValue |> expander "root")

// Test
[ for l in weatherHistory.List -> series [ "It" => l ] ]
|> Frame.ofRowsOrdinal
|> Frame.expandAllCols 10

[tpetricek]

@mandest I completely forgot about this. Do you know what the issue was with my original attempt? (Or was the only issue the fact that it only handled records?)

[jrwdexter]

@tpetricek I honestly am not 100% sure. If you handle just the record cases, many more items get expanded (eg. .IsNumber). By handling all of the cases, this sort of behavior is avoided. That being said, my example above forces complete expansion, as the expander itself is recursive (array & record call the expander). I was hoping to explore this further at some point, but I haven't gotten to it.

[dsyme]

FsLab is now changing to be an incubation space for F# data science projects, per discussion in #137 (comment)

Closing this out as it relates to the old FsLab package collection.