indexing_part2_advanced.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A deep dive into DataFrames.jl indexing\n",
    "# Part 2: implementation of indexing in DataFrames.jl\n",
    "### Bogumił Kamiński"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part we will not cover all scenarios of implementation of indexing in DataFrames.jl, but rather I will focus on scenarios that are non-obvious (at least for me)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial is tested under Julia 1.9.0."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In general to provide support for indexing and broadcasting for your type you should follow instructions contained in the [Julia manual](https://docs.julialang.org/en/v1/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Actually, effectively, this tutorial is mostly about how you can dig into what Julia is doing under the hood when processing your code."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also I hope it will show package developers how hard it is to define your own types that fully support indexing/broadcasting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, this notebook is more advanced and I refer to the source code a lot. I expect that it will be hard to follow it without watching the video recording of the tutorial during JuliaCon2020."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "using DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 1: Consequences of the fact that `DataFrame` can be resized"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>0×0 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|}\n",
       "\t& \\\\\n",
       "\t\\hline\n",
       "\t& \\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m0×0 DataFrame\u001b[0m"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0, 0)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "size(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "we get that number of rows is `0` but actually for `setindex!` and `setproperty!` it is treated as *undefined*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.x = [1, 2, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>3×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">3</td><td style = \"text-align: right;\">3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\t3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2\n",
       "   3 │     3"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "ArgumentError: New columns must have the same length as old columns",
     "output_type": "error",
     "traceback": [
      "ArgumentError: New columns must have the same length as old columns",
      "",
      "Stacktrace:",
      " [1] insert_single_column!(df::DataFrame, v::Vector{Int64}, col_ind::Symbol)",
      "   @ DataFrames C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\dataframe\\dataframe.jl:637",
      " [2] setindex!",
      "   @ C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\dataframe\\dataframe.jl:669 [inlined]",
      " [3] setproperty!(df::DataFrame, col_ind::Symbol, v::Vector{Int64})",
      "   @ DataFrames C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\dataframe\\dataframe.jl:675",
      " [4] top-level scope",
      "   @ In[6]:1"
     ]
    }
   ],
   "source": [
    "df.y = [1, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Base.setproperty!(df::DataFrame, col_ind::Symbol, v::AbstractVector) =\r\n",
      "    (df[!, col_ind] = v)\r\n",
      "Base.setproperty!(df::DataFrame, col_ind::AbstractString, v::AbstractVector) =\r\n",
      "    (df[!, col_ind] = v)\r\n",
      "Base.setproperty!(::DataFrame, col_ind::Symbol, v::Any) =\r\n",
      "    throw(ArgumentError(\"It is only allowed to pass a vector as a column of a DataFrame. \" *\r\n",
      "                        \"Instead use `df[!, col_ind] .= v` if you want to use broadcasting.\"))\r\n",
      "Base.setproperty!(::DataFrame, col_ind::AbstractString, v::Any) =\r\n",
      "    throw(ArgumentError(\"It is only allowed to pass a vector as a column of a DataFrame. \" *\r\n",
      "                        \"Instead use `df[!, col_ind] .= v` if you want to use broadcasting.\"))\r\n",
      "\r\n",
      "# df[SingleRowIndex, SingleColumnIndex] = Single Item\r\n",
      "function Base.setindex!(df::DataFrame, v::Any, row_ind::Integer, col_ind::ColumnIndex)\r\n",
      "    insert_single_entry!(df, v, row_ind, col_ind)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# df[SingleRowIndex, MultiColumnIndex] = value\r\n",
      "# the method for value of type DataFrameRow, AbstractDict and NamedTuple\r\n",
      "# is defined in dataframerow.jl\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::Union{Tuple, AbstractArray},\r\n",
      "                                  row_ind::Integer,\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if length(v) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"$(length(idxs)) columns were selected but the assigned \" *\r\n",
      "                                    \"collection contains $(length(v)) elements\"))\r\n",
      "        end\r\n",
      "        for (i, x) in zip(idxs, v)\r\n",
      "            df[!, i][row_ind] = x\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, SingleColumnIndex] = AbstractVector\r\n",
      "for T in (:AbstractVector, :Not, :Colon)\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::AbstractVector,\r\n",
      "                                  row_inds::$T,\r\n",
      "                                  col_ind::ColumnIndex)\r\n",
      "        if row_inds isa Colon && !haskey(index(df), col_ind)\r\n",
      "            df[!, col_ind] = copy(v)\r\n",
      "            return df\r\n",
      "        end\r\n",
      "        x = df[!, col_ind]\r\n",
      "        x[row_inds] = v\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractDataFrame\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"column names in source and target do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            df[!, col][row_inds] = new_df[!, j]\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::typeof(!),\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"Column names in source and target data frames do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # make sure we make a copy on assignment\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[!, col] = new_df[:, j]\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractMatrix\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon, :(typeof(!))),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  mx::AbstractMatrix,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if size(mx, 2) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"number of selected columns ($(length(idxs))) \" *\r\n",
      "                                    \"and number of columns in \" *\r\n",
      "                                    \"matrix ($(size(mx, 2))) do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[row_inds, col] = (row_inds === !) ? mx[:, j] : view(mx, :, j)\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    copy(df::DataFrame; copycols::Bool=true)\r\n",
      "\r\n",
      "Copy data frame `df`.\r\n",
      "If `copycols=true` (the default), return a new  `DataFrame` holding\r\n",
      "copies of column vectors in `df`.\r\n",
      "If `copycols=false`, return a new `DataFrame` sharing column vectors with `df`.\r\n",
      "\r\n",
      "Metadata: this function preserves all table-level and column-level metadata.\r\n",
      "\"\"\"\r\n",
      "function Base.copy(df::DataFrame; copycols::Bool=true)\r\n",
      "    cdf = DataFrame(copy(_columns(df)), copy(index(df)), copycols=copycols)\r\n",
      "\r\n",
      "    df_metadata = getfield(df, :metadata)\r\n",
      "    if !isnothing(df_metadata)\r\n",
      "        setfield!(cdf, :metadata, copy(df_metadata))\r\n",
      "    end\r\n",
      "    df_colmetadata = getfield(df, :colmetadata)\r\n",
      "    if !isnothing(df_colmetadata)\r\n",
      "        cdf_colmetadata = copy(df_colmetadata)\r\n",
      "        map!(copy, values(cdf_colmetadata))\r\n",
      "        setfield!(cdf, :colmetadata, cdf_colmetadata)\r\n",
      "    end\r\n",
      "    setfield!(cdf, :allnotemetadata, getfield(df, :allnotemetadata))\r\n",
      "\r\n",
      "    return cdf\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows specified by `inds` from a `DataFrame` `df` in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> deleteat!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, ::Colon) = empty!(df)\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    size(df, 2) == 0 && throw(BoundsError(df, (inds, :)))\r\n",
      "    return _deleteat!_helper(df, Int[inds])\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    if isempty(inds)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    elseif size(df, 2) == 0\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !(eltype(inds) <: Integer || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && inds isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to deleteat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector{Bool})\r\n",
      "    if length(inds) != size(df, 1)\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && drop isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, inds::Not) =\r\n",
      "    _deleteat!_helper(df, axes(df, 1)[inds])\r\n",
      "\r\n",
      "function _deleteat!_helper(df::DataFrame, drop)\r\n",
      "    cols = _columns(df)\r\n",
      "    if isempty(cols)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "\r\n",
      "    n = nrow(df)\r\n",
      "    col1 = cols[1]\r\n",
      "    deleteat!(col1, drop)\r\n",
      "    newn = length(col1)\r\n",
      "    @assert newn <= n\r\n",
      "    # the 0.06 threshold is heuristic; based on tests\r\n",
      "    # the assignment makes the code type-unstable but it is a small union\r\n",
      "    # so the overhead should be small\r\n",
      "    if drop isa AbstractVector{Bool} && newn < 0.06 * n\r\n",
      "        drop = findall(drop)\r\n",
      "    end\r\n",
      "    for i in 2:length(cols)\r\n",
      "        col = cols[i]\r\n",
      "        # this check is done to handle column aliases\r\n",
      "        if length(col) == n\r\n",
      "            deleteat!(col, drop)\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    for i in 1:length(cols)\r\n",
      "        # this should never happen, but we add it for safety\r\n",
      "        @assert length(cols[i]) == newn corrupt_msg(df, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows at all indices not specified by `inds` from a `DataFrame` `df`\r\n",
      "in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> keepat!(df, [1, 3])\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, ::Colon)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    isempty(inds) && return empty!(df)\r\n",
      "\r\n",
      "    # this is required because of https://github.com/JuliaData/InvertedIndices.jl/issues/31\r\n",
      "    if !((eltype(inds) <: Integer) || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        throw(ArgumentError(\"Indices passed to keepat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return deleteat!(df, Not(inds))\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    return deleteat!(df, Not(Int[inds]))\r\n",
      "end\r\n",
      "\r\n",
      "keepat!(df::DataFrame, inds::AbstractVector{Bool}) = deleteat!(df, .!inds)\r\n",
      "keepat!(df::DataFrame, inds::Not) = deleteat!(df, Not(inds))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    empty!(df::DataFrame)\r\n",
      "\r\n",
      "Remove all rows from `df`, making each of its columns empty.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> empty!(df)\r\n",
      "0×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┴──────────────\r\n",
      "\r\n",
      "julia> df.a, df.b\r\n",
      "(Int64[], Int64[])\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "function Base.empty!(df::DataFrame)\r\n",
      "    foreach(empty!, eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    resize!(df::DataFrame, n::Integer)\r\n",
      "\r\n",
      "Resize `df` to have `n` rows by calling `resize!` on all columns of `df`.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> resize!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.resize!(df::DataFrame, n::Integer)\r\n",
      "    if ncol(df) == 0 && n != 0\r\n",
      "        throw(ArgumentError(\"data frame has no columns and requested number \" *\r\n",
      "                            \"of rows is not zero\"))\r\n",
      "    end\r\n",
      "    foreach(col -> resize!(col, n), eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    pop!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the last row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> pop!(df)\r\n",
      "(a = 3, b = 6)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "Base.pop!(df::DataFrame) = popat!(df, nrow(df))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popfirst!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the first row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popfirst!(df)\r\n",
      "(a = 1, b = 4)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     2      5\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.popfirst!(df::DataFrame) = popat!(df, 1)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popat!(df::DataFrame, i::Integer)\r\n",
      "\r\n",
      "Remove the `i`-th row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popat!(df, 2)\r\n",
      "(a = 2, b = 5)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.popat!(df::DataFrame, i::Integer)\r\n",
      "    i isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    nt = NamedTuple(df[i, :])\r\n",
      "    deleteat!(df, i)\r\n",
      "    return nt\r\n",
      "end\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## hcat!\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\r\n",
      "# hcat! for 2 arguments, only a vector or a data frame is allowed\r\n",
      "function hcat!(df1::DataFrame, df2::AbstractDataFrame;\r\n",
      "               makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    u = add_names(index(df1), index(df2), makeunique=makeunique)\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df1)\r\n",
      "    _keep_matching_table_note_metadata!(df1, df2)\r\n",
      "    for i in 1:length(u)\r\n",
      "        df1[!, u[i]] = copycols ? df2[:, i] : df2[!, i]\r\n",
      "        _copy_col_note_metadata!(df1, u[i], df2, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df1\r\n",
      "end\r\n",
      "\r\n",
      "# TODO: after deprecation remove AbstractVector methods\r\n",
      "\r\n",
      "function hcat!(df::DataFrame, x::AbstractVector; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(df, DataFrame(AbstractVector[x], [:x1], copycols=false),\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "function hcat!(x::AbstractVector, df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(DataFrame(AbstractVector[x], [:x1], copycols=copycols), df,\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "# hcat! for 1-n arguments\r\n",
      "function hcat!(df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "hcat!(a::DataFrame, b::Union{AbstractDataFrame, AbstractVector},\r\n",
      "      c::Union{AbstractDataFrame, AbstractVector}...;\r\n",
      "      makeunique::Bool=false, copycols::Bool=true) =\r\n",
      "    hcat!(hcat!(a, b, makeunique=makeunique, copycols=copycols),\r\n",
      "          c..., makeunique=makeunique, copycols=copycols)\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## Missing values support\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\"\"\"\r\n",
      "    allowmissing!(df::DataFrame, cols=:)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `T` to\r\n",
      "`Union{T, Missing}` to support missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function allowmissing! end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, col::ColumnIndex)\r\n",
      "    df[!, col] = allowmissing(df[!, col])\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex})\r\n",
      "    for col in cols\r\n",
      "        allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{Bool})\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::MultiColumnIndex) =\r\n",
      "    allowmissing!(df, index(df)[cols])\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::Colon=:) =\r\n",
      "    allowmissing!(df, axes(df, 2))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    disallowmissing!(df::DataFrame, cols=:; error::Bool=true)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `Union{T, Missing}` to\r\n",
      "`T` to drop support for missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "If `error=false` then columns containing a `missing` value will be skipped instead\r\n",
      "of throwing an error.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function disallowmissing! end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, col::ColumnIndex; error::Bool=true)\r\n",
      "    x = df[!, col]\r\n",
      "    if Missing <: eltype(x)\r\n",
      "        try\r\n",
      "            df[!, col] = disallowmissing(x)\r\n",
      "        catch e\r\n",
      "            row = findfirst(ismissing, x)\r\n",
      "            if row !== nothing\r\n",
      "                if error\r\n",
      "                    col_name = only(names(df, col))\r\n",
      "                    throw(ArgumentError(\"Missing value found in column \" *\r\n",
      "                                        \":$col_name in row $row\"))\r\n",
      "                end\r\n",
      "            else\r\n",
      "                rethrow(e)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex};\r\n",
      "                          error::Bool=true)\r\n",
      "    for col in cols\r\n",
      "        disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{Bool}; error::Bool=true)\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::MultiColumnIndex; error::Bool=true) =\r\n",
      "    disallowmissing!(df, index(df)[cols], error=error)\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::Colon=:; error::Bool=true) =\r\n",
      "    disallowmissing!(df, axes(df, 2), error=error)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows. `inner` specifies how many\r\n",
      "times each row is repeated, and `outer` specifies how many times the full set\r\n",
      "of rows is repeated. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat!(df, inner=2, outer=3);\r\n",
      "\r\n",
      "julia> df\r\n",
      "12×2 DataFrame\r\n",
      " Row │ a      b\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     1      3\r\n",
      "   3 │     2      4\r\n",
      "   4 │     2      4\r\n",
      "   5 │     1      3\r\n",
      "   6 │     1      3\r\n",
      "   7 │     2      4\r\n",
      "   8 │     2      4\r\n",
      "   9 │     1      3\r\n",
      "  10 │     1      3\r\n",
      "  11 │     2      4\r\n",
      "  12 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "    inner < 0 && throw(ArgumentError(\"inner keyword argument must be non-negative\"))\r\n",
      "    outer < 0 && throw(ArgumentError(\"outer keyword argument must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, inner=Int(inner), outer=Int(outer))\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame, count::Integer)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows the number of times\r\n",
      "specified by `count`. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat(df, 2)\r\n",
      "4×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "   3 │     1      3\r\n",
      "   4 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame, count::Integer)\r\n",
      "    count < 0 && throw(ArgumentError(\"count must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, count)\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# This is not exactly copy! as in general we allow axes to be different\r\n",
      "# Also no table metadata needs to be copied as we use _replace_columns!\r\n",
      "# only in situations when table metadata for `df` must be left as-is\r\n",
      "function _replace_columns!(df::DataFrame, newdf::DataFrame)\r\n",
      "    # for DataFrame object here we do not support keep_present keyword argument\r\n",
      "    # like for SubDataFrame because here transform! always falls back to select!\r\n",
      "    @assert ncol(newdf) == 0 || nrow(df) == nrow(newdf)\r\n",
      "    copy!(_columns(df), _columns(newdf))\r\n",
      "    copy!(_names(index(df)), _names(newdf))\r\n",
      "    copy!(index(df).lookup, index(newdf).lookup)\r\n",
      "\r\n",
      "    emptycolmetadata!(df)\r\n",
      "    for (col, col_keys) in colmetadatakeys(newdf)\r\n",
      "        if hasproperty(df, col)\r\n",
      "            for key in col_keys\r\n",
      "                val, style = colmetadata(newdf, col, key, style=true)\r\n",
      "                style === :note && colmetadata!(df, col, key, val, style=:note)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}; kwargs...) =\r\n",
      "    isempty(kwargs) ? DataFrame() : allcombinations(DataFrame, kwargs...)\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}, pairs::Pair{<:AbstractString, <:Any}...) =\r\n",
      "    allcombinations(DataFrame, (Symbol(k) => v for (k, v) in pairs)...)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    allcombinations(DataFrame, pairs::Pair...)\r\n",
      "    allcombinations(DataFrame; kwargs...)\r\n",
      "\r\n",
      "Create a `DataFrame` from all combinations of values in passed arguments.\r\n",
      "The first passed values vary fastest.\r\n",
      "\r\n",
      "Arguments associating a column name with values to expand can be specified\r\n",
      "either as `Pair`s passed as positional arguments, or as keyword arguments.\r\n",
      "Column names must be `Symbol`s or strings and must be unique.\r\n",
      "\r\n",
      "Column value can be a vector which is consumed as is or an object of any other\r\n",
      "type (except `AbstractArray`). In the latter case the passed value is treated\r\n",
      "as having length one for expansion. As a particular rule values stored in a `Ref`\r\n",
      "or a `0`-dimensional `AbstractArray` are unwrapped and treated as having length one.\r\n",
      "\r\n",
      "See also: [`crossjoin`](@ref) can be used to get the cartesian product\r\n",
      "of rows from passed data frames.\r\n",
      "\r\n",
      "# Examples\r\n",
      "\r\n",
      "```jldoctest\r\n",
      "julia> allcombinations(DataFrame, a=1:2, b='a':'c')\r\n",
      "6×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Char\r\n",
      "─────┼─────────────\r\n",
      "   1 │     1  a\r\n",
      "   2 │     2  a\r\n",
      "   3 │     1  b\r\n",
      "   4 │     2  b\r\n",
      "   5 │     1  c\r\n",
      "   6 │     2  c\r\n",
      "\r\n",
      "julia> allcombinations(DataFrame, \"a\" => 1:2, \"b\" => 'a':'c', \"c\" => \"const\")\r\n",
      "6×3 DataFrame\r\n",
      " Row │ a      b     c\r\n",
      "     │ Int64  Char  String\r\n",
      "─────┼─────────────────────\r\n",
      "   1 │     1  a     const\r\n",
      "   2 │     2  a     const\r\n",
      "   3 │     1  b     const\r\n",
      "   4 │     2  b     const\r\n",
      "   5 │     1  c     const\r\n",
      "   6 │     2  c     const\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function allcombinations(::Type{DataFrame}, pairs::Pair{Symbol, <:Any}...)\r\n",
      "    colnames = first.(pairs)\r\n",
      "    if !allunique(colnames)\r\n",
      "        throw(ArgumentError(\"All column names passed to allcombinations must be unique\"))\r\n",
      "    end\r\n",
      "    colvalues = map(pairs) do p\r\n",
      "        v = last(p)\r\n",
      "        if v isa AbstractVector\r\n",
      "            return v\r\n",
      "        elseif v isa Union{AbstractArray{<:Any, 0}, Ref}\r\n",
      "            x = v[]\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(x), 1), x)\r\n",
      "        elseif v isa AbstractArray\r\n",
      "            throw(ArgumentError(\"adding AbstractArray other than AbstractVector \" *\r\n",
      "                                \"as a column of a data frame is not allowed\"))\r\n",
      "        else\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(v), 1), v)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    @assert length(colvalues) == length(colnames)\r\n",
      "    @assert all(x -> x isa AbstractVector, colvalues)\r\n",
      "\r\n",
      "    target_rows = Int(prod(x -> big(length(x)), colvalues))\r\n",
      "    out_df = DataFrame()\r\n",
      "    inner = 1\r\n",
      "    for (val, cname) in zip(colvalues, colnames)\r\n",
      "        len = length(val)\r\n",
      "        last_inner = inner\r\n",
      "        inner *= len\r\n",
      "        outer, remv = inner == 0 ? (0, 0) : divrem(target_rows, inner)\r\n",
      "        @assert iszero(remv)\r\n",
      "        out_df[!, cname] = repeat(val, inner=last_inner, outer=outer)\r\n",
      "    end\r\n",
      "    @assert inner == target_rows\r\n",
      "    @assert size(out_df) == (target_rows, length(colnames))\r\n",
      "    return out_df\r\n",
      "end\r\n",
      "\r\n",
      "_try_select_no_copy(df::DataFrame, cols) = select(df, cols, copycols=false)\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "@less df.y = [1, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function Base.setindex!(df::DataFrame, v::AbstractVector, ::typeof(!), col_ind::ColumnIndex)\r\n",
      "    insert_single_column!(df, v, col_ind)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# df.col = AbstractVector\r\n",
      "# separate methods are needed due to dispatch ambiguity\r\n",
      "Base.setproperty!(df::DataFrame, col_ind::Symbol, v::AbstractVector) =\r\n",
      "    (df[!, col_ind] = v)\r\n",
      "Base.setproperty!(df::DataFrame, col_ind::AbstractString, v::AbstractVector) =\r\n",
      "    (df[!, col_ind] = v)\r\n",
      "Base.setproperty!(::DataFrame, col_ind::Symbol, v::Any) =\r\n",
      "    throw(ArgumentError(\"It is only allowed to pass a vector as a column of a DataFrame. \" *\r\n",
      "                        \"Instead use `df[!, col_ind] .= v` if you want to use broadcasting.\"))\r\n",
      "Base.setproperty!(::DataFrame, col_ind::AbstractString, v::Any) =\r\n",
      "    throw(ArgumentError(\"It is only allowed to pass a vector as a column of a DataFrame. \" *\r\n",
      "                        \"Instead use `df[!, col_ind] .= v` if you want to use broadcasting.\"))\r\n",
      "\r\n",
      "# df[SingleRowIndex, SingleColumnIndex] = Single Item\r\n",
      "function Base.setindex!(df::DataFrame, v::Any, row_ind::Integer, col_ind::ColumnIndex)\r\n",
      "    insert_single_entry!(df, v, row_ind, col_ind)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# df[SingleRowIndex, MultiColumnIndex] = value\r\n",
      "# the method for value of type DataFrameRow, AbstractDict and NamedTuple\r\n",
      "# is defined in dataframerow.jl\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::Union{Tuple, AbstractArray},\r\n",
      "                                  row_ind::Integer,\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if length(v) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"$(length(idxs)) columns were selected but the assigned \" *\r\n",
      "                                    \"collection contains $(length(v)) elements\"))\r\n",
      "        end\r\n",
      "        for (i, x) in zip(idxs, v)\r\n",
      "            df[!, i][row_ind] = x\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, SingleColumnIndex] = AbstractVector\r\n",
      "for T in (:AbstractVector, :Not, :Colon)\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::AbstractVector,\r\n",
      "                                  row_inds::$T,\r\n",
      "                                  col_ind::ColumnIndex)\r\n",
      "        if row_inds isa Colon && !haskey(index(df), col_ind)\r\n",
      "            df[!, col_ind] = copy(v)\r\n",
      "            return df\r\n",
      "        end\r\n",
      "        x = df[!, col_ind]\r\n",
      "        x[row_inds] = v\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractDataFrame\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"column names in source and target do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            df[!, col][row_inds] = new_df[!, j]\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::typeof(!),\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"Column names in source and target data frames do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # make sure we make a copy on assignment\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[!, col] = new_df[:, j]\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractMatrix\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon, :(typeof(!))),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  mx::AbstractMatrix,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if size(mx, 2) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"number of selected columns ($(length(idxs))) \" *\r\n",
      "                                    \"and number of columns in \" *\r\n",
      "                                    \"matrix ($(size(mx, 2))) do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[row_inds, col] = (row_inds === !) ? mx[:, j] : view(mx, :, j)\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    copy(df::DataFrame; copycols::Bool=true)\r\n",
      "\r\n",
      "Copy data frame `df`.\r\n",
      "If `copycols=true` (the default), return a new  `DataFrame` holding\r\n",
      "copies of column vectors in `df`.\r\n",
      "If `copycols=false`, return a new `DataFrame` sharing column vectors with `df`.\r\n",
      "\r\n",
      "Metadata: this function preserves all table-level and column-level metadata.\r\n",
      "\"\"\"\r\n",
      "function Base.copy(df::DataFrame; copycols::Bool=true)\r\n",
      "    cdf = DataFrame(copy(_columns(df)), copy(index(df)), copycols=copycols)\r\n",
      "\r\n",
      "    df_metadata = getfield(df, :metadata)\r\n",
      "    if !isnothing(df_metadata)\r\n",
      "        setfield!(cdf, :metadata, copy(df_metadata))\r\n",
      "    end\r\n",
      "    df_colmetadata = getfield(df, :colmetadata)\r\n",
      "    if !isnothing(df_colmetadata)\r\n",
      "        cdf_colmetadata = copy(df_colmetadata)\r\n",
      "        map!(copy, values(cdf_colmetadata))\r\n",
      "        setfield!(cdf, :colmetadata, cdf_colmetadata)\r\n",
      "    end\r\n",
      "    setfield!(cdf, :allnotemetadata, getfield(df, :allnotemetadata))\r\n",
      "\r\n",
      "    return cdf\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows specified by `inds` from a `DataFrame` `df` in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> deleteat!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, ::Colon) = empty!(df)\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    size(df, 2) == 0 && throw(BoundsError(df, (inds, :)))\r\n",
      "    return _deleteat!_helper(df, Int[inds])\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    if isempty(inds)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    elseif size(df, 2) == 0\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !(eltype(inds) <: Integer || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && inds isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to deleteat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector{Bool})\r\n",
      "    if length(inds) != size(df, 1)\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && drop isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, inds::Not) =\r\n",
      "    _deleteat!_helper(df, axes(df, 1)[inds])\r\n",
      "\r\n",
      "function _deleteat!_helper(df::DataFrame, drop)\r\n",
      "    cols = _columns(df)\r\n",
      "    if isempty(cols)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "\r\n",
      "    n = nrow(df)\r\n",
      "    col1 = cols[1]\r\n",
      "    deleteat!(col1, drop)\r\n",
      "    newn = length(col1)\r\n",
      "    @assert newn <= n\r\n",
      "    # the 0.06 threshold is heuristic; based on tests\r\n",
      "    # the assignment makes the code type-unstable but it is a small union\r\n",
      "    # so the overhead should be small\r\n",
      "    if drop isa AbstractVector{Bool} && newn < 0.06 * n\r\n",
      "        drop = findall(drop)\r\n",
      "    end\r\n",
      "    for i in 2:length(cols)\r\n",
      "        col = cols[i]\r\n",
      "        # this check is done to handle column aliases\r\n",
      "        if length(col) == n\r\n",
      "            deleteat!(col, drop)\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    for i in 1:length(cols)\r\n",
      "        # this should never happen, but we add it for safety\r\n",
      "        @assert length(cols[i]) == newn corrupt_msg(df, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows at all indices not specified by `inds` from a `DataFrame` `df`\r\n",
      "in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> keepat!(df, [1, 3])\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, ::Colon)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    isempty(inds) && return empty!(df)\r\n",
      "\r\n",
      "    # this is required because of https://github.com/JuliaData/InvertedIndices.jl/issues/31\r\n",
      "    if !((eltype(inds) <: Integer) || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to keepat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return deleteat!(df, Not(inds))\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    return deleteat!(df, Not(Int[inds]))\r\n",
      "end\r\n",
      "\r\n",
      "keepat!(df::DataFrame, inds::AbstractVector{Bool}) = deleteat!(df, .!inds)\r\n",
      "keepat!(df::DataFrame, inds::Not) = deleteat!(df, Not(inds))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    empty!(df::DataFrame)\r\n",
      "\r\n",
      "Remove all rows from `df`, making each of its columns empty.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> empty!(df)\r\n",
      "0×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┴──────────────\r\n",
      "\r\n",
      "julia> df.a, df.b\r\n",
      "(Int64[], Int64[])\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "function Base.empty!(df::DataFrame)\r\n",
      "    foreach(empty!, eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    resize!(df::DataFrame, n::Integer)\r\n",
      "\r\n",
      "Resize `df` to have `n` rows by calling `resize!` on all columns of `df`.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> resize!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.resize!(df::DataFrame, n::Integer)\r\n",
      "    if ncol(df) == 0 && n != 0\r\n",
      "        throw(ArgumentError(\"data frame has no columns and requested number \" *\r\n",
      "                            \"of rows is not zero\"))\r\n",
      "    end\r\n",
      "    foreach(col -> resize!(col, n), eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    pop!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the last row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> pop!(df)\r\n",
      "(a = 3, b = 6)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "Base.pop!(df::DataFrame) = popat!(df, nrow(df))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popfirst!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the first row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popfirst!(df)\r\n",
      "(a = 1, b = 4)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     2      5\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.popfirst!(df::DataFrame) = popat!(df, 1)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popat!(df::DataFrame, i::Integer)\r\n",
      "\r\n",
      "Remove the `i`-th row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popat!(df, 2)\r\n",
      "(a = 2, b = 5)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.popat!(df::DataFrame, i::Integer)\r\n",
      "    i isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    nt = NamedTuple(df[i, :])\r\n",
      "    deleteat!(df, i)\r\n",
      "    return nt\r\n",
      "end\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## hcat!\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\r\n",
      "# hcat! for 2 arguments, only a vector or a data frame is allowed\r\n",
      "function hcat!(df1::DataFrame, df2::AbstractDataFrame;\r\n",
      "               makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    u = add_names(index(df1), index(df2), makeunique=makeunique)\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df1)\r\n",
      "    _keep_matching_table_note_metadata!(df1, df2)\r\n",
      "    for i in 1:length(u)\r\n",
      "        df1[!, u[i]] = copycols ? df2[:, i] : df2[!, i]\r\n",
      "        _copy_col_note_metadata!(df1, u[i], df2, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df1\r\n",
      "end\r\n",
      "\r\n",
      "# TODO: after deprecation remove AbstractVector methods\r\n",
      "\r\n",
      "function hcat!(df::DataFrame, x::AbstractVector; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(df, DataFrame(AbstractVector[x], [:x1], copycols=false),\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "function hcat!(x::AbstractVector, df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(DataFrame(AbstractVector[x], [:x1], copycols=copycols), df,\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "# hcat! for 1-n arguments\r\n",
      "function hcat!(df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "hcat!(a::DataFrame, b::Union{AbstractDataFrame, AbstractVector},\r\n",
      "      c::Union{AbstractDataFrame, AbstractVector}...;\r\n",
      "      makeunique::Bool=false, copycols::Bool=true) =\r\n",
      "    hcat!(hcat!(a, b, makeunique=makeunique, copycols=copycols),\r\n",
      "          c..., makeunique=makeunique, copycols=copycols)\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## Missing values support\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\"\"\"\r\n",
      "    allowmissing!(df::DataFrame, cols=:)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `T` to\r\n",
      "`Union{T, Missing}` to support missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function allowmissing! end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, col::ColumnIndex)\r\n",
      "    df[!, col] = allowmissing(df[!, col])\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex})\r\n",
      "    for col in cols\r\n",
      "        allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{Bool})\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::MultiColumnIndex) =\r\n",
      "    allowmissing!(df, index(df)[cols])\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::Colon=:) =\r\n",
      "    allowmissing!(df, axes(df, 2))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    disallowmissing!(df::DataFrame, cols=:; error::Bool=true)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `Union{T, Missing}` to\r\n",
      "`T` to drop support for missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "If `error=false` then columns containing a `missing` value will be skipped instead\r\n",
      "of throwing an error.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function disallowmissing! end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, col::ColumnIndex; error::Bool=true)\r\n",
      "    x = df[!, col]\r\n",
      "    if Missing <: eltype(x)\r\n",
      "        try\r\n",
      "            df[!, col] = disallowmissing(x)\r\n",
      "        catch e\r\n",
      "            row = findfirst(ismissing, x)\r\n",
      "            if row !== nothing\r\n",
      "                if error\r\n",
      "                    col_name = only(names(df, col))\r\n",
      "                    throw(ArgumentError(\"Missing value found in column \" *\r\n",
      "                                        \":$col_name in row $row\"))\r\n",
      "                end\r\n",
      "            else\r\n",
      "                rethrow(e)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex};\r\n",
      "                          error::Bool=true)\r\n",
      "    for col in cols\r\n",
      "        disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{Bool}; error::Bool=true)\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::MultiColumnIndex; error::Bool=true) =\r\n",
      "    disallowmissing!(df, index(df)[cols], error=error)\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::Colon=:; error::Bool=true) =\r\n",
      "    disallowmissing!(df, axes(df, 2), error=error)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows. `inner` specifies how many\r\n",
      "times each row is repeated, and `outer` specifies how many times the full set\r\n",
      "of rows is repeated. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat!(df, inner=2, outer=3);\r\n",
      "\r\n",
      "julia> df\r\n",
      "12×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     1      3\r\n",
      "   3 │     2      4\r\n",
      "   4 │     2      4\r\n",
      "   5 │     1      3\r\n",
      "   6 │     1      3\r\n",
      "   7 │     2      4\r\n",
      "   8 │     2      4\r\n",
      "   9 │     1      3\r\n",
      "  10 │     1      3\r\n",
      "  11 │     2      4\r\n",
      "  12 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "    inner < 0 && throw(ArgumentError(\"inner keyword argument must be non-negative\"))\r\n",
      "    outer < 0 && throw(ArgumentError(\"outer keyword argument must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, inner=Int(inner), outer=Int(outer))\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame, count::Integer)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows the number of times\r\n",
      "specified by `count`. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat(df, 2)\r\n",
      "4×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "   3 │     1      3\r\n",
      "   4 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame, count::Integer)\r\n",
      "    count < 0 && throw(ArgumentError(\"count must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, count)\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# This is not exactly copy! as in general we allow axes to be different\r\n",
      "# Also no table metadata needs to be copied as we use _replace_columns!\r\n",
      "# only in situations when table metadata for `df` must be left as-is\r\n",
      "function _replace_columns!(df::DataFrame, newdf::DataFrame)\r\n",
      "    # for DataFrame object here we do not support keep_present keyword argument\r\n",
      "    # like for SubDataFrame because here transform! always falls back to select!\r\n",
      "    @assert ncol(newdf) == 0 || nrow(df) == nrow(newdf)\r\n",
      "    copy!(_columns(df), _columns(newdf))\r\n",
      "    copy!(_names(index(df)), _names(newdf))\r\n",
      "    copy!(index(df).lookup, index(newdf).lookup)\r\n",
      "\r\n",
      "    emptycolmetadata!(df)\r\n",
      "    for (col, col_keys) in colmetadatakeys(newdf)\r\n",
      "        if hasproperty(df, col)\r\n",
      "            for key in col_keys\r\n",
      "                val, style = colmetadata(newdf, col, key, style=true)\r\n",
      "                style === :note && colmetadata!(df, col, key, val, style=:note)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}; kwargs...) =\r\n",
      "    isempty(kwargs) ? DataFrame() : allcombinations(DataFrame, kwargs...)\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}, pairs::Pair{<:AbstractString, <:Any}...) =\r\n",
      "    allcombinations(DataFrame, (Symbol(k) => v for (k, v) in pairs)...)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    allcombinations(DataFrame, pairs::Pair...)\r\n",
      "    allcombinations(DataFrame; kwargs...)\r\n",
      "\r\n",
      "Create a `DataFrame` from all combinations of values in passed arguments.\r\n",
      "The first passed values vary fastest.\r\n",
      "\r\n",
      "Arguments associating a column name with values to expand can be specified\r\n",
      "either as `Pair`s passed as positional arguments, or as keyword arguments.\r\n",
      "Column names must be `Symbol`s or strings and must be unique.\r\n",
      "\r\n",
      "Column value can be a vector which is consumed as is or an object of any other\r\n",
      "type (except `AbstractArray`). In the latter case the passed value is treated\r\n",
      "as having length one for expansion. As a particular rule values stored in a `Ref`\r\n",
      "or a `0`-dimensional `AbstractArray` are unwrapped and treated as having length one.\r\n",
      "\r\n",
      "See also: [`crossjoin`](@ref) can be used to get the cartesian product\r\n",
      "of rows from passed data frames.\r\n",
      "\r\n",
      "# Examples\r\n",
      "\r\n",
      "```jldoctest\r\n",
      "julia> allcombinations(DataFrame, a=1:2, b='a':'c')\r\n",
      "6×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Char\r\n",
      "─────┼─────────────\r\n",
      "   1 │     1  a\r\n",
      "   2 │     2  a\r\n",
      "   3 │     1  b\r\n",
      "   4 │     2  b\r\n",
      "   5 │     1  c\r\n",
      "   6 │     2  c\r\n",
      "\r\n",
      "julia> allcombinations(DataFrame, \"a\" => 1:2, \"b\" => 'a':'c', \"c\" => \"const\")\r\n",
      "6×3 DataFrame\r\n",
      " Row │ a      b     c\r\n",
      "     │ Int64  Char  String\r\n",
      "─────┼─────────────────────\r\n",
      "   1 │     1  a     const\r\n",
      "   2 │     2  a     const\r\n",
      "   3 │     1  b     const\r\n",
      "   4 │     2  b     const\r\n",
      "   5 │     1  c     const\r\n",
      "   6 │     2  c     const\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function allcombinations(::Type{DataFrame}, pairs::Pair{Symbol, <:Any}...)\r\n",
      "    colnames = first.(pairs)\r\n",
      "    if !allunique(colnames)\r\n",
      "        throw(ArgumentError(\"All column names passed to allcombinations must be unique\"))\r\n",
      "    end\r\n",
      "    colvalues = map(pairs) do p\r\n",
      "        v = last(p)\r\n",
      "        if v isa AbstractVector\r\n",
      "            return v\r\n",
      "        elseif v isa Union{AbstractArray{<:Any, 0}, Ref}\r\n",
      "            x = v[]\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(x), 1), x)\r\n",
      "        elseif v isa AbstractArray\r\n",
      "            throw(ArgumentError(\"adding AbstractArray other than AbstractVector \" *\r\n",
      "                                \"as a column of a data frame is not allowed\"))\r\n",
      "        else\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(v), 1), v)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    @assert length(colvalues) == length(colnames)\r\n",
      "    @assert all(x -> x isa AbstractVector, colvalues)\r\n",
      "\r\n",
      "    target_rows = Int(prod(x -> big(length(x)), colvalues))\r\n",
      "    out_df = DataFrame()\r\n",
      "    inner = 1\r\n",
      "    for (val, cname) in zip(colvalues, colnames)\r\n",
      "        len = length(val)\r\n",
      "        last_inner = inner\r\n",
      "        inner *= len\r\n",
      "        outer, remv = inner == 0 ? (0, 0) : divrem(target_rows, inner)\r\n",
      "        @assert iszero(remv)\r\n",
      "        out_df[!, cname] = repeat(val, inner=last_inner, outer=outer)\r\n",
      "    end\r\n",
      "    @assert inner == target_rows\r\n",
      "    @assert size(out_df) == (target_rows, length(colnames))\r\n",
      "    return out_df\r\n",
      "end\r\n",
      "\r\n",
      "_try_select_no_copy(df::DataFrame, cols) = select(df, cols, copycols=false)\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "@less df[!, :y] = [1, 2]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function insert_single_column!(df::DataFrame, v::AbstractVector, col_ind::ColumnIndex)\r\n",
      "    if ncol(df) != 0 && nrow(df) != length(v)\r\n",
      "        throw(ArgumentError(\"New columns must have the same length as old columns\"))\r\n",
      "    end\r\n",
      "    dv = isa(v, AbstractRange) ? collect(v) : v\r\n",
      "    firstindex(dv) != 1 && _onebased_check_error()\r\n",
      "\r\n",
      "    if haskey(index(df), col_ind)\r\n",
      "        j = index(df)[col_ind]\r\n",
      "        _columns(df)[j] = dv\r\n",
      "    else\r\n",
      "        if col_ind isa SymbolOrString\r\n",
      "            push!(index(df), Symbol(col_ind))\r\n",
      "            push!(_columns(df), dv)\r\n",
      "        else\r\n",
      "            throw(ArgumentError(\"Cannot assign to non-existent column: $col_ind\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return dv\r\n",
      "end\r\n",
      "\r\n",
      "function insert_single_entry!(df::DataFrame, v::Any, row_ind::Integer, col_ind::ColumnIndex)\r\n",
      "    if haskey(index(df), col_ind)\r\n",
      "        _columns(df)[index(df)[col_ind]][row_ind] = v\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return v\r\n",
      "    else\r\n",
      "        throw(ArgumentError(\"Cannot assign to non-existent column: $col_ind\"))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[!, SingleColumnIndex] = AbstractVector\r\n",
      "function Base.setindex!(df::DataFrame, v::AbstractVector, ::typeof(!), col_ind::ColumnIndex)\r\n",
      "    insert_single_column!(df, v, col_ind)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# df.col = AbstractVector\r\n",
      "# separate methods are needed due to dispatch ambiguity\r\n",
      "Base.setproperty!(df::DataFrame, col_ind::Symbol, v::AbstractVector) =\r\n",
      "    (df[!, col_ind] = v)\r\n",
      "Base.setproperty!(df::DataFrame, col_ind::AbstractString, v::AbstractVector) =\r\n",
      "    (df[!, col_ind] = v)\r\n",
      "Base.setproperty!(::DataFrame, col_ind::Symbol, v::Any) =\r\n",
      "    throw(ArgumentError(\"It is only allowed to pass a vector as a column of a DataFrame. \" *\r\n",
      "                        \"Instead use `df[!, col_ind] .= v` if you want to use broadcasting.\"))\r\n",
      "Base.setproperty!(::DataFrame, col_ind::AbstractString, v::Any) =\r\n",
      "    throw(ArgumentError(\"It is only allowed to pass a vector as a column of a DataFrame. \" *\r\n",
      "                        \"Instead use `df[!, col_ind] .= v` if you want to use broadcasting.\"))\r\n",
      "\r\n",
      "# df[SingleRowIndex, SingleColumnIndex] = Single Item\r\n",
      "function Base.setindex!(df::DataFrame, v::Any, row_ind::Integer, col_ind::ColumnIndex)\r\n",
      "    insert_single_entry!(df, v, row_ind, col_ind)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# df[SingleRowIndex, MultiColumnIndex] = value\r\n",
      "# the method for value of type DataFrameRow, AbstractDict and NamedTuple\r\n",
      "# is defined in dataframerow.jl\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::Union{Tuple, AbstractArray},\r\n",
      "                                  row_ind::Integer,\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if length(v) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"$(length(idxs)) columns were selected but the assigned \" *\r\n",
      "                                    \"collection contains $(length(v)) elements\"))\r\n",
      "        end\r\n",
      "        for (i, x) in zip(idxs, v)\r\n",
      "            df[!, i][row_ind] = x\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, SingleColumnIndex] = AbstractVector\r\n",
      "for T in (:AbstractVector, :Not, :Colon)\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::AbstractVector,\r\n",
      "                                  row_inds::$T,\r\n",
      "                                  col_ind::ColumnIndex)\r\n",
      "        if row_inds isa Colon && !haskey(index(df), col_ind)\r\n",
      "            df[!, col_ind] = copy(v)\r\n",
      "            return df\r\n",
      "        end\r\n",
      "        x = df[!, col_ind]\r\n",
      "        x[row_inds] = v\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractDataFrame\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"column names in source and target do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            df[!, col][row_inds] = new_df[!, j]\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::typeof(!),\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"Column names in source and target data frames do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # make sure we make a copy on assignment\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[!, col] = new_df[:, j]\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractMatrix\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon, :(typeof(!))),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  mx::AbstractMatrix,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if size(mx, 2) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"number of selected columns ($(length(idxs))) \" *\r\n",
      "                                    \"and number of columns in \" *\r\n",
      "                                    \"matrix ($(size(mx, 2))) do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[row_inds, col] = (row_inds === !) ? mx[:, j] : view(mx, :, j)\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    copy(df::DataFrame; copycols::Bool=true)\r\n",
      "\r\n",
      "Copy data frame `df`.\r\n",
      "If `copycols=true` (the default), return a new  `DataFrame` holding\r\n",
      "copies of column vectors in `df`.\r\n",
      "If `copycols=false`, return a new `DataFrame` sharing column vectors with `df`.\r\n",
      "\r\n",
      "Metadata: this function preserves all table-level and column-level metadata.\r\n",
      "\"\"\"\r\n",
      "function Base.copy(df::DataFrame; copycols::Bool=true)\r\n",
      "    cdf = DataFrame(copy(_columns(df)), copy(index(df)), copycols=copycols)\r\n",
      "\r\n",
      "    df_metadata = getfield(df, :metadata)\r\n",
      "    if !isnothing(df_metadata)\r\n",
      "        setfield!(cdf, :metadata, copy(df_metadata))\r\n",
      "    end\r\n",
      "    df_colmetadata = getfield(df, :colmetadata)\r\n",
      "    if !isnothing(df_colmetadata)\r\n",
      "        cdf_colmetadata = copy(df_colmetadata)\r\n",
      "        map!(copy, values(cdf_colmetadata))\r\n",
      "        setfield!(cdf, :colmetadata, cdf_colmetadata)\r\n",
      "    end\r\n",
      "    setfield!(cdf, :allnotemetadata, getfield(df, :allnotemetadata))\r\n",
      "\r\n",
      "    return cdf\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows specified by `inds` from a `DataFrame` `df` in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> deleteat!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, ::Colon) = empty!(df)\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    size(df, 2) == 0 && throw(BoundsError(df, (inds, :)))\r\n",
      "    return _deleteat!_helper(df, Int[inds])\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    if isempty(inds)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    elseif size(df, 2) == 0\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !(eltype(inds) <: Integer || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && inds isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to deleteat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector{Bool})\r\n",
      "    if length(inds) != size(df, 1)\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && drop isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, inds::Not) =\r\n",
      "    _deleteat!_helper(df, axes(df, 1)[inds])\r\n",
      "\r\n",
      "function _deleteat!_helper(df::DataFrame, drop)\r\n",
      "    cols = _columns(df)\r\n",
      "    if isempty(cols)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "\r\n",
      "    n = nrow(df)\r\n",
      "    col1 = cols[1]\r\n",
      "    deleteat!(col1, drop)\r\n",
      "    newn = length(col1)\r\n",
      "    @assert newn <= n\r\n",
      "    # the 0.06 threshold is heuristic; based on tests\r\n",
      "    # the assignment makes the code type-unstable but it is a small union\r\n",
      "    # so the overhead should be small\r\n",
      "    if drop isa AbstractVector{Bool} && newn < 0.06 * n\r\n",
      "        drop = findall(drop)\r\n",
      "    end\r\n",
      "    for i in 2:length(cols)\r\n",
      "        col = cols[i]\r\n",
      "        # this check is done to handle column aliases\r\n",
      "        if length(col) == n\r\n",
      "            deleteat!(col, drop)\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    for i in 1:length(cols)\r\n",
      "        # this should never happen, but we add it for safety\r\n",
      "        @assert length(cols[i]) == newn corrupt_msg(df, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows at all indices not specified by `inds` from a `DataFrame` `df`\r\n",
      "in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> keepat!(df, [1, 3])\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, ::Colon)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    isempty(inds) && return empty!(df)\r\n",
      "\r\n",
      "    # this is required because of https://github.com/JuliaData/InvertedIndices.jl/issues/31\r\n",
      "    if !((eltype(inds) <: Integer) || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to keepat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return deleteat!(df, Not(inds))\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    return deleteat!(df, Not(Int[inds]))\r\n",
      "end\r\n",
      "\r\n",
      "keepat!(df::DataFrame, inds::AbstractVector{Bool}) = deleteat!(df, .!inds)\r\n",
      "keepat!(df::DataFrame, inds::Not) = deleteat!(df, Not(inds))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    empty!(df::DataFrame)\r\n",
      "\r\n",
      "Remove all rows from `df`, making each of its columns empty.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> empty!(df)\r\n",
      "0×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┴──────────────\r\n",
      "\r\n",
      "julia> df.a, df.b\r\n",
      "(Int64[], Int64[])\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "function Base.empty!(df::DataFrame)\r\n",
      "    foreach(empty!, eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    resize!(df::DataFrame, n::Integer)\r\n",
      "\r\n",
      "Resize `df` to have `n` rows by calling `resize!` on all columns of `df`.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> resize!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.resize!(df::DataFrame, n::Integer)\r\n",
      "    if ncol(df) == 0 && n != 0\r\n",
      "        throw(ArgumentError(\"data frame has no columns and requested number \" *\r\n",
      "                            \"of rows is not zero\"))\r\n",
      "    end\r\n",
      "    foreach(col -> resize!(col, n), eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    pop!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the last row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> pop!(df)\r\n",
      "(a = 3, b = 6)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "Base.pop!(df::DataFrame) = popat!(df, nrow(df))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popfirst!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the first row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popfirst!(df)\r\n",
      "(a = 1, b = 4)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     2      5\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.popfirst!(df::DataFrame) = popat!(df, 1)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popat!(df::DataFrame, i::Integer)\r\n",
      "\r\n",
      "Remove the `i`-th row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popat!(df, 2)\r\n",
      "(a = 2, b = 5)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.popat!(df::DataFrame, i::Integer)\r\n",
      "    i isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    nt = NamedTuple(df[i, :])\r\n",
      "    deleteat!(df, i)\r\n",
      "    return nt\r\n",
      "end\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## hcat!\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\r\n",
      "# hcat! for 2 arguments, only a vector or a data frame is allowed\r\n",
      "function hcat!(df1::DataFrame, df2::AbstractDataFrame;\r\n",
      "               makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    u = add_names(index(df1), index(df2), makeunique=makeunique)\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df1)\r\n",
      "    _keep_matching_table_note_metadata!(df1, df2)\r\n",
      "    for i in 1:length(u)\r\n",
      "        df1[!, u[i]] = copycols ? df2[:, i] : df2[!, i]\r\n",
      "        _copy_col_note_metadata!(df1, u[i], df2, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df1\r\n",
      "end\r\n",
      "\r\n",
      "# TODO: after deprecation remove AbstractVector methods\r\n",
      "\r\n",
      "function hcat!(df::DataFrame, x::AbstractVector; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(df, DataFrame(AbstractVector[x], [:x1], copycols=false),\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "function hcat!(x::AbstractVector, df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(DataFrame(AbstractVector[x], [:x1], copycols=copycols), df,\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "# hcat! for 1-n arguments\r\n",
      "function hcat!(df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "hcat!(a::DataFrame, b::Union{AbstractDataFrame, AbstractVector},\r\n",
      "      c::Union{AbstractDataFrame, AbstractVector}...;\r\n",
      "      makeunique::Bool=false, copycols::Bool=true) =\r\n",
      "    hcat!(hcat!(a, b, makeunique=makeunique, copycols=copycols),\r\n",
      "          c..., makeunique=makeunique, copycols=copycols)\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## Missing values support\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\"\"\"\r\n",
      "    allowmissing!(df::DataFrame, cols=:)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `T` to\r\n",
      "`Union{T, Missing}` to support missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function allowmissing! end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, col::ColumnIndex)\r\n",
      "    df[!, col] = allowmissing(df[!, col])\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex})\r\n",
      "    for col in cols\r\n",
      "        allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{Bool})\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::MultiColumnIndex) =\r\n",
      "    allowmissing!(df, index(df)[cols])\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::Colon=:) =\r\n",
      "    allowmissing!(df, axes(df, 2))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    disallowmissing!(df::DataFrame, cols=:; error::Bool=true)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `Union{T, Missing}` to\r\n",
      "`T` to drop support for missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "If `error=false` then columns containing a `missing` value will be skipped instead\r\n",
      "of throwing an error.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function disallowmissing! end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, col::ColumnIndex; error::Bool=true)\r\n",
      "    x = df[!, col]\r\n",
      "    if Missing <: eltype(x)\r\n",
      "        try\r\n",
      "            df[!, col] = disallowmissing(x)\r\n",
      "        catch e\r\n",
      "            row = findfirst(ismissing, x)\r\n",
      "            if row !== nothing\r\n",
      "                if error\r\n",
      "                    col_name = only(names(df, col))\r\n",
      "                    throw(ArgumentError(\"Missing value found in column \" *\r\n",
      "                                        \":$col_name in row $row\"))\r\n",
      "                end\r\n",
      "            else\r\n",
      "                rethrow(e)\r\n",
      "            end\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex};\r\n",
      "                          error::Bool=true)\r\n",
      "    for col in cols\r\n",
      "        disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{Bool}; error::Bool=true)\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::MultiColumnIndex; error::Bool=true) =\r\n",
      "    disallowmissing!(df, index(df)[cols], error=error)\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::Colon=:; error::Bool=true) =\r\n",
      "    disallowmissing!(df, axes(df, 2), error=error)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows. `inner` specifies how many\r\n",
      "times each row is repeated, and `outer` specifies how many times the full set\r\n",
      "of rows is repeated. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat!(df, inner=2, outer=3);\r\n",
      "\r\n",
      "julia> df\r\n",
      "12×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     1      3\r\n",
      "   3 │     2      4\r\n",
      "   4 │     2      4\r\n",
      "   5 │     1      3\r\n",
      "   6 │     1      3\r\n",
      "   7 │     2      4\r\n",
      "   8 │     2      4\r\n",
      "   9 │     1      3\r\n",
      "  10 │     1      3\r\n",
      "  11 │     2      4\r\n",
      "  12 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "    inner < 0 && throw(ArgumentError(\"inner keyword argument must be non-negative\"))\r\n",
      "    outer < 0 && throw(ArgumentError(\"outer keyword argument must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, inner=Int(inner), outer=Int(outer))\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame, count::Integer)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows the number of times\r\n",
      "specified by `count`. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat(df, 2)\r\n",
      "4×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "   3 │     1      3\r\n",
      "   4 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame, count::Integer)\r\n",
      "    count < 0 && throw(ArgumentError(\"count must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, count)\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# This is not exactly copy! as in general we allow axes to be different\r\n",
      "# Also no table metadata needs to be copied as we use _replace_columns!\r\n",
      "# only in situations when table metadata for `df` must be left as-is\r\n",
      "function _replace_columns!(df::DataFrame, newdf::DataFrame)\r\n",
      "    # for DataFrame object here we do not support keep_present keyword argument\r\n",
      "    # like for SubDataFrame because here transform! always falls back to select!\r\n",
      "    @assert ncol(newdf) == 0 || nrow(df) == nrow(newdf)\r\n",
      "    copy!(_columns(df), _columns(newdf))\r\n",
      "    copy!(_names(index(df)), _names(newdf))\r\n",
      "    copy!(index(df).lookup, index(newdf).lookup)\r\n",
      "\r\n",
      "    emptycolmetadata!(df)\r\n",
      "    for (col, col_keys) in colmetadatakeys(newdf)\r\n",
      "        if hasproperty(df, col)\r\n",
      "            for key in col_keys\r\n",
      "                val, style = colmetadata(newdf, col, key, style=true)\r\n",
      "                style === :note && colmetadata!(df, col, key, val, style=:note)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}; kwargs...) =\r\n",
      "    isempty(kwargs) ? DataFrame() : allcombinations(DataFrame, kwargs...)\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}, pairs::Pair{<:AbstractString, <:Any}...) =\r\n",
      "    allcombinations(DataFrame, (Symbol(k) => v for (k, v) in pairs)...)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    allcombinations(DataFrame, pairs::Pair...)\r\n",
      "    allcombinations(DataFrame; kwargs...)\r\n",
      "\r\n",
      "Create a `DataFrame` from all combinations of values in passed arguments.\r\n",
      "The first passed values vary fastest.\r\n",
      "\r\n",
      "Arguments associating a column name with values to expand can be specified\r\n",
      "either as `Pair`s passed as positional arguments, or as keyword arguments.\r\n",
      "Column names must be `Symbol`s or strings and must be unique.\r\n",
      "\r\n",
      "Column value can be a vector which is consumed as is or an object of any other\r\n",
      "type (except `AbstractArray`). In the latter case the passed value is treated\r\n",
      "as having length one for expansion. As a particular rule values stored in a `Ref`\r\n",
      "or a `0`-dimensional `AbstractArray` are unwrapped and treated as having length one.\r\n",
      "\r\n",
      "See also: [`crossjoin`](@ref) can be used to get the cartesian product\r\n",
      "of rows from passed data frames.\r\n",
      "\r\n",
      "# Examples\r\n",
      "\r\n",
      "```jldoctest\r\n",
      "julia> allcombinations(DataFrame, a=1:2, b='a':'c')\r\n",
      "6×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Char\r\n",
      "─────┼─────────────\r\n",
      "   1 │     1  a\r\n",
      "   2 │     2  a\r\n",
      "   3 │     1  b\r\n",
      "   4 │     2  b\r\n",
      "   5 │     1  c\r\n",
      "   6 │     2  c\r\n",
      "\r\n",
      "julia> allcombinations(DataFrame, \"a\" => 1:2, \"b\" => 'a':'c', \"c\" => \"const\")\r\n",
      "6×3 DataFrame\r\n",
      " Row │ a      b     c\r\n",
      "     │ Int64  Char  String\r\n",
      "─────┼─────────────────────\r\n",
      "   1 │     1  a     const\r\n",
      "   2 │     2  a     const\r\n",
      "   3 │     1  b     const\r\n",
      "   4 │     2  b     const\r\n",
      "   5 │     1  c     const\r\n",
      "   6 │     2  c     const\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function allcombinations(::Type{DataFrame}, pairs::Pair{Symbol, <:Any}...)\r\n",
      "    colnames = first.(pairs)\r\n",
      "    if !allunique(colnames)\r\n",
      "        throw(ArgumentError(\"All column names passed to allcombinations must be unique\"))\r\n",
      "    end\r\n",
      "    colvalues = map(pairs) do p\r\n",
      "        v = last(p)\r\n",
      "        if v isa AbstractVector\r\n",
      "            return v\r\n",
      "        elseif v isa Union{AbstractArray{<:Any, 0}, Ref}\r\n",
      "            x = v[]\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(x), 1), x)\r\n",
      "        elseif v isa AbstractArray\r\n",
      "            throw(ArgumentError(\"adding AbstractArray other than AbstractVector \" *\r\n",
      "                                \"as a column of a data frame is not allowed\"))\r\n",
      "        else\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(v), 1), v)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    @assert length(colvalues) == length(colnames)\r\n",
      "    @assert all(x -> x isa AbstractVector, colvalues)\r\n",
      "\r\n",
      "    target_rows = Int(prod(x -> big(length(x)), colvalues))\r\n",
      "    out_df = DataFrame()\r\n",
      "    inner = 1\r\n",
      "    for (val, cname) in zip(colvalues, colnames)\r\n",
      "        len = length(val)\r\n",
      "        last_inner = inner\r\n",
      "        inner *= len\r\n",
      "        outer, remv = inner == 0 ? (0, 0) : divrem(target_rows, inner)\r\n",
      "        @assert iszero(remv)\r\n",
      "        out_df[!, cname] = repeat(val, inner=last_inner, outer=outer)\r\n",
      "    end\r\n",
      "    @assert inner == target_rows\r\n",
      "    @assert size(out_df) == (target_rows, length(colnames))\r\n",
      "    return out_df\r\n",
      "end\r\n",
      "\r\n",
      "_try_select_no_copy(df::DataFrame, cols) = select(df, cols, copycols=false)\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "@less DataFrames.insert_single_column!(df, [1, 2], :y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that for `broadcast!` it is treated as `0` rows to be consistent with the value returned by `size`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64[]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame()\n",
    "df[!, :x] .= 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>0×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m0×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┴───────"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, pseudo-broadcasting provided by DataFrames.jl in `DataFrame`, `insertcols!` and `combine` broadcasts scalars into 1-row, as usually this is what the user expects."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">a</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(:a => 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">a</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(DataFrame(), :a => 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">nrow</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">0</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& nrow\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m nrow  \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     0"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combine(DataFrame(), nrow)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "but not in `select` and `transform` as in this case we keep the number of rows in the source:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>0×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">nrow</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& nrow\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m0×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m nrow  \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┴───────"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(DataFrame(), nrow)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>0×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">nrow</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& nrow\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m0×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m nrow  \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┴───────"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform(DataFrame(), nrow)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 2: broadcasting assignment of getproperty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>2×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(x=1:2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since Julia 1.7 this works (`:y` is a new column):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{Int64}:\n",
       " 2\n",
       " 2"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.y .= 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and this works:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{Int64}:\n",
       " 1\n",
       " 1"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[!, :z] .= 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>2×3 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th><th style = \"text-align: left;\">y</th><th style = \"text-align: left;\">z</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td><td style = \"text-align: right;\">2</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td><td style = \"text-align: right;\">2</td><td style = \"text-align: right;\">1</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x & y & z\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 1 \\\\\n",
       "\t2 & 2 & 2 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\u001b[1m y     \u001b[0m\u001b[1m z     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      1\n",
       "   2 │     2      2      1"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the way to check what is going on:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MethodInstance for (::var\"#3#4\")(::DataFrame)\n",
      "  from (::var\"#3#4\")(\u001b[90mdf\u001b[39m)\u001b[90m @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[21]:1\u001b[24m\u001b[39m\n",
      "Arguments\n",
      "  #self#\u001b[36m::Core.Const(var\"#3#4\"())\u001b[39m\n",
      "  df\u001b[36m::DataFrame\u001b[39m\n",
      "Body\u001b[36m::Vector{Int64}\u001b[39m\n",
      "\u001b[90m1 ─\u001b[39m %1 = Base.dotgetproperty(df, :zz)\u001b[36m::Core.PartialStruct(DataFrames.LazyNewColDataFrame{Symbol, DataFrame}, Any[DataFrame, Core.Const(:zz)])\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %2 = Base.broadcasted(Base.identity, 1)\u001b[36m::Core.Const(Base.Broadcast.Broadcasted(identity, (1,)))\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %3 = Base.materialize!(%1, %2)\u001b[36m::Vector{Int64}\u001b[39m\n",
      "\u001b[90m└──\u001b[39m      return %3\n",
      "\n"
     ]
    }
   ],
   "source": [
    "@code_warntype (df -> df.zz .= 1)(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "vs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MethodInstance for (::var\"#5#6\")(::DataFrame)\n",
      "  from (::var\"#5#6\")(\u001b[90mdf\u001b[39m)\u001b[90m @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[22]:1\u001b[24m\u001b[39m\n",
      "Arguments\n",
      "  #self#\u001b[36m::Core.Const(var\"#5#6\"())\u001b[39m\n",
      "  df\u001b[36m::DataFrame\u001b[39m\n",
      "Body\u001b[91m\u001b[1m::Any\u001b[22m\u001b[39m\n",
      "\u001b[90m1 ─\u001b[39m %1 = Base.dotview(df, Main.:(:), :zz)\u001b[91m\u001b[1m::Any\u001b[22m\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %2 = Base.broadcasted(Base.identity, 1)\u001b[36m::Core.Const(Base.Broadcast.Broadcasted(identity, (1,)))\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %3 = Base.materialize!(%1, %2)\u001b[91m\u001b[1m::Any\u001b[22m\u001b[39m\n",
      "\u001b[90m└──\u001b[39m      return %3\n",
      "\n"
     ]
    }
   ],
   "source": [
    "@code_warntype (df -> df[:, :zz] .= 1)(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that in `df.z .= 1` Julia does the following steps:\n",
    "1. tries to take a property `:z` from `df`\n",
    "2. does broadcasting into the result of `df.z`\n",
    "\n",
    "And since `:z` does not exist in `df` the `dotgetproperty` correctly identifies that a new column needs to be allocated."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Tip:</b>\n",
    "\n",
    "This behavior is available starting from Julia 1.7. In earlier versions of Julia this is an error.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If the column exists it gets replaced:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element Vector{String}:\n",
       " \"a\"\n",
       " \"a\""
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.x .= \"a\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>2×3 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th><th style = \"text-align: left;\">y</th><th style = \"text-align: left;\">z</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"String\" style = \"text-align: left;\">String</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: left;\">a</td><td style = \"text-align: right;\">2</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: left;\">a</td><td style = \"text-align: right;\">2</td><td style = \"text-align: right;\">1</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x & y & z\\\\\n",
       "\t\\hline\n",
       "\t& String & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & a & 2 & 1 \\\\\n",
       "\t2 & a & 2 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x      \u001b[0m\u001b[1m y     \u001b[0m\u001b[1m z     \u001b[0m\n",
       "     │\u001b[90m String \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────\n",
       "   1 │ a           2      1\n",
       "   2 │ a           2      1"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now what happens in `df[:, :z] .= 1` is that try to broadcast into a result of `Base.dotview(df, :, :z)` instead.\n",
    "\n",
    "Let us check what it returns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element view(::Vector{Int64}, :) with eltype Int64:\n",
       " 1\n",
       " 1"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.dotview(df, :, :z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2-element view(::Vector{String}, :) with eltype String:\n",
       " \"a\"\n",
       " \"a\""
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.dotview(df, :, :x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DataFrames.LazyNewColDataFrame{Symbol, DataFrame}(\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x      \u001b[0m\u001b[1m y     \u001b[0m\u001b[1m z     \u001b[0m\n",
       "     │\u001b[90m String \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────\n",
       "   1 │ a           2      1\n",
       "   2 │ a           2      1, :z)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.dotview(df, !, :z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DataFrames.LazyNewColDataFrame{Symbol, DataFrame}(\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x      \u001b[0m\u001b[1m y     \u001b[0m\u001b[1m z     \u001b[0m\n",
       "     │\u001b[90m String \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────\n",
       "   1 │ a           2      1\n",
       "   2 │ a           2      1, :x)"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.dotview(df, !, :x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function Base.dotview(df::AbstractDataFrame, ::typeof(!), cols)\r\n",
      "    if !(cols isa ColumnIndex)\r\n",
      "        return ColReplaceDataFrame(df, convert(Vector{Int}, index(df)[cols]))\r\n",
      "    end\r\n",
      "    if cols isa SymbolOrString\r\n",
      "        if columnindex(df, cols) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "    elseif !(1 <= cols <= ncol(df))\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, cols isa AbstractString ? Symbol(cols) : cols)\r\n",
      "end\r\n",
      "\r\n",
      "if isdefined(Base, :dotgetproperty) # Introduced in Julia 1.7\r\n",
      "    function Base.dotgetproperty(df::AbstractDataFrame, col::SymbolOrString)\r\n",
      "        if columnindex(df, col) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "        return LazyNewColDataFrame(df, Symbol(col))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(lazydf::LazyNewColDataFrame, bc::Base.Broadcast.Broadcasted{T}) where T\r\n",
      "    df = lazydf.df\r\n",
      "    if !haskey(index(df), lazydf.col) && df isa SubDataFrame && lazydf.col isa SymbolOrString\r\n",
      "        @assert is_column_insertion_allowed(df)\r\n",
      "    end\r\n",
      "    if bc isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "        bc_tmp = Base.Broadcast.Broadcasted{T}(bc.f, bc.args, ())\r\n",
      "        v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "        col = similar(Vector{typeof(v)}, nrow(df))\r\n",
      "        copyto!(col, bc)\r\n",
      "    else\r\n",
      "        col = Base.Broadcast.materialize(bc)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df[!, lazydf.col] = col\r\n",
      "end\r\n",
      "\r\n",
      "function _copyto_helper!(dfcol::AbstractVector, bc::Base.Broadcast.Broadcasted, col::Int)\r\n",
      "    if axes(dfcol, 1) != axes(bc)[1]\r\n",
      "        # this should never happen unless data frame is corrupted (has unequal column lengths)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting. The updated\" *\r\n",
      "                                \" data frame is invalid and should not be used\"))\r\n",
      "    end\r\n",
      "    @inbounds for row in eachindex(dfcol)\r\n",
      "        dfcol[row] = bc[CartesianIndex(row, col)]\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src)\r\n",
      "    for col in eachcol(dest)\r\n",
      "        src = Base.Broadcast.unalias(col, src)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.broadcast_unalias(::Nothing, src::AbstractDataFrame) = src\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest, src::AbstractDataFrame)\r\n",
      "    wascopied = false\r\n",
      "    for (i, col) in enumerate(eachcol(src))\r\n",
      "        if Base.mightalias(dest, col)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src = SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                       index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), i)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, i] = Base.unaliascopy(col)\r\n",
      "            end\r\n",
      "            wascopied = true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function _broadcast_unalias_helper(dest::AbstractDataFrame, scol::AbstractVector,\r\n",
      "                                   src::AbstractDataFrame, col2::Int, wascopied::Bool)\r\n",
      "    # col1 can be checked till col2 point as we are writing broadcasting\r\n",
      "    # results from 1 to ncol\r\n",
      "    # we go downwards because aliasing when col1 == col2 is most probable\r\n",
      "    for col1 in col2:-1:1\r\n",
      "        dcol = dest[!, col1]\r\n",
      "        if Base.mightalias(dcol, scol)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src =SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                      index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), col2)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, col2] = Base.unaliascopy(scol)\r\n",
      "            end\r\n",
      "            return src, true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src, wascopied\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src::AbstractDataFrame)\r\n",
      "    if size(dest, 2) != size(src, 2)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting.\"))\r\n",
      "    end\r\n",
      "    wascopied = false\r\n",
      "    for col2 in axes(dest, 2)\r\n",
      "        scol = src[!, col2]\r\n",
      "        src, wascopied = _broadcast_unalias_helper(dest, scol, src, col2, wascopied)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n",
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "@less Base.dotview(df, !, :x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `dotview` is defined only when a special treatement is needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "# 2 methods for generic function <b>dotview</b> from \u001b[90mBase.Broadcast\u001b[39m:<ul><li> dotview(df::<b>AbstractDataFrame</b>, ::<b>typeof(!)</b>, cols) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:171</a></li> <li> dotview(df::<b>AbstractDataFrame</b>, ::<b>Colon</b>, cols::<b>Union{AbstractString, Signed, Symbol, Unsigned}</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:156</a></li> </ul>"
      ],
      "text/plain": [
       "# 2 methods for generic function \"dotview\" from \u001b[90mBase.Broadcast\u001b[39m:\n",
       " [1] dotview(\u001b[90mdf\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m, ::\u001b[1mtypeof(!)\u001b[22m, \u001b[90mcols\u001b[39m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:171\u001b[24m\u001b[39m\n",
       " [2] dotview(\u001b[90mdf\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m, ::\u001b[1mColon\u001b[22m, \u001b[90mcols\u001b[39m::\u001b[1mUnion\u001b[22m\u001b[0m{AbstractString, Signed, Symbol, Unsigned})\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:156\u001b[24m\u001b[39m"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "methods(Base.dotview, DataFrames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "as \"normally\" the default implementation is just enough:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×1 SubDataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"String\" style = \"text-align: left;\">String</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: left;\">a</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& String\\\\\n",
       "\t\\hline\n",
       "\t1 & a \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×1 SubDataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x      \u001b[0m\n",
       "     │\u001b[90m String \u001b[0m\n",
       "─────┼────────\n",
       "   1 │ a"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.dotview(df, 1:1, 1:1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SubDataFrame{DataFrame, DataFrames.SubIndex{DataFrames.Index, UnitRange{Int64}, UnitRange{Int64}}, UnitRange{Int64}}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "typeof(Base.dotview(df, 1:1, 1:1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we can see that:\n",
    "1. if we use `df[:, :x]` (an existing column) - we get just a view into it; a particular consequence is that we cannot cheange the `eltype` of the column (just like with `df.x .= 1`)\n",
    "2. if we use `df[!, ...]` (any column) or `df[:, :z]` (non existing column) we get a `LazyNewColDataFrame` object.\n",
    "\n",
    "Importantly note that in indexing context `x[y] .= z` the meaning of `x[y]` can be controlled by the package developer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us try to understand what `LazyNewColDataFrame` does.\n",
    "\n",
    "For this we need to dig into how broadcasting assignment works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>2×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(x = [1, 2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to manually recreate the process of execution of `df[:, :z] .= 1`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DataFrames.LazyNewColDataFrame{Symbol, DataFrame}(\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2, :z)"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dest = Base.dotview(df, :, :z)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Base.Broadcast.Broadcasted(identity, (1,))"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bc = Base.broadcasted(identity, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@inline function materialize!(dest, bc::Broadcasted{Style}) where {Style}\r\n",
      "    return materialize!(combine_styles(dest, bc), dest, bc)\r\n",
      "end\r\n",
      "@inline function materialize!(::BroadcastStyle, dest, bc::Broadcasted{Style}) where {Style}\r\n",
      "    return copyto!(dest, instantiate(Broadcasted{Style}(bc.f, bc.args, axes(dest))))\r\n",
      "end\r\n",
      "\r\n",
      "## general `copy` methods\r\n",
      "@inline copy(bc::Broadcasted{<:AbstractArrayStyle{0}}) = bc[CartesianIndex()]\r\n",
      "copy(bc::Broadcasted{<:Union{Nothing,Unknown}}) =\r\n",
      "    throw(ArgumentError(\"broadcasting requires an assigned BroadcastStyle\"))\r\n",
      "\r\n",
      "const NonleafHandlingStyles = Union{DefaultArrayStyle,ArrayConflict}\r\n",
      "\r\n",
      "@inline function copy(bc::Broadcasted{Style}) where {Style}\r\n",
      "    ElType = combine_eltypes(bc.f, bc.args)\r\n",
      "    if Base.isconcretetype(ElType)\r\n",
      "        # We can trust it and defer to the simpler `copyto!`\r\n",
      "        return copyto!(similar(bc, ElType), bc)\r\n",
      "    end\r\n",
      "    # When ElType is not concrete, use narrowing. Use the first output\r\n",
      "    # value to determine the starting output eltype; copyto_nonleaf!\r\n",
      "    # will widen `dest` as needed to accommodate later values.\r\n",
      "    bc′ = preprocess(nothing, bc)\r\n",
      "    iter = eachindex(bc′)\r\n",
      "    y = iterate(iter)\r\n",
      "    if y === nothing\r\n",
      "        # if empty, take the ElType at face value\r\n",
      "        return similar(bc′, ElType)\r\n",
      "    end\r\n",
      "    # Initialize using the first value\r\n",
      "    I, state = y\r\n",
      "    @inbounds val = bc′[I]\r\n",
      "    dest = similar(bc′, typeof(val))\r\n",
      "    @inbounds dest[I] = val\r\n",
      "    # Now handle the remaining values\r\n",
      "    # The typeassert gives inference a helping hand on the element type and dimensionality\r\n",
      "    # (work-around for #28382)\r\n",
      "    ElType′ = ElType === Union{} ? Any : ElType <: Type ? Type : ElType\r\n",
      "    RT = dest isa AbstractArray ? AbstractArray{<:ElType′, ndims(dest)} : Any\r\n",
      "    return copyto_nonleaf!(dest, bc′, iter, state, 1)::RT\r\n",
      "end\r\n",
      "\r\n",
      "## general `copyto!` methods\r\n",
      "# The most general method falls back to a method that replaces Style->Nothing\r\n",
      "# This permits specialization on typeof(dest) without introducing ambiguities\r\n",
      "@inline copyto!(dest::AbstractArray, bc::Broadcasted) = copyto!(dest, convert(Broadcasted{Nothing}, bc))\r\n",
      "\r\n",
      "# Performance optimization for the common identity scalar case: dest .= val\r\n",
      "@inline function copyto!(dest::AbstractArray, bc::Broadcasted{<:AbstractArrayStyle{0}})\r\n",
      "    # Typically, we must independently execute bc for every storage location in `dest`, but:\r\n",
      "    # IF we're in the common no-op identity case with no nested args (like `dest .= val`),\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && isflat(bc)\r\n",
      "        # THEN we can just extract the argument and `fill!` the destination with it\r\n",
      "        return fill!(dest, bc.args[1][])\r\n",
      "    else\r\n",
      "        # Otherwise, fall back to the default implementation like above\r\n",
      "        return copyto!(dest, convert(Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# For broadcasted assignments like `broadcast!(f, A, ..., A, ...)`, where `A`\r\n",
      "# appears on both the LHS and the RHS of the `.=`, then we know we're only\r\n",
      "# going to make one pass through the array, and even though `A` is aliasing\r\n",
      "# against itself, the mutations won't affect the result as the indices on the\r\n",
      "# LHS and RHS will always match. This is not true in general, but with the `.op=`\r\n",
      "# syntax it's fairly common for an argument to be `===` a source.\r\n",
      "broadcast_unalias(dest, src) = dest === src ? src : unalias(dest, src)\r\n",
      "broadcast_unalias(::Nothing, src) = src\r\n",
      "\r\n",
      "# Preprocessing a `Broadcasted` does two things:\r\n",
      "# * unaliases any arguments from `dest`\r\n",
      "# * \"extrudes\" the arguments where it is advantageous to pre-compute the broadcasted indices\r\n",
      "@inline preprocess(dest, bc::Broadcasted{Style}) where {Style} = Broadcasted{Style}(bc.f, preprocess_args(dest, bc.args), bc.axes)\r\n",
      "preprocess(dest, x) = extrude(broadcast_unalias(dest, x))\r\n",
      "\r\n",
      "@inline preprocess_args(dest, args::Tuple) = (preprocess(dest, args[1]), preprocess_args(dest, tail(args))...)\r\n",
      "@inline preprocess_args(dest, args::Tuple{Any}) = (preprocess(dest, args[1]),)\r\n",
      "@inline preprocess_args(dest, args::Tuple{}) = ()\r\n",
      "\r\n",
      "# Specialize this method if all you want to do is specialize on typeof(dest)\r\n",
      "@inline function copyto!(dest::AbstractArray, bc::Broadcasted{Nothing})\r\n",
      "    axes(dest) == axes(bc) || throwdm(axes(dest), axes(bc))\r\n",
      "    # Performance optimization: broadcast!(identity, dest, A) is equivalent to copyto!(dest, A) if indices match\r\n",
      "    if bc.f === identity && bc.args isa Tuple{AbstractArray} # only a single input argument to broadcast!\r\n",
      "        A = bc.args[1]\r\n",
      "        if axes(dest) == axes(A)\r\n",
      "            return copyto!(dest, A)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    bc′ = preprocess(dest, bc)\r\n",
      "    # Performance may vary depending on whether `@inbounds` is placed outside the\r\n",
      "    # for loop or not. (cf. https://github.com/JuliaLang/julia/issues/38086)\r\n",
      "    @inbounds @simd for I in eachindex(bc′)\r\n",
      "        dest[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "# Performance optimization: for BitArray outputs, we cache the result\r\n",
      "# in a \"small\" Vector{Bool}, and then copy in chunks into the output\r\n",
      "@inline function copyto!(dest::BitArray, bc::Broadcasted{Nothing})\r\n",
      "    axes(dest) == axes(bc) || throwdm(axes(dest), axes(bc))\r\n",
      "    ischunkedbroadcast(dest, bc) && return chunkedcopyto!(dest, bc)\r\n",
      "    length(dest) < 256 && return invoke(copyto!, Tuple{AbstractArray, Broadcasted{Nothing}}, dest, bc)\r\n",
      "    tmp = Vector{Bool}(undef, bitcache_size)\r\n",
      "    destc = dest.chunks\r\n",
      "    cind = 1\r\n",
      "    bc′ = preprocess(dest, bc)\r\n",
      "    @inbounds for P in Iterators.partition(eachindex(bc′), bitcache_size)\r\n",
      "        ind = 1\r\n",
      "        @simd for I in P\r\n",
      "            tmp[ind] = bc′[I]\r\n",
      "            ind += 1\r\n",
      "        end\r\n",
      "        @simd for i in ind:bitcache_size\r\n",
      "            tmp[i] = false\r\n",
      "        end\r\n",
      "        dumpbitcache(destc, cind, tmp)\r\n",
      "        cind += bitcache_chunks\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "# For some BitArray operations, we can work at the level of chunks. The trivial\r\n",
      "# implementation just walks over the UInt64 chunks in a linear fashion.\r\n",
      "# This requires three things:\r\n",
      "#   1. The function must be known to work at the level of chunks (or can be converted to do so)\r\n",
      "#   2. The only arrays involved must be BitArrays or scalar Bools\r\n",
      "#   3. There must not be any broadcasting beyond scalar — all array sizes must match\r\n",
      "# We could eventually allow for all broadcasting and other array types, but that\r\n",
      "# requires very careful consideration of all the edge effects.\r\n",
      "const ChunkableOp = Union{typeof(&), typeof(|), typeof(xor), typeof(~), typeof(identity),\r\n",
      "    typeof(!), typeof(*), typeof(==)} # these are convertible to chunkable ops by liftfuncs\r\n",
      "const BroadcastedChunkableOp{Style<:Union{Nothing,BroadcastStyle}, Axes, F<:ChunkableOp, Args<:Tuple} = Broadcasted{Style,Axes,F,Args}\r\n",
      "ischunkedbroadcast(R, bc::BroadcastedChunkableOp) = ischunkedbroadcast(R, bc.args)\r\n",
      "ischunkedbroadcast(R, args) = false\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:BitArray,Vararg{Any}}) = size(R) == size(args[1]) && ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:Bool,Vararg{Any}}) = ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:BroadcastedChunkableOp,Vararg{Any}}) = ischunkedbroadcast(R, args[1]) && ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{}) = true\r\n",
      "\r\n",
      "# Convert compatible functions to chunkable ones. They must also be green-lighted as ChunkableOps\r\n",
      "liftfuncs(bc::Broadcasted{Style}) where {Style} = Broadcasted{Style}(bc.f, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(sign)}) where {Style} = Broadcasted{Style}(identity, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(!)}) where {Style} = Broadcasted{Style}(~, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(*)}) where {Style} = Broadcasted{Style}(&, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(==)}) where {Style} = Broadcasted{Style}((~)∘(xor), map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(x) = x\r\n",
      "\r\n",
      "liftchunks(::Tuple{}) = ()\r\n",
      "liftchunks(args::Tuple{<:BitArray,Vararg{Any}}) = (args[1].chunks, liftchunks(tail(args))...)\r\n",
      "# Transform scalars to repeated scalars the size of a chunk\r\n",
      "liftchunks(args::Tuple{<:Bool,Vararg{Any}}) = (ifelse(args[1], typemax(UInt64), UInt64(0)), liftchunks(tail(args))...)\r\n",
      "ithchunk(i) = ()\r\n",
      "Base.@propagate_inbounds ithchunk(i, c::Vector{UInt64}, args...) = (c[i], ithchunk(i, args...)...)\r\n",
      "Base.@propagate_inbounds ithchunk(i, b::UInt64, args...) = (b, ithchunk(i, args...)...)\r\n",
      "@inline function chunkedcopyto!(dest::BitArray, bc::Broadcasted)\r\n",
      "    isempty(dest) && return dest\r\n",
      "    f = flatten(liftfuncs(bc))\r\n",
      "    args = liftchunks(f.args)\r\n",
      "    dc = dest.chunks\r\n",
      "    @simd for i in eachindex(dc)\r\n",
      "        @inbounds dc[i] = f.f(ithchunk(i, args...)...)\r\n",
      "    end\r\n",
      "    @inbounds dc[end] &= Base._msk_end(dest)\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "\r\n",
      "@noinline throwdm(axdest, axsrc) =\r\n",
      "    throw(DimensionMismatch(\"destination axes $axdest are not compatible with source axes $axsrc\"))\r\n",
      "\r\n",
      "function restart_copyto_nonleaf!(newdest, dest, bc, val, I, iter, state, count)\r\n",
      "    # Function barrier that makes the copying to newdest type stable\r\n",
      "    for II in Iterators.take(iter, count)\r\n",
      "        newdest[II] = dest[II]\r\n",
      "    end\r\n",
      "    newdest[I] = val\r\n",
      "    return copyto_nonleaf!(newdest, bc, iter, state, count+1)\r\n",
      "end\r\n",
      "\r\n",
      "function copyto_nonleaf!(dest, bc::Broadcasted, iter, state, count)\r\n",
      "    T = eltype(dest)\r\n",
      "    while true\r\n",
      "        y = iterate(iter, state)\r\n",
      "        y === nothing && break\r\n",
      "        I, state = y\r\n",
      "        @inbounds val = bc[I]\r\n",
      "        if val isa T\r\n",
      "            @inbounds dest[I] = val\r\n",
      "        else\r\n",
      "            # This element type doesn't fit in dest. Allocate a new dest with wider eltype,\r\n",
      "            # copy over old values, and continue\r\n",
      "            newdest = Base.similar(bc, promote_typejoin(T, typeof(val)))\r\n",
      "            return restart_copyto_nonleaf!(newdest, dest, bc, val, I, iter, state, count)\r\n",
      "        end\r\n",
      "        count += 1\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "## Tuple methods\r\n",
      "\r\n",
      "@inline function copy(bc::Broadcasted{Style{Tuple}})\r\n",
      "    dim = axes(bc)\r\n",
      "    length(dim) == 1 || throw(DimensionMismatch(\"tuple only supports one dimension\"))\r\n",
      "    N = length(dim[1])\r\n",
      "    return ntuple(k -> @inbounds(_broadcast_getindex(bc, k)), Val(N))\r\n",
      "end\r\n",
      "\r\n",
      "## scalar-range broadcast operations ##\r\n",
      "# DefaultArrayStyle and \\ are not available at the time of range.jl\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractRange) = r\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractRange) = range(-first(r), step=negate(step(r)), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::OrdinalRange) = range(-first(r), -last(r), step=negate(step(r)))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::StepRangeLen) = StepRangeLen(-r.ref, negate(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::LinRange) = LinRange(-r.start, -r.stop, length(r))\r\n",
      "\r\n",
      "# For #18336 we need to prevent promotion of the step type:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractRange, x::Number) = range(first(r) + x, step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::AbstractRange) = range(x + first(r), step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::OrdinalRange, x::Integer) = range(first(r) + x, last(r) + x, step=step(r))\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Integer, r::OrdinalRange) = range(x + first(r), x + last(r), step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractUnitRange, x::Integer) = range(first(r) + x, last(r) + x)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Integer, r::AbstractUnitRange) = range(x + first(r), x + last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractUnitRange, x::Real) = range(first(r) + x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Real, r::AbstractUnitRange) = range(x + first(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::StepRangeLen{T}, x::Number) where T =\r\n",
      "    StepRangeLen{typeof(T(r.ref)+x)}(r.ref + x, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::StepRangeLen{T}) where T =\r\n",
      "    StepRangeLen{typeof(x+T(r.ref))}(x + r.ref, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::LinRange, x::Number) = LinRange(r.start + x, r.stop + x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::LinRange) = LinRange(x + r.start, x + r.stop, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r1::AbstractRange, r2::AbstractRange) = r1 + r2\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractRange, x::Number) = range(first(r) - x, step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::AbstractRange) = range(x - first(r), step=negate(step(r)), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::OrdinalRange, x::Integer) = range(first(r) - x, last(r) - x, step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Integer, r::OrdinalRange) = range(x - first(r), x - last(r), step=negate(step(r)))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractUnitRange, x::Integer) = range(first(r) - x, last(r) - x)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractUnitRange, x::Real) = range(first(r) - x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::StepRangeLen{T}, x::Number) where T =\r\n",
      "    StepRangeLen{typeof(T(r.ref)-x)}(r.ref - x, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::StepRangeLen{T}) where T =\r\n",
      "    StepRangeLen{typeof(x-T(r.ref))}(x - r.ref, negate(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::LinRange, x::Number) = LinRange(r.start - x, r.stop - x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::LinRange) = LinRange(x - r.start, x - r.stop, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r1::AbstractRange, r2::AbstractRange) = r1 - r2\r\n",
      "\r\n",
      "# at present Base.range_start_step_length(1,0,5) is an error, so for 0 .* (-2:2) we explicitly construct StepRangeLen:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::AbstractRange) = StepRangeLen(x*first(r), x*step(r), length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::StepRangeLen{T}) where {T} =\r\n",
      "    StepRangeLen{typeof(x*T(r.ref))}(x*r.ref, x*r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::LinRange) = LinRange(x * r.start, x * r.stop, r.len)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::AbstractFloat, r::OrdinalRange) =\r\n",
      "    Base.range_start_step_length(x*first(r), x*step(r), length(r))  # 0.2 .* (-2:2) needs TwicePrecision\r\n",
      "# separate in case of noncommutative multiplication:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::AbstractRange, x::Number) = StepRangeLen(first(r)*x, step(r)*x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::StepRangeLen{T}, x::Number) where {T} =\r\n",
      "    StepRangeLen{typeof(T(r.ref)*x)}(r.ref*x, r.step*x, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::LinRange, x::Number) = LinRange(r.start * x, r.stop * x, r.len)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::OrdinalRange, x::AbstractFloat) =\r\n",
      "    Base.range_start_step_length(first(r)*x, step(r)*x, length(r))\r\n",
      "\r\n",
      "#broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::AbstractRange, x::Number) = range(first(r)/x, last(r)/x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::AbstractRange, x::Number) = range(first(r)/x, step=step(r)/x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::StepRangeLen{T}, x::Number) where {T} =\r\n",
      "    StepRangeLen{typeof(T(r.ref)/x)}(r.ref/x, r.step/x, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::LinRange, x::Number) = LinRange(r.start / x, r.stop / x, r.len)\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::AbstractRange) = range(x\\first(r), step=x\\step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::StepRangeLen) = StepRangeLen(x\\r.ref, x\\r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::LinRange) = LinRange(x \\ r.start, x \\ r.stop, r.len)\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::UnitRange) = big(r.start):big(last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::StepRange) = big(r.start):big(r.step):big(last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::StepRangeLen) = StepRangeLen(big(r.ref), big(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::LinRange) = LinRange(big(r.start), big(r.stop), length(r))\r\n",
      "\r\n",
      "## CartesianIndices\r\n",
      "broadcasted(::typeof(+), I::CartesianIndices{N}, j::CartesianIndex{N}) where N =\r\n",
      "    CartesianIndices(map((rng, offset)->rng .+ offset, I.indices, Tuple(j)))\r\n",
      "broadcasted(::typeof(+), j::CartesianIndex{N}, I::CartesianIndices{N}) where N =\r\n",
      "    I .+ j\r\n",
      "broadcasted(::typeof(-), I::CartesianIndices{N}, j::CartesianIndex{N}) where N =\r\n",
      "    CartesianIndices(map((rng, offset)->rng .- offset, I.indices, Tuple(j)))\r\n",
      "function broadcasted(::typeof(-), j::CartesianIndex{N}, I::CartesianIndices{N}) where N\r\n",
      "    diffrange(offset, rng) = range(offset-last(rng), length=length(rng), step=step(rng))\r\n",
      "    Iterators.reverse(CartesianIndices(map(diffrange, Tuple(j), I.indices)))\r\n",
      "end\r\n",
      "\r\n",
      "## In specific instances, we can broadcast masked BitArrays whole chunks at a time\r\n",
      "# Very intentionally do not support much functionality here: scalar indexing would be O(n)\r\n",
      "struct BitMaskedBitArray{N,M}\r\n",
      "    parent::BitArray{N}\r\n",
      "    mask::BitArray{M}\r\n",
      "    BitMaskedBitArray{N,M}(parent, mask) where {N,M} = new(parent, mask)\r\n",
      "end\r\n",
      "@inline function BitMaskedBitArray(parent::BitArray{N}, mask::BitArray{M}) where {N,M}\r\n",
      "    @boundscheck checkbounds(parent, mask)\r\n",
      "    BitMaskedBitArray{N,M}(parent, mask)\r\n",
      "end\r\n",
      "Base.@propagate_inbounds dotview(B::BitArray, i::BitArray) = BitMaskedBitArray(B, i)\r\n",
      "Base.show(io::IO, B::BitMaskedBitArray) = foreach(arg->show(io, arg), (typeof(B), (B.parent, B.mask)))\r\n",
      "# Override materialize! to prevent the BitMaskedBitArray from escaping to an overridable method\r\n",
      "@inline materialize!(B::BitMaskedBitArray, bc::Broadcasted{<:Any,<:Any,typeof(identity),Tuple{Bool}}) = fill!(B, bc.args[1])\r\n",
      "@inline materialize!(B::BitMaskedBitArray, bc::Broadcasted{<:Any}) = materialize!(@inbounds(view(B.parent, B.mask)), bc)\r\n",
      "function Base.fill!(B::BitMaskedBitArray, b::Bool)\r\n",
      "    Bc = B.parent.chunks\r\n",
      "    Ic = B.mask.chunks\r\n",
      "    @inbounds if b\r\n",
      "        for i = 1:length(Bc)\r\n",
      "            Bc[i] |= Ic[i]\r\n",
      "        end\r\n",
      "    else\r\n",
      "        for i = 1:length(Bc)\r\n",
      "            Bc[i] &= ~Ic[i]\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return B\r\n",
      "end\r\n",
      "\r\n",
      "\r\n",
      "\r\n",
      "############################################################\r\n",
      "\r\n",
      "# x[...] .= f.(y...) ---> broadcast!(f, dotview(x, ...), y...).\r\n",
      "# The dotview function defaults to getindex, but we override it in\r\n",
      "# a few cases to get the expected in-place behavior without affecting\r\n",
      "# explicit calls to view.   (All of this can go away if slices\r\n",
      "# are changed to generate views by default.)\r\n",
      "\r\n",
      "Base.@propagate_inbounds dotview(args...) = Base.maybeview(args...)\r\n",
      "\r\n",
      "############################################################\r\n",
      "# The parser turns @. into a call to the __dot__ macro,\r\n",
      "# which converts all function calls and assignments into\r\n",
      "# broadcasting \"dot\" calls/assignments:\r\n",
      "\r\n",
      "dottable(x) = false # avoid dotting spliced objects (e.g. view calls inserted by @view)\r\n",
      "# don't add dots to dot operators\r\n",
      "dottable(x::Symbol) = (!isoperator(x) || first(string(x)) != '.' || x === :..) && x !== :(:)\r\n",
      "dottable(x::Expr) = x.head !== :$\r\n",
      "undot(x) = x\r\n",
      "function undot(x::Expr)\r\n",
      "    if x.head === :.=\r\n",
      "        Expr(:(=), x.args...)\r\n",
      "    elseif x.head === :block # occurs in for x=..., y=...\r\n",
      "        Expr(:block, Base.mapany(undot, x.args)...)\r\n",
      "    else\r\n",
      "        x\r\n",
      "    end\r\n",
      "end\r\n",
      "__dot__(x) = x\r\n",
      "function __dot__(x::Expr)\r\n",
      "    dotargs = Base.mapany(__dot__, x.args)\r\n",
      "    if x.head === :call && dottable(x.args[1])\r\n",
      "        Expr(:., dotargs[1], Expr(:tuple, dotargs[2:end]...))\r\n",
      "    elseif x.head === :comparison\r\n",
      "        Expr(:comparison, (iseven(i) && dottable(arg) && arg isa Symbol && isoperator(arg) ?\r\n",
      "                               Symbol('.', arg) : arg for (i, arg) in pairs(dotargs))...)\r\n",
      "    elseif x.head === :$\r\n",
      "        x.args[1]\r\n",
      "    elseif x.head === :let # don't add dots to `let x=...` assignments\r\n",
      "        Expr(:let, undot(dotargs[1]), dotargs[2])\r\n",
      "    elseif x.head === :for # don't add dots to for x=... assignments\r\n",
      "        Expr(:for, undot(dotargs[1]), dotargs[2])\r\n",
      "    elseif (x.head === :(=) || x.head === :function || x.head === :macro) &&\r\n",
      "           Meta.isexpr(x.args[1], :call) # function or macro definition\r\n",
      "        Expr(x.head, x.args[1], dotargs[2])\r\n",
      "    elseif x.head === :(<:) || x.head === :(>:)\r\n",
      "        tmp = x.head === :(<:) ? :.<: : :.>:\r\n",
      "        Expr(:call, tmp, dotargs...)\r\n",
      "    else\r\n",
      "        head = String(x.head)::String\r\n",
      "        if last(head) == '=' && first(head) != '.' || head == \"&&\" || head == \"||\"\r\n",
      "            Expr(Symbol('.', head), dotargs...)\r\n",
      "        else\r\n",
      "            Expr(x.head, dotargs...)\r\n",
      "        end\r\n",
      "    end\r\n",
      "end\r\n",
      "\"\"\"\r\n",
      "    @. expr\r\n",
      "\r\n",
      "Convert every function call or operator in `expr` into a \"dot call\"\r\n",
      "(e.g. convert `f(x)` to `f.(x)`), and convert every assignment in `expr`\r\n",
      "to a \"dot assignment\" (e.g. convert `+=` to `.+=`).\r\n",
      "\r\n",
      "If you want to *avoid* adding dots for selected function calls in\r\n",
      "`expr`, splice those function calls in with `\\$`.  For example,\r\n",
      "`@. sqrt(abs(\\$sort(x)))` is equivalent to `sqrt.(abs.(sort(x)))`\r\n",
      "(no dot for `sort`).\r\n",
      "\r\n",
      "(`@.` is equivalent to a call to `@__dot__`.)\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> x = 1.0:3.0; y = similar(x);\r\n",
      "\r\n",
      "julia> @. y = x + 3 * sin(x)\r\n",
      "3-element Vector{Float64}:\r\n",
      " 3.5244129544236893\r\n",
      " 4.727892280477045\r\n",
      " 3.4233600241796016\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "macro __dot__(x)\r\n",
      "    esc(__dot__(x))\r\n",
      "end\r\n",
      "\r\n",
      "@inline function broadcasted_kwsyntax(f, args...; kwargs...)\r\n",
      "    if isempty(kwargs) # some BroadcastStyles dispatch on `f`, so try to preserve its type\r\n",
      "        return broadcasted(f, args...)\r\n",
      "    else\r\n",
      "        return broadcasted((args...) -> f(args...; kwargs...), args...)\r\n",
      "    end\r\n",
      "end\r\n",
      "@inline function broadcasted(f, args...)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(args′...), f, args′...)\r\n",
      "end\r\n",
      "# Due to the current Type{T}/DataType specialization heuristics within Tuples,\r\n",
      "# the totally generic varargs broadcasted(f, args...) method above loses Type{T}s in\r\n",
      "# mapping broadcastable across the args. These additional methods with explicit\r\n",
      "# arguments ensure we preserve Type{T}s in the first or second argument position.\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@inline function broadcasted(f, arg1, args...)\r\n",
      "    arg1′ = broadcastable(arg1)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(arg1′, args′...), f, arg1′, args′...)\r\n",
      "end\r\n",
      "@inline function broadcasted(f, arg1, arg2, args...)\r\n",
      "    arg1′ = broadcastable(arg1)\r\n",
      "    arg2′ = broadcastable(arg2)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(arg1′, arg2′, args′...), f, arg1′, arg2′, args′...)\r\n",
      "end\r\n",
      "@inline broadcasted(::S, f, args...) where S<:BroadcastStyle = Broadcasted{S}(f, args)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    BroadcastFunction{F} <: Function\r\n",
      "\r\n",
      "Represents the \"dotted\" version of an operator, which broadcasts the operator over its\r\n",
      "arguments, so `BroadcastFunction(op)` is functionally equivalent to `(x...) -> (op).(x...)`.\r\n",
      "\r\n",
      "Can be created by just passing an operator preceded by a dot to a higher-order function.\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> a = [[1 3; 2 4], [5 7; 6 8]];\r\n",
      "\r\n",
      "julia> b = [[9 11; 10 12], [13 15; 14 16]];\r\n",
      "\r\n",
      "julia> map(.*, a, b)\r\n",
      "2-element Vector{Matrix{Int64}}:\r\n",
      " [9 33; 20 48]\r\n",
      " [65 105; 84 128]\r\n",
      "\r\n",
      "julia> Base.BroadcastFunction(+)(a, b) == a .+ b\r\n",
      "true\r\n",
      "```\r\n",
      "\r\n",
      "!!! compat \"Julia 1.6\"\r\n",
      "    `BroadcastFunction` and the standalone `.op` syntax are available as of Julia 1.6.\r\n",
      "\"\"\"\r\n",
      "struct BroadcastFunction{F} <: Function\r\n",
      "    f::F\r\n",
      "end\r\n",
      "\r\n",
      "@inline (op::BroadcastFunction)(x...; kwargs...) = op.f.(x...; kwargs...)\r\n",
      "\r\n",
      "function Base.show(io::IO, op::BroadcastFunction)\r\n",
      "    print(io, BroadcastFunction, '(')\r\n",
      "    show(io, op.f)\r\n",
      "    print(io, ')')\r\n",
      "    nothing\r\n",
      "end\r\n",
      "Base.show(io::IO, ::MIME\"text/plain\", op::BroadcastFunction) = show(io, op)\r\n",
      "\r\n",
      "end # module\r\n"
     ]
    }
   ],
   "source": [
    "@less Base.materialize!(dest, bc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we see that first Base checks what should be style of the output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Base.Broadcast.DefaultArrayStyle{1}()"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.Broadcast.combine_styles(dest, bc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "but e.g."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DataFrames.DataFrameStyle()"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Base.Broadcast.combine_styles(df, bc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "as we insist that if a data frame takes part in broadcasting the result should be a data frame (more on this later)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@inline function materialize!(::BroadcastStyle, dest, bc::Broadcasted{Style}) where {Style}\r\n",
      "    return copyto!(dest, instantiate(Broadcasted{Style}(bc.f, bc.args, axes(dest))))\r\n",
      "end\r\n",
      "\r\n",
      "## general `copy` methods\r\n",
      "@inline copy(bc::Broadcasted{<:AbstractArrayStyle{0}}) = bc[CartesianIndex()]\r\n",
      "copy(bc::Broadcasted{<:Union{Nothing,Unknown}}) =\r\n",
      "    throw(ArgumentError(\"broadcasting requires an assigned BroadcastStyle\"))\r\n",
      "\r\n",
      "const NonleafHandlingStyles = Union{DefaultArrayStyle,ArrayConflict}\r\n",
      "\r\n",
      "@inline function copy(bc::Broadcasted{Style}) where {Style}\r\n",
      "    ElType = combine_eltypes(bc.f, bc.args)\r\n",
      "    if Base.isconcretetype(ElType)\r\n",
      "        # We can trust it and defer to the simpler `copyto!`\r\n",
      "        return copyto!(similar(bc, ElType), bc)\r\n",
      "    end\r\n",
      "    # When ElType is not concrete, use narrowing. Use the first output\r\n",
      "    # value to determine the starting output eltype; copyto_nonleaf!\r\n",
      "    # will widen `dest` as needed to accommodate later values.\r\n",
      "    bc′ = preprocess(nothing, bc)\r\n",
      "    iter = eachindex(bc′)\r\n",
      "    y = iterate(iter)\r\n",
      "    if y === nothing\r\n",
      "        # if empty, take the ElType at face value\r\n",
      "        return similar(bc′, ElType)\r\n",
      "    end\r\n",
      "    # Initialize using the first value\r\n",
      "    I, state = y\r\n",
      "    @inbounds val = bc′[I]\r\n",
      "    dest = similar(bc′, typeof(val))\r\n",
      "    @inbounds dest[I] = val\r\n",
      "    # Now handle the remaining values\r\n",
      "    # The typeassert gives inference a helping hand on the element type and dimensionality\r\n",
      "    # (work-around for #28382)\r\n",
      "    ElType′ = ElType === Union{} ? Any : ElType <: Type ? Type : ElType\r\n",
      "    RT = dest isa AbstractArray ? AbstractArray{<:ElType′, ndims(dest)} : Any\r\n",
      "    return copyto_nonleaf!(dest, bc′, iter, state, 1)::RT\r\n",
      "end\r\n",
      "\r\n",
      "## general `copyto!` methods\r\n",
      "# The most general method falls back to a method that replaces Style->Nothing\r\n",
      "# This permits specialization on typeof(dest) without introducing ambiguities\r\n",
      "@inline copyto!(dest::AbstractArray, bc::Broadcasted) = copyto!(dest, convert(Broadcasted{Nothing}, bc))\r\n",
      "\r\n",
      "# Performance optimization for the common identity scalar case: dest .= val\r\n",
      "@inline function copyto!(dest::AbstractArray, bc::Broadcasted{<:AbstractArrayStyle{0}})\r\n",
      "    # Typically, we must independently execute bc for every storage location in `dest`, but:\r\n",
      "    # IF we're in the common no-op identity case with no nested args (like `dest .= val`),\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && isflat(bc)\r\n",
      "        # THEN we can just extract the argument and `fill!` the destination with it\r\n",
      "        return fill!(dest, bc.args[1][])\r\n",
      "    else\r\n",
      "        # Otherwise, fall back to the default implementation like above\r\n",
      "        return copyto!(dest, convert(Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# For broadcasted assignments like `broadcast!(f, A, ..., A, ...)`, where `A`\r\n",
      "# appears on both the LHS and the RHS of the `.=`, then we know we're only\r\n",
      "# going to make one pass through the array, and even though `A` is aliasing\r\n",
      "# against itself, the mutations won't affect the result as the indices on the\r\n",
      "# LHS and RHS will always match. This is not true in general, but with the `.op=`\r\n",
      "# syntax it's fairly common for an argument to be `===` a source.\r\n",
      "broadcast_unalias(dest, src) = dest === src ? src : unalias(dest, src)\r\n",
      "broadcast_unalias(::Nothing, src) = src\r\n",
      "\r\n",
      "# Preprocessing a `Broadcasted` does two things:\r\n",
      "# * unaliases any arguments from `dest`\r\n",
      "# * \"extrudes\" the arguments where it is advantageous to pre-compute the broadcasted indices\r\n",
      "@inline preprocess(dest, bc::Broadcasted{Style}) where {Style} = Broadcasted{Style}(bc.f, preprocess_args(dest, bc.args), bc.axes)\r\n",
      "preprocess(dest, x) = extrude(broadcast_unalias(dest, x))\r\n",
      "\r\n",
      "@inline preprocess_args(dest, args::Tuple) = (preprocess(dest, args[1]), preprocess_args(dest, tail(args))...)\r\n",
      "@inline preprocess_args(dest, args::Tuple{Any}) = (preprocess(dest, args[1]),)\r\n",
      "@inline preprocess_args(dest, args::Tuple{}) = ()\r\n",
      "\r\n",
      "# Specialize this method if all you want to do is specialize on typeof(dest)\r\n",
      "@inline function copyto!(dest::AbstractArray, bc::Broadcasted{Nothing})\r\n",
      "    axes(dest) == axes(bc) || throwdm(axes(dest), axes(bc))\r\n",
      "    # Performance optimization: broadcast!(identity, dest, A) is equivalent to copyto!(dest, A) if indices match\r\n",
      "    if bc.f === identity && bc.args isa Tuple{AbstractArray} # only a single input argument to broadcast!\r\n",
      "        A = bc.args[1]\r\n",
      "        if axes(dest) == axes(A)\r\n",
      "            return copyto!(dest, A)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    bc′ = preprocess(dest, bc)\r\n",
      "    # Performance may vary depending on whether `@inbounds` is placed outside the\r\n",
      "    # for loop or not. (cf. https://github.com/JuliaLang/julia/issues/38086)\r\n",
      "    @inbounds @simd for I in eachindex(bc′)\r\n",
      "        dest[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "# Performance optimization: for BitArray outputs, we cache the result\r\n",
      "# in a \"small\" Vector{Bool}, and then copy in chunks into the output\r\n",
      "@inline function copyto!(dest::BitArray, bc::Broadcasted{Nothing})\r\n",
      "    axes(dest) == axes(bc) || throwdm(axes(dest), axes(bc))\r\n",
      "    ischunkedbroadcast(dest, bc) && return chunkedcopyto!(dest, bc)\r\n",
      "    length(dest) < 256 && return invoke(copyto!, Tuple{AbstractArray, Broadcasted{Nothing}}, dest, bc)\r\n",
      "    tmp = Vector{Bool}(undef, bitcache_size)\r\n",
      "    destc = dest.chunks\r\n",
      "    cind = 1\r\n",
      "    bc′ = preprocess(dest, bc)\r\n",
      "    @inbounds for P in Iterators.partition(eachindex(bc′), bitcache_size)\r\n",
      "        ind = 1\r\n",
      "        @simd for I in P\r\n",
      "            tmp[ind] = bc′[I]\r\n",
      "            ind += 1\r\n",
      "        end\r\n",
      "        @simd for i in ind:bitcache_size\r\n",
      "            tmp[i] = false\r\n",
      "        end\r\n",
      "        dumpbitcache(destc, cind, tmp)\r\n",
      "        cind += bitcache_chunks\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "# For some BitArray operations, we can work at the level of chunks. The trivial\r\n",
      "# implementation just walks over the UInt64 chunks in a linear fashion.\r\n",
      "# This requires three things:\r\n",
      "#   1. The function must be known to work at the level of chunks (or can be converted to do so)\r\n",
      "#   2. The only arrays involved must be BitArrays or scalar Bools\r\n",
      "#   3. There must not be any broadcasting beyond scalar — all array sizes must match\r\n",
      "# We could eventually allow for all broadcasting and other array types, but that\r\n",
      "# requires very careful consideration of all the edge effects.\r\n",
      "const ChunkableOp = Union{typeof(&), typeof(|), typeof(xor), typeof(~), typeof(identity),\r\n",
      "    typeof(!), typeof(*), typeof(==)} # these are convertible to chunkable ops by liftfuncs\r\n",
      "const BroadcastedChunkableOp{Style<:Union{Nothing,BroadcastStyle}, Axes, F<:ChunkableOp, Args<:Tuple} = Broadcasted{Style,Axes,F,Args}\r\n",
      "ischunkedbroadcast(R, bc::BroadcastedChunkableOp) = ischunkedbroadcast(R, bc.args)\r\n",
      "ischunkedbroadcast(R, args) = false\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:BitArray,Vararg{Any}}) = size(R) == size(args[1]) && ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:Bool,Vararg{Any}}) = ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:BroadcastedChunkableOp,Vararg{Any}}) = ischunkedbroadcast(R, args[1]) && ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{}) = true\r\n",
      "\r\n",
      "# Convert compatible functions to chunkable ones. They must also be green-lighted as ChunkableOps\r\n",
      "liftfuncs(bc::Broadcasted{Style}) where {Style} = Broadcasted{Style}(bc.f, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(sign)}) where {Style} = Broadcasted{Style}(identity, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(!)}) where {Style} = Broadcasted{Style}(~, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(*)}) where {Style} = Broadcasted{Style}(&, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(==)}) where {Style} = Broadcasted{Style}((~)∘(xor), map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(x) = x\r\n",
      "\r\n",
      "liftchunks(::Tuple{}) = ()\r\n",
      "liftchunks(args::Tuple{<:BitArray,Vararg{Any}}) = (args[1].chunks, liftchunks(tail(args))...)\r\n",
      "# Transform scalars to repeated scalars the size of a chunk\r\n",
      "liftchunks(args::Tuple{<:Bool,Vararg{Any}}) = (ifelse(args[1], typemax(UInt64), UInt64(0)), liftchunks(tail(args))...)\r\n",
      "ithchunk(i) = ()\r\n",
      "Base.@propagate_inbounds ithchunk(i, c::Vector{UInt64}, args...) = (c[i], ithchunk(i, args...)...)\r\n",
      "Base.@propagate_inbounds ithchunk(i, b::UInt64, args...) = (b, ithchunk(i, args...)...)\r\n",
      "@inline function chunkedcopyto!(dest::BitArray, bc::Broadcasted)\r\n",
      "    isempty(dest) && return dest\r\n",
      "    f = flatten(liftfuncs(bc))\r\n",
      "    args = liftchunks(f.args)\r\n",
      "    dc = dest.chunks\r\n",
      "    @simd for i in eachindex(dc)\r\n",
      "        @inbounds dc[i] = f.f(ithchunk(i, args...)...)\r\n",
      "    end\r\n",
      "    @inbounds dc[end] &= Base._msk_end(dest)\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "\r\n",
      "@noinline throwdm(axdest, axsrc) =\r\n",
      "    throw(DimensionMismatch(\"destination axes $axdest are not compatible with source axes $axsrc\"))\r\n",
      "\r\n",
      "function restart_copyto_nonleaf!(newdest, dest, bc, val, I, iter, state, count)\r\n",
      "    # Function barrier that makes the copying to newdest type stable\r\n",
      "    for II in Iterators.take(iter, count)\r\n",
      "        newdest[II] = dest[II]\r\n",
      "    end\r\n",
      "    newdest[I] = val\r\n",
      "    return copyto_nonleaf!(newdest, bc, iter, state, count+1)\r\n",
      "end\r\n",
      "\r\n",
      "function copyto_nonleaf!(dest, bc::Broadcasted, iter, state, count)\r\n",
      "    T = eltype(dest)\r\n",
      "    while true\r\n",
      "        y = iterate(iter, state)\r\n",
      "        y === nothing && break\r\n",
      "        I, state = y\r\n",
      "        @inbounds val = bc[I]\r\n",
      "        if val isa T\r\n",
      "            @inbounds dest[I] = val\r\n",
      "        else\r\n",
      "            # This element type doesn't fit in dest. Allocate a new dest with wider eltype,\r\n",
      "            # copy over old values, and continue\r\n",
      "            newdest = Base.similar(bc, promote_typejoin(T, typeof(val)))\r\n",
      "            return restart_copyto_nonleaf!(newdest, dest, bc, val, I, iter, state, count)\r\n",
      "        end\r\n",
      "        count += 1\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "## Tuple methods\r\n",
      "\r\n",
      "@inline function copy(bc::Broadcasted{Style{Tuple}})\r\n",
      "    dim = axes(bc)\r\n",
      "    length(dim) == 1 || throw(DimensionMismatch(\"tuple only supports one dimension\"))\r\n",
      "    N = length(dim[1])\r\n",
      "    return ntuple(k -> @inbounds(_broadcast_getindex(bc, k)), Val(N))\r\n",
      "end\r\n",
      "\r\n",
      "## scalar-range broadcast operations ##\r\n",
      "# DefaultArrayStyle and \\ are not available at the time of range.jl\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractRange) = r\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractRange) = range(-first(r), step=negate(step(r)), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::OrdinalRange) = range(-first(r), -last(r), step=negate(step(r)))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::StepRangeLen) = StepRangeLen(-r.ref, negate(r.step), length(r), r.offset)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::LinRange) = LinRange(-r.start, -r.stop, length(r))\r\n",
      "\r\n",
      "# For #18336 we need to prevent promotion of the step type:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractRange, x::Number) = range(first(r) + x, step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::AbstractRange) = range(x + first(r), step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::OrdinalRange, x::Integer) = range(first(r) + x, last(r) + x, step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Integer, r::OrdinalRange) = range(x + first(r), x + last(r), step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractUnitRange, x::Integer) = range(first(r) + x, last(r) + x)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Integer, r::AbstractUnitRange) = range(x + first(r), x + last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractUnitRange, x::Real) = range(first(r) + x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Real, r::AbstractUnitRange) = range(x + first(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::StepRangeLen{T}, x::Number) where T =\r\n",
      "    StepRangeLen{typeof(T(r.ref)+x)}(r.ref + x, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::StepRangeLen{T}) where T =\r\n",
      "    StepRangeLen{typeof(x+T(r.ref))}(x + r.ref, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::LinRange, x::Number) = LinRange(r.start + x, r.stop + x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::LinRange) = LinRange(x + r.start, x + r.stop, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r1::AbstractRange, r2::AbstractRange) = r1 + r2\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractRange, x::Number) = range(first(r) - x, step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::AbstractRange) = range(x - first(r), step=negate(step(r)), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::OrdinalRange, x::Integer) = range(first(r) - x, last(r) - x, step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Integer, r::OrdinalRange) = range(x - first(r), x - last(r), step=negate(step(r)))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractUnitRange, x::Integer) = range(first(r) - x, last(r) - x)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractUnitRange, x::Real) = range(first(r) - x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::StepRangeLen{T}, x::Number) where T =\r\n",
      "    StepRangeLen{typeof(T(r.ref)-x)}(r.ref - x, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::StepRangeLen{T}) where T =\r\n",
      "    StepRangeLen{typeof(x-T(r.ref))}(x - r.ref, negate(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::LinRange, x::Number) = LinRange(r.start - x, r.stop - x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::LinRange) = LinRange(x - r.start, x - r.stop, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r1::AbstractRange, r2::AbstractRange) = r1 - r2\r\n",
      "\r\n",
      "# at present Base.range_start_step_length(1,0,5) is an error, so for 0 .* (-2:2) we explicitly construct StepRangeLen:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::AbstractRange) = StepRangeLen(x*first(r), x*step(r), length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::StepRangeLen{T}) where {T} =\r\n",
      "    StepRangeLen{typeof(x*T(r.ref))}(x*r.ref, x*r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::LinRange) = LinRange(x * r.start, x * r.stop, r.len)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::AbstractFloat, r::OrdinalRange) =\r\n",
      "    Base.range_start_step_length(x*first(r), x*step(r), length(r))  # 0.2 .* (-2:2) needs TwicePrecision\r\n",
      "# separate in case of noncommutative multiplication:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::AbstractRange, x::Number) = StepRangeLen(first(r)*x, step(r)*x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::StepRangeLen{T}, x::Number) where {T} =\r\n",
      "    StepRangeLen{typeof(T(r.ref)*x)}(r.ref*x, r.step*x, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::LinRange, x::Number) = LinRange(r.start * x, r.stop * x, r.len)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::OrdinalRange, x::AbstractFloat) =\r\n",
      "    Base.range_start_step_length(first(r)*x, step(r)*x, length(r))\r\n",
      "\r\n",
      "#broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::AbstractRange, x::Number) = range(first(r)/x, last(r)/x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::AbstractRange, x::Number) = range(first(r)/x, step=step(r)/x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::StepRangeLen{T}, x::Number) where {T} =\r\n",
      "    StepRangeLen{typeof(T(r.ref)/x)}(r.ref/x, r.step/x, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::LinRange, x::Number) = LinRange(r.start / x, r.stop / x, r.len)\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::AbstractRange) = range(x\\first(r), step=x\\step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::StepRangeLen) = StepRangeLen(x\\r.ref, x\\r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::LinRange) = LinRange(x \\ r.start, x \\ r.stop, r.len)\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::UnitRange) = big(r.start):big(last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::StepRange) = big(r.start):big(r.step):big(last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::StepRangeLen) = StepRangeLen(big(r.ref), big(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::LinRange) = LinRange(big(r.start), big(r.stop), length(r))\r\n",
      "\r\n",
      "## CartesianIndices\r\n",
      "broadcasted(::typeof(+), I::CartesianIndices{N}, j::CartesianIndex{N}) where N =\r\n",
      "    CartesianIndices(map((rng, offset)->rng .+ offset, I.indices, Tuple(j)))\r\n",
      "broadcasted(::typeof(+), j::CartesianIndex{N}, I::CartesianIndices{N}) where N =\r\n",
      "    I .+ j\r\n",
      "broadcasted(::typeof(-), I::CartesianIndices{N}, j::CartesianIndex{N}) where N =\r\n",
      "    CartesianIndices(map((rng, offset)->rng .- offset, I.indices, Tuple(j)))\r\n",
      "function broadcasted(::typeof(-), j::CartesianIndex{N}, I::CartesianIndices{N}) where N\r\n",
      "    diffrange(offset, rng) = range(offset-last(rng), length=length(rng), step=step(rng))\r\n",
      "    Iterators.reverse(CartesianIndices(map(diffrange, Tuple(j), I.indices)))\r\n",
      "end\r\n",
      "\r\n",
      "## In specific instances, we can broadcast masked BitArrays whole chunks at a time\r\n",
      "# Very intentionally do not support much functionality here: scalar indexing would be O(n)\r\n",
      "struct BitMaskedBitArray{N,M}\r\n",
      "    parent::BitArray{N}\r\n",
      "    mask::BitArray{M}\r\n",
      "    BitMaskedBitArray{N,M}(parent, mask) where {N,M} = new(parent, mask)\r\n",
      "end\r\n",
      "@inline function BitMaskedBitArray(parent::BitArray{N}, mask::BitArray{M}) where {N,M}\r\n",
      "    @boundscheck checkbounds(parent, mask)\r\n",
      "    BitMaskedBitArray{N,M}(parent, mask)\r\n",
      "end\r\n",
      "Base.@propagate_inbounds dotview(B::BitArray, i::BitArray) = BitMaskedBitArray(B, i)\r\n",
      "Base.show(io::IO, B::BitMaskedBitArray) = foreach(arg->show(io, arg), (typeof(B), (B.parent, B.mask)))\r\n",
      "# Override materialize! to prevent the BitMaskedBitArray from escaping to an overridable method\r\n",
      "@inline materialize!(B::BitMaskedBitArray, bc::Broadcasted{<:Any,<:Any,typeof(identity),Tuple{Bool}}) = fill!(B, bc.args[1])\r\n",
      "@inline materialize!(B::BitMaskedBitArray, bc::Broadcasted{<:Any}) = materialize!(@inbounds(view(B.parent, B.mask)), bc)\r\n",
      "function Base.fill!(B::BitMaskedBitArray, b::Bool)\r\n",
      "    Bc = B.parent.chunks\r\n",
      "    Ic = B.mask.chunks\r\n",
      "    @inbounds if b\r\n",
      "        for i = 1:length(Bc)\r\n",
      "            Bc[i] |= Ic[i]\r\n",
      "        end\r\n",
      "    else\r\n",
      "        for i = 1:length(Bc)\r\n",
      "            Bc[i] &= ~Ic[i]\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return B\r\n",
      "end\r\n",
      "\r\n",
      "\r\n",
      "\r\n",
      "############################################################\r\n",
      "\r\n",
      "# x[...] .= f.(y...) ---> broadcast!(f, dotview(x, ...), y...).\r\n",
      "# The dotview function defaults to getindex, but we override it in\r\n",
      "# a few cases to get the expected in-place behavior without affecting\r\n",
      "# explicit calls to view.   (All of this can go away if slices\r\n",
      "# are changed to generate views by default.)\r\n",
      "\r\n",
      "Base.@propagate_inbounds dotview(args...) = Base.maybeview(args...)\r\n",
      "\r\n",
      "############################################################\r\n",
      "# The parser turns @. into a call to the __dot__ macro,\r\n",
      "# which converts all function calls and assignments into\r\n",
      "# broadcasting \"dot\" calls/assignments:\r\n",
      "\r\n",
      "dottable(x) = false # avoid dotting spliced objects (e.g. view calls inserted by @view)\r\n",
      "# don't add dots to dot operators\r\n",
      "dottable(x::Symbol) = (!isoperator(x) || first(string(x)) != '.' || x === :..) && x !== :(:)\r\n",
      "dottable(x::Expr) = x.head !== :$\r\n",
      "undot(x) = x\r\n",
      "function undot(x::Expr)\r\n",
      "    if x.head === :.=\r\n",
      "        Expr(:(=), x.args...)\r\n",
      "    elseif x.head === :block # occurs in for x=..., y=...\r\n",
      "        Expr(:block, Base.mapany(undot, x.args)...)\r\n",
      "    else\r\n",
      "        x\r\n",
      "    end\r\n",
      "end\r\n",
      "__dot__(x) = x\r\n",
      "function __dot__(x::Expr)\r\n",
      "    dotargs = Base.mapany(__dot__, x.args)\r\n",
      "    if x.head === :call && dottable(x.args[1])\r\n",
      "        Expr(:., dotargs[1], Expr(:tuple, dotargs[2:end]...))\r\n",
      "    elseif x.head === :comparison\r\n",
      "        Expr(:comparison, (iseven(i) && dottable(arg) && arg isa Symbol && isoperator(arg) ?\r\n",
      "                               Symbol('.', arg) : arg for (i, arg) in pairs(dotargs))...)\r\n",
      "    elseif x.head === :$\r\n",
      "        x.args[1]\r\n",
      "    elseif x.head === :let # don't add dots to `let x=...` assignments\r\n",
      "        Expr(:let, undot(dotargs[1]), dotargs[2])\r\n",
      "    elseif x.head === :for # don't add dots to for x=... assignments\r\n",
      "        Expr(:for, undot(dotargs[1]), dotargs[2])\r\n",
      "    elseif (x.head === :(=) || x.head === :function || x.head === :macro) &&\r\n",
      "           Meta.isexpr(x.args[1], :call) # function or macro definition\r\n",
      "        Expr(x.head, x.args[1], dotargs[2])\r\n",
      "    elseif x.head === :(<:) || x.head === :(>:)\r\n",
      "        tmp = x.head === :(<:) ? :.<: : :.>:\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        Expr(:call, tmp, dotargs...)\r\n",
      "    else\r\n",
      "        head = String(x.head)::String\r\n",
      "        if last(head) == '=' && first(head) != '.' || head == \"&&\" || head == \"||\"\r\n",
      "            Expr(Symbol('.', head), dotargs...)\r\n",
      "        else\r\n",
      "            Expr(x.head, dotargs...)\r\n",
      "        end\r\n",
      "    end\r\n",
      "end\r\n",
      "\"\"\"\r\n",
      "    @. expr\r\n",
      "\r\n",
      "Convert every function call or operator in `expr` into a \"dot call\"\r\n",
      "(e.g. convert `f(x)` to `f.(x)`), and convert every assignment in `expr`\r\n",
      "to a \"dot assignment\" (e.g. convert `+=` to `.+=`).\r\n",
      "\r\n",
      "If you want to *avoid* adding dots for selected function calls in\r\n",
      "`expr`, splice those function calls in with `\\$`.  For example,\r\n",
      "`@. sqrt(abs(\\$sort(x)))` is equivalent to `sqrt.(abs.(sort(x)))`\r\n",
      "(no dot for `sort`).\r\n",
      "\r\n",
      "(`@.` is equivalent to a call to `@__dot__`.)\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> x = 1.0:3.0; y = similar(x);\r\n",
      "\r\n",
      "julia> @. y = x + 3 * sin(x)\r\n",
      "3-element Vector{Float64}:\r\n",
      " 3.5244129544236893\r\n",
      " 4.727892280477045\r\n",
      " 3.4233600241796016\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "macro __dot__(x)\r\n",
      "    esc(__dot__(x))\r\n",
      "end\r\n",
      "\r\n",
      "@inline function broadcasted_kwsyntax(f, args...; kwargs...)\r\n",
      "    if isempty(kwargs) # some BroadcastStyles dispatch on `f`, so try to preserve its type\r\n",
      "        return broadcasted(f, args...)\r\n",
      "    else\r\n",
      "        return broadcasted((args...) -> f(args...; kwargs...), args...)\r\n",
      "    end\r\n",
      "end\r\n",
      "@inline function broadcasted(f, args...)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(args′...), f, args′...)\r\n",
      "end\r\n",
      "# Due to the current Type{T}/DataType specialization heuristics within Tuples,\r\n",
      "# the totally generic varargs broadcasted(f, args...) method above loses Type{T}s in\r\n",
      "# mapping broadcastable across the args. These additional methods with explicit\r\n",
      "# arguments ensure we preserve Type{T}s in the first or second argument position.\r\n",
      "@inline function broadcasted(f, arg1, args...)\r\n",
      "    arg1′ = broadcastable(arg1)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(arg1′, args′...), f, arg1′, args′...)\r\n",
      "end\r\n",
      "@inline function broadcasted(f, arg1, arg2, args...)\r\n",
      "    arg1′ = broadcastable(arg1)\r\n",
      "    arg2′ = broadcastable(arg2)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(arg1′, arg2′, args′...), f, arg1′, arg2′, args′...)\r\n",
      "end\r\n",
      "@inline broadcasted(::S, f, args...) where S<:BroadcastStyle = Broadcasted{S}(f, args)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    BroadcastFunction{F} <: Function\r\n",
      "\r\n",
      "Represents the \"dotted\" version of an operator, which broadcasts the operator over its\r\n",
      "arguments, so `BroadcastFunction(op)` is functionally equivalent to `(x...) -> (op).(x...)`.\r\n",
      "\r\n",
      "Can be created by just passing an operator preceded by a dot to a higher-order function.\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> a = [[1 3; 2 4], [5 7; 6 8]];\r\n",
      "\r\n",
      "julia> b = [[9 11; 10 12], [13 15; 14 16]];\r\n",
      "\r\n",
      "julia> map(.*, a, b)\r\n",
      "2-element Vector{Matrix{Int64}}:\r\n",
      " [9 33; 20 48]\r\n",
      " [65 105; 84 128]\r\n",
      "\r\n",
      "julia> Base.BroadcastFunction(+)(a, b) == a .+ b\r\n",
      "true\r\n",
      "```\r\n",
      "\r\n",
      "!!! compat \"Julia 1.6\"\r\n",
      "    `BroadcastFunction` and the standalone `.op` syntax are available as of Julia 1.6.\r\n",
      "\"\"\"\r\n",
      "struct BroadcastFunction{F} <: Function\r\n",
      "    f::F\r\n",
      "end\r\n",
      "\r\n",
      "@inline (op::BroadcastFunction)(x...; kwargs...) = op.f.(x...; kwargs...)\r\n",
      "\r\n",
      "function Base.show(io::IO, op::BroadcastFunction)\r\n",
      "    print(io, BroadcastFunction, '(')\r\n",
      "    show(io, op.f)\r\n",
      "    print(io, ')')\r\n",
      "    nothing\r\n",
      "end\r\n",
      "Base.show(io::IO, ::MIME\"text/plain\", op::BroadcastFunction) = show(io, op)\r\n",
      "\r\n",
      "end # module\r\n"
     ]
    }
   ],
   "source": [
    "@less Base.materialize!(Base.Broadcast.combine_styles(dest, bc), dest, bc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, typeof(identity), Tuple{Int64}}"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "typeof(bc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Base.axes(x::LazyNewColDataFrame) = (Base.OneTo(nrow(x.df)),)\r\n",
      "Base.ndims(::Type{<:LazyNewColDataFrame}) = 1\r\n",
      "\r\n",
      "struct ColReplaceDataFrame{T<:AbstractDataFrame}\r\n",
      "    df::T\r\n",
      "    cols::Vector{Int}\r\n",
      "end\r\n",
      "\r\n",
      "Base.axes(x::ColReplaceDataFrame) = (axes(x.df, 1), Base.OneTo(length(x.cols)))\r\n",
      "Base.ndims(::Type{<:ColReplaceDataFrame}) = 2\r\n",
      "\r\n",
      "# In the functions below we need to call _drop_all_nonnote_metadata!\r\n",
      "# upfront as the rest of the operations is handled by Base Julia\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, idx::CartesianIndex{2})\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df[idx]\r\n",
      "end\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, row::Integer, col::ColumnIndex)\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df[row, col]\r\n",
      "end\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, rows, cols)\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return view(df, rows, cols)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.dotview(df::AbstractDataFrame, ::Colon, cols::ColumnIndex)\r\n",
      "    if haskey(index(df), cols)\r\n",
      "        _drop_all_nonnote_metadata!(parent(df))\r\n",
      "        return view(df, :, cols)\r\n",
      "    end\r\n",
      "    if !(cols isa SymbolOrString)\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    if !is_column_insertion_allowed(df)\r\n",
      "        throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                            \"columns of its parent data frame is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, Symbol(cols))\r\n",
      "end\r\n",
      "\r\n",
      "function Base.dotview(df::AbstractDataFrame, ::typeof(!), cols)\r\n",
      "    if !(cols isa ColumnIndex)\r\n",
      "        return ColReplaceDataFrame(df, convert(Vector{Int}, index(df)[cols]))\r\n",
      "    end\r\n",
      "    if cols isa SymbolOrString\r\n",
      "        if columnindex(df, cols) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "    elseif !(1 <= cols <= ncol(df))\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, cols isa AbstractString ? Symbol(cols) : cols)\r\n",
      "end\r\n",
      "\r\n",
      "if isdefined(Base, :dotgetproperty) # Introduced in Julia 1.7\r\n",
      "    function Base.dotgetproperty(df::AbstractDataFrame, col::SymbolOrString)\r\n",
      "        if columnindex(df, col) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "        return LazyNewColDataFrame(df, Symbol(col))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(lazydf::LazyNewColDataFrame, bc::Base.Broadcast.Broadcasted{T}) where T\r\n",
      "    df = lazydf.df\r\n",
      "    if !haskey(index(df), lazydf.col) && df isa SubDataFrame && lazydf.col isa SymbolOrString\r\n",
      "        @assert is_column_insertion_allowed(df)\r\n",
      "    end\r\n",
      "    if bc isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "        bc_tmp = Base.Broadcast.Broadcasted{T}(bc.f, bc.args, ())\r\n",
      "        v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "        col = similar(Vector{typeof(v)}, nrow(df))\r\n",
      "        copyto!(col, bc)\r\n",
      "    else\r\n",
      "        col = Base.Broadcast.materialize(bc)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df[!, lazydf.col] = col\r\n",
      "end\r\n",
      "\r\n",
      "function _copyto_helper!(dfcol::AbstractVector, bc::Base.Broadcast.Broadcasted, col::Int)\r\n",
      "    if axes(dfcol, 1) != axes(bc)[1]\r\n",
      "        # this should never happen unless data frame is corrupted (has unequal column lengths)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting. The updated\" *\r\n",
      "                                \" data frame is invalid and should not be used\"))\r\n",
      "    end\r\n",
      "    @inbounds for row in eachindex(dfcol)\r\n",
      "        dfcol[row] = bc[CartesianIndex(row, col)]\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src)\r\n",
      "    for col in eachcol(dest)\r\n",
      "        src = Base.Broadcast.unalias(col, src)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.broadcast_unalias(::Nothing, src::AbstractDataFrame) = src\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest, src::AbstractDataFrame)\r\n",
      "    wascopied = false\r\n",
      "    for (i, col) in enumerate(eachcol(src))\r\n",
      "        if Base.mightalias(dest, col)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src = SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                       index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), i)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, i] = Base.unaliascopy(col)\r\n",
      "            end\r\n",
      "            wascopied = true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function _broadcast_unalias_helper(dest::AbstractDataFrame, scol::AbstractVector,\r\n",
      "                                   src::AbstractDataFrame, col2::Int, wascopied::Bool)\r\n",
      "    # col1 can be checked till col2 point as we are writing broadcasting\r\n",
      "    # results from 1 to ncol\r\n",
      "    # we go downwards because aliasing when col1 == col2 is most probable\r\n",
      "    for col1 in col2:-1:1\r\n",
      "        dcol = dest[!, col1]\r\n",
      "        if Base.mightalias(dcol, scol)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src =SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                      index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), col2)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, col2] = Base.unaliascopy(scol)\r\n",
      "            end\r\n",
      "            return src, true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src, wascopied\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src::AbstractDataFrame)\r\n",
      "    if size(dest, 2) != size(src, 2)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting.\"))\r\n",
      "    end\r\n",
      "    wascopied = false\r\n",
      "    for col2 in axes(dest, 2)\r\n",
      "        scol = src[!, col2]\r\n",
      "        src, wascopied = _broadcast_unalias_helper(dest, scol, src, col2, wascopied)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "@less axes(dest)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Base.Broadcast.Broadcasted((identity, (1,)), (Base.OneTo(2),))"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inst = Base.Broadcast.instantiate(Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}}((bc.f, bc.args), axes(dest)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function Base.copyto!(lazydf::LazyNewColDataFrame, bc::Base.Broadcast.Broadcasted{T}) where T\r\n",
      "    df = lazydf.df\r\n",
      "    if !haskey(index(df), lazydf.col) && df isa SubDataFrame && lazydf.col isa SymbolOrString\r\n",
      "        @assert is_column_insertion_allowed(df)\r\n",
      "    end\r\n",
      "    if bc isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "        bc_tmp = Base.Broadcast.Broadcasted{T}(bc.f, bc.args, ())\r\n",
      "        v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "        col = similar(Vector{typeof(v)}, nrow(df))\r\n",
      "        copyto!(col, bc)\r\n",
      "    else\r\n",
      "        col = Base.Broadcast.materialize(bc)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df[!, lazydf.col] = col\r\n",
      "end\r\n",
      "\r\n",
      "function _copyto_helper!(dfcol::AbstractVector, bc::Base.Broadcast.Broadcasted, col::Int)\r\n",
      "    if axes(dfcol, 1) != axes(bc)[1]\r\n",
      "        # this should never happen unless data frame is corrupted (has unequal column lengths)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting. The updated\" *\r\n",
      "                                \" data frame is invalid and should not be used\"))\r\n",
      "    end\r\n",
      "    @inbounds for row in eachindex(dfcol)\r\n",
      "        dfcol[row] = bc[CartesianIndex(row, col)]\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src)\r\n",
      "    for col in eachcol(dest)\r\n",
      "        src = Base.Broadcast.unalias(col, src)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.broadcast_unalias(::Nothing, src::AbstractDataFrame) = src\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest, src::AbstractDataFrame)\r\n",
      "    wascopied = false\r\n",
      "    for (i, col) in enumerate(eachcol(src))\r\n",
      "        if Base.mightalias(dest, col)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src = SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                       index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), i)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, i] = Base.unaliascopy(col)\r\n",
      "            end\r\n",
      "            wascopied = true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function _broadcast_unalias_helper(dest::AbstractDataFrame, scol::AbstractVector,\r\n",
      "                                   src::AbstractDataFrame, col2::Int, wascopied::Bool)\r\n",
      "    # col1 can be checked till col2 point as we are writing broadcasting\r\n",
      "    # results from 1 to ncol\r\n",
      "    # we go downwards because aliasing when col1 == col2 is most probable\r\n",
      "    for col1 in col2:-1:1\r\n",
      "        dcol = dest[!, col1]\r\n",
      "        if Base.mightalias(dcol, scol)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src =SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                      index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), col2)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, col2] = Base.unaliascopy(scol)\r\n",
      "            end\r\n",
      "            return src, true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src, wascopied\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src::AbstractDataFrame)\r\n",
      "    if size(dest, 2) != size(src, 2)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting.\"))\r\n",
      "    end\r\n",
      "    wascopied = false\r\n",
      "    for col2 in axes(dest, 2)\r\n",
      "        scol = src[!, col2]\r\n",
      "        src, wascopied = _broadcast_unalias_helper(dest, scol, src, col2, wascopied)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n",
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "@less copyto!(dest, inst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why a special path for 0-dimensional objects is required?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "MethodError: objects of type Tuple{typeof(identity), Tuple{Int64}} are not callable",
     "output_type": "error",
     "traceback": [
      "MethodError: objects of type Tuple{typeof(identity), Tuple{Int64}} are not callable",
      "",
      "Stacktrace:",
      " [1] _broadcast_getindex_evalf",
      "   @ .\\broadcast.jl:683 [inlined]",
      " [2] _broadcast_getindex",
      "   @ .\\broadcast.jl:656 [inlined]",
      " [3] getindex",
      "   @ .\\broadcast.jl:610 [inlined]",
      " [4] copy",
      "   @ .\\broadcast.jl:888 [inlined]",
      " [5] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{0}, Nothing, Tuple{typeof(identity), Tuple{Int64}}, Tuple{Base.OneTo{Int64}}})",
      "   @ Base.Broadcast .\\broadcast.jl:873",
      " [6] top-level scope",
      "   @ In[44]:1"
     ]
    }
   ],
   "source": [
    "Base.Broadcast.materialize(inst)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 3: avoiding dispatch ambiguity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×4 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x1</th><th style = \"text-align: left;\">x2</th><th style = \"text-align: left;\">x3</th><th style = \"text-align: left;\">x4</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td><td style = \"text-align: right;\">2</td><td style = \"text-align: right;\">3</td><td style = \"text-align: right;\">4</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      2      3      4"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame([1 2 3 4], :auto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 11\n",
       " 12\n",
       " 13"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[1, Not(1)] = [11, 12, 13]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×4 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x1</th><th style = \"text-align: left;\">x2</th><th style = \"text-align: left;\">x3</th><th style = \"text-align: left;\">x4</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td><td style = \"text-align: right;\">11</td><td style = \"text-align: right;\">12</td><td style = \"text-align: right;\">13</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 11 & 12 & 13 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1     11     12     13"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::Union{Tuple, AbstractArray},\r\n",
      "                                  row_ind::Integer,\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if length(v) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"$(length(idxs)) columns were selected but the assigned \" *\r\n",
      "                                    \"collection contains $(length(v)) elements\"))\r\n",
      "        end\r\n",
      "        for (i, x) in zip(idxs, v)\r\n",
      "            df[!, i][row_ind] = x\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, SingleColumnIndex] = AbstractVector\r\n",
      "for T in (:AbstractVector, :Not, :Colon)\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  v::AbstractVector,\r\n",
      "                                  row_inds::$T,\r\n",
      "                                  col_ind::ColumnIndex)\r\n",
      "        if row_inds isa Colon && !haskey(index(df), col_ind)\r\n",
      "            df[!, col_ind] = copy(v)\r\n",
      "            return df\r\n",
      "        end\r\n",
      "        x = df[!, col_ind]\r\n",
      "        x[row_inds] = v\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractDataFrame\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"column names in source and target do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            df[!, col][row_inds] = new_df[!, j]\r\n",
      "        end\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "for T in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  new_df::AbstractDataFrame,\r\n",
      "                                  row_inds::typeof(!),\r\n",
      "                                  col_inds::$T)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if view(_names(df), idxs) != _names(new_df)\r\n",
      "            throw(ArgumentError(\"Column names in source and target data frames do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # make sure we make a copy on assignment\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[!, col] = new_df[:, j]\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# df[MultiRowIndex, MultiColumnIndex] = AbstractMatrix\r\n",
      "for T1 in (:AbstractVector, :Not, :Colon, :(typeof(!))),\r\n",
      "    T2 in MULTICOLUMNINDEX_TUPLE\r\n",
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  mx::AbstractMatrix,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if size(mx, 2) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"number of selected columns ($(length(idxs))) \" *\r\n",
      "                                    \"and number of columns in \" *\r\n",
      "                                    \"matrix ($(size(mx, 2))) do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[row_inds, col] = (row_inds === !) ? mx[:, j] : view(mx, :, j)\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    copy(df::DataFrame; copycols::Bool=true)\r\n",
      "\r\n",
      "Copy data frame `df`.\r\n",
      "If `copycols=true` (the default), return a new  `DataFrame` holding\r\n",
      "copies of column vectors in `df`.\r\n",
      "If `copycols=false`, return a new `DataFrame` sharing column vectors with `df`.\r\n",
      "\r\n",
      "Metadata: this function preserves all table-level and column-level metadata.\r\n",
      "\"\"\"\r\n",
      "function Base.copy(df::DataFrame; copycols::Bool=true)\r\n",
      "    cdf = DataFrame(copy(_columns(df)), copy(index(df)), copycols=copycols)\r\n",
      "\r\n",
      "    df_metadata = getfield(df, :metadata)\r\n",
      "    if !isnothing(df_metadata)\r\n",
      "        setfield!(cdf, :metadata, copy(df_metadata))\r\n",
      "    end\r\n",
      "    df_colmetadata = getfield(df, :colmetadata)\r\n",
      "    if !isnothing(df_colmetadata)\r\n",
      "        cdf_colmetadata = copy(df_colmetadata)\r\n",
      "        map!(copy, values(cdf_colmetadata))\r\n",
      "        setfield!(cdf, :colmetadata, cdf_colmetadata)\r\n",
      "    end\r\n",
      "    setfield!(cdf, :allnotemetadata, getfield(df, :allnotemetadata))\r\n",
      "\r\n",
      "    return cdf\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows specified by `inds` from a `DataFrame` `df` in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> deleteat!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, ::Colon) = empty!(df)\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    size(df, 2) == 0 && throw(BoundsError(df, (inds, :)))\r\n",
      "    return _deleteat!_helper(df, Int[inds])\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    if isempty(inds)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    elseif size(df, 2) == 0\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !(eltype(inds) <: Integer || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && inds isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to deleteat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector{Bool})\r\n",
      "    if length(inds) != size(df, 1)\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && drop isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, inds::Not) =\r\n",
      "    _deleteat!_helper(df, axes(df, 1)[inds])\r\n",
      "\r\n",
      "function _deleteat!_helper(df::DataFrame, drop)\r\n",
      "    cols = _columns(df)\r\n",
      "    if isempty(cols)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "\r\n",
      "    n = nrow(df)\r\n",
      "    col1 = cols[1]\r\n",
      "    deleteat!(col1, drop)\r\n",
      "    newn = length(col1)\r\n",
      "    @assert newn <= n\r\n",
      "    # the 0.06 threshold is heuristic; based on tests\r\n",
      "    # the assignment makes the code type-unstable but it is a small union\r\n",
      "    # so the overhead should be small\r\n",
      "    if drop isa AbstractVector{Bool} && newn < 0.06 * n\r\n",
      "        drop = findall(drop)\r\n",
      "    end\r\n",
      "    for i in 2:length(cols)\r\n",
      "        col = cols[i]\r\n",
      "        # this check is done to handle column aliases\r\n",
      "        if length(col) == n\r\n",
      "            deleteat!(col, drop)\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    for i in 1:length(cols)\r\n",
      "        # this should never happen, but we add it for safety\r\n",
      "        @assert length(cols[i]) == newn corrupt_msg(df, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows at all indices not specified by `inds` from a `DataFrame` `df`\r\n",
      "in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> keepat!(df, [1, 3])\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, ::Colon)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    isempty(inds) && return empty!(df)\r\n",
      "\r\n",
      "    # this is required because of https://github.com/JuliaData/InvertedIndices.jl/issues/31\r\n",
      "    if !((eltype(inds) <: Integer) || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to keepat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return deleteat!(df, Not(inds))\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    return deleteat!(df, Not(Int[inds]))\r\n",
      "end\r\n",
      "\r\n",
      "keepat!(df::DataFrame, inds::AbstractVector{Bool}) = deleteat!(df, .!inds)\r\n",
      "keepat!(df::DataFrame, inds::Not) = deleteat!(df, Not(inds))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    empty!(df::DataFrame)\r\n",
      "\r\n",
      "Remove all rows from `df`, making each of its columns empty.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> empty!(df)\r\n",
      "0×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┴──────────────\r\n",
      "\r\n",
      "julia> df.a, df.b\r\n",
      "(Int64[], Int64[])\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "function Base.empty!(df::DataFrame)\r\n",
      "    foreach(empty!, eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    resize!(df::DataFrame, n::Integer)\r\n",
      "\r\n",
      "Resize `df` to have `n` rows by calling `resize!` on all columns of `df`.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> resize!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.resize!(df::DataFrame, n::Integer)\r\n",
      "    if ncol(df) == 0 && n != 0\r\n",
      "        throw(ArgumentError(\"data frame has no columns and requested number \" *\r\n",
      "                            \"of rows is not zero\"))\r\n",
      "    end\r\n",
      "    foreach(col -> resize!(col, n), eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    pop!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the last row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> pop!(df)\r\n",
      "(a = 3, b = 6)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "Base.pop!(df::DataFrame) = popat!(df, nrow(df))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popfirst!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the first row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popfirst!(df)\r\n",
      "(a = 1, b = 4)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     2      5\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.popfirst!(df::DataFrame) = popat!(df, 1)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popat!(df::DataFrame, i::Integer)\r\n",
      "\r\n",
      "Remove the `i`-th row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popat!(df, 2)\r\n",
      "(a = 2, b = 5)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.popat!(df::DataFrame, i::Integer)\r\n",
      "    i isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    nt = NamedTuple(df[i, :])\r\n",
      "    deleteat!(df, i)\r\n",
      "    return nt\r\n",
      "end\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## hcat!\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\r\n",
      "# hcat! for 2 arguments, only a vector or a data frame is allowed\r\n",
      "function hcat!(df1::DataFrame, df2::AbstractDataFrame;\r\n",
      "               makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    u = add_names(index(df1), index(df2), makeunique=makeunique)\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df1)\r\n",
      "    _keep_matching_table_note_metadata!(df1, df2)\r\n",
      "    for i in 1:length(u)\r\n",
      "        df1[!, u[i]] = copycols ? df2[:, i] : df2[!, i]\r\n",
      "        _copy_col_note_metadata!(df1, u[i], df2, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df1\r\n",
      "end\r\n",
      "\r\n",
      "# TODO: after deprecation remove AbstractVector methods\r\n",
      "\r\n",
      "function hcat!(df::DataFrame, x::AbstractVector; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(df, DataFrame(AbstractVector[x], [:x1], copycols=false),\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "function hcat!(x::AbstractVector, df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(DataFrame(AbstractVector[x], [:x1], copycols=copycols), df,\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "# hcat! for 1-n arguments\r\n",
      "function hcat!(df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "hcat!(a::DataFrame, b::Union{AbstractDataFrame, AbstractVector},\r\n",
      "      c::Union{AbstractDataFrame, AbstractVector}...;\r\n",
      "      makeunique::Bool=false, copycols::Bool=true) =\r\n",
      "    hcat!(hcat!(a, b, makeunique=makeunique, copycols=copycols),\r\n",
      "          c..., makeunique=makeunique, copycols=copycols)\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## Missing values support\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\"\"\"\r\n",
      "    allowmissing!(df::DataFrame, cols=:)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `T` to\r\n",
      "`Union{T, Missing}` to support missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function allowmissing! end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, col::ColumnIndex)\r\n",
      "    df[!, col] = allowmissing(df[!, col])\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex})\r\n",
      "    for col in cols\r\n",
      "        allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{Bool})\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::MultiColumnIndex) =\r\n",
      "    allowmissing!(df, index(df)[cols])\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::Colon=:) =\r\n",
      "    allowmissing!(df, axes(df, 2))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    disallowmissing!(df::DataFrame, cols=:; error::Bool=true)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `Union{T, Missing}` to\r\n",
      "`T` to drop support for missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "If `error=false` then columns containing a `missing` value will be skipped instead\r\n",
      "of throwing an error.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function disallowmissing! end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, col::ColumnIndex; error::Bool=true)\r\n",
      "    x = df[!, col]\r\n",
      "    if Missing <: eltype(x)\r\n",
      "        try\r\n",
      "            df[!, col] = disallowmissing(x)\r\n",
      "        catch e\r\n",
      "            row = findfirst(ismissing, x)\r\n",
      "            if row !== nothing\r\n",
      "                if error\r\n",
      "                    col_name = only(names(df, col))\r\n",
      "                    throw(ArgumentError(\"Missing value found in column \" *\r\n",
      "                                        \":$col_name in row $row\"))\r\n",
      "                end\r\n",
      "            else\r\n",
      "                rethrow(e)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex};\r\n",
      "                          error::Bool=true)\r\n",
      "    for col in cols\r\n",
      "        disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{Bool}; error::Bool=true)\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::MultiColumnIndex; error::Bool=true) =\r\n",
      "    disallowmissing!(df, index(df)[cols], error=error)\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::Colon=:; error::Bool=true) =\r\n",
      "    disallowmissing!(df, axes(df, 2), error=error)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows. `inner` specifies how many\r\n",
      "times each row is repeated, and `outer` specifies how many times the full set\r\n",
      "of rows is repeated. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat!(df, inner=2, outer=3);\r\n",
      "\r\n",
      "julia> df\r\n",
      "12×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     1      3\r\n",
      "   3 │     2      4\r\n",
      "   4 │     2      4\r\n",
      "   5 │     1      3\r\n",
      "   6 │     1      3\r\n",
      "   7 │     2      4\r\n",
      "   8 │     2      4\r\n",
      "   9 │     1      3\r\n",
      "  10 │     1      3\r\n",
      "  11 │     2      4\r\n",
      "  12 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "    inner < 0 && throw(ArgumentError(\"inner keyword argument must be non-negative\"))\r\n",
      "    outer < 0 && throw(ArgumentError(\"outer keyword argument must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, inner=Int(inner), outer=Int(outer))\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame, count::Integer)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows the number of times\r\n",
      "specified by `count`. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat(df, 2)\r\n",
      "4×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "   3 │     1      3\r\n",
      "   4 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame, count::Integer)\r\n",
      "    count < 0 && throw(ArgumentError(\"count must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, count)\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# This is not exactly copy! as in general we allow axes to be different\r\n",
      "# Also no table metadata needs to be copied as we use _replace_columns!\r\n",
      "# only in situations when table metadata for `df` must be left as-is\r\n",
      "function _replace_columns!(df::DataFrame, newdf::DataFrame)\r\n",
      "    # for DataFrame object here we do not support keep_present keyword argument\r\n",
      "    # like for SubDataFrame because here transform! always falls back to select!\r\n",
      "    @assert ncol(newdf) == 0 || nrow(df) == nrow(newdf)\r\n",
      "    copy!(_columns(df), _columns(newdf))\r\n",
      "    copy!(_names(index(df)), _names(newdf))\r\n",
      "    copy!(index(df).lookup, index(newdf).lookup)\r\n",
      "\r\n",
      "    emptycolmetadata!(df)\r\n",
      "    for (col, col_keys) in colmetadatakeys(newdf)\r\n",
      "        if hasproperty(df, col)\r\n",
      "            for key in col_keys\r\n",
      "                val, style = colmetadata(newdf, col, key, style=true)\r\n",
      "                style === :note && colmetadata!(df, col, key, val, style=:note)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}; kwargs...) =\r\n",
      "    isempty(kwargs) ? DataFrame() : allcombinations(DataFrame, kwargs...)\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}, pairs::Pair{<:AbstractString, <:Any}...) =\r\n",
      "    allcombinations(DataFrame, (Symbol(k) => v for (k, v) in pairs)...)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    allcombinations(DataFrame, pairs::Pair...)\r\n",
      "    allcombinations(DataFrame; kwargs...)\r\n",
      "\r\n",
      "Create a `DataFrame` from all combinations of values in passed arguments.\r\n",
      "The first passed values vary fastest.\r\n",
      "\r\n",
      "Arguments associating a column name with values to expand can be specified\r\n",
      "either as `Pair`s passed as positional arguments, or as keyword arguments.\r\n",
      "Column names must be `Symbol`s or strings and must be unique.\r\n",
      "\r\n",
      "Column value can be a vector which is consumed as is or an object of any other\r\n",
      "type (except `AbstractArray`). In the latter case the passed value is treated\r\n",
      "as having length one for expansion. As a particular rule values stored in a `Ref`\r\n",
      "or a `0`-dimensional `AbstractArray` are unwrapped and treated as having length one.\r\n",
      "\r\n",
      "See also: [`crossjoin`](@ref) can be used to get the cartesian product\r\n",
      "of rows from passed data frames.\r\n",
      "\r\n",
      "# Examples\r\n",
      "\r\n",
      "```jldoctest\r\n",
      "julia> allcombinations(DataFrame, a=1:2, b='a':'c')\r\n",
      "6×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Char\r\n",
      "─────┼─────────────\r\n",
      "   1 │     1  a\r\n",
      "   2 │     2  a\r\n",
      "   3 │     1  b\r\n",
      "   4 │     2  b\r\n",
      "   5 │     1  c\r\n",
      "   6 │     2  c\r\n",
      "\r\n",
      "julia> allcombinations(DataFrame, \"a\" => 1:2, \"b\" => 'a':'c', \"c\" => \"const\")\r\n",
      "6×3 DataFrame\r\n",
      " Row │ a      b     c\r\n",
      "     │ Int64  Char  String\r\n",
      "─────┼─────────────────────\r\n",
      "   1 │     1  a     const\r\n",
      "   2 │     2  a     const\r\n",
      "   3 │     1  b     const\r\n",
      "   4 │     2  b     const\r\n",
      "   5 │     1  c     const\r\n",
      "   6 │     2  c     const\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function allcombinations(::Type{DataFrame}, pairs::Pair{Symbol, <:Any}...)\r\n",
      "    colnames = first.(pairs)\r\n",
      "    if !allunique(colnames)\r\n",
      "        throw(ArgumentError(\"All column names passed to allcombinations must be unique\"))\r\n",
      "    end\r\n",
      "    colvalues = map(pairs) do p\r\n",
      "        v = last(p)\r\n",
      "        if v isa AbstractVector\r\n",
      "            return v\r\n",
      "        elseif v isa Union{AbstractArray{<:Any, 0}, Ref}\r\n",
      "            x = v[]\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(x), 1), x)\r\n",
      "        elseif v isa AbstractArray\r\n",
      "            throw(ArgumentError(\"adding AbstractArray other than AbstractVector \" *\r\n",
      "                                \"as a column of a data frame is not allowed\"))\r\n",
      "        else\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(v), 1), v)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    @assert length(colvalues) == length(colnames)\r\n",
      "    @assert all(x -> x isa AbstractVector, colvalues)\r\n",
      "\r\n",
      "    target_rows = Int(prod(x -> big(length(x)), colvalues))\r\n",
      "    out_df = DataFrame()\r\n",
      "    inner = 1\r\n",
      "    for (val, cname) in zip(colvalues, colnames)\r\n",
      "        len = length(val)\r\n",
      "        last_inner = inner\r\n",
      "        inner *= len\r\n",
      "        outer, remv = inner == 0 ? (0, 0) : divrem(target_rows, inner)\r\n",
      "        @assert iszero(remv)\r\n",
      "        out_df[!, cname] = repeat(val, inner=last_inner, outer=outer)\r\n",
      "    end\r\n",
      "    @assert inner == target_rows\r\n",
      "    @assert size(out_df) == (target_rows, length(colnames))\r\n",
      "    return out_df\r\n",
      "end\r\n",
      "\r\n",
      "_try_select_no_copy(df::DataFrame, cols) = select(df, cols, copycols=false)\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "@less df[1, Not(1)] = [11, 12, 13] # note @eval in the source code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Why is this needed?\n",
    "Because we are flexible in both row indexing and column indexing options."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is a simple worked example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "f (generic function with 2 methods)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f(x::Union{Float64, Int64}, y::Int64) = 1\n",
    "f(x::Int64, y) = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "MethodError: f(::Int64, ::Int64) is ambiguous.\n\nCandidates:\n  f(\u001b[90mx\u001b[39m::\u001b[1mUnion\u001b[22m\u001b[0m{Float64, Int64}, \u001b[90my\u001b[39m::\u001b[1mInt64\u001b[22m)\n\u001b[90m    @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[49]:1\u001b[24m\u001b[39m\n  f(\u001b[90mx\u001b[39m::\u001b[1mInt64\u001b[22m, \u001b[90my\u001b[39m)\n\u001b[90m    @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[49]:2\u001b[24m\u001b[39m\n\nPossible fix, define\n  f(::Int64, ::Int64)\n",
     "output_type": "error",
     "traceback": [
      "MethodError: f(::Int64, ::Int64) is ambiguous.\n\nCandidates:\n  f(\u001b[90mx\u001b[39m::\u001b[1mUnion\u001b[22m\u001b[0m{Float64, Int64}, \u001b[90my\u001b[39m::\u001b[1mInt64\u001b[22m)\n\u001b[90m    @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[49]:1\u001b[24m\u001b[39m\n  f(\u001b[90mx\u001b[39m::\u001b[1mInt64\u001b[22m, \u001b[90my\u001b[39m)\n\u001b[90m    @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[49]:2\u001b[24m\u001b[39m\n\nPossible fix, define\n  f(::Int64, ::Int64)\n",
      "",
      "Stacktrace:",
      " [1] top-level scope",
      "   @ In[50]:1"
     ]
    }
   ],
   "source": [
    "f(1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "g (generic function with 3 methods)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "for T in (Float64, Int)\n",
    "    @eval g(x::$T, y::Int64) = 1\n",
    "end\n",
    "g(x::Int64, y) = 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "g(1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In more complex scenarios it gets very complicated to ensure that you cover every possible ambiguity (you have to think of a cartesian index of options), so it is simpler to unwrap `Union`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also have a look at this one to see how to define non-standard indices:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>1×4 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x1</th><th style = \"text-align: left;\">x2</th><th style = \"text-align: left;\">x3</th><th style = \"text-align: left;\">x4</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td><td style = \"text-align: right;\">11</td><td style = \"text-align: right;\">12</td><td style = \"text-align: right;\">13</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 11 & 12 & 13 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1     11     12     13"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    @eval function Base.setindex!(df::DataFrame,\r\n",
      "                                  mx::AbstractMatrix,\r\n",
      "                                  row_inds::$T1,\r\n",
      "                                  col_inds::$T2)\r\n",
      "        idxs = index(df)[col_inds]\r\n",
      "        if size(mx, 2) != length(idxs)\r\n",
      "            throw(DimensionMismatch(\"number of selected columns ($(length(idxs))) \" *\r\n",
      "                                    \"and number of columns in \" *\r\n",
      "                                    \"matrix ($(size(mx, 2))) do not match\"))\r\n",
      "        end\r\n",
      "        for (j, col) in enumerate(idxs)\r\n",
      "            # this will drop metadata appropriately\r\n",
      "            df[row_inds, col] = (row_inds === !) ? mx[:, j] : view(mx, :, j)\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    copy(df::DataFrame; copycols::Bool=true)\r\n",
      "\r\n",
      "Copy data frame `df`.\r\n",
      "If `copycols=true` (the default), return a new  `DataFrame` holding\r\n",
      "copies of column vectors in `df`.\r\n",
      "If `copycols=false`, return a new `DataFrame` sharing column vectors with `df`.\r\n",
      "\r\n",
      "Metadata: this function preserves all table-level and column-level metadata.\r\n",
      "\"\"\"\r\n",
      "function Base.copy(df::DataFrame; copycols::Bool=true)\r\n",
      "    cdf = DataFrame(copy(_columns(df)), copy(index(df)), copycols=copycols)\r\n",
      "\r\n",
      "    df_metadata = getfield(df, :metadata)\r\n",
      "    if !isnothing(df_metadata)\r\n",
      "        setfield!(cdf, :metadata, copy(df_metadata))\r\n",
      "    end\r\n",
      "    df_colmetadata = getfield(df, :colmetadata)\r\n",
      "    if !isnothing(df_colmetadata)\r\n",
      "        cdf_colmetadata = copy(df_colmetadata)\r\n",
      "        map!(copy, values(cdf_colmetadata))\r\n",
      "        setfield!(cdf, :colmetadata, cdf_colmetadata)\r\n",
      "    end\r\n",
      "    setfield!(cdf, :allnotemetadata, getfield(df, :allnotemetadata))\r\n",
      "\r\n",
      "    return cdf\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows specified by `inds` from a `DataFrame` `df` in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> deleteat!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.deleteat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, ::Colon) = empty!(df)\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    size(df, 2) == 0 && throw(BoundsError(df, (inds, :)))\r\n",
      "    return _deleteat!_helper(df, Int[inds])\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    if isempty(inds)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    elseif size(df, 2) == 0\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !(eltype(inds) <: Integer || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && inds isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to deleteat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.deleteat!(df::DataFrame, inds::AbstractVector{Bool})\r\n",
      "    if length(inds) != size(df, 1)\r\n",
      "        throw(BoundsError(df, (inds, :)))\r\n",
      "    end\r\n",
      "    # workaround https://github.com/JuliaLang/julia/pull/41646\r\n",
      "    if VERSION <= v\"1.6.2\" && drop isa UnitRange{<:Integer}\r\n",
      "        inds = collect(inds)\r\n",
      "    end\r\n",
      "    return _deleteat!_helper(df, inds)\r\n",
      "end\r\n",
      "\r\n",
      "Base.deleteat!(df::DataFrame, inds::Not) =\r\n",
      "    _deleteat!_helper(df, axes(df, 1)[inds])\r\n",
      "\r\n",
      "function _deleteat!_helper(df::DataFrame, drop)\r\n",
      "    cols = _columns(df)\r\n",
      "    if isempty(cols)\r\n",
      "        _drop_all_nonnote_metadata!(df)\r\n",
      "        return df\r\n",
      "    end\r\n",
      "\r\n",
      "    n = nrow(df)\r\n",
      "    col1 = cols[1]\r\n",
      "    deleteat!(col1, drop)\r\n",
      "    newn = length(col1)\r\n",
      "    @assert newn <= n\r\n",
      "    # the 0.06 threshold is heuristic; based on tests\r\n",
      "    # the assignment makes the code type-unstable but it is a small union\r\n",
      "    # so the overhead should be small\r\n",
      "    if drop isa AbstractVector{Bool} && newn < 0.06 * n\r\n",
      "        drop = findall(drop)\r\n",
      "    end\r\n",
      "    for i in 2:length(cols)\r\n",
      "        col = cols[i]\r\n",
      "        # this check is done to handle column aliases\r\n",
      "        if length(col) == n\r\n",
      "            deleteat!(col, drop)\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    for i in 1:length(cols)\r\n",
      "        # this should never happen, but we add it for safety\r\n",
      "        @assert length(cols[i]) == newn corrupt_msg(df, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "Delete rows at all indices not specified by `inds` from a `DataFrame` `df`\r\n",
      "in place and return it.\r\n",
      "\r\n",
      "Internally `deleteat!` is called for all columns so `inds` must be:\r\n",
      "a vector of sorted and unique integers, a boolean vector, an integer,\r\n",
      "or `Not` wrapping any valid selector.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> keepat!(df, [1, 3])\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "keepat!(df::DataFrame, inds)\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, ::Colon)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::AbstractVector)\r\n",
      "    isempty(inds) && return empty!(df)\r\n",
      "\r\n",
      "    # this is required because of https://github.com/JuliaData/InvertedIndices.jl/issues/31\r\n",
      "    if !((eltype(inds) <: Integer) || all(x -> x isa Integer, inds))\r\n",
      "        throw(ArgumentError(\"unsupported index $inds\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if Bool <: eltype(inds) && any(x -> x isa Bool, inds)\r\n",
      "        throw(ArgumentError(\"invalid index of type Bool\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    if !issorted(inds, lt=<=)\r\n",
      "        throw(ArgumentError(\"Indices passed to keepat! must be unique and sorted\"))\r\n",
      "    end\r\n",
      "\r\n",
      "    return deleteat!(df, Not(inds))\r\n",
      "end\r\n",
      "\r\n",
      "function keepat!(df::DataFrame, inds::Integer)\r\n",
      "    inds isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    return deleteat!(df, Not(Int[inds]))\r\n",
      "end\r\n",
      "\r\n",
      "keepat!(df::DataFrame, inds::AbstractVector{Bool}) = deleteat!(df, .!inds)\r\n",
      "keepat!(df::DataFrame, inds::Not) = deleteat!(df, Not(inds))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    empty!(df::DataFrame)\r\n",
      "\r\n",
      "Remove all rows from `df`, making each of its columns empty.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> empty!(df)\r\n",
      "0×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┴──────────────\r\n",
      "\r\n",
      "julia> df.a, df.b\r\n",
      "(Int64[], Int64[])\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "function Base.empty!(df::DataFrame)\r\n",
      "    foreach(empty!, eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    resize!(df::DataFrame, n::Integer)\r\n",
      "\r\n",
      "Resize `df` to have `n` rows by calling `resize!` on all columns of `df`.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> resize!(df, 2)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.resize!(df::DataFrame, n::Integer)\r\n",
      "    if ncol(df) == 0 && n != 0\r\n",
      "        throw(ArgumentError(\"data frame has no columns and requested number \" *\r\n",
      "                            \"of rows is not zero\"))\r\n",
      "    end\r\n",
      "    foreach(col -> resize!(col, n), eachcol(df))\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    pop!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the last row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> pop!(df)\r\n",
      "(a = 3, b = 6)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "```\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "Base.pop!(df::DataFrame) = popat!(df, nrow(df))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popfirst!(df::DataFrame)\r\n",
      "\r\n",
      "Remove the first row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popfirst!(df)\r\n",
      "(a = 1, b = 4)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     2      5\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "Base.popfirst!(df::DataFrame) = popat!(df, 1)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    popat!(df::DataFrame, i::Integer)\r\n",
      "\r\n",
      "Remove the `i`-th row from `df` and return a `NamedTuple` created from this row.\r\n",
      "\r\n",
      "!!! note\r\n",
      "\r\n",
      "    Using this method for very wide data frames may lead to expensive compilation.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "julia> df = DataFrame(a=1:3, b=4:6)\r\n",
      "3×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     2      5\r\n",
      "   3 │     3      6\r\n",
      "\r\n",
      "julia> popat!(df, 2)\r\n",
      "(a = 2, b = 5)\r\n",
      "\r\n",
      "julia> df\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      4\r\n",
      "   2 │     3      6\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function Base.popat!(df::DataFrame, i::Integer)\r\n",
      "    i isa Bool && throw(ArgumentError(\"Invalid index of type Bool\"))\r\n",
      "    nt = NamedTuple(df[i, :])\r\n",
      "    deleteat!(df, i)\r\n",
      "    return nt\r\n",
      "end\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## hcat!\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\r\n",
      "# hcat! for 2 arguments, only a vector or a data frame is allowed\r\n",
      "function hcat!(df1::DataFrame, df2::AbstractDataFrame;\r\n",
      "               makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    u = add_names(index(df1), index(df2), makeunique=makeunique)\r\n",
      "\r\n",
      "    _drop_all_nonnote_metadata!(df1)\r\n",
      "    _keep_matching_table_note_metadata!(df1, df2)\r\n",
      "    for i in 1:length(u)\r\n",
      "        df1[!, u[i]] = copycols ? df2[:, i] : df2[!, i]\r\n",
      "        _copy_col_note_metadata!(df1, u[i], df2, i)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df1\r\n",
      "end\r\n",
      "\r\n",
      "# TODO: after deprecation remove AbstractVector methods\r\n",
      "\r\n",
      "function hcat!(df::DataFrame, x::AbstractVector; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(df, DataFrame(AbstractVector[x], [:x1], copycols=false),\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "function hcat!(x::AbstractVector, df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    Base.depwarn(\"horizontal concatenation of data frame with a vector is deprecated. \" *\r\n",
      "                 \"Pass DataFrame(x1=x) instead.\", :hcat!)\r\n",
      "    return hcat!(DataFrame(AbstractVector[x], [:x1], copycols=copycols), df,\r\n",
      "                 makeunique=makeunique, copycols=copycols)\r\n",
      "end\r\n",
      "\r\n",
      "# hcat! for 1-n arguments\r\n",
      "function hcat!(df::DataFrame; makeunique::Bool=false, copycols::Bool=true)\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "hcat!(a::DataFrame, b::Union{AbstractDataFrame, AbstractVector},\r\n",
      "      c::Union{AbstractDataFrame, AbstractVector}...;\r\n",
      "      makeunique::Bool=false, copycols::Bool=true) =\r\n",
      "    hcat!(hcat!(a, b, makeunique=makeunique, copycols=copycols),\r\n",
      "          c..., makeunique=makeunique, copycols=copycols)\r\n",
      "\r\n",
      "##############################################################################\r\n",
      "##\r\n",
      "## Missing values support\r\n",
      "##\r\n",
      "##############################################################################\r\n",
      "\"\"\"\r\n",
      "    allowmissing!(df::DataFrame, cols=:)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `T` to\r\n",
      "`Union{T, Missing}` to support missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function allowmissing! end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, col::ColumnIndex)\r\n",
      "    df[!, col] = allowmissing(df[!, col])\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex})\r\n",
      "    for col in cols\r\n",
      "        allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function allowmissing!(df::DataFrame, cols::AbstractVector{Bool})\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && allowmissing!(df, col)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::MultiColumnIndex) =\r\n",
      "    allowmissing!(df, index(df)[cols])\r\n",
      "\r\n",
      "allowmissing!(df::DataFrame, cols::Colon=:) =\r\n",
      "    allowmissing!(df, axes(df, 2))\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    disallowmissing!(df::DataFrame, cols=:; error::Bool=true)\r\n",
      "\r\n",
      "Convert columns `cols` of data frame `df` from element type `Union{T, Missing}` to\r\n",
      "`T` to drop support for missing values.\r\n",
      "\r\n",
      "`cols` can be any column selector ($COLUMNINDEX_STR; $MULTICOLUMNINDEX_STR).\r\n",
      "\r\n",
      "If `cols` is omitted all columns in the data frame are converted.\r\n",
      "\r\n",
      "If `error=false` then columns containing a `missing` value will be skipped instead\r\n",
      "of throwing an error.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\"\"\"\r\n",
      "function disallowmissing! end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, col::ColumnIndex; error::Bool=true)\r\n",
      "    x = df[!, col]\r\n",
      "    if Missing <: eltype(x)\r\n",
      "        try\r\n",
      "            df[!, col] = disallowmissing(x)\r\n",
      "        catch e\r\n",
      "            row = findfirst(ismissing, x)\r\n",
      "            if row !== nothing\r\n",
      "                if error\r\n",
      "                    col_name = only(names(df, col))\r\n",
      "                    throw(ArgumentError(\"Missing value found in column \" *\r\n",
      "                                        \":$col_name in row $row\"))\r\n",
      "                end\r\n",
      "            else\r\n",
      "                rethrow(e)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{<:ColumnIndex};\r\n",
      "                          error::Bool=true)\r\n",
      "    for col in cols\r\n",
      "        disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function disallowmissing!(df::DataFrame, cols::AbstractVector{Bool}; error::Bool=true)\r\n",
      "    length(cols) == size(df, 2) || throw(BoundsError(df, (!, cols)))\r\n",
      "    for (col, cond) in enumerate(cols)\r\n",
      "        cond && disallowmissing!(df, col, error=error)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::MultiColumnIndex; error::Bool=true) =\r\n",
      "    disallowmissing!(df, index(df)[cols], error=error)\r\n",
      "\r\n",
      "disallowmissing!(df::DataFrame, cols::Colon=:; error::Bool=true) =\r\n",
      "    disallowmissing!(df, axes(df, 2), error=error)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows. `inner` specifies how many\r\n",
      "times each row is repeated, and `outer` specifies how many times the full set\r\n",
      "of rows is repeated. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat!(df, inner=2, outer=3);\r\n",
      "\r\n",
      "julia> df\r\n",
      "12×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     1      3\r\n",
      "   3 │     2      4\r\n",
      "   4 │     2      4\r\n",
      "   5 │     1      3\r\n",
      "   6 │     1      3\r\n",
      "   7 │     2      4\r\n",
      "   8 │     2      4\r\n",
      "   9 │     1      3\r\n",
      "  10 │     1      3\r\n",
      "  11 │     2      4\r\n",
      "  12 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame; inner::Integer=1, outer::Integer=1)\r\n",
      "    inner < 0 && throw(ArgumentError(\"inner keyword argument must be non-negative\"))\r\n",
      "    outer < 0 && throw(ArgumentError(\"outer keyword argument must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, inner=Int(inner), outer=Int(outer))\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    repeat!(df::DataFrame, count::Integer)\r\n",
      "\r\n",
      "Update a data frame `df` in-place by repeating its rows the number of times\r\n",
      "specified by `count`. Columns of `df` are freshly allocated.\r\n",
      "\r\n",
      "$METADATA_FIXED\r\n",
      "\r\n",
      "# Example\r\n",
      "```jldoctest\r\n",
      "julia> df = DataFrame(a=1:2, b=3:4)\r\n",
      "2×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "\r\n",
      "julia> repeat(df, 2)\r\n",
      "4×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Int64\r\n",
      "─────┼──────────────\r\n",
      "   1 │     1      3\r\n",
      "   2 │     2      4\r\n",
      "   3 │     1      3\r\n",
      "   4 │     2      4\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function repeat!(df::DataFrame, count::Integer)\r\n",
      "    count < 0 && throw(ArgumentError(\"count must be non-negative\"))\r\n",
      "    cols = _columns(df)\r\n",
      "    for (i, col) in enumerate(cols)\r\n",
      "        col_new = repeat(col, count)\r\n",
      "        firstindex(col_new) != 1 && _onebased_check_error(i, col_new)\r\n",
      "        cols[i] = col_new\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "# This is not exactly copy! as in general we allow axes to be different\r\n",
      "# Also no table metadata needs to be copied as we use _replace_columns!\r\n",
      "# only in situations when table metadata for `df` must be left as-is\r\n",
      "function _replace_columns!(df::DataFrame, newdf::DataFrame)\r\n",
      "    # for DataFrame object here we do not support keep_present keyword argument\r\n",
      "    # like for SubDataFrame because here transform! always falls back to select!\r\n",
      "    @assert ncol(newdf) == 0 || nrow(df) == nrow(newdf)\r\n",
      "    copy!(_columns(df), _columns(newdf))\r\n",
      "    copy!(_names(index(df)), _names(newdf))\r\n",
      "    copy!(index(df).lookup, index(newdf).lookup)\r\n",
      "\r\n",
      "    emptycolmetadata!(df)\r\n",
      "    for (col, col_keys) in colmetadatakeys(newdf)\r\n",
      "        if hasproperty(df, col)\r\n",
      "            for key in col_keys\r\n",
      "                val, style = colmetadata(newdf, col, key, style=true)\r\n",
      "                style === :note && colmetadata!(df, col, key, val, style=:note)\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(df)\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}; kwargs...) =\r\n",
      "    isempty(kwargs) ? DataFrame() : allcombinations(DataFrame, kwargs...)\r\n",
      "\r\n",
      "allcombinations(::Type{DataFrame}, pairs::Pair{<:AbstractString, <:Any}...) =\r\n",
      "    allcombinations(DataFrame, (Symbol(k) => v for (k, v) in pairs)...)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    allcombinations(DataFrame, pairs::Pair...)\r\n",
      "    allcombinations(DataFrame; kwargs...)\r\n",
      "\r\n",
      "Create a `DataFrame` from all combinations of values in passed arguments.\r\n",
      "The first passed values vary fastest.\r\n",
      "\r\n",
      "Arguments associating a column name with values to expand can be specified\r\n",
      "either as `Pair`s passed as positional arguments, or as keyword arguments.\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Column names must be `Symbol`s or strings and must be unique.\r\n",
      "\r\n",
      "Column value can be a vector which is consumed as is or an object of any other\r\n",
      "type (except `AbstractArray`). In the latter case the passed value is treated\r\n",
      "as having length one for expansion. As a particular rule values stored in a `Ref`\r\n",
      "or a `0`-dimensional `AbstractArray` are unwrapped and treated as having length one.\r\n",
      "\r\n",
      "See also: [`crossjoin`](@ref) can be used to get the cartesian product\r\n",
      "of rows from passed data frames.\r\n",
      "\r\n",
      "# Examples\r\n",
      "\r\n",
      "```jldoctest\r\n",
      "julia> allcombinations(DataFrame, a=1:2, b='a':'c')\r\n",
      "6×2 DataFrame\r\n",
      " Row │ a      b\r\n",
      "     │ Int64  Char\r\n",
      "─────┼─────────────\r\n",
      "   1 │     1  a\r\n",
      "   2 │     2  a\r\n",
      "   3 │     1  b\r\n",
      "   4 │     2  b\r\n",
      "   5 │     1  c\r\n",
      "   6 │     2  c\r\n",
      "\r\n",
      "julia> allcombinations(DataFrame, \"a\" => 1:2, \"b\" => 'a':'c', \"c\" => \"const\")\r\n",
      "6×3 DataFrame\r\n",
      " Row │ a      b     c\r\n",
      "     │ Int64  Char  String\r\n",
      "─────┼─────────────────────\r\n",
      "   1 │     1  a     const\r\n",
      "   2 │     2  a     const\r\n",
      "   3 │     1  b     const\r\n",
      "   4 │     2  b     const\r\n",
      "   5 │     1  c     const\r\n",
      "   6 │     2  c     const\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "function allcombinations(::Type{DataFrame}, pairs::Pair{Symbol, <:Any}...)\r\n",
      "    colnames = first.(pairs)\r\n",
      "    if !allunique(colnames)\r\n",
      "        throw(ArgumentError(\"All column names passed to allcombinations must be unique\"))\r\n",
      "    end\r\n",
      "    colvalues = map(pairs) do p\r\n",
      "        v = last(p)\r\n",
      "        if v isa AbstractVector\r\n",
      "            return v\r\n",
      "        elseif v isa Union{AbstractArray{<:Any, 0}, Ref}\r\n",
      "            x = v[]\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(x), 1), x)\r\n",
      "        elseif v isa AbstractArray\r\n",
      "            throw(ArgumentError(\"adding AbstractArray other than AbstractVector \" *\r\n",
      "                                \"as a column of a data frame is not allowed\"))\r\n",
      "        else\r\n",
      "            return fill!(Tables.allocatecolumn(typeof(v), 1), v)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    @assert length(colvalues) == length(colnames)\r\n",
      "    @assert all(x -> x isa AbstractVector, colvalues)\r\n",
      "\r\n",
      "    target_rows = Int(prod(x -> big(length(x)), colvalues))\r\n",
      "    out_df = DataFrame()\r\n",
      "    inner = 1\r\n",
      "    for (val, cname) in zip(colvalues, colnames)\r\n",
      "        len = length(val)\r\n",
      "        last_inner = inner\r\n",
      "        inner *= len\r\n",
      "        outer, remv = inner == 0 ? (0, 0) : divrem(target_rows, inner)\r\n",
      "        @assert iszero(remv)\r\n",
      "        out_df[!, cname] = repeat(val, inner=last_inner, outer=outer)\r\n",
      "    end\r\n",
      "    @assert inner == target_rows\r\n",
      "    @assert size(out_df) == (target_rows, length(colnames))\r\n",
      "    return out_df\r\n",
      "end\r\n",
      "\r\n",
      "_try_select_no_copy(df::DataFrame, cols) = select(df, cols, copycols=false)\r\n",
      "\r\n"
     ]
    }
   ],
   "source": [
    "@less df[:, :] = rand(Int, 1, 4) # note how `!` or `Not` are referenced to"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 4: defining broadcasting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your type should support `CartesianIndex` indexing because it later can get used in broadcasting mechanics (which was not obvious for me initially)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Base.setindex!(df::AbstractDataFrame, val, idx::CartesianIndex{2}) =\r\n",
      "    (df[idx[1], idx[2]] = val)\r\n",
      "\r\n",
      "Base.broadcastable(df::AbstractDataFrame) = df\r\n",
      "\r\n",
      "struct DataFrameStyle <: Base.Broadcast.BroadcastStyle end\r\n",
      "\r\n",
      "Base.Broadcast.BroadcastStyle(::Type{<:AbstractDataFrame}) =\r\n",
      "    DataFrameStyle()\r\n",
      "\r\n",
      "Base.Broadcast.BroadcastStyle(::DataFrameStyle, ::Base.Broadcast.BroadcastStyle) =\r\n",
      "    DataFrameStyle()\r\n",
      "Base.Broadcast.BroadcastStyle(::Base.Broadcast.BroadcastStyle, ::DataFrameStyle) =\r\n",
      "    DataFrameStyle()\r\n",
      "Base.Broadcast.BroadcastStyle(::DataFrameStyle, ::DataFrameStyle) = DataFrameStyle()\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.BroadcastStyle(::DataFrameStyle, ::Base.Broadcast.Unknown) =\r\n",
      "    DataFrameStyle()\r\n",
      "\r\n",
      "function copyto_widen!(res::AbstractVector{T}, bc::Base.Broadcast.Broadcasted,\r\n",
      "                       pos, col) where T\r\n",
      "    for i in pos:length(axes(bc)[1])\r\n",
      "        val = bc[CartesianIndex(i, col)]\r\n",
      "        S = typeof(val)\r\n",
      "        if S <: T || promote_type(S, T) <: T\r\n",
      "            res[i] = val\r\n",
      "        else\r\n",
      "            newres = similar(Vector{promote_type(S, T)}, length(res))\r\n",
      "            copyto!(newres, 1, res, 1, i-1)\r\n",
      "            newres[i] = val\r\n",
      "            return copyto_widen!(newres, bc, i + 1, col)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return res\r\n",
      "end\r\n",
      "\r\n",
      "function getcolbc(bcf::Base.Broadcast.Broadcasted{Style}, colind) where {Style}\r\n",
      "    # we assume that bcf is already flattened and unaliased\r\n",
      "    newargs = map(bcf.args) do x\r\n",
      "        Base.Broadcast.extrude(x isa AbstractDataFrame ? x[!, colind] : x)\r\n",
      "    end\r\n",
      "    return Base.Broadcast.Broadcasted{Style}(bcf.f, newargs, bcf.axes)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copy(bc::Base.Broadcast.Broadcasted{DataFrameStyle})\r\n",
      "    ndim = length(axes(bc))\r\n",
      "    if ndim != 2\r\n",
      "        throw(DimensionMismatch(\"cannot broadcast a data frame into $ndim dimensions\"))\r\n",
      "    end\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(df) for df in bcf.args if df isa AbstractDataFrame])\r\n",
      "    if length(colnames) != 1\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "    nrows = length(axes(bcf)[1])\r\n",
      "    df = DataFrame()\r\n",
      "    for i in axes(bcf)[2]\r\n",
      "        if nrows == 0\r\n",
      "            col = Any[]\r\n",
      "        else\r\n",
      "            bcf′ = getcolbc(bcf, i)\r\n",
      "            v1 = bcf′[CartesianIndex(1, i)]\r\n",
      "            startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "            startcol[1] = v1\r\n",
      "            col = copyto_widen!(startcol, bcf′, 2, i)\r\n",
      "        end\r\n",
      "        df[!, colnames[1][i]] = col\r\n",
      "    end\r\n",
      "\r\n",
      "    dfs = AbstractDataFrame[df for df in bcf.args if df isa AbstractDataFrame]\r\n",
      "    @assert !isempty(dfs)\r\n",
      "\r\n",
      "    _merge_matching_table_note_metadata!(df, dfs)\r\n",
      "\r\n",
      "    if all(x -> !isempty(colmetadatakeys(x)), dfs)\r\n",
      "        for colname in _names(df)\r\n",
      "            if length(dfs) == 1\r\n",
      "                _copy_col_note_metadata!(df, colname, only(dfs), colname)\r\n",
      "            else\r\n",
      "                if all(x -> !isempty(colmetadatakeys(x, colname)), dfs)\r\n",
      "                    for key1 in colmetadatakeys(dfs[1], colname)\r\n",
      "                        val1, style1 = colmetadata(dfs[1], colname, key1, style=true)\r\n",
      "                        if style1 === :note\r\n",
      "                            add_meta = true\r\n",
      "                            for i in 2:length(dfs)\r\n",
      "                                if key1 in colmetadatakeys(dfs[i], colname)\r\n",
      "                                    vali, stylei = colmetadata(dfs[i], colname, key1, style=true)\r\n",
      "                                    if !(stylei === :note && isequal(val1, vali))\r\n",
      "                                        add_meta = false\r\n",
      "                                        break\r\n",
      "                                    end\r\n",
      "                                else\r\n",
      "                                    add_meta = false\r\n",
      "                                    break\r\n",
      "                                end\r\n",
      "                            end\r\n",
      "                            add_meta && colmetadata!(df, colname, key1, val1, style=:note)\r\n",
      "                        end\r\n",
      "                    end\r\n",
      "                end\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "### Broadcasting assignment\r\n",
      "\r\n",
      "struct LazyNewColDataFrame{T,D}\r\n",
      "    df::D\r\n",
      "    col::T\r\n",
      "end\r\n",
      "\r\n",
      "Base.axes(x::LazyNewColDataFrame) = (Base.OneTo(nrow(x.df)),)\r\n",
      "Base.ndims(::Type{<:LazyNewColDataFrame}) = 1\r\n",
      "\r\n",
      "struct ColReplaceDataFrame{T<:AbstractDataFrame}\r\n",
      "    df::T\r\n",
      "    cols::Vector{Int}\r\n",
      "end\r\n",
      "\r\n",
      "Base.axes(x::ColReplaceDataFrame) = (axes(x.df, 1), Base.OneTo(length(x.cols)))\r\n",
      "Base.ndims(::Type{<:ColReplaceDataFrame}) = 2\r\n",
      "\r\n",
      "# In the functions below we need to call _drop_all_nonnote_metadata!\r\n",
      "# upfront as the rest of the operations is handled by Base Julia\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, idx::CartesianIndex{2})\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df[idx]\r\n",
      "end\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, row::Integer, col::ColumnIndex)\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df[row, col]\r\n",
      "end\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, rows, cols)\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return view(df, rows, cols)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.dotview(df::AbstractDataFrame, ::Colon, cols::ColumnIndex)\r\n",
      "    if haskey(index(df), cols)\r\n",
      "        _drop_all_nonnote_metadata!(parent(df))\r\n",
      "        return view(df, :, cols)\r\n",
      "    end\r\n",
      "    if !(cols isa SymbolOrString)\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    if !is_column_insertion_allowed(df)\r\n",
      "        throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                            \"columns of its parent data frame is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, Symbol(cols))\r\n",
      "end\r\n",
      "\r\n",
      "function Base.dotview(df::AbstractDataFrame, ::typeof(!), cols)\r\n",
      "    if !(cols isa ColumnIndex)\r\n",
      "        return ColReplaceDataFrame(df, convert(Vector{Int}, index(df)[cols]))\r\n",
      "    end\r\n",
      "    if cols isa SymbolOrString\r\n",
      "        if columnindex(df, cols) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "    elseif !(1 <= cols <= ncol(df))\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, cols isa AbstractString ? Symbol(cols) : cols)\r\n",
      "end\r\n",
      "\r\n",
      "if isdefined(Base, :dotgetproperty) # Introduced in Julia 1.7\r\n",
      "    function Base.dotgetproperty(df::AbstractDataFrame, col::SymbolOrString)\r\n",
      "        if columnindex(df, col) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "        return LazyNewColDataFrame(df, Symbol(col))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(lazydf::LazyNewColDataFrame, bc::Base.Broadcast.Broadcasted{T}) where T\r\n",
      "    df = lazydf.df\r\n",
      "    if !haskey(index(df), lazydf.col) && df isa SubDataFrame && lazydf.col isa SymbolOrString\r\n",
      "        @assert is_column_insertion_allowed(df)\r\n",
      "    end\r\n",
      "    if bc isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "        bc_tmp = Base.Broadcast.Broadcasted{T}(bc.f, bc.args, ())\r\n",
      "        v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "        col = similar(Vector{typeof(v)}, nrow(df))\r\n",
      "        copyto!(col, bc)\r\n",
      "    else\r\n",
      "        col = Base.Broadcast.materialize(bc)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df[!, lazydf.col] = col\r\n",
      "end\r\n",
      "\r\n",
      "function _copyto_helper!(dfcol::AbstractVector, bc::Base.Broadcast.Broadcasted, col::Int)\r\n",
      "    if axes(dfcol, 1) != axes(bc)[1]\r\n",
      "        # this should never happen unless data frame is corrupted (has unequal column lengths)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting. The updated\" *\r\n",
      "                                \" data frame is invalid and should not be used\"))\r\n",
      "    end\r\n",
      "    @inbounds for row in eachindex(dfcol)\r\n",
      "        dfcol[row] = bc[CartesianIndex(row, col)]\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src)\r\n",
      "    for col in eachcol(dest)\r\n",
      "        src = Base.Broadcast.unalias(col, src)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.broadcast_unalias(::Nothing, src::AbstractDataFrame) = src\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest, src::AbstractDataFrame)\r\n",
      "    wascopied = false\r\n",
      "    for (i, col) in enumerate(eachcol(src))\r\n",
      "        if Base.mightalias(dest, col)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src = SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                       index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), i)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, i] = Base.unaliascopy(col)\r\n",
      "            end\r\n",
      "            wascopied = true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function _broadcast_unalias_helper(dest::AbstractDataFrame, scol::AbstractVector,\r\n",
      "                                   src::AbstractDataFrame, col2::Int, wascopied::Bool)\r\n",
      "    # col1 can be checked till col2 point as we are writing broadcasting\r\n",
      "    # results from 1 to ncol\r\n",
      "    # we go downwards because aliasing when col1 == col2 is most probable\r\n",
      "    for col1 in col2:-1:1\r\n",
      "        dcol = dest[!, col1]\r\n",
      "        if Base.mightalias(dcol, scol)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src =SubDataFrame(copy(parent(src), copycols=false),\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                      index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), col2)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, col2] = Base.unaliascopy(scol)\r\n",
      "            end\r\n",
      "            return src, true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src, wascopied\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src::AbstractDataFrame)\r\n",
      "    if size(dest, 2) != size(src, 2)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting.\"))\r\n",
      "    end\r\n",
      "    wascopied = false\r\n",
      "    for col2 in axes(dest, 2)\r\n",
      "        scol = src[!, col2]\r\n",
      "        src, wascopied = _broadcast_unalias_helper(dest, scol, src, col2, wascopied)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n",
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "@less df[CartesianIndex(1, 1)] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also below you can see how we force broadcasting to make sure the result is a `DataFrame` using `BroadcastStyle`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now in order for broadcasting to overcome the problem that `DataFrame` column access is not type stable we have to process it column by column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "f (generic function with 3 methods)"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "f(df) = df .+ 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MethodInstance for f(::DataFrame)\n",
      "  from f(\u001b[90mdf\u001b[39m)\u001b[90m @\u001b[39m \u001b[90mMain\u001b[39m \u001b[90m\u001b[4mIn[56]:1\u001b[24m\u001b[39m\n",
      "Arguments\n",
      "  #self#\u001b[36m::Core.Const(f)\u001b[39m\n",
      "  df\u001b[36m::DataFrame\u001b[39m\n",
      "Body\u001b[36m::DataFrame\u001b[39m\n",
      "\u001b[90m1 ─\u001b[39m %1 = Base.broadcasted(Main.:+, df, 1)\u001b[36m::Core.PartialStruct(Base.Broadcast.Broadcasted{DataFrames.DataFrameStyle, Nothing, typeof(+), Tuple{DataFrame, Int64}}, Any[Core.Const(+), Core.PartialStruct(Tuple{DataFrame, Int64}, Any[DataFrame, Core.Const(1)]), Core.Const(nothing)])\u001b[39m\n",
      "\u001b[90m│  \u001b[39m %2 = Base.materialize(%1)\u001b[36m::DataFrame\u001b[39m\n",
      "\u001b[90m└──\u001b[39m      return %2\n",
      "\n"
     ]
    }
   ],
   "source": [
    "@code_warntype f(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@inline materialize(bc::Broadcasted) = copy(instantiate(bc))\r\n",
      "materialize(x) = x\r\n",
      "\r\n",
      "@inline function materialize!(dest, x)\r\n",
      "    return materialize!(dest, instantiate(Broadcasted(identity, (x,), axes(dest))))\r\n",
      "end\r\n",
      "\r\n",
      "@inline function materialize!(dest, bc::Broadcasted{Style}) where {Style}\r\n",
      "    return materialize!(combine_styles(dest, bc), dest, bc)\r\n",
      "end\r\n",
      "@inline function materialize!(::BroadcastStyle, dest, bc::Broadcasted{Style}) where {Style}\r\n",
      "    return copyto!(dest, instantiate(Broadcasted{Style}(bc.f, bc.args, axes(dest))))\r\n",
      "end\r\n",
      "\r\n",
      "## general `copy` methods\r\n",
      "@inline copy(bc::Broadcasted{<:AbstractArrayStyle{0}}) = bc[CartesianIndex()]\r\n",
      "copy(bc::Broadcasted{<:Union{Nothing,Unknown}}) =\r\n",
      "    throw(ArgumentError(\"broadcasting requires an assigned BroadcastStyle\"))\r\n",
      "\r\n",
      "const NonleafHandlingStyles = Union{DefaultArrayStyle,ArrayConflict}\r\n",
      "\r\n",
      "@inline function copy(bc::Broadcasted{Style}) where {Style}\r\n",
      "    ElType = combine_eltypes(bc.f, bc.args)\r\n",
      "    if Base.isconcretetype(ElType)\r\n",
      "        # We can trust it and defer to the simpler `copyto!`\r\n",
      "        return copyto!(similar(bc, ElType), bc)\r\n",
      "    end\r\n",
      "    # When ElType is not concrete, use narrowing. Use the first output\r\n",
      "    # value to determine the starting output eltype; copyto_nonleaf!\r\n",
      "    # will widen `dest` as needed to accommodate later values.\r\n",
      "    bc′ = preprocess(nothing, bc)\r\n",
      "    iter = eachindex(bc′)\r\n",
      "    y = iterate(iter)\r\n",
      "    if y === nothing\r\n",
      "        # if empty, take the ElType at face value\r\n",
      "        return similar(bc′, ElType)\r\n",
      "    end\r\n",
      "    # Initialize using the first value\r\n",
      "    I, state = y\r\n",
      "    @inbounds val = bc′[I]\r\n",
      "    dest = similar(bc′, typeof(val))\r\n",
      "    @inbounds dest[I] = val\r\n",
      "    # Now handle the remaining values\r\n",
      "    # The typeassert gives inference a helping hand on the element type and dimensionality\r\n",
      "    # (work-around for #28382)\r\n",
      "    ElType′ = ElType === Union{} ? Any : ElType <: Type ? Type : ElType\r\n",
      "    RT = dest isa AbstractArray ? AbstractArray{<:ElType′, ndims(dest)} : Any\r\n",
      "    return copyto_nonleaf!(dest, bc′, iter, state, 1)::RT\r\n",
      "end\r\n",
      "\r\n",
      "## general `copyto!` methods\r\n",
      "# The most general method falls back to a method that replaces Style->Nothing\r\n",
      "# This permits specialization on typeof(dest) without introducing ambiguities\r\n",
      "@inline copyto!(dest::AbstractArray, bc::Broadcasted) = copyto!(dest, convert(Broadcasted{Nothing}, bc))\r\n",
      "\r\n",
      "# Performance optimization for the common identity scalar case: dest .= val\r\n",
      "@inline function copyto!(dest::AbstractArray, bc::Broadcasted{<:AbstractArrayStyle{0}})\r\n",
      "    # Typically, we must independently execute bc for every storage location in `dest`, but:\r\n",
      "    # IF we're in the common no-op identity case with no nested args (like `dest .= val`),\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && isflat(bc)\r\n",
      "        # THEN we can just extract the argument and `fill!` the destination with it\r\n",
      "        return fill!(dest, bc.args[1][])\r\n",
      "    else\r\n",
      "        # Otherwise, fall back to the default implementation like above\r\n",
      "        return copyto!(dest, convert(Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "# For broadcasted assignments like `broadcast!(f, A, ..., A, ...)`, where `A`\r\n",
      "# appears on both the LHS and the RHS of the `.=`, then we know we're only\r\n",
      "# going to make one pass through the array, and even though `A` is aliasing\r\n",
      "# against itself, the mutations won't affect the result as the indices on the\r\n",
      "# LHS and RHS will always match. This is not true in general, but with the `.op=`\r\n",
      "# syntax it's fairly common for an argument to be `===` a source.\r\n",
      "broadcast_unalias(dest, src) = dest === src ? src : unalias(dest, src)\r\n",
      "broadcast_unalias(::Nothing, src) = src\r\n",
      "\r\n",
      "# Preprocessing a `Broadcasted` does two things:\r\n",
      "# * unaliases any arguments from `dest`\r\n",
      "# * \"extrudes\" the arguments where it is advantageous to pre-compute the broadcasted indices\r\n",
      "@inline preprocess(dest, bc::Broadcasted{Style}) where {Style} = Broadcasted{Style}(bc.f, preprocess_args(dest, bc.args), bc.axes)\r\n",
      "preprocess(dest, x) = extrude(broadcast_unalias(dest, x))\r\n",
      "\r\n",
      "@inline preprocess_args(dest, args::Tuple) = (preprocess(dest, args[1]), preprocess_args(dest, tail(args))...)\r\n",
      "@inline preprocess_args(dest, args::Tuple{Any}) = (preprocess(dest, args[1]),)\r\n",
      "@inline preprocess_args(dest, args::Tuple{}) = ()\r\n",
      "\r\n",
      "# Specialize this method if all you want to do is specialize on typeof(dest)\r\n",
      "@inline function copyto!(dest::AbstractArray, bc::Broadcasted{Nothing})\r\n",
      "    axes(dest) == axes(bc) || throwdm(axes(dest), axes(bc))\r\n",
      "    # Performance optimization: broadcast!(identity, dest, A) is equivalent to copyto!(dest, A) if indices match\r\n",
      "    if bc.f === identity && bc.args isa Tuple{AbstractArray} # only a single input argument to broadcast!\r\n",
      "        A = bc.args[1]\r\n",
      "        if axes(dest) == axes(A)\r\n",
      "            return copyto!(dest, A)\r\n",
      "        end\r\n",
      "    end\r\n",
      "    bc′ = preprocess(dest, bc)\r\n",
      "    # Performance may vary depending on whether `@inbounds` is placed outside the\r\n",
      "    # for loop or not. (cf. https://github.com/JuliaLang/julia/issues/38086)\r\n",
      "    @inbounds @simd for I in eachindex(bc′)\r\n",
      "        dest[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "# Performance optimization: for BitArray outputs, we cache the result\r\n",
      "# in a \"small\" Vector{Bool}, and then copy in chunks into the output\r\n",
      "@inline function copyto!(dest::BitArray, bc::Broadcasted{Nothing})\r\n",
      "    axes(dest) == axes(bc) || throwdm(axes(dest), axes(bc))\r\n",
      "    ischunkedbroadcast(dest, bc) && return chunkedcopyto!(dest, bc)\r\n",
      "    length(dest) < 256 && return invoke(copyto!, Tuple{AbstractArray, Broadcasted{Nothing}}, dest, bc)\r\n",
      "    tmp = Vector{Bool}(undef, bitcache_size)\r\n",
      "    destc = dest.chunks\r\n",
      "    cind = 1\r\n",
      "    bc′ = preprocess(dest, bc)\r\n",
      "    @inbounds for P in Iterators.partition(eachindex(bc′), bitcache_size)\r\n",
      "        ind = 1\r\n",
      "        @simd for I in P\r\n",
      "            tmp[ind] = bc′[I]\r\n",
      "            ind += 1\r\n",
      "        end\r\n",
      "        @simd for i in ind:bitcache_size\r\n",
      "            tmp[i] = false\r\n",
      "        end\r\n",
      "        dumpbitcache(destc, cind, tmp)\r\n",
      "        cind += bitcache_chunks\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "# For some BitArray operations, we can work at the level of chunks. The trivial\r\n",
      "# implementation just walks over the UInt64 chunks in a linear fashion.\r\n",
      "# This requires three things:\r\n",
      "#   1. The function must be known to work at the level of chunks (or can be converted to do so)\r\n",
      "#   2. The only arrays involved must be BitArrays or scalar Bools\r\n",
      "#   3. There must not be any broadcasting beyond scalar — all array sizes must match\r\n",
      "# We could eventually allow for all broadcasting and other array types, but that\r\n",
      "# requires very careful consideration of all the edge effects.\r\n",
      "const ChunkableOp = Union{typeof(&), typeof(|), typeof(xor), typeof(~), typeof(identity),\r\n",
      "    typeof(!), typeof(*), typeof(==)} # these are convertible to chunkable ops by liftfuncs\r\n",
      "const BroadcastedChunkableOp{Style<:Union{Nothing,BroadcastStyle}, Axes, F<:ChunkableOp, Args<:Tuple} = Broadcasted{Style,Axes,F,Args}\r\n",
      "ischunkedbroadcast(R, bc::BroadcastedChunkableOp) = ischunkedbroadcast(R, bc.args)\r\n",
      "ischunkedbroadcast(R, args) = false\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:BitArray,Vararg{Any}}) = size(R) == size(args[1]) && ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:Bool,Vararg{Any}}) = ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{<:BroadcastedChunkableOp,Vararg{Any}}) = ischunkedbroadcast(R, args[1]) && ischunkedbroadcast(R, tail(args))\r\n",
      "ischunkedbroadcast(R, args::Tuple{}) = true\r\n",
      "\r\n",
      "# Convert compatible functions to chunkable ones. They must also be green-lighted as ChunkableOps\r\n",
      "liftfuncs(bc::Broadcasted{Style}) where {Style} = Broadcasted{Style}(bc.f, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(sign)}) where {Style} = Broadcasted{Style}(identity, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(!)}) where {Style} = Broadcasted{Style}(~, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(*)}) where {Style} = Broadcasted{Style}(&, map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(bc::Broadcasted{Style,<:Any,typeof(==)}) where {Style} = Broadcasted{Style}((~)∘(xor), map(liftfuncs, bc.args), bc.axes)\r\n",
      "liftfuncs(x) = x\r\n",
      "\r\n",
      "liftchunks(::Tuple{}) = ()\r\n",
      "liftchunks(args::Tuple{<:BitArray,Vararg{Any}}) = (args[1].chunks, liftchunks(tail(args))...)\r\n",
      "# Transform scalars to repeated scalars the size of a chunk\r\n",
      "liftchunks(args::Tuple{<:Bool,Vararg{Any}}) = (ifelse(args[1], typemax(UInt64), UInt64(0)), liftchunks(tail(args))...)\r\n",
      "ithchunk(i) = ()\r\n",
      "Base.@propagate_inbounds ithchunk(i, c::Vector{UInt64}, args...) = (c[i], ithchunk(i, args...)...)\r\n",
      "Base.@propagate_inbounds ithchunk(i, b::UInt64, args...) = (b, ithchunk(i, args...)...)\r\n",
      "@inline function chunkedcopyto!(dest::BitArray, bc::Broadcasted)\r\n",
      "    isempty(dest) && return dest\r\n",
      "    f = flatten(liftfuncs(bc))\r\n",
      "    args = liftchunks(f.args)\r\n",
      "    dc = dest.chunks\r\n",
      "    @simd for i in eachindex(dc)\r\n",
      "        @inbounds dc[i] = f.f(ithchunk(i, args...)...)\r\n",
      "    end\r\n",
      "    @inbounds dc[end] &= Base._msk_end(dest)\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "\r\n",
      "@noinline throwdm(axdest, axsrc) =\r\n",
      "    throw(DimensionMismatch(\"destination axes $axdest are not compatible with source axes $axsrc\"))\r\n",
      "\r\n",
      "function restart_copyto_nonleaf!(newdest, dest, bc, val, I, iter, state, count)\r\n",
      "    # Function barrier that makes the copying to newdest type stable\r\n",
      "    for II in Iterators.take(iter, count)\r\n",
      "        newdest[II] = dest[II]\r\n",
      "    end\r\n",
      "    newdest[I] = val\r\n",
      "    return copyto_nonleaf!(newdest, bc, iter, state, count+1)\r\n",
      "end\r\n",
      "\r\n",
      "function copyto_nonleaf!(dest, bc::Broadcasted, iter, state, count)\r\n",
      "    T = eltype(dest)\r\n",
      "    while true\r\n",
      "        y = iterate(iter, state)\r\n",
      "        y === nothing && break\r\n",
      "        I, state = y\r\n",
      "        @inbounds val = bc[I]\r\n",
      "        if val isa T\r\n",
      "            @inbounds dest[I] = val\r\n",
      "        else\r\n",
      "            # This element type doesn't fit in dest. Allocate a new dest with wider eltype,\r\n",
      "            # copy over old values, and continue\r\n",
      "            newdest = Base.similar(bc, promote_typejoin(T, typeof(val)))\r\n",
      "            return restart_copyto_nonleaf!(newdest, dest, bc, val, I, iter, state, count)\r\n",
      "        end\r\n",
      "        count += 1\r\n",
      "    end\r\n",
      "    return dest\r\n",
      "end\r\n",
      "\r\n",
      "## Tuple methods\r\n",
      "\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "@inline function copy(bc::Broadcasted{Style{Tuple}})\r\n",
      "    dim = axes(bc)\r\n",
      "    length(dim) == 1 || throw(DimensionMismatch(\"tuple only supports one dimension\"))\r\n",
      "    N = length(dim[1])\r\n",
      "    return ntuple(k -> @inbounds(_broadcast_getindex(bc, k)), Val(N))\r\n",
      "end\r\n",
      "\r\n",
      "## scalar-range broadcast operations ##\r\n",
      "# DefaultArrayStyle and \\ are not available at the time of range.jl\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractRange) = r\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractRange) = range(-first(r), step=negate(step(r)), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::OrdinalRange) = range(-first(r), -last(r), step=negate(step(r)))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::StepRangeLen) = StepRangeLen(-r.ref, negate(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::LinRange) = LinRange(-r.start, -r.stop, length(r))\r\n",
      "\r\n",
      "# For #18336 we need to prevent promotion of the step type:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractRange, x::Number) = range(first(r) + x, step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::AbstractRange) = range(x + first(r), step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::OrdinalRange, x::Integer) = range(first(r) + x, last(r) + x, step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Integer, r::OrdinalRange) = range(x + first(r), x + last(r), step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractUnitRange, x::Integer) = range(first(r) + x, last(r) + x)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Integer, r::AbstractUnitRange) = range(x + first(r), x + last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::AbstractUnitRange, x::Real) = range(first(r) + x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Real, r::AbstractUnitRange) = range(x + first(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::StepRangeLen{T}, x::Number) where T =\r\n",
      "    StepRangeLen{typeof(T(r.ref)+x)}(r.ref + x, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::StepRangeLen{T}) where T =\r\n",
      "    StepRangeLen{typeof(x+T(r.ref))}(x + r.ref, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r::LinRange, x::Number) = LinRange(r.start + x, r.stop + x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), x::Number, r::LinRange) = LinRange(x + r.start, x + r.stop, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(+), r1::AbstractRange, r2::AbstractRange) = r1 + r2\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractRange, x::Number) = range(first(r) - x, step=step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::AbstractRange) = range(x - first(r), step=negate(step(r)), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::OrdinalRange, x::Integer) = range(first(r) - x, last(r) - x, step=step(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Integer, r::OrdinalRange) = range(x - first(r), x - last(r), step=negate(step(r)))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractUnitRange, x::Integer) = range(first(r) - x, last(r) - x)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::AbstractUnitRange, x::Real) = range(first(r) - x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::StepRangeLen{T}, x::Number) where T =\r\n",
      "    StepRangeLen{typeof(T(r.ref)-x)}(r.ref - x, r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::StepRangeLen{T}) where T =\r\n",
      "    StepRangeLen{typeof(x-T(r.ref))}(x - r.ref, negate(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r::LinRange, x::Number) = LinRange(r.start - x, r.stop - x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), x::Number, r::LinRange) = LinRange(x - r.start, x - r.stop, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(-), r1::AbstractRange, r2::AbstractRange) = r1 - r2\r\n",
      "\r\n",
      "# at present Base.range_start_step_length(1,0,5) is an error, so for 0 .* (-2:2) we explicitly construct StepRangeLen:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::AbstractRange) = StepRangeLen(x*first(r), x*step(r), length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::StepRangeLen{T}) where {T} =\r\n",
      "    StepRangeLen{typeof(x*T(r.ref))}(x*r.ref, x*r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::Number, r::LinRange) = LinRange(x * r.start, x * r.stop, r.len)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), x::AbstractFloat, r::OrdinalRange) =\r\n",
      "    Base.range_start_step_length(x*first(r), x*step(r), length(r))  # 0.2 .* (-2:2) needs TwicePrecision\r\n",
      "# separate in case of noncommutative multiplication:\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::AbstractRange, x::Number) = StepRangeLen(first(r)*x, step(r)*x, length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::StepRangeLen{T}, x::Number) where {T} =\r\n",
      "    StepRangeLen{typeof(T(r.ref)*x)}(r.ref*x, r.step*x, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::LinRange, x::Number) = LinRange(r.start * x, r.stop * x, r.len)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(*), r::OrdinalRange, x::AbstractFloat) =\r\n",
      "    Base.range_start_step_length(first(r)*x, step(r)*x, length(r))\r\n",
      "\r\n",
      "#broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::AbstractRange, x::Number) = range(first(r)/x, last(r)/x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::AbstractRange, x::Number) = range(first(r)/x, step=step(r)/x, length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::StepRangeLen{T}, x::Number) where {T} =\r\n",
      "    StepRangeLen{typeof(T(r.ref)/x)}(r.ref/x, r.step/x, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(/), r::LinRange, x::Number) = LinRange(r.start / x, r.stop / x, r.len)\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::AbstractRange) = range(x\\first(r), step=x\\step(r), length=length(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::StepRangeLen) = StepRangeLen(x\\r.ref, x\\r.step, length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(\\), x::Number, r::LinRange) = LinRange(x \\ r.start, x \\ r.stop, r.len)\r\n",
      "\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::UnitRange) = big(r.start):big(last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::StepRange) = big(r.start):big(r.step):big(last(r))\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::StepRangeLen) = StepRangeLen(big(r.ref), big(r.step), length(r), r.offset)\r\n",
      "broadcasted(::DefaultArrayStyle{1}, ::typeof(big), r::LinRange) = LinRange(big(r.start), big(r.stop), length(r))\r\n",
      "\r\n",
      "## CartesianIndices\r\n",
      "broadcasted(::typeof(+), I::CartesianIndices{N}, j::CartesianIndex{N}) where N =\r\n",
      "    CartesianIndices(map((rng, offset)->rng .+ offset, I.indices, Tuple(j)))\r\n",
      "broadcasted(::typeof(+), j::CartesianIndex{N}, I::CartesianIndices{N}) where N =\r\n",
      "    I .+ j\r\n",
      "broadcasted(::typeof(-), I::CartesianIndices{N}, j::CartesianIndex{N}) where N =\r\n",
      "    CartesianIndices(map((rng, offset)->rng .- offset, I.indices, Tuple(j)))\r\n",
      "function broadcasted(::typeof(-), j::CartesianIndex{N}, I::CartesianIndices{N}) where N\r\n",
      "    diffrange(offset, rng) = range(offset-last(rng), length=length(rng), step=step(rng))\r\n",
      "    Iterators.reverse(CartesianIndices(map(diffrange, Tuple(j), I.indices)))\r\n",
      "end\r\n",
      "\r\n",
      "## In specific instances, we can broadcast masked BitArrays whole chunks at a time\r\n",
      "# Very intentionally do not support much functionality here: scalar indexing would be O(n)\r\n",
      "struct BitMaskedBitArray{N,M}\r\n",
      "    parent::BitArray{N}\r\n",
      "    mask::BitArray{M}\r\n",
      "    BitMaskedBitArray{N,M}(parent, mask) where {N,M} = new(parent, mask)\r\n",
      "end\r\n",
      "@inline function BitMaskedBitArray(parent::BitArray{N}, mask::BitArray{M}) where {N,M}\r\n",
      "    @boundscheck checkbounds(parent, mask)\r\n",
      "    BitMaskedBitArray{N,M}(parent, mask)\r\n",
      "end\r\n",
      "Base.@propagate_inbounds dotview(B::BitArray, i::BitArray) = BitMaskedBitArray(B, i)\r\n",
      "Base.show(io::IO, B::BitMaskedBitArray) = foreach(arg->show(io, arg), (typeof(B), (B.parent, B.mask)))\r\n",
      "# Override materialize! to prevent the BitMaskedBitArray from escaping to an overridable method\r\n",
      "@inline materialize!(B::BitMaskedBitArray, bc::Broadcasted{<:Any,<:Any,typeof(identity),Tuple{Bool}}) = fill!(B, bc.args[1])\r\n",
      "@inline materialize!(B::BitMaskedBitArray, bc::Broadcasted{<:Any}) = materialize!(@inbounds(view(B.parent, B.mask)), bc)\r\n",
      "function Base.fill!(B::BitMaskedBitArray, b::Bool)\r\n",
      "    Bc = B.parent.chunks\r\n",
      "    Ic = B.mask.chunks\r\n",
      "    @inbounds if b\r\n",
      "        for i = 1:length(Bc)\r\n",
      "            Bc[i] |= Ic[i]\r\n",
      "        end\r\n",
      "    else\r\n",
      "        for i = 1:length(Bc)\r\n",
      "            Bc[i] &= ~Ic[i]\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return B\r\n",
      "end\r\n",
      "\r\n",
      "\r\n",
      "\r\n",
      "############################################################\r\n",
      "\r\n",
      "# x[...] .= f.(y...) ---> broadcast!(f, dotview(x, ...), y...).\r\n",
      "# The dotview function defaults to getindex, but we override it in\r\n",
      "# a few cases to get the expected in-place behavior without affecting\r\n",
      "# explicit calls to view.   (All of this can go away if slices\r\n",
      "# are changed to generate views by default.)\r\n",
      "\r\n",
      "Base.@propagate_inbounds dotview(args...) = Base.maybeview(args...)\r\n",
      "\r\n",
      "############################################################\r\n",
      "# The parser turns @. into a call to the __dot__ macro,\r\n",
      "# which converts all function calls and assignments into\r\n",
      "# broadcasting \"dot\" calls/assignments:\r\n",
      "\r\n",
      "dottable(x) = false # avoid dotting spliced objects (e.g. view calls inserted by @view)\r\n",
      "# don't add dots to dot operators\r\n",
      "dottable(x::Symbol) = (!isoperator(x) || first(string(x)) != '.' || x === :..) && x !== :(:)\r\n",
      "dottable(x::Expr) = x.head !== :$\r\n",
      "undot(x) = x\r\n",
      "function undot(x::Expr)\r\n",
      "    if x.head === :.=\r\n",
      "        Expr(:(=), x.args...)\r\n",
      "    elseif x.head === :block # occurs in for x=..., y=...\r\n",
      "        Expr(:block, Base.mapany(undot, x.args)...)\r\n",
      "    else\r\n",
      "        x\r\n",
      "    end\r\n",
      "end\r\n",
      "__dot__(x) = x\r\n",
      "function __dot__(x::Expr)\r\n",
      "    dotargs = Base.mapany(__dot__, x.args)\r\n",
      "    if x.head === :call && dottable(x.args[1])\r\n",
      "        Expr(:., dotargs[1], Expr(:tuple, dotargs[2:end]...))\r\n",
      "    elseif x.head === :comparison\r\n",
      "        Expr(:comparison, (iseven(i) && dottable(arg) && arg isa Symbol && isoperator(arg) ?\r\n",
      "                               Symbol('.', arg) : arg for (i, arg) in pairs(dotargs))...)\r\n",
      "    elseif x.head === :$\r\n",
      "        x.args[1]\r\n",
      "    elseif x.head === :let # don't add dots to `let x=...` assignments\r\n",
      "        Expr(:let, undot(dotargs[1]), dotargs[2])\r\n",
      "    elseif x.head === :for # don't add dots to for x=... assignments\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "        Expr(:for, undot(dotargs[1]), dotargs[2])\r\n",
      "    elseif (x.head === :(=) || x.head === :function || x.head === :macro) &&\r\n",
      "           Meta.isexpr(x.args[1], :call) # function or macro definition\r\n",
      "        Expr(x.head, x.args[1], dotargs[2])\r\n",
      "    elseif x.head === :(<:) || x.head === :(>:)\r\n",
      "        tmp = x.head === :(<:) ? :.<: : :.>:\r\n",
      "        Expr(:call, tmp, dotargs...)\r\n",
      "    else\r\n",
      "        head = String(x.head)::String\r\n",
      "        if last(head) == '=' && first(head) != '.' || head == \"&&\" || head == \"||\"\r\n",
      "            Expr(Symbol('.', head), dotargs...)\r\n",
      "        else\r\n",
      "            Expr(x.head, dotargs...)\r\n",
      "        end\r\n",
      "    end\r\n",
      "end\r\n",
      "\"\"\"\r\n",
      "    @. expr\r\n",
      "\r\n",
      "Convert every function call or operator in `expr` into a \"dot call\"\r\n",
      "(e.g. convert `f(x)` to `f.(x)`), and convert every assignment in `expr`\r\n",
      "to a \"dot assignment\" (e.g. convert `+=` to `.+=`).\r\n",
      "\r\n",
      "If you want to *avoid* adding dots for selected function calls in\r\n",
      "`expr`, splice those function calls in with `\\$`.  For example,\r\n",
      "`@. sqrt(abs(\\$sort(x)))` is equivalent to `sqrt.(abs.(sort(x)))`\r\n",
      "(no dot for `sort`).\r\n",
      "\r\n",
      "(`@.` is equivalent to a call to `@__dot__`.)\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> x = 1.0:3.0; y = similar(x);\r\n",
      "\r\n",
      "julia> @. y = x + 3 * sin(x)\r\n",
      "3-element Vector{Float64}:\r\n",
      " 3.5244129544236893\r\n",
      " 4.727892280477045\r\n",
      " 3.4233600241796016\r\n",
      "```\r\n",
      "\"\"\"\r\n",
      "macro __dot__(x)\r\n",
      "    esc(__dot__(x))\r\n",
      "end\r\n",
      "\r\n",
      "@inline function broadcasted_kwsyntax(f, args...; kwargs...)\r\n",
      "    if isempty(kwargs) # some BroadcastStyles dispatch on `f`, so try to preserve its type\r\n",
      "        return broadcasted(f, args...)\r\n",
      "    else\r\n",
      "        return broadcasted((args...) -> f(args...; kwargs...), args...)\r\n",
      "    end\r\n",
      "end\r\n",
      "@inline function broadcasted(f, args...)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(args′...), f, args′...)\r\n",
      "end\r\n",
      "# Due to the current Type{T}/DataType specialization heuristics within Tuples,\r\n",
      "# the totally generic varargs broadcasted(f, args...) method above loses Type{T}s in\r\n",
      "# mapping broadcastable across the args. These additional methods with explicit\r\n",
      "# arguments ensure we preserve Type{T}s in the first or second argument position.\r\n",
      "@inline function broadcasted(f, arg1, args...)\r\n",
      "    arg1′ = broadcastable(arg1)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(arg1′, args′...), f, arg1′, args′...)\r\n",
      "end\r\n",
      "@inline function broadcasted(f, arg1, arg2, args...)\r\n",
      "    arg1′ = broadcastable(arg1)\r\n",
      "    arg2′ = broadcastable(arg2)\r\n",
      "    args′ = map(broadcastable, args)\r\n",
      "    broadcasted(combine_styles(arg1′, arg2′, args′...), f, arg1′, arg2′, args′...)\r\n",
      "end\r\n",
      "@inline broadcasted(::S, f, args...) where S<:BroadcastStyle = Broadcasted{S}(f, args)\r\n",
      "\r\n",
      "\"\"\"\r\n",
      "    BroadcastFunction{F} <: Function\r\n",
      "\r\n",
      "Represents the \"dotted\" version of an operator, which broadcasts the operator over its\r\n",
      "arguments, so `BroadcastFunction(op)` is functionally equivalent to `(x...) -> (op).(x...)`.\r\n",
      "\r\n",
      "Can be created by just passing an operator preceded by a dot to a higher-order function.\r\n",
      "\r\n",
      "# Examples\r\n",
      "```jldoctest\r\n",
      "julia> a = [[1 3; 2 4], [5 7; 6 8]];\r\n",
      "\r\n",
      "julia> b = [[9 11; 10 12], [13 15; 14 16]];\r\n",
      "\r\n",
      "julia> map(.*, a, b)\r\n",
      "2-element Vector{Matrix{Int64}}:\r\n",
      " [9 33; 20 48]\r\n",
      " [65 105; 84 128]\r\n",
      "\r\n",
      "julia> Base.BroadcastFunction(+)(a, b) == a .+ b\r\n",
      "true\r\n",
      "```\r\n",
      "\r\n",
      "!!! compat \"Julia 1.6\"\r\n",
      "    `BroadcastFunction` and the standalone `.op` syntax are available as of Julia 1.6.\r\n",
      "\"\"\"\r\n",
      "struct BroadcastFunction{F} <: Function\r\n",
      "    f::F\r\n",
      "end\r\n",
      "\r\n",
      "@inline (op::BroadcastFunction)(x...; kwargs...) = op.f.(x...; kwargs...)\r\n",
      "\r\n",
      "function Base.show(io::IO, op::BroadcastFunction)\r\n",
      "    print(io, BroadcastFunction, '(')\r\n",
      "    show(io, op.f)\r\n",
      "    print(io, ')')\r\n",
      "    nothing\r\n",
      "end\r\n",
      "Base.show(io::IO, ::MIME\"text/plain\", op::BroadcastFunction) = show(io, op)\r\n",
      "\r\n",
      "end # module\r\n"
     ]
    }
   ],
   "source": [
    "@less Base.materialize(Base.broadcasted(+, df, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So we see that essentially we need to define `copy`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function Base.copy(bc::Base.Broadcast.Broadcasted{DataFrameStyle})\r\n",
      "    ndim = length(axes(bc))\r\n",
      "    if ndim != 2\r\n",
      "        throw(DimensionMismatch(\"cannot broadcast a data frame into $ndim dimensions\"))\r\n",
      "    end\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(df) for df in bcf.args if df isa AbstractDataFrame])\r\n",
      "    if length(colnames) != 1\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "    nrows = length(axes(bcf)[1])\r\n",
      "    df = DataFrame()\r\n",
      "    for i in axes(bcf)[2]\r\n",
      "        if nrows == 0\r\n",
      "            col = Any[]\r\n",
      "        else\r\n",
      "            bcf′ = getcolbc(bcf, i)\r\n",
      "            v1 = bcf′[CartesianIndex(1, i)]\r\n",
      "            startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "            startcol[1] = v1\r\n",
      "            col = copyto_widen!(startcol, bcf′, 2, i)\r\n",
      "        end\r\n",
      "        df[!, colnames[1][i]] = col\r\n",
      "    end\r\n",
      "\r\n",
      "    dfs = AbstractDataFrame[df for df in bcf.args if df isa AbstractDataFrame]\r\n",
      "    @assert !isempty(dfs)\r\n",
      "\r\n",
      "    _merge_matching_table_note_metadata!(df, dfs)\r\n",
      "\r\n",
      "    if all(x -> !isempty(colmetadatakeys(x)), dfs)\r\n",
      "        for colname in _names(df)\r\n",
      "            if length(dfs) == 1\r\n",
      "                _copy_col_note_metadata!(df, colname, only(dfs), colname)\r\n",
      "            else\r\n",
      "                if all(x -> !isempty(colmetadatakeys(x, colname)), dfs)\r\n",
      "                    for key1 in colmetadatakeys(dfs[1], colname)\r\n",
      "                        val1, style1 = colmetadata(dfs[1], colname, key1, style=true)\r\n",
      "                        if style1 === :note\r\n",
      "                            add_meta = true\r\n",
      "                            for i in 2:length(dfs)\r\n",
      "                                if key1 in colmetadatakeys(dfs[i], colname)\r\n",
      "                                    vali, stylei = colmetadata(dfs[i], colname, key1, style=true)\r\n",
      "                                    if !(stylei === :note && isequal(val1, vali))\r\n",
      "                                        add_meta = false\r\n",
      "                                        break\r\n",
      "                                    end\r\n",
      "                                else\r\n",
      "                                    add_meta = false\r\n",
      "                                    break\r\n",
      "                                end\r\n",
      "                            end\r\n",
      "                            add_meta && colmetadata!(df, colname, key1, val1, style=:note)\r\n",
      "                        end\r\n",
      "                    end\r\n",
      "                end\r\n",
      "            end\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "### Broadcasting assignment\r\n",
      "\r\n",
      "struct LazyNewColDataFrame{T,D}\r\n",
      "    df::D\r\n",
      "    col::T\r\n",
      "end\r\n",
      "\r\n",
      "Base.axes(x::LazyNewColDataFrame) = (Base.OneTo(nrow(x.df)),)\r\n",
      "Base.ndims(::Type{<:LazyNewColDataFrame}) = 1\r\n",
      "\r\n",
      "struct ColReplaceDataFrame{T<:AbstractDataFrame}\r\n",
      "    df::T\r\n",
      "    cols::Vector{Int}\r\n",
      "end\r\n",
      "\r\n",
      "Base.axes(x::ColReplaceDataFrame) = (axes(x.df, 1), Base.OneTo(length(x.cols)))\r\n",
      "Base.ndims(::Type{<:ColReplaceDataFrame}) = 2\r\n",
      "\r\n",
      "# In the functions below we need to call _drop_all_nonnote_metadata!\r\n",
      "# upfront as the rest of the operations is handled by Base Julia\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, idx::CartesianIndex{2})\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df[idx]\r\n",
      "end\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, row::Integer, col::ColumnIndex)\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df[row, col]\r\n",
      "end\r\n",
      "\r\n",
      "function Base.maybeview(df::AbstractDataFrame, rows, cols)\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return view(df, rows, cols)\r\n",
      "end\r\n",
      "\r\n",
      "function Base.dotview(df::AbstractDataFrame, ::Colon, cols::ColumnIndex)\r\n",
      "    if haskey(index(df), cols)\r\n",
      "        _drop_all_nonnote_metadata!(parent(df))\r\n",
      "        return view(df, :, cols)\r\n",
      "    end\r\n",
      "    if !(cols isa SymbolOrString)\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    if !is_column_insertion_allowed(df)\r\n",
      "        throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                            \"columns of its parent data frame is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, Symbol(cols))\r\n",
      "end\r\n",
      "\r\n",
      "function Base.dotview(df::AbstractDataFrame, ::typeof(!), cols)\r\n",
      "    if !(cols isa ColumnIndex)\r\n",
      "        return ColReplaceDataFrame(df, convert(Vector{Int}, index(df)[cols]))\r\n",
      "    end\r\n",
      "    if cols isa SymbolOrString\r\n",
      "        if columnindex(df, cols) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "    elseif !(1 <= cols <= ncol(df))\r\n",
      "        throw(ArgumentError(\"creating new columns using an integer index is disallowed\"))\r\n",
      "    end\r\n",
      "    return LazyNewColDataFrame(df, cols isa AbstractString ? Symbol(cols) : cols)\r\n",
      "end\r\n",
      "\r\n",
      "if isdefined(Base, :dotgetproperty) # Introduced in Julia 1.7\r\n",
      "    function Base.dotgetproperty(df::AbstractDataFrame, col::SymbolOrString)\r\n",
      "        if columnindex(df, col) == 0 && !is_column_insertion_allowed(df)\r\n",
      "            throw(ArgumentError(\"creating new columns in a SubDataFrame that subsets \" *\r\n",
      "                                \"columns of its parent data frame is disallowed\"))\r\n",
      "        end\r\n",
      "        return LazyNewColDataFrame(df, Symbol(col))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(lazydf::LazyNewColDataFrame, bc::Base.Broadcast.Broadcasted{T}) where T\r\n",
      "    df = lazydf.df\r\n",
      "    if !haskey(index(df), lazydf.col) && df isa SubDataFrame && lazydf.col isa SymbolOrString\r\n",
      "        @assert is_column_insertion_allowed(df)\r\n",
      "    end\r\n",
      "    if bc isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "        bc_tmp = Base.Broadcast.Broadcasted{T}(bc.f, bc.args, ())\r\n",
      "        v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "        col = similar(Vector{typeof(v)}, nrow(df))\r\n",
      "        copyto!(col, bc)\r\n",
      "    else\r\n",
      "        col = Base.Broadcast.materialize(bc)\r\n",
      "    end\r\n",
      "\r\n",
      "    return df[!, lazydf.col] = col\r\n",
      "end\r\n",
      "\r\n",
      "function _copyto_helper!(dfcol::AbstractVector, bc::Base.Broadcast.Broadcasted, col::Int)\r\n",
      "    if axes(dfcol, 1) != axes(bc)[1]\r\n",
      "        # this should never happen unless data frame is corrupted (has unequal column lengths)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting. The updated\" *\r\n",
      "                                \" data frame is invalid and should not be used\"))\r\n",
      "    end\r\n",
      "    @inbounds for row in eachindex(dfcol)\r\n",
      "        dfcol[row] = bc[CartesianIndex(row, col)]\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src)\r\n",
      "    for col in eachcol(dest)\r\n",
      "        src = Base.Broadcast.unalias(col, src)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.broadcast_unalias(::Nothing, src::AbstractDataFrame) = src\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest, src::AbstractDataFrame)\r\n",
      "    wascopied = false\r\n",
      "    for (i, col) in enumerate(eachcol(src))\r\n",
      "        if Base.mightalias(dest, col)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src = SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                       index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), i)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, i] = Base.unaliascopy(col)\r\n",
      "            end\r\n",
      "            wascopied = true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function _broadcast_unalias_helper(dest::AbstractDataFrame, scol::AbstractVector,\r\n",
      "                                   src::AbstractDataFrame, col2::Int, wascopied::Bool)\r\n",
      "    # col1 can be checked till col2 point as we are writing broadcasting\r\n",
      "    # results from 1 to ncol\r\n",
      "    # we go downwards because aliasing when col1 == col2 is most probable\r\n",
      "    for col1 in col2:-1:1\r\n",
      "        dcol = dest[!, col1]\r\n",
      "        if Base.mightalias(dcol, scol)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src =SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                      index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), col2)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, col2] = Base.unaliascopy(scol)\r\n",
      "            end\r\n",
      "            return src, true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src, wascopied\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src::AbstractDataFrame)\r\n",
      "    if size(dest, 2) != size(src, 2)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting.\"))\r\n",
      "    end\r\n",
      "    wascopied = false\r\n",
      "    for col2 in axes(dest, 2)\r\n",
      "        scol = src[!, col2]\r\n",
      "        src, wascopied = _broadcast_unalias_helper(dest, scol, src, col2, wascopied)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n",
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "less(copy, (Base.Broadcast.Broadcasted{DataFrames.DataFrameStyle},)) # note getcolbc! and copyto_widen!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Example 5: unaliasing in broadcasting assignment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is aliasing?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Assume we have:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = [1, 2, 3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element view(::Vector{Int64}, 3:-1:1) with eltype Int64:\n",
       " 3\n",
       " 2\n",
       " 1"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = @view x[3:-1:1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "now we call:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 3\n",
       " 2\n",
       " 1"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x .= y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 3\n",
       " 2\n",
       " 1"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and all is OK."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But assume we have a naive broadcasting implemented:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element view(::Vector{Int64}, 3:-1:1) with eltype Int64:\n",
       " 3\n",
       " 2\n",
       " 1"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = [1, 2, 3]\n",
    "y = @view x[3:-1:1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "naive_broadcast! (generic function with 1 method)"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "naive_broadcast!(x, y) = foreach(i -> x[i] = y[i], eachindex(x, y))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "naive_broadcast!(x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Vector{Int64}:\n",
       " 3\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is ensured to be avoided by broadcasting mechanism in Base in `Base.Broadcast.preprocess` function (which should be called before performing assignment of source to target). This function intenally calls `Base.Broadcast.broadcast_unalias` that should be implemented for your custom type."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "# 8 methods for generic function <b>broadcast_unalias</b> from \u001b[90mBase.Broadcast\u001b[39m:<ul><li> broadcast_unalias(dest::<b>DataFrameRow</b>, src::<b>AbstractDataFrame</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:381</a></li> <li> broadcast_unalias(dest::<b>DataFrameRow</b>, src) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:377</a></li> <li> broadcast_unalias(dest::<b>AbstractDataFrame</b>, src::<b>AbstractDataFrame</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:284</a></li> <li> broadcast_unalias(::<b>Nothing</b>, src::<b>AbstractDataFrame</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:232</a></li> <li> broadcast_unalias(dest, src::<b>AbstractDataFrame</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:234</a></li> <li> broadcast_unalias(dest::<b>AbstractDataFrame</b>, src) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:224</a></li> <li> broadcast_unalias(::<b>Nothing</b>, src) in Base.Broadcast at <a href=\"https://github.com/JuliaLang/julia/tree/b75ddb787ff1838deb46b624ce8c8470aa75e85c/base/broadcast.jl#L948\" target=\"_blank\">broadcast.jl:948</a></li> <li> broadcast_unalias(dest, src) in Base.Broadcast at <a href=\"https://github.com/JuliaLang/julia/tree/b75ddb787ff1838deb46b624ce8c8470aa75e85c/base/broadcast.jl#L947\" target=\"_blank\">broadcast.jl:947</a></li> </ul>"
      ],
      "text/plain": [
       "# 8 methods for generic function \"broadcast_unalias\" from \u001b[90mBase.Broadcast\u001b[39m:\n",
       " [1] broadcast_unalias(\u001b[90mdest\u001b[39m::\u001b[1mDataFrameRow\u001b[22m, \u001b[90msrc\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:381\u001b[24m\u001b[39m\n",
       " [2] broadcast_unalias(\u001b[90mdest\u001b[39m::\u001b[1mDataFrameRow\u001b[22m, \u001b[90msrc\u001b[39m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:377\u001b[24m\u001b[39m\n",
       " [3] broadcast_unalias(\u001b[90mdest\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m, \u001b[90msrc\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:284\u001b[24m\u001b[39m\n",
       " [4] broadcast_unalias(::\u001b[1mNothing\u001b[22m, \u001b[90msrc\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:232\u001b[24m\u001b[39m\n",
       " [5] broadcast_unalias(\u001b[90mdest\u001b[39m, \u001b[90msrc\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:234\u001b[24m\u001b[39m\n",
       " [6] broadcast_unalias(\u001b[90mdest\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m, \u001b[90msrc\u001b[39m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:224\u001b[24m\u001b[39m\n",
       " [7] broadcast_unalias(::\u001b[1mNothing\u001b[22m, \u001b[90msrc\u001b[39m)\n",
       "\u001b[90m     @\u001b[39m \u001b[90m\u001b[4mbroadcast.jl:948\u001b[24m\u001b[39m\n",
       " [8] broadcast_unalias(\u001b[90mdest\u001b[39m, \u001b[90msrc\u001b[39m)\n",
       "\u001b[90m     @\u001b[39m \u001b[90m\u001b[4mbroadcast.jl:947\u001b[24m\u001b[39m"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "methods(Base.Broadcast.broadcast_unalias)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src)\r\n",
      "    for col in eachcol(dest)\r\n",
      "        src = Base.Broadcast.unalias(col, src)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "# The method below is added to avoid dispatch ambiguity\r\n",
      "Base.Broadcast.broadcast_unalias(::Nothing, src::AbstractDataFrame) = src\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest, src::AbstractDataFrame)\r\n",
      "    wascopied = false\r\n",
      "    for (i, col) in enumerate(eachcol(src))\r\n",
      "        if Base.mightalias(dest, col)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src = SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                       index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), i)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, i] = Base.unaliascopy(col)\r\n",
      "            end\r\n",
      "            wascopied = true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function _broadcast_unalias_helper(dest::AbstractDataFrame, scol::AbstractVector,\r\n",
      "                                   src::AbstractDataFrame, col2::Int, wascopied::Bool)\r\n",
      "    # col1 can be checked till col2 point as we are writing broadcasting\r\n",
      "    # results from 1 to ncol\r\n",
      "    # we go downwards because aliasing when col1 == col2 is most probable\r\n",
      "    for col1 in col2:-1:1\r\n",
      "        dcol = dest[!, col1]\r\n",
      "        if Base.mightalias(dcol, scol)\r\n",
      "            if src isa SubDataFrame\r\n",
      "                if !wascopied\r\n",
      "                    src =SubDataFrame(copy(parent(src), copycols=false),\r\n",
      "                                      index(src), rows(src))\r\n",
      "                end\r\n",
      "                parentidx = parentcols(index(src), col2)\r\n",
      "                parent(src)[!, parentidx] = Base.unaliascopy(parent(src)[!, parentidx])\r\n",
      "            else\r\n",
      "                if !wascopied\r\n",
      "                    src = copy(src, copycols=false)\r\n",
      "                end\r\n",
      "                src[!, col2] = Base.unaliascopy(scol)\r\n",
      "            end\r\n",
      "            return src, true\r\n",
      "        end\r\n",
      "    end\r\n",
      "    return src, wascopied\r\n",
      "end\r\n",
      "\r\n",
      "function Base.Broadcast.broadcast_unalias(dest::AbstractDataFrame, src::AbstractDataFrame)\r\n",
      "    if size(dest, 2) != size(src, 2)\r\n",
      "        throw(DimensionMismatch(\"Dimension mismatch in broadcasting.\"))\r\n",
      "    end\r\n",
      "    wascopied = false\r\n",
      "    for col2 in axes(dest, 2)\r\n",
      "        scol = src[!, col2]\r\n",
      "        src, wascopied = _broadcast_unalias_helper(dest, scol, src, col2, wascopied)\r\n",
      "    end\r\n",
      "    return src\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n",
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "less(Base.Broadcast.broadcast_unalias, (AbstractDataFrame, Any)) # this is a first method of several"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that this process is expensive unfortunately, but we want to stay safe:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>3×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">3</td><td style = \"text-align: right;\">3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\t3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2\n",
       "   3 │     3"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(x=[1,2,3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element view(::Vector{Int64}, 3:-1:1) with eltype Int64:\n",
       " 3\n",
       " 2\n",
       " 1"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = view(df, 3:-1:1, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>3×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">3</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">3</td><td style = \"text-align: right;\">1</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 3 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\t3 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     3\n",
       "   2 │     2\n",
       "   3 │     1"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df .= y\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element view(::Vector{Int64}, 3:-1:1) with eltype Int64:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div style = \"float: left;\"><span>3×1 DataFrame</span></div><div style = \"clear: both;\"></div></div><div class = \"data-frame\" style = \"overflow-x: scroll;\"><table class = \"data-frame\" style = \"margin-bottom: 6px;\"><thead><tr class = \"header\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">Row</th><th style = \"text-align: left;\">x</th></tr><tr class = \"subheader headerLastRow\"><th class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\"></th><th title = \"Int64\" style = \"text-align: left;\">Int64</th></tr></thead><tbody><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">1</td><td style = \"text-align: right;\">1</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">2</td><td style = \"text-align: right;\">2</td></tr><tr><td class = \"rowNumber\" style = \"font-weight: bold; text-align: right;\">3</td><td style = \"text-align: right;\">3</td></tr></tbody></table></div>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\t3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x     \u001b[0m\n",
       "     │\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2\n",
       "   3 │     3"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df .= y\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When is unaliasing triggered by DataFrames.jl?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well - we already know that ultimately `copyto!` is called in broadcasting assignment:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "# 5 methods for generic function <b>copyto!</b> from \u001b[90mBase\u001b[39m:<ul><li> copyto!(df::<b>AbstractDataFrame</b>, bc::<b>Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:320</a></li> <li> copyto!(df::<b>AbstractDataFrame</b>, bc::<b>Base.Broadcast.Broadcasted</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:296</a></li> <li> copyto!(lazydf::<b>DataFrames.LazyNewColDataFrame</b>, bc::<b>Base.Broadcast.Broadcasted{T}</b>)<i> where T</i> in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:196</a></li> <li> copyto!(crdf::<b>DataFrames.ColReplaceDataFrame</b>, bc::<b>Base.Broadcast.Broadcasted</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:336</a></li> <li> copyto!(dfr::<b>DataFrameRow</b>, bc::<b>Base.Broadcast.Broadcasted</b>) in DataFrames at <a href=\"file://C:/Users/bogum/.julia/packages/DataFrames/LteEl/src/other/broadcasting.jl\" target=\"_blank\">C:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\broadcasting.jl:384</a></li> </ul>"
      ],
      "text/plain": [
       "# 5 methods for generic function \"copyto!\" from \u001b[90mBase\u001b[39m:\n",
       " [1] copyto!(\u001b[90mdf\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m, \u001b[90mbc\u001b[39m::\u001b[1mBase.Broadcast.Broadcasted\u001b[22m\u001b[0m{<:Base.Broadcast.AbstractArrayStyle{0}})\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:320\u001b[24m\u001b[39m\n",
       " [2] copyto!(\u001b[90mdf\u001b[39m::\u001b[1mAbstractDataFrame\u001b[22m, \u001b[90mbc\u001b[39m::\u001b[1mBase.Broadcast.Broadcasted\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:296\u001b[24m\u001b[39m\n",
       " [3] copyto!(\u001b[90mlazydf\u001b[39m::\u001b[1mDataFrames.LazyNewColDataFrame\u001b[22m, \u001b[90mbc\u001b[39m::\u001b[1mBase.Broadcast.Broadcasted\u001b[22m\u001b[0m{T}) where T\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:196\u001b[24m\u001b[39m\n",
       " [4] copyto!(\u001b[90mcrdf\u001b[39m::\u001b[1mDataFrames.ColReplaceDataFrame\u001b[22m, \u001b[90mbc\u001b[39m::\u001b[1mBase.Broadcast.Broadcasted\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:336\u001b[24m\u001b[39m\n",
       " [5] copyto!(\u001b[90mdfr\u001b[39m::\u001b[1mDataFrameRow\u001b[22m, \u001b[90mbc\u001b[39m::\u001b[1mBase.Broadcast.Broadcasted\u001b[22m)\n",
       "\u001b[90m     @\u001b[39m \u001b[35mDataFrames\u001b[39m \u001b[90mC:\\Users\\bogum\\.julia\\packages\\DataFrames\\LteEl\\src\\other\\\u001b[39m\u001b[90m\u001b[4mbroadcasting.jl:384\u001b[24m\u001b[39m"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "methods(copyto!, DataFrames)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let us have a look how they are implemented:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "function Base.copyto!(df::AbstractDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 || (length(colnames) == 1 && _names(df) != colnames[1])\r\n",
      "        push!(colnames, _names(df))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(df, bcf)\r\n",
      "    for i in axes(df, 2)\r\n",
      "        _copyto_helper!(df[!, i], getcolbc(bcf′, i), i)\r\n",
      "    end\r\n",
      "    _drop_all_nonnote_metadata!(parent(df))\r\n",
      "    return df\r\n",
      "end\r\n",
      "\r\n",
      "function Base.copyto!(df::AbstractDataFrame,\r\n",
      "                      bc::Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}})\r\n",
      "    # special case of fast approach when bc is providing an untransformed scalar\r\n",
      "    if bc.f === identity && bc.args isa Tuple{Any} && Base.Broadcast.isflat(bc)\r\n",
      "        for col in axes(df, 2)\r\n",
      "            fill!(df[!, col], bc.args[1][])\r\n",
      "        end\r\n",
      "        return df\r\n",
      "    else\r\n",
      "        return copyto!(df, convert(Base.Broadcast.Broadcasted{Nothing}, bc))\r\n",
      "    end\r\n",
      "end\r\n",
      "\r\n",
      "create_bc_tmp(bcf′_col::Base.Broadcast.Broadcasted{T}) where {T} =\r\n",
      "    Base.Broadcast.Broadcasted{T}(bcf′_col.f, bcf′_col.args, ())\r\n",
      "\r\n",
      "function Base.copyto!(crdf::ColReplaceDataFrame, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bcf = Base.Broadcast.flatten(bc)\r\n",
      "    colnames = unique!(Any[_names(x) for x in bcf.args if x isa AbstractDataFrame])\r\n",
      "    if length(colnames) > 1 ||\r\n",
      "        (length(colnames) == 1 && view(_names(crdf.df), crdf.cols) != colnames[1])\r\n",
      "        push!(colnames, view(_names(crdf.df), crdf.cols))\r\n",
      "        wrongnames = setdiff(union(colnames...), intersect(colnames...))\r\n",
      "        if isempty(wrongnames)\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames \" *\r\n",
      "                                \"must have the same order\"))\r\n",
      "        else\r\n",
      "            msg = join(wrongnames, \", \", \" and \")\r\n",
      "            throw(ArgumentError(\"Column names in broadcasted data frames must match. \" *\r\n",
      "                                \"Non matching column names are $msg\"))\r\n",
      "        end\r\n",
      "    end\r\n",
      "\r\n",
      "    bcf′ = Base.Broadcast.preprocess(crdf, bcf)\r\n",
      "    nrows = length(axes(bcf′)[1])\r\n",
      "    for (i, col_idx) in enumerate(crdf.cols)\r\n",
      "        bcf′_col = getcolbc(bcf′, i)\r\n",
      "        if bcf′_col isa Base.Broadcast.Broadcasted{<:Base.Broadcast.AbstractArrayStyle{0}}\r\n",
      "            bc_tmp = create_bc_tmp(bcf′_col)\r\n",
      "            v = Base.Broadcast.materialize(bc_tmp)\r\n",
      "            newcol = similar(Vector{typeof(v)}, nrow(crdf.df))\r\n",
      "            copyto!(newcol, bc)\r\n",
      "        else\r\n",
      "            if nrows == 0\r\n",
      "                newcol = Any[]\r\n",
      "            else\r\n",
      "                v1 = bcf′_col[CartesianIndex(1, i)]\r\n",
      "                startcol = similar(Vector{typeof(v1)}, nrows)\r\n",
      "                startcol[1] = v1\r\n",
      "                newcol = copyto_widen!(startcol, bcf′_col, 2, i)\r\n",
      "            end\r\n",
      "        end\r\n",
      "        crdf.df[!, col_idx] = newcol\r\n",
      "    end\r\n",
      "    return crdf.df\r\n",
      "end\r\n",
      "\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "# this is currently impossible but is added to avoid potential dispatch ambiguity in the future\r\n",
      "Base.Broadcast.broadcast_unalias(dest::DataFrameRow, src::AbstractDataFrame) =\r\n",
      "    Base.Broadcast.broadcast_unalias(parent(dest), src)\r\n",
      "\r\n",
      "function Base.copyto!(dfr::DataFrameRow, bc::Base.Broadcast.Broadcasted)\r\n",
      "    bc′ = Base.Broadcast.preprocess(dfr, bc)\r\n",
      "    for I in eachindex(bc′)\r\n",
      "        dfr[I] = bc′[I]\r\n",
      "    end\r\n",
      "    return dfr\r\n",
      "end\r\n"
     ]
    }
   ],
   "source": [
    "less(Base.copyto!, (AbstractDataFrame, Base.Broadcast.Broadcasted))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### That is all for today!\n",
    "\n",
    "I hope this part of the tutorial gave you some insight how indexing and broadcasting is implemented in DataFrames.jl and what things you should take into account when designing your own types that are expected to support indexing/broadcasting."
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Julia 1.9.0",
   "language": "julia",
   "name": "julia-1.9"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}