diff --git a/versions/v0.34.1/search.json b/versions/v0.34.1/search.json index a63fe714b..6851d64aa 100644 --- a/versions/v0.34.1/search.json +++ b/versions/v0.34.1/search.json @@ -583,7 +583,7 @@ "href": "tutorials/16-contexts/index.html#contexts-within-contexts", "title": "A Mini Turing Implementation II: Contexts", "section": "Contexts within contexts", - "text": "Contexts within contexts\nLet’s use the above two contexts to provide a slightly more general definition of the SamplingContext and the Metropolis-Hastings sampler we wrote in the mini Turing tutorial.\n\nstruct SamplingContext{S<:AbstractMCMC.AbstractSampler,R<:Random.AbstractRNG}\n rng::R\n sampler::S\n subcontext::Union{PriorContext, JointContext}\nend\n\nThe new aspect here is the subcontext field. Note that this is a context within a context! The idea is that we don’t need to hard code how the MCMC sampler evaluates the log probability, but rather can pass that work onto the subcontext. This way the same sampler can be used to sample from either the joint or the prior distribution.\nThe methods for SamplingContext are largely as in the our earlier mini Turing case, except they now pass some of the work onto the subcontext:\n\nfunction observe(context::SamplingContext, args...)\n # Sampling doesn't affect the observed values, so nothing to do here other than pass to\n # the subcontext.\n return observe(context.subcontext, args...)\nend\n\nstruct PriorSampler <: AbstractMCMC.AbstractSampler end\n\nfunction assume(context::SamplingContext{PriorSampler}, varinfo, dist, var_id)\n sample = Random.rand(context.rng, dist)\n varinfo[var_id] = (sample, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The subcontext field of the MHSampler determines which distribution this sampler\n# samples from.\nstruct MHSampler{D, T<:Real} <: AbstractMCMC.AbstractSampler\n sigma::T\n subcontext::D\nend\n\nMHSampler(subcontext) = MHSampler(1, subcontext)\n\nfunction assume(context::SamplingContext{<:MHSampler}, varinfo, dist, var_id)\n sampler = context.sampler\n old_value = varinfo.values[var_id]\n\n # propose a random-walk step, i.e, add the current value to a random \n # value sampled from a Normal distribution centered at 0\n value = rand(context.rng, Normal(old_value, sampler.sigma))\n varinfo[var_id] = (value, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The following three methods are identical to before, except for passing\n# `sampler.subcontext` to the context SamplingContext.\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG, model::MiniModel, sampler::MHSampler; kwargs...\n)\n vi = VarInfo()\n ctx = SamplingContext(rng, PriorSampler(), sampler.subcontext)\n model.f(vi, ctx, values(model.data)...)\n return vi, vi\nend\n\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG,\n model::MiniModel,\n sampler::MHSampler,\n prev_state::VarInfo; # is just the old trace\n kwargs...,\n)\n vi = prev_state\n new_vi = deepcopy(vi)\n ctx = SamplingContext(rng, sampler, sampler.subcontext)\n model.f(new_vi, ctx, values(model.data)...)\n\n # Compute log acceptance probability\n # Since the proposal is symmetric the computation can be simplified\n logα = sum(values(new_vi.logps)) - sum(values(vi.logps))\n\n # Accept proposal with computed acceptance probability\n if -Random.randexp(rng) < logα\n return new_vi, new_vi\n else\n return prev_state, prev_state\n end\nend;\n\nfunction AbstractMCMC.bundle_samples(\n samples, model::MiniModel, ::MHSampler, ::Any, ::Type{Chains}; kwargs...\n)\n # We get a vector of traces\n values = [sample.values for sample in samples]\n params = [key for key in keys(values[1]) if key ∉ keys(model.data)]\n vals = reduce(hcat, [value[p] for value in values] for p in params)\n # Composing the `Chains` data-structure, of which analyzing infrastructure is provided\n chains = Chains(vals, params)\n return chains\nend;\n\nWe can use this to sample from the joint distribution just like before:\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(JointContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rh ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float ⋯\n\n a 0.9756 0.9002 0.0031 81938.3955 120104.0916 1.00 ⋯\n b 2.8807 0.4872 0.0012 170261.0295 212854.8961 1.00 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -0.7930 0.3689 0.9760 1.5845 2.7403\n b 1.9278 2.5519 2.8801 3.2088 3.8362\n\n\nor we can choose to sample from the prior instead\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(PriorContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rha ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float6 ⋯\n\n a 0.4968 0.9976 0.0040 63599.5342 127012.8951 1.000 ⋯\n b 0.4965 2.2315 0.0137 26513.8959 50948.6364 1.000 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -1.4610 -0.1750 0.4960 1.1698 2.4493\n b -3.8737 -1.0094 0.4960 2.0009 4.8792\n\n\nOf course, using an MCMC algorithm to sample from the prior is unnecessary and silly (PriorSampler exists, after all), but the point is to illustrate the flexibility of the context system. We could, for instance, use the same setup to implement an Approximate Bayesian Computation (ABC) algorithm.\nThe use of contexts also goes far beyond just evaluating log probabilities and sampling. Some examples from Turing are\n\nFixedContext, which fixes some variables to given values and removes them completely from the evaluation of any log probabilities. They power the Turing.fix and Turing.unfix functions.\nConditionContext conditions the model on fixed values for some parameters. They are used by Turing.condition and Turing.uncondition, i.e. the model | (parameter=value,) syntax. The difference between fix and condition is whether the log probability for the corresponding variable is included in the overall log density.\nPriorExtractorContext collects information about what the prior distribution of each variable is.\nPrefixContext adds prefixes to variable names, allowing models to be used within other models without variable name collisions.\nPointwiseLikelihoodContext records the log likelihood of each individual variable.\nDebugContext collects useful debugging information while executing the model.\n\nAll of the above are what Turing calls parent contexts, which is to say that they all keep a subcontext just like our above SamplingContext did. Their implementations of assume and observe call the implementation of the subcontext once they are done doing their own work of fixing/conditioning/prefixing/etc. Contexts are often chained, so that e.g. a DebugContext may wrap within it a PrefixContext, which may in turn wrap a ConditionContext, etc. The only contexts that don’t have a subcontext in the Turing are the ones for evaluating the prior, likelihood, and joint distributions. These are called leaf contexts.\nThe above version of mini Turing is still much simpler than the full Turing language, but the principles of how contexts are used are the same.", + "text": "Contexts within contexts\nLet’s use the above two contexts to provide a slightly more general definition of the SamplingContext and the Metropolis-Hastings sampler we wrote in the mini Turing tutorial.\n\nstruct SamplingContext{S<:AbstractMCMC.AbstractSampler,R<:Random.AbstractRNG}\n rng::R\n sampler::S\n subcontext::Union{PriorContext, JointContext}\nend\n\nThe new aspect here is the subcontext field. Note that this is a context within a context! The idea is that we don’t need to hard code how the MCMC sampler evaluates the log probability, but rather can pass that work onto the subcontext. This way the same sampler can be used to sample from either the joint or the prior distribution.\nThe methods for SamplingContext are largely as in the our earlier mini Turing case, except they now pass some of the work onto the subcontext:\n\nfunction observe(context::SamplingContext, args...)\n # Sampling doesn't affect the observed values, so nothing to do here other than pass to\n # the subcontext.\n return observe(context.subcontext, args...)\nend\n\nstruct PriorSampler <: AbstractMCMC.AbstractSampler end\n\nfunction assume(context::SamplingContext{PriorSampler}, varinfo, dist, var_id)\n sample = Random.rand(context.rng, dist)\n varinfo[var_id] = (sample, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The subcontext field of the MHSampler determines which distribution this sampler\n# samples from.\nstruct MHSampler{D, T<:Real} <: AbstractMCMC.AbstractSampler\n sigma::T\n subcontext::D\nend\n\nMHSampler(subcontext) = MHSampler(1, subcontext)\n\nfunction assume(context::SamplingContext{<:MHSampler}, varinfo, dist, var_id)\n sampler = context.sampler\n old_value = varinfo.values[var_id]\n\n # propose a random-walk step, i.e, add the current value to a random \n # value sampled from a Normal distribution centered at 0\n value = rand(context.rng, Normal(old_value, sampler.sigma))\n varinfo[var_id] = (value, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The following three methods are identical to before, except for passing\n# `sampler.subcontext` to the context SamplingContext.\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG, model::MiniModel, sampler::MHSampler; kwargs...\n)\n vi = VarInfo()\n ctx = SamplingContext(rng, PriorSampler(), sampler.subcontext)\n model.f(vi, ctx, values(model.data)...)\n return vi, vi\nend\n\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG,\n model::MiniModel,\n sampler::MHSampler,\n prev_state::VarInfo; # is just the old trace\n kwargs...,\n)\n vi = prev_state\n new_vi = deepcopy(vi)\n ctx = SamplingContext(rng, sampler, sampler.subcontext)\n model.f(new_vi, ctx, values(model.data)...)\n\n # Compute log acceptance probability\n # Since the proposal is symmetric the computation can be simplified\n logα = sum(values(new_vi.logps)) - sum(values(vi.logps))\n\n # Accept proposal with computed acceptance probability\n if -Random.randexp(rng) < logα\n return new_vi, new_vi\n else\n return prev_state, prev_state\n end\nend;\n\nfunction AbstractMCMC.bundle_samples(\n samples, model::MiniModel, ::MHSampler, ::Any, ::Type{Chains}; kwargs...\n)\n # We get a vector of traces\n values = [sample.values for sample in samples]\n params = [key for key in keys(values[1]) if key ∉ keys(model.data)]\n vals = reduce(hcat, [value[p] for value in values] for p in params)\n # Composing the `Chains` data-structure, of which analyzing infrastructure is provided\n chains = Chains(vals, params)\n return chains\nend;\n\nWe can use this to sample from the joint distribution just like before:\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(JointContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rh ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float ⋯\n\n a 0.9800 0.8998 0.0032 79854.8573 122024.0634 1.00 ⋯\n b 2.8804 0.4874 0.0012 173435.9080 213392.1069 1.00 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -0.7817 0.3711 0.9787 1.5872 2.7449\n b 1.9247 2.5521 2.8799 3.2079 3.8390\n\n\nor we can choose to sample from the prior instead\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(PriorContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rha ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float6 ⋯\n\n a 0.4928 0.9975 0.0039 64665.7870 126806.7300 1.000 ⋯\n b 0.4622 2.2379 0.0135 27501.9212 52628.0032 1.000 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -1.4660 -0.1785 0.4930 1.1652 2.4435\n b -3.9475 -1.0457 0.4666 1.9788 4.8259\n\n\nOf course, using an MCMC algorithm to sample from the prior is unnecessary and silly (PriorSampler exists, after all), but the point is to illustrate the flexibility of the context system. We could, for instance, use the same setup to implement an Approximate Bayesian Computation (ABC) algorithm.\nThe use of contexts also goes far beyond just evaluating log probabilities and sampling. Some examples from Turing are\n\nFixedContext, which fixes some variables to given values and removes them completely from the evaluation of any log probabilities. They power the Turing.fix and Turing.unfix functions.\nConditionContext conditions the model on fixed values for some parameters. They are used by Turing.condition and Turing.uncondition, i.e. the model | (parameter=value,) syntax. The difference between fix and condition is whether the log probability for the corresponding variable is included in the overall log density.\nPriorExtractorContext collects information about what the prior distribution of each variable is.\nPrefixContext adds prefixes to variable names, allowing models to be used within other models without variable name collisions.\nPointwiseLikelihoodContext records the log likelihood of each individual variable.\nDebugContext collects useful debugging information while executing the model.\n\nAll of the above are what Turing calls parent contexts, which is to say that they all keep a subcontext just like our above SamplingContext did. Their implementations of assume and observe call the implementation of the subcontext once they are done doing their own work of fixing/conditioning/prefixing/etc. Contexts are often chained, so that e.g. a DebugContext may wrap within it a PrefixContext, which may in turn wrap a ConditionContext, etc. The only contexts that don’t have a subcontext in the Turing are the ones for evaluating the prior, likelihood, and joint distributions. These are called leaf contexts.\nThe above version of mini Turing is still much simpler than the full Turing language, but the principles of how contexts are used are the same.", "crumbs": [ "Get Started", "Developers", diff --git a/versions/v0.34.1/search_original.json b/versions/v0.34.1/search_original.json index d2302d521..132cae87a 100644 --- a/versions/v0.34.1/search_original.json +++ b/versions/v0.34.1/search_original.json @@ -583,7 +583,7 @@ "href": "tutorials/16-contexts/index.html#contexts-within-contexts", "title": "A Mini Turing Implementation II: Contexts", "section": "Contexts within contexts", - "text": "Contexts within contexts\nLet’s use the above two contexts to provide a slightly more general definition of the SamplingContext and the Metropolis-Hastings sampler we wrote in the mini Turing tutorial.\n\nstruct SamplingContext{S<:AbstractMCMC.AbstractSampler,R<:Random.AbstractRNG}\n rng::R\n sampler::S\n subcontext::Union{PriorContext, JointContext}\nend\n\nThe new aspect here is the subcontext field. Note that this is a context within a context! The idea is that we don’t need to hard code how the MCMC sampler evaluates the log probability, but rather can pass that work onto the subcontext. This way the same sampler can be used to sample from either the joint or the prior distribution.\nThe methods for SamplingContext are largely as in the our earlier mini Turing case, except they now pass some of the work onto the subcontext:\n\nfunction observe(context::SamplingContext, args...)\n # Sampling doesn't affect the observed values, so nothing to do here other than pass to\n # the subcontext.\n return observe(context.subcontext, args...)\nend\n\nstruct PriorSampler <: AbstractMCMC.AbstractSampler end\n\nfunction assume(context::SamplingContext{PriorSampler}, varinfo, dist, var_id)\n sample = Random.rand(context.rng, dist)\n varinfo[var_id] = (sample, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The subcontext field of the MHSampler determines which distribution this sampler\n# samples from.\nstruct MHSampler{D, T<:Real} <: AbstractMCMC.AbstractSampler\n sigma::T\n subcontext::D\nend\n\nMHSampler(subcontext) = MHSampler(1, subcontext)\n\nfunction assume(context::SamplingContext{<:MHSampler}, varinfo, dist, var_id)\n sampler = context.sampler\n old_value = varinfo.values[var_id]\n\n # propose a random-walk step, i.e, add the current value to a random \n # value sampled from a Normal distribution centered at 0\n value = rand(context.rng, Normal(old_value, sampler.sigma))\n varinfo[var_id] = (value, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The following three methods are identical to before, except for passing\n# `sampler.subcontext` to the context SamplingContext.\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG, model::MiniModel, sampler::MHSampler; kwargs...\n)\n vi = VarInfo()\n ctx = SamplingContext(rng, PriorSampler(), sampler.subcontext)\n model.f(vi, ctx, values(model.data)...)\n return vi, vi\nend\n\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG,\n model::MiniModel,\n sampler::MHSampler,\n prev_state::VarInfo; # is just the old trace\n kwargs...,\n)\n vi = prev_state\n new_vi = deepcopy(vi)\n ctx = SamplingContext(rng, sampler, sampler.subcontext)\n model.f(new_vi, ctx, values(model.data)...)\n\n # Compute log acceptance probability\n # Since the proposal is symmetric the computation can be simplified\n logα = sum(values(new_vi.logps)) - sum(values(vi.logps))\n\n # Accept proposal with computed acceptance probability\n if -Random.randexp(rng) < logα\n return new_vi, new_vi\n else\n return prev_state, prev_state\n end\nend;\n\nfunction AbstractMCMC.bundle_samples(\n samples, model::MiniModel, ::MHSampler, ::Any, ::Type{Chains}; kwargs...\n)\n # We get a vector of traces\n values = [sample.values for sample in samples]\n params = [key for key in keys(values[1]) if key ∉ keys(model.data)]\n vals = reduce(hcat, [value[p] for value in values] for p in params)\n # Composing the `Chains` data-structure, of which analyzing infrastructure is provided\n chains = Chains(vals, params)\n return chains\nend;\n\nWe can use this to sample from the joint distribution just like before:\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(JointContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rh ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float ⋯\n\n a 0.9756 0.9002 0.0031 81938.3955 120104.0916 1.00 ⋯\n b 2.8807 0.4872 0.0012 170261.0295 212854.8961 1.00 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -0.7930 0.3689 0.9760 1.5845 2.7403\n b 1.9278 2.5519 2.8801 3.2088 3.8362\n\n\nor we can choose to sample from the prior instead\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(PriorContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rha ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float6 ⋯\n\n a 0.4968 0.9976 0.0040 63599.5342 127012.8951 1.000 ⋯\n b 0.4965 2.2315 0.0137 26513.8959 50948.6364 1.000 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -1.4610 -0.1750 0.4960 1.1698 2.4493\n b -3.8737 -1.0094 0.4960 2.0009 4.8792\n\n\nOf course, using an MCMC algorithm to sample from the prior is unnecessary and silly (PriorSampler exists, after all), but the point is to illustrate the flexibility of the context system. We could, for instance, use the same setup to implement an Approximate Bayesian Computation (ABC) algorithm.\nThe use of contexts also goes far beyond just evaluating log probabilities and sampling. Some examples from Turing are\n\nFixedContext, which fixes some variables to given values and removes them completely from the evaluation of any log probabilities. They power the Turing.fix and Turing.unfix functions.\nConditionContext conditions the model on fixed values for some parameters. They are used by Turing.condition and Turing.uncondition, i.e. the model | (parameter=value,) syntax. The difference between fix and condition is whether the log probability for the corresponding variable is included in the overall log density.\nPriorExtractorContext collects information about what the prior distribution of each variable is.\nPrefixContext adds prefixes to variable names, allowing models to be used within other models without variable name collisions.\nPointwiseLikelihoodContext records the log likelihood of each individual variable.\nDebugContext collects useful debugging information while executing the model.\n\nAll of the above are what Turing calls parent contexts, which is to say that they all keep a subcontext just like our above SamplingContext did. Their implementations of assume and observe call the implementation of the subcontext once they are done doing their own work of fixing/conditioning/prefixing/etc. Contexts are often chained, so that e.g. a DebugContext may wrap within it a PrefixContext, which may in turn wrap a ConditionContext, etc. The only contexts that don’t have a subcontext in the Turing are the ones for evaluating the prior, likelihood, and joint distributions. These are called leaf contexts.\nThe above version of mini Turing is still much simpler than the full Turing language, but the principles of how contexts are used are the same.", + "text": "Contexts within contexts\nLet’s use the above two contexts to provide a slightly more general definition of the SamplingContext and the Metropolis-Hastings sampler we wrote in the mini Turing tutorial.\n\nstruct SamplingContext{S<:AbstractMCMC.AbstractSampler,R<:Random.AbstractRNG}\n rng::R\n sampler::S\n subcontext::Union{PriorContext, JointContext}\nend\n\nThe new aspect here is the subcontext field. Note that this is a context within a context! The idea is that we don’t need to hard code how the MCMC sampler evaluates the log probability, but rather can pass that work onto the subcontext. This way the same sampler can be used to sample from either the joint or the prior distribution.\nThe methods for SamplingContext are largely as in the our earlier mini Turing case, except they now pass some of the work onto the subcontext:\n\nfunction observe(context::SamplingContext, args...)\n # Sampling doesn't affect the observed values, so nothing to do here other than pass to\n # the subcontext.\n return observe(context.subcontext, args...)\nend\n\nstruct PriorSampler <: AbstractMCMC.AbstractSampler end\n\nfunction assume(context::SamplingContext{PriorSampler}, varinfo, dist, var_id)\n sample = Random.rand(context.rng, dist)\n varinfo[var_id] = (sample, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The subcontext field of the MHSampler determines which distribution this sampler\n# samples from.\nstruct MHSampler{D, T<:Real} <: AbstractMCMC.AbstractSampler\n sigma::T\n subcontext::D\nend\n\nMHSampler(subcontext) = MHSampler(1, subcontext)\n\nfunction assume(context::SamplingContext{<:MHSampler}, varinfo, dist, var_id)\n sampler = context.sampler\n old_value = varinfo.values[var_id]\n\n # propose a random-walk step, i.e, add the current value to a random \n # value sampled from a Normal distribution centered at 0\n value = rand(context.rng, Normal(old_value, sampler.sigma))\n varinfo[var_id] = (value, NaN)\n # Once the value has been sampled, let the subcontext handle evaluating the log\n # probability.\n return assume(context.subcontext, varinfo, dist, var_id)\nend;\n\n# The following three methods are identical to before, except for passing\n# `sampler.subcontext` to the context SamplingContext.\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG, model::MiniModel, sampler::MHSampler; kwargs...\n)\n vi = VarInfo()\n ctx = SamplingContext(rng, PriorSampler(), sampler.subcontext)\n model.f(vi, ctx, values(model.data)...)\n return vi, vi\nend\n\nfunction AbstractMCMC.step(\n rng::Random.AbstractRNG,\n model::MiniModel,\n sampler::MHSampler,\n prev_state::VarInfo; # is just the old trace\n kwargs...,\n)\n vi = prev_state\n new_vi = deepcopy(vi)\n ctx = SamplingContext(rng, sampler, sampler.subcontext)\n model.f(new_vi, ctx, values(model.data)...)\n\n # Compute log acceptance probability\n # Since the proposal is symmetric the computation can be simplified\n logα = sum(values(new_vi.logps)) - sum(values(vi.logps))\n\n # Accept proposal with computed acceptance probability\n if -Random.randexp(rng) < logα\n return new_vi, new_vi\n else\n return prev_state, prev_state\n end\nend;\n\nfunction AbstractMCMC.bundle_samples(\n samples, model::MiniModel, ::MHSampler, ::Any, ::Type{Chains}; kwargs...\n)\n # We get a vector of traces\n values = [sample.values for sample in samples]\n params = [key for key in keys(values[1]) if key ∉ keys(model.data)]\n vals = reduce(hcat, [value[p] for value in values] for p in params)\n # Composing the `Chains` data-structure, of which analyzing infrastructure is provided\n chains = Chains(vals, params)\n return chains\nend;\n\nWe can use this to sample from the joint distribution just like before:\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(JointContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rh ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float ⋯\n\n a 0.9800 0.8998 0.0032 79854.8573 122024.0634 1.00 ⋯\n b 2.8804 0.4874 0.0012 173435.9080 213392.1069 1.00 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -0.7817 0.3711 0.9787 1.5872 2.7449\n b 1.9247 2.5521 2.8799 3.2079 3.8390\n\n\nor we can choose to sample from the prior instead\n\nsample(MiniModel(m, (x=3.0,)), MHSampler(PriorContext()), 1_000_000; chain_type=Chains, progress=false)\n\nChains MCMC chain (1000000×2×1 Array{Float64, 3}):\n\nIterations = 1:1:1000000\nNumber of chains = 1\nSamples per chain = 1000000\nparameters = a, b\n\nSummary Statistics\n parameters mean std mcse ess_bulk ess_tail rha ⋯\n Symbol Float64 Float64 Float64 Float64 Float64 Float6 ⋯\n\n a 0.4928 0.9975 0.0039 64665.7870 126806.7300 1.000 ⋯\n b 0.4622 2.2379 0.0135 27501.9212 52628.0032 1.000 ⋯\n 2 columns omitted\n\nQuantiles\n parameters 2.5% 25.0% 50.0% 75.0% 97.5%\n Symbol Float64 Float64 Float64 Float64 Float64\n\n a -1.4660 -0.1785 0.4930 1.1652 2.4435\n b -3.9475 -1.0457 0.4666 1.9788 4.8259\n\n\nOf course, using an MCMC algorithm to sample from the prior is unnecessary and silly (PriorSampler exists, after all), but the point is to illustrate the flexibility of the context system. We could, for instance, use the same setup to implement an Approximate Bayesian Computation (ABC) algorithm.\nThe use of contexts also goes far beyond just evaluating log probabilities and sampling. Some examples from Turing are\n\nFixedContext, which fixes some variables to given values and removes them completely from the evaluation of any log probabilities. They power the Turing.fix and Turing.unfix functions.\nConditionContext conditions the model on fixed values for some parameters. They are used by Turing.condition and Turing.uncondition, i.e. the model | (parameter=value,) syntax. The difference between fix and condition is whether the log probability for the corresponding variable is included in the overall log density.\nPriorExtractorContext collects information about what the prior distribution of each variable is.\nPrefixContext adds prefixes to variable names, allowing models to be used within other models without variable name collisions.\nPointwiseLikelihoodContext records the log likelihood of each individual variable.\nDebugContext collects useful debugging information while executing the model.\n\nAll of the above are what Turing calls parent contexts, which is to say that they all keep a subcontext just like our above SamplingContext did. Their implementations of assume and observe call the implementation of the subcontext once they are done doing their own work of fixing/conditioning/prefixing/etc. Contexts are often chained, so that e.g. a DebugContext may wrap within it a PrefixContext, which may in turn wrap a ConditionContext, etc. The only contexts that don’t have a subcontext in the Turing are the ones for evaluating the prior, likelihood, and joint distributions. These are called leaf contexts.\nThe above version of mini Turing is still much simpler than the full Turing language, but the principles of how contexts are used are the same.", "crumbs": [ "Get Started", "Developers", diff --git a/versions/v0.34.1/sitemap.xml b/versions/v0.34.1/sitemap.xml index a98a35e94..b4e513234 100644 --- a/versions/v0.34.1/sitemap.xml +++ b/versions/v0.34.1/sitemap.xml @@ -2,154 +2,154 @@ https://turinglang.org/tutorials/02-logistic-regression/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-16-using-turing-external-samplers/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/docs-17-implementing-samplers/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/dev-model-manual/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-04-for-developers-abstractmcmc-turing/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-09-using-turing-advanced/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-07-for-developers-variational-inference/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/usage-modifying-logprob/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/01-gaussian-mixture-model/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/10-bayesian-differential-equations/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/12-gplvm/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/16-contexts/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-01-contributing-guide/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/00-introduction/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.805Z https://turinglang.org/tutorials/07-poisson-regression/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-15-using-turing-sampler-viz/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/13-seasonal-time-series/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/14-minituring/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-17-mode-estimation/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/docs-00-getting-started/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/09-variational-inference/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-10-using-turing-autodiff/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/11-probabilistic-pca/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-11-using-turing-dynamichmc/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-08-using-turing/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-12-using-turing-guide/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/docs-06-for-developers-interface/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/usage-custom-distribution/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/usage-generated-quantities/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/docs-05-for-developers-compiler/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/usage-probability-interface/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/08-multinomial-logistic-regression/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/15-gaussian-processes/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/04-hidden-markov-model/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/05-linear-regression/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/03-bayesian-neural-network/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.809Z https://turinglang.org/tutorials/docs-13-using-turing-performance-tips/index.html - 2024-11-17T13:18:04.655Z + 2024-11-19T16:36:34.813Z https://turinglang.org/tutorials/06-infinite-mixture-model/index.html - 2024-11-17T13:18:04.651Z + 2024-11-19T16:36:34.809Z diff --git a/versions/v0.34.1/tutorials/16-contexts/index.html b/versions/v0.34.1/tutorials/16-contexts/index.html index c24d75786..a4f613bc2 100644 --- a/versions/v0.34.1/tutorials/16-contexts/index.html +++ b/versions/v0.34.1/tutorials/16-contexts/index.html @@ -533,10 +533,10 @@

A Mini Turing Implementation II: Contexts

-

In the Mini Turing tutorial we developed a miniature version of the Turing language, to illustrate its core design. A passing mention was made of contexts. In this tutorial we develop that aspect of our mini Turing language further to demonstrate how and why contexts are an important part of Turing’s design.

+

In the Mini Turing tutorial we developed a miniature version of the Turing language, to illustrate its core design. A passing mention was made of contexts. In this tutorial we develop that aspect of our mini Turing language further to demonstrate how and why contexts are an important part of Turing’s design.

Mini Turing expanded, now with more contexts

-

If you haven’t read Mini Turing yet, you should do that first. We start by repeating verbatim much of the code from there. Define the type for holding values for variables:

+

If you haven’t read Mini Turing yet, you should do that first. We start by repeating verbatim much of the code from there. Define the type for holding values for variables:

import MacroTools, Random, AbstractMCMC
 using Distributions: Normal, logpdf
@@ -799,16 +799,16 @@ 
Contexts within c
   parameters      mean       std      mcse      ess_bulk      ess_tail      rh ⋯
       Symbol   Float64   Float64   Float64       Float64       Float64   Float ⋯
 
-           a    0.9756    0.9002    0.0031    81938.3955   120104.0916    1.00 ⋯
-           b    2.8807    0.4872    0.0012   170261.0295   212854.8961    1.00 ⋯
+           a    0.9800    0.8998    0.0032    79854.8573   122024.0634    1.00 ⋯
+           b    2.8804    0.4874    0.0012   173435.9080   213392.1069    1.00 ⋯
                                                                2 columns omitted
 
 Quantiles
   parameters      2.5%     25.0%     50.0%     75.0%     97.5%
       Symbol   Float64   Float64   Float64   Float64   Float64
 
-           a   -0.7930    0.3689    0.9760    1.5845    2.7403
-           b    1.9278    2.5519    2.8801    3.2088    3.8362
+ a -0.7817 0.3711 0.9787 1.5872 2.7449 + b 1.9247 2.5521 2.8799 3.2079 3.8390

or we can choose to sample from the prior instead

@@ -826,16 +826,16 @@

Contexts within c parameters mean std mcse ess_bulk ess_tail rha ⋯ Symbol Float64 Float64 Float64 Float64 Float64 Float6 ⋯ - a 0.4968 0.9976 0.0040 63599.5342 127012.8951 1.000 ⋯ - b 0.4965 2.2315 0.0137 26513.8959 50948.6364 1.000 ⋯ + a 0.4928 0.9975 0.0039 64665.7870 126806.7300 1.000 ⋯ + b 0.4622 2.2379 0.0135 27501.9212 52628.0032 1.000 ⋯ 2 columns omitted Quantiles parameters 2.5% 25.0% 50.0% 75.0% 97.5% Symbol Float64 Float64 Float64 Float64 Float64 - a -1.4610 -0.1750 0.4960 1.1698 2.4493 - b -3.8737 -1.0094 0.4960 2.0009 4.8792 + a -1.4660 -0.1785 0.4930 1.1652 2.4435 + b -3.9475 -1.0457 0.4666 1.9788 4.8259

Of course, using an MCMC algorithm to sample from the prior is unnecessary and silly (PriorSampler exists, after all), but the point is to illustrate the flexibility of the context system. We could, for instance, use the same setup to implement an Approximate Bayesian Computation (ABC) algorithm.

diff --git a/versions/v0.34.1/tutorials/docs-09-using-turing-advanced/index.html b/versions/v0.34.1/tutorials/docs-09-using-turing-advanced/index.html index 6e474952c..0a93f8ba9 100644 --- a/versions/v0.34.1/tutorials/docs-09-using-turing-advanced/index.html +++ b/versions/v0.34.1/tutorials/docs-09-using-turing-advanced/index.html @@ -490,10 +490,10 @@

Advanced Usage

This page has been separated into new sections. Please update any bookmarks you might have: