Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to sample unguided choices from prior during forward sampling. #757

Open
null-a opened this issue Jan 12, 2017 · 4 comments
Open

Comments

@null-a
Copy link
Member

null-a commented Jan 12, 2017

This would simplify sampling from the posterior predictive after performing optimization. @stuhlmueller suggested that forward sampling should have an option analogous to the importance option of SMC.

@null-a
Copy link
Member Author

null-a commented Jan 13, 2017

There are 6 possible strategies we might want to implement when sampling from a random choice. Each strategy specifies what should happen when the random choice does/does not specify an explicit guide distribution.

SMC makes three of these available via the following values of the importance argument:

    Guide given
    Use Prior Use Guide Auto guide
No guide Use Prior 'ignoreGuide' 'default'
Auto guide 'autoGuide'

Forward sampling supports two of these via the guide argument:

    Guide given
    Use Prior Use Guide Auto guide
No guide Use Prior false
Auto guide true

After the change proposed in this issue is implemented, both SMC and forward sampling will support the same strategies, so it's probably sensible to have an argument that takes the same values across both. Here's one suggestion:

    Guide given
    Use Prior Use Guide Auto guide
No guide Use Prior 'no' 'yes'
Auto guide 'auto'

Perhaps the name for this would continue to be importance for SMC and guide for forward sampling.

One concern I have with this is that a user may say guide='yes' when they really needed guide='auto', since the names aren't particularly intuitive. If we can't fix this by coming up with better names (or some other nice way), perhaps forward sampling should issue a warning when guide='yes' and we notice we sampled from the prior at an unguided choice.

This also suggests a couple of ways we might make this more flexible:

  • It might be useful to override the global strategy, per choice.
  • It's possible that some of the strategies we don't currently implement would be useful to have. For example, the bottom right of this table is "use mean field everywhere, ignoring any handwritten guides".

@null-a
Copy link
Member Author

null-a commented Jan 25, 2017

If we were to cover all possible cases, perhaps we'd have two options -- one for what to do with (explicity) guided choices, one for unguided choices. Maybe:

Infer({method: 'SMC', guided: 'prior|guide|auto', unguided: 'prior|auto'});

(For forward sampling it might be better to use 'model' or 'target' rather than 'prior', but either way I think we probably want the same options for SMC and forward sampling for ease of remembering.)

@stuhlmueller
Copy link
Member

I tend to like the solution with two options best. Being explicit makes it easier to think about what the options do. If there are sensible defaults, we might still only need to specify one option most of the time.

@null-a
Copy link
Member Author

null-a commented Jan 30, 2017

The downside of switching to using two args as suggested above is that the thing we currently get with Infer({method: 'forward', guide: true}) becomes more cumbersome to ask for. Specifically it would be Infer({method: 'forward', guided: 'guide', unguided: 'auto'}).

@stuhlmueller suggested that we might alleviate this by having the new args structured like so:

Infer({guide: {guided: ..., unguided: ...}, ...})

... and then interpret Infer({guide: true, ...}) as meaning Infer({guide: {guided: 'guide', unguided: 'auto'}}).

This structure seems pretty reasonable to me. Maybe we could make it nicer to use by having the word 'guide' appear less frequently. Having 'guide', 'guided', 'guide', 'unguided' appear one after the other seems awkward? I don't have any good suggestions for alternatives at the moment, but I'll think about it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants