2018-10-23-bobu18a.md

title	abstract	keywords	layout	series	id	month	tex_title	firstpage	lastpage	page	order	cycles	bibtex_author	author	date	address	publisher	container-title	volume	genre	issued	pdf	extras
Learning under Misspecified Objective Spaces	Learning robot objective functions from human input has become increasingly important, but state-of-the-art techniques assume that the human’s desired objective lies within the robot’s hypothesis space. When this is not true, even methods that keep track of uncertainty over the objective fail because they reason about which hypothesis might be correct, and not whether any of the hypotheses are correct. We focus specifically on learning from physical human corrections during the robot’s task execution, where not having a rich enough hypothesis space leads to the robot updating its objective in ways that the person did not actually intend. We observe that such corrections appear irrelevant to the robot, because they are not the best way of achieving any of the candidate objectives. Instead of naively trusting and learning from every human interaction, we propose robots learn conservatively by reasoning in real time about how relevant the human’s correction is for the robot’s hypothesis space. We test our inference method in an experiment with human interaction data, and demonstrate that this alleviates unintended learning in an in-person user study with a robot manipulator.	model misspecification, Bayesian inference, reward learning, online learning, physical human-robot interaction, inverse reinforcement learning	inproceedings	Proceedings of Machine Learning Research	bobu18a	0	Learning under Misspecified Objective Spaces	796	805	796-805	796	false	Bobu, Andreea and Bajcsy, Andrea and Fisac, Jaime F. and Dragan, Anca D.	given family Andreea Bobu given family Andrea Bajcsy given family Jaime F. Fisac given family Anca D. Dragan	2018-10-23		PMLR	Proceedings of The 2nd Conference on Robot Learning	87	inproceedings	date-parts 2018 10 23	http://proceedings.mlr.press/v87/bobu18a/bobu18a.pdf	label link Supplementary video https://youtu.be/stnFye8HdcU label link Source code https://github.com/andreea7b/beta_adaptive_pHRI