A Review of Language-Guided Dexterous Grasping with Human-in-the-Loop Adaptation

초록

Language-guided dexterous grasping aims to translate natural-language instructions and multimodal observations into multi-fingered grasps that are physically admissible, semantically aligned, functionally suitable, user-adaptive, and robust under execution uncertainty. Existing research remains fragmented across language-conditioned manipulation, dexterous grasp generation, human-in-the-loop learning, and simulation-based validation. This structured scoping review synthesizes these areas through a formulation-oriented analytical lens and organizes prior work around the main decisions involved in language-guided dexterous grasping: candidate generation, admissibility verification, multi-objective evaluation, human-feedback-based adaptation, and simulation-based or digital twin-based validation. We compare language, vision-language, and vision-language-action models, optimization-based and generative grasp methods, feedback mechanisms, and validation strategies in terms of their representations, assumptions, data requirements, evaluation settings, and failure modes. The synthesis indicates that no single paradigm currently provides reliable end-to-end language-guided dexterous grasping: semantic reasoning lacks contact-level guarantees, physically grounded generators provide limited task and user conditioning, human feedback introduces scalability costs, and simulation remains vulnerable to model mismatch. Finally, we identify benchmark requirements and research directions for language-conditioned datasets, semantic alignment, functional suitability, tactile integration, low-burden feedback, execution-risk estimation, and calibrated sim-to-real validation.

키워드