Reactive whole-body locomotion-integrated manipulation based on combined learning and optimization

Collaborative robots, including fixed-base and mobile base, have become increasingly vital in contemporary settings due to their transformative impact on industrial and service sectors. In today’s rapidly evolving landscape, characterized by dynamic market demands and technological advancements, collaborative robots must offer a flexible and adaptive solution to various industries and work closely with humans in unstructured environments. Therefore, with the rapid development of artificial intelligence, reactive planning and control play a pivotal role in collaborative robots, enabling them to respond promptly to changes in their surroundings, such as unexpected obstacles or alterations in the task requirements. Furthermore, integrating reactive control mechanisms allows collaborative robots to seamlessly interact with their environment, human counterparts, and other robots, fostering a collaborative ecosystem characterized by fluid adaptability and enhanced performance.

 

Fixed-base collaborative robotic manipulators only need to handle the changes inside their limited workspace. A lot of computer version techniques have been applied to detect task changes in the workspace. For example, a reactive motion planning and control approach based on a cascade of dependent quadratic programmings (QP) in some studies is proposed to ensure the end-effector (EE) motion of a 7 degree of freedom (DoF) collaborative arm is always inside the view of the camera (namely, obey this constraint) and is compliant to the randomly applied human’s external force at running time. Based on hierarchical quadratic programming (HQP), an adaptative motion controller for robotic arm in some studies can respond to human disturbance online and generate compliant behavior, subjecting to the joint limits. Besides, model predictive control (MPC) in some studies is also applied for reactive motion planning for manipulators. In particular, this method can generate real-time collision-free trajectories following a moving target from a joystick in dynamic environments.

 

The HQP and MPC methods mentioned above work well for fixed-based robotic arms. However, reactive planning and control for collaborative mobile manipulators (CMM) is more complicated due to the high redundancies. CMM includes a mobile platform and a collaborative robotic arm mounted on top of the former, combining mobility and dexterity. With enhanced mobility, CMM can work in larger workspaces and play a more and more important role in various applications, such as smart manufacturing factories, warehouses, domestic, and health care. Therefore, the learning/planning and control for CMM are the hot topics in the current robotics field. In general, the planning and control techniques for CMMs can be categorized into two groups based on how they handle the two components: Either as two distinct subsystems or as a unified whole-body system. For the former category, usually, the mobile base moves close to the target object (locomotion mode) first, and the robotic arm starts to move after the base stopping (manipulation mode). To ensure the feasibility of the manipulation phase of the robotic arm, the inverse reachability map (IRM) is applied to determine the final mobile base placement during the locomotion period. Once given a desired EE pose, the IRM can find a feasible base placement for the mobile base. Although the IRM can be constructed offline and used online, it varies from one CMM to another. Meanwhile, it is non-trivial to construct the IRM.

 

For CMM, it is straightforward that the tasks’ efficiency and performance can be improved if the arm and mobile base can move simultaneously without stopping, namely, in loco-manipulation mode. To this end, many unified whole-body CMM methods have been proposed recently. For example, a whole-body kinematics perceptive MPC framework for CMM is proposed in some studies, where the visual information and haptic sensing are used to avoid obstacles and control interaction forces separately. A similar MPC method is used to achieve nonprehensile object transportation and simultaneously avoid the static and dynamic obstacles for CMM in some studies. Besides, a holistic approach based on QP for CMM in some studies maximizes the manipulability of the arm and minimizes the angle between the mobile base and end-effector. The MPC/QP-based whole-body approaches can generate feasible motion for CMM as the arm, and base joint limits are set as constraints inside the optimization scheme. Besides, the MPC/QP approaches for collaborative robots (both fixed-base and mobile base) can respond to task changes by adapting trajectories online for the short (predicted) horizon. However, these methods can only be used as local planners since the long-horizon reference trajectory still needs to be provided. To generate the reference trajectory, a sampling-based method is proposed in some studies as a high-level planner, which is tracked by an optimization-based low-level controller. However, it is challenging for these methods to pass by a via-point with specific velocity requirements, like the contact velocity with the target object in the pick-and-place task in some studies (i.e., the trajectory modulation capability).

 

Besides, the learning-from-humans demonstration approach is applied to generate the reference trajectory for collaborative robots. For example, based on an impedance controller, the reference trajectory and the variable impedance profile are learned for fixed-base robots in some studies. For CMM, the EE trajectory and the desired interaction force in a table-cleaning task are learned from human demonstrations in some studies. The learned EE reference trajectory is sent to a whole-body Cartesian impedance controller to generate the overall joint-level command for CMM. Differently, the EE and mobile base motion in a door-opening task are learned from human demonstrations for CMM in some studies. However, the IRM is still necessary as a constraint to ensure the feasibility of the EE movements, making it challenging to apply this approach to other CMMs. Recently, a combined learning and optimization approach has been proposed in some studies, transferring human whole-body loco-manipulation skills to a CMM with different geometry. Specifically, the kernelized movement primitives (KMP) encode the human demonstrated whole-body trajectory, where the waist and pelvis motion are mapped to the EE and mobile base, respectively. Then, the learned trajectories are sent to the HQP controller, which separately sets the EE and mobile base tracking as primary and secondary tasks. This approach successfully learns human-demonstrated fluent and single whole-body movement (i.e., unified loco-manipulation mode) for CMM and adapts the learned trajectory to a new initial point. However, the target object position is the same (fixed) for all the experiments in some studies. It is still an open question of how to generate the whole-body reactive locomotion-integrated manipulation skills based on the learned reference trajectories for CMM when the target object is moved on the fly.

 

To address these issues, this paper published in Machine Intelligence Research proposes an online reactive whole-body locomotion-integrated manipulation framework based on combined learning and optimization. In addition, to address the novelty of the proposed method, there is one table that compares researchers’ method with the most relevant works in the literature. According to the task changes, researchers aim to adapt the learned whole-body locomotion-integrated manipulation skills online reactively. Specifically, once the desired EE pose is changed, the corresponding feasible base placement will be found first, and the learned whole-body reference trajectory will be updated online later. The generated whole-body reference trajectory is sent to an HQP controller to generate the joint-level commands for CMM. The main contributions of this paper are as follows:

 

1) An overall online reactive whole-body locomotion-integrated manipulation framework that adapts to the task’s changes online;

 

2) A reactive task space whole-body reference trajectory learning and adaptation formulation with trajectory modulation capability based on human demonstrations;

 

3) A tailored HQP controller with deeper ablation investigation that transfers the updated task space whole-body trajectory to optimal joint-level commands for CMM.

 

The overall approach and the details of the proposed methodology are illustrated in Section 2. Section 2 outlines the proposed framework, comprising reactive whole-body trajectory learning, hierarchical quadratic programming (HQP) formulation, and joint-level controller. In reactive whole-body trajectory learning, reference trajectories are extracted from human demonstrations offline using GMM/GMR, with T-KMP encoding whole-body trajectories and M-KMP learning EE-mobile base spatial relations, enabling online adaptation by inserting new desired points. The HQP controller, with two priority levels, treats EE tracking as the primary task and mobile base tracking with arm joint velocity minimization as the secondary task, subject to joint constraints. Additionally, there are also some technical details.

 

Section 3 presents experimental results and discussions of the proposed approach on the MOCA platform, including an ablation experiment of the HQP controller, reactive pick-and-place tasks, and reactive reaching tasks. This section includes human demonstrations collection, experimental setup, experimental results and analysis, and discussions.

 

Section 4 is the conclusion. This paper aimed to address the issue of reactive planning and control for CMM based on human demonstrations. In particular, the whole-body demonstrated reference trajectories were encoded by T-KMP, and the relationship between EE pose and the related mobile base pose was learned by M-KMP. Therefore, once the desired EE poses changed due to task requirements, the M-KMP updated first and generated the corresponding feasible base pose. Then, the T-KMP updated with inserted new points and generated the whole-body trajectory. The whole-body reference trajectories were sent to the HQP to generate the joint-level command for CMM, where the base pose tracking was essential to imitate the whole-body motion of human locomotion-integrate manipulation behavior. The successful reactive pick-and-place and reactive reaching tasks showed that the proposed approach could learn and adapt human whole-body behavior to CMM online, even outside the demonstration region.

 

The OptiTrack system tracked the bottle in the current research, which should be inside the workspace of this visual tracking device. Future work will include mounting an RGB-D camera on MOCA to detect the target object, which can make the learned skills used in larger space. Besides, although the whole-body reference trajectory was generalized online, obstacle avoidance was not considered. Therefore, dynamic obstacle avoidance could be another direction of further research by further exploring the capacity of KMP, ensuring CMMs work in more complex unstructured environments. However, the onboard perception will increase the overall computation load. To ensure the real-time perception and online learned skills generalization and execution for CMM, algorithmic optimizations and the utilization of dedicated hardware accelerators should also be considered in the future. Based on the enhanced perception and dynamic obstacle avoidance capacity, another interesting future research direction is to explore multi-object, multi-step loco-manipulation tasks with CMMs, further validating the generality of the proposed framework.

 

See the article:

Reactive Whole-body Locomotion-integrated Manipulation Based on Combined Learning and Optimization

http://doi.org/10.1007/s11633-024-1538-9

---

---