Skip to main content
SpringerLink
Log in

Semantic and geometric reasoning for robotic grasping: a probabilistic logic approach

  • Published:
Autonomous Robots Aims and scope Submit manuscript

Abstract

While any grasp must satisfy the grasping stability criteria, good grasps depend on the specific manipulation scenario: the object, its properties and functionalities, as well as the task and grasp constraints. We propose a probabilistic logic approach for robot grasping, which improves grasping capabilities by leveraging semantic object parts. It provides the robot with semantic reasoning skills about the most likely object part to be grasped, given the task constraints and object properties, while also dealing with the uncertainty of visual perception and grasp planning. The probabilistic logic framework is task-dependent. It semantically reasons about pre-grasp configurations with respect to the intended task and employs object-task affordances and object/task ontologies to encode rules that generalize over similar object parts and object/task categories. The use of probabilistic logic for task-dependent grasping contrasts with current approaches that usually learn direct mappings from visual perceptions to task-dependent grasping points. The logic-based module receives data from a low-level module that extracts semantic objects parts, and sends information to the low-level grasp planner. These three modules define our probabilistic logic framework, which is able to perform robotic grasping in realistic kitchen-related scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. Available at: https://github.com/vislab-tecnico-lisboa/object_recognition/tree/master/ist_object_details.

  2. The work can be extended to consider several objects simultaneously.

  3. If the sum is less than 1, there is a non-zero probability that nothing happens.

  4. The notation p&p is abbreviation for pick and place.

  5. Available at: www.theaffordances.net.

  6. Available at: https://people.cs.kuleuven.be/~laura.antanas/.

  7. Available at: https://github.com/vislab-tecnico-lisboa/grasping/tree/master/ist_grasp_generation/ist_grasping_point_prediction.

  8. The synthetic dataset and part of ORCA datasets are available for download at http://www.first-mm.eu/data.html.

References

  • Abdo, N., Kretzschmar, H., & Stachniss, C. (2012). From low-level trajectory demonstrations to symbolic actions for planning. In ICAPS workshop on combining task and motion planning for real-world applications (pp. 1–8).

  • Aleotti, J., & Caselli, S. (2011). Part-based robot grasp planning from human demonstration. In ICRA (pp. 4554–4560).

  • Antanas, L., Dries, A., Moreno, P., & De Raedt, L. (2018). Relational affordance learning for task-dependent robot grasping. In N. Lachiche & C. Vrain (Eds.), Inductive logic programming (pp. 1–15). Cham: Springer.

    Google Scholar 

  • ASUS, Asus xtion pro. http://www.asus.com/Multimedia/Motion_Sensor/Xtion_PRO/. Accessed 15 July 2014.

  • Baltzakis, H. Orca simulator. http://www.ics.forth.gr/cvrl/_software/orca_setup.exe. Accessed 15 July 2014.

  • Barck-Holst, C., Ralph, M., Holmar, F., & Kragic, D. (2009). Learning grasping affordance using probabilistic and ontological approaches. In International conference on advanced robotics (pp. 1–6).

  • Besl, P. J., & McKay, N. D. (1992). A method for registration of 3-d shapes. TPAMI, 14(2), 239–256.

    Article  Google Scholar 

  • Bohg, J., Johnson-Roberson, M., León, B., Felip, J., Gratal, X., Bergström, N., et al. (2011). Mind the gap—robotic grasping under incomplete observation. In ICRA (pp. 686–693).

  • Bohg, J., & Kragic, D. (2010). Learning grasping points with shape context. RAS, 58(4), 362–377.

    Google Scholar 

  • Bohg, J., Welke, K., Leon, B., Do, M., Song, D., Wohlkinger, W., et al. (2012). Task-based grasp adaptation on a humanoid robot. In IFAC symposium on robot control (pp. 779–786).

  • Cocora, A., Kersting, K., Plagemann, C., Burgard, W., & Raedt, L. D. (2006). Learning relational navigation policies. In IEEE/RSJ ICIRS (pp. 2792–2797).

  • Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297.

    MATH  Google Scholar 

  • Dang, H., & Allen, P. K. (2012). Semantic grasping: Planning robotic grasps functionally suitable for an object manipulation task. In IEEE/RSJ ICIRS (pp. 1311–1317).

  • Detry, R., Ek, C. H., Madry, M., & Kragic, D. (2012a). Compressing grasping experience into a dictionary of prototypical grasp-predicting parts. In International workshop on human-friendly robotics (pp. 1–1).

  • Detry, R., Ek, C. H., Madry, M., & Kragic, D. (2013). Learning a dictionary of prototypical grasp-predicting parts from grasping experience. In ICRA (pp. 601–608).

  • Detry, R., Ek, C. H., Madry, M., Piater, J. H., & Kragic, D. (2012b). Generalizing grasps across partly similar objects. In ICRA (pp. 3791–3797).

  • Fierens, D., Van den Broeck, G., Thon, I., Gutmann, B., & De Raedt, L. (2011). Inference in probabilistic logic programs using weighted CNF’s. In F. Gagliardi Cozman, A. Pfeffer (Eds.) UAI (pp. 211–220).

  • Figueiredo, R., Moreno, P., & Bernardino, A. (2017). Automatic object shape completion from 3d point clouds for object manipulation. In International joint conference on computer vision, imaging and computer graphics theory and applications (Vol. 4: VISAPP, pp. 565–570).

  • Fischinger, D., Vincze, M., & Jiang, Y. (2013). Learning grasps for unknown objects in cluttered scenes. In ICRA (pp. 609–616).

  • Flanagan, J. R., Bowman, M. C., & Johansson, R. S. (2006). Control strategies in object manipulation tasks. Current Opinion in Neurobiology, 16(6), 650–659.

    Article  Google Scholar 

  • Gibson, J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.

    Google Scholar 

  • Hanheide, M., Gretton, C., Dearden, R., Hawes, N., Wyatt, J. L., Pronobis, A., et al. (2011). Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour. In IJCAI (pp. 2442–2449).

  • Hart, S., Grupen, R. A., & Jensen, D. (2005). A relational representation for procedural task knowledge. In AAAI (pp. 1280–1285).

  • Jerez, J., & Suero, A. Newton game dynamics. Open-source physics engine. http://www.newtondynamics.com. Accessed 15 July 2014.

  • Jiang, Y., Moseson, S., & Saxena, A. (2011). Efficient grasping from rgbd images: Learning using a new rectangle representation. In ICRA (pp. 3304–3311).

  • Kroemer, O., Amor, H. B., Ewerton, M., & Peters, J. (2012). Point cloud completion using extrusions. In Humanoids (pp. 680–685).

  • Kulick, J., Toussaint, M., Lang, T., & Lopes, M. (2013). Active learning for teaching a robot grounded relational symbols. In IJCAI (pp. 1451–1457). AAAI Press.

  • K. Robotics, Kuka lightweight robot (LWR). http://www.kuka-robotics.com/en/products/addons/lwr/. Accessed 15 July 2014.

  • Lang, T., & Toussaint, M. (2010). Planning with noisy probabilistic relational rules. JAIR, 39, 1–49.

    Article  MATH  Google Scholar 

  • Lenz, I., Lee, H., & Saxena, A. (2015). Deep learning for detecting robotic grasps. CoRR arXiv:1301.3592.

  • Limketkai, B., Liao, L., & Fox, D. (2005). Relational object maps for mobile robots. In IJCAI (pp. 1471–1476).

  • Madry, M., Song, D., Ek, C. H., & Kragic, D. (2012a). Robot bring me something to drink from: Object representation for transferring task specific grasps. In ICRA Workshop on semantic perception, mapping and exploration (pp. 1–6).

  • Madry, M., Song, D., & Kragic, D. (2012b). From object categories to grasp transfer using probabilistic reasoning. In ICRA (pp. 1716–1723).

  • Marton, Z. C., Rusu, R. B., Jain, D., Klank, U., & Beetz, M. (2009). Probabilistic categorization of kitchen objects in table settings with a composite sensor. In IEEE/RSJ ICIRS (pp. 4777–4784).

  • Meert, W., Struyf, J., & Blockeel, H. (2008). Learning ground CP-logic theories by leveraging bayesian network learning techniques. Fundamenta Informaticae, 89(1), 131–160.

    MathSciNet  MATH  Google Scholar 

  • Mitra, N. J., Guibas, L., & Pauly, M. (2006). Partial and approximate symmetry detection for 3D geometry. ACM Transactions on Graphics, 25(3), 560–568.

    Article  Google Scholar 

  • Moldovan, B., Antanas, L., & Hoffmann, M. (2013a). Opening doors: An initial SRL approach. Lecture Notes in Computer Science, 7842, 178–192.

    Article  MathSciNet  Google Scholar 

  • Moldovan, B., Moreno, P., Nitti, D., Santos-Victor, J., & De Raedt, L. (2018). Relational affordances for multiple-object manipulation. Autonomous Robots, 42(1), 19–44.

    Article  Google Scholar 

  • Moldovan, B., Moreno, P., & van Otterlo, M. (2013b) On the use of probabilistic relational affordance models for sequential manipulation tasks in robotics. In ICRA (pp. 1290–1295).

  • Moldovan, B., Moreno, P., van Otterlo, M., Santos-Victor, J., & De Raedt, L. (2012). Learning relational affordance models for robots in multi-object manipulation tasks. In ICRA (pp. 4373–4378).

  • Montesano, L., & Lopes, M. (2012). Active learning of visual descriptors for grasping using non-parametric smoothed beta distributions. RAS, 60(3), 452–462.

    Google Scholar 

  • Muja, M., & Ciocarlie, M. Table top segmentation package. http://www.ros.org/wiki/tabletop_object_detector. Accessed 15 July 2014.

  • Neumann, M., Garnett, R., Bauckhage, C., & Kersting, K. (2016). Propagation kernels: Efficient graph kernels from propagated information. Machine Learning, 102(2), 209–245.

    Article  MathSciNet  MATH  Google Scholar 

  • Neumann, M., Moreno, P., Antanas, L., Garnett, R., & Kersting, K. (2013). Graph kernels for object category prediction in task—dependent robot grasping. In MLG (pp. 1–6).

  • Neumann, M., Patricia, N., Garnett, R., & Kersting, K. (2012). Efficient graph kernels by randomization. In ECML/PKDD (pp. 378–393).

  • Nyga, D., Balint-Benczedi, F., & Beetz, M. (2014). PR2 Looking at things: Ensemble learning for unstructured information processing with Markov logic networks. In IEEE international conference on robotics and automation (ICRA), Hong Kong, China.

  • Palinko, O., Sciutti, A., Rea, F., & Sandini, G. (2014). Weight-aware robot motion planning for lift-to-pass action. International conference on human-agent interaction (pp. 193–196). New York, NY, USA: ACM.

    Google Scholar 

  • Platt, J. C. (1999). Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In A. Smola, P. Bartlett, B. Schölkopf, & D. Schuurmans (Eds.), Advances in large margin classifiers (pp. 61–74). Cambridge: MIT Press.

    Google Scholar 

  • Rusinkiewicz, S., & Levoy, M. (2001). Efficient variants of the ICP algorithm. In 3DIM (pp. 145–152).

  • Sánchez, G., & Latombe, J.-C. (2003). A single-query bi-directional probabilistic roadmap planner with lazy collision checking. Robotics research, Volume 6 of Springer tracts in advanced robotics (pp. 403–417). Berlin: Springer.

    Google Scholar 

  • Saxena, A., Wong, L. L. S., & Ng, A. Y. (2008). Learning grasp strategies with partial shape information. In AAAI (pp. 1491–1494). AAAI Press.

  • Song, D., Huebner, K., Kyrki, V., & Kragic, D. (2010). Learning task constraints for robot grasping using graphical models. In IEEE/RSJ ICIRS (pp. 1579–1585).

  • Şucan, I. A., Moll, M., & Kavraki, L. E. (2012). The open motion planning library. Robotics & Automation Magazine, 19(4), 72–82.

    Article  Google Scholar 

  • Sweeney, J., & Grupen, R. A. (2007). A model of shared grasp affordances from demonstration. In Humanoids (pp. 27–35).

  • Tenorth, M., & Beetz, M. (2009). Knowrob knowledge processing for autonomous personal robots. In IEEE/RSJ ICIRS (pp. 4261–4266).

  • Tenorth, M., Profanter, S., Balint-Benczedi, F., & Beetz, M. (2013). Decomposing cad models of objects of daily use and reasoning about their functional parts. In 2013 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 5943–5949). IEEE.

  • Thrun, S., & Wegbreit, B. (2005). Shape from symmetry. In ICCV (pp. 1824–1831).

  • Toussaint, M., Plath, N., Lang, T., & Jetchev, N. (2010). Integrated motor control, planning, grasping and high-level reasoning in a blocks world using probabilistic inference. In ICRA (pp. 385–391).

  • Vennekens, J., Denecker, M., & Bruynooghe, M. (2009). CP-logic: A language of causal probabilistic events and its relation to logic programming. Theory and Practice of Logic Programming, 9(3), 245–308.

    Article  MathSciNet  MATH  Google Scholar 

  • Winkler, J., Bartels, G., Msenlechner, L., & Beetz, M. (2012). Knowledge enabled high-level task abstraction and execution. Conference for Advances in Cognitive Systems, 2(1), 131–148.

    Google Scholar 

  • W. Robotics. Universal gripper WSG 50. http://www.weiss-robotics.de/gripper-systems/gripper-modules/universal-gripper-wsg-50.html. Accessed 15 July 2014.

  • Yousef, H., Boukallel, M., & Althoefer, K. (2011). Tactile sensing for dexterous in-hand manipulation in roboticsa review. Sensors and Actuators A: Physical, 167(2), 171–187 (Solid-State Sensors. Actuators and Microsystems Workshop).

  • Zhu, X. (2013). Persistent homology: An introduction and a new text representation for natural language processing. In IJCAI (pp. 1953–1959). AAAI Press.

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, KULeuven, Heverlee, Belgium

    Laura Antanas & Luc De Raedt

  2. Institute for Systems and Robotics, Lisbon, Portugal

    Plinio Moreno, Rui Pimentel de Figueiredo & José Santos-Victor

  3. Department of Computer Science and Engineering, Washington University in St Louis, St Louis, USA

    Marion Neumann

  4. Computer Science Department, Technical University of Dortmund, Dortmund, Germany

    Kristian Kersting

Authors
  1. Laura Antanas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Plinio Moreno
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marion Neumann
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rui Pimentel de Figueiredo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kristian Kersting
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. José Santos-Victor
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Luc De Raedt
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Laura Antanas.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Antanas, L., Moreno, P., Neumann, M. et al. Semantic and geometric reasoning for robotic grasping: a probabilistic logic approach. Auton Robot 43, 1393–1418 (2019). https://doi.org/10.1007/s10514-018-9784-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10514-018-9784-8

Keywords

Search

Navigation