skip to main content
10.1145/3379337.3415593acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Optimal Control for Electromagnetic Haptic Guidance Systems

Published:20 October 2020Publication History

ABSTRACT

We introduce an optimal control method for electromagnetic haptic guidance systems. Our real-time approach assists users in pen-based tasks such as drawing, sketching or designing. The key to our control method is that it guides users, yet does not take away agency. Existing approaches force the stylus to a continuously advancing setpoint on a target trajectory, leading to undesirable behavior such as loss of haptic guidance or unintended snapping. Our control approach, in contrast, gently pulls users towards the target trajectory, allowing them to always easily override the system to adapt their input spontaneously and draw at their own speed. To achieve this flexible guidance, our optimization iteratively predicts the motion of an input device such as a pen, and adjusts the position and strength of an underlying dynamic electromagnetic actuator accordingly. To enable real-time computation, we additionally introduce a novel and fast approximate model of an electromagnet. We demonstrate the applicability of our approach by implementing it on a prototypical hardware platform based on an electromagnet moving on a bi-axial linear stage, as well as a set of applications. Experimental results show that our approach is more accurate and preferred by users compared to open-loop and time-dependent closed-loop approaches.

Skip Supplemental Material Section

Supplemental Material

ufp7909pv.mp4

mp4

19.2 MB

ufp7909vf.mp4

mp4

90.5 MB

3379337.3415593.mp4

Presentation Video

mp4

27 MB

References

  1. David A. Abbink, Mark Mulder, and Erwin R. Boer. 2012. Haptic shared control: smoothly shifting control authority? Cognition, Technology & Work 14, 1 (2012), 19--28. DOI: http://dx.doi.org/10.1007/s10111-011-0192--5Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. A. Pedro Aguiar, Joao P. Hespanha, and Petar V. Kokotovic. 2008. Performance limitations in reference tracking and path following for nonlinear systems. Automatica 44, 3 (2008), 598 -- 610. DOI: http://dx.doi.org/10.1016/j.automatica.2007.06.030Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Emre Aksan, Fabrizio Pece, and Otmar Hilliges. 2018. DeepWriting: Making Digital Ink Editable via Deep Generative Modeling. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--14. DOI: http://dx.doi.org/10.1145/3173574.3173779Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Karl Johan Åström and Tore Hägglund. 1995. PID controllers: theory, design, and tuning. Vol. 2. Instrument society of America Research Triangle Park, NC.Google ScholarGoogle Scholar
  5. COMSOL Multiphysics. COMSOL Multiphysics. (????). www.comsol.com.Google ScholarGoogle Scholar
  6. M. Da Silva, Y. Abe, and J. Popovic.´ 2008. Simulation of Human Motion Data using Short-Horizon Model-Predictive Control. Computer Graphics Forum 27, 2 (2008), 371--380. DOI: http://dx.doi.org/10.1111/j.1467--8659.2008.01134.xGoogle ScholarGoogle ScholarCross RefCross Ref
  7. Alexander Domahidi and Juan Jerez. 2014. FORCES Professional. embotech GmbH (http://embotech. com/FORCES-Pro). (2014).Google ScholarGoogle Scholar
  8. T. Faulwasser, B. Kern, and R. Findeisen. 2009. Model predictive path-following for constrained nonlinear systems. In Proceedings of the 48h IEEE Conference on Decision and Control (CDC). 8642--8647. DOI: http://dx.doi.org/10.1109/CDC.2009.5399744Google ScholarGoogle Scholar
  9. Piyum Fernando, Roshan Lalintha Peiris, and Suranga Nanayakkara. 2014. I-Draw: Towards a Freehand Drawing Assistant. In Proceedings of the 26th Australian Computer-Human Interaction Conference on Designing Futures: The Future of Design (OzCHI '14). Association for Computing Machinery, New York, NY, USA, 208--211. DOI: http://dx.doi.org/10.1145/2686612.2686644Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Benjamin A. C. Forsyth and Karon E. MacLean. 2006. Predictive Haptic Guidance: Intelligent User Assistance for the Control of Dynamic Tasks. IEEE Transactions on Visualization and Computer Graphics 12, 1 (Jan. 2006), 103--113. DOI:http://dx.doi.org/10.1109/TVCG.2006.11Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Christoph Gebhardt,, and Otmar Hilliges. 2018. Optimizing for Aesthetically Pleasing Quadrotor Camera Motion. ACM Trans. Graph. 37, 4, Article 90 (July 2018), 11 pages. DOI: http://dx.doi.org/10.1145/3197517.3201390Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Bruce P Gibbs. 2011. Advanced Kalman fltering, least-squares and modeling: a practical handbook. John Wiley & Sons.Google ScholarGoogle Scholar
  13. Soheil Kianzad, Yuxiang Huang, Robert Xiao, and Karon E. MacLean. 2020. Phasking on Paper: Accessing a Continuum of PHysically Assisted SKetchING. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI '20). Association for Computing Machinery, New York, NY, USA, 1--12. DOI:http://dx.doi.org/10.1145/3313831.3376134Google ScholarGoogle Scholar
  14. Hyunjung Kim, Seoktae Kim, Boram Lee, Jinhee Pak, Minjung Sohn, Geehyuk Lee, and Woohun Lee. 2008. Digital Rubbing: Playful and Intuitive Interaction Technique for Transferring a Graphic Image onto Paper with Pen-Based Computing. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08). Association for Computing Machinery, New York, NY, USA, 2337--2342. DOI: http://dx.doi.org/10.1145/1358628.1358680Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Andrey Kolmogorov. 1933. Sulla determinazione empirica di una lgge di distribuzione. Inst. Ital. Attuari, Giorn. 4 (1933), 83--91.Google ScholarGoogle Scholar
  16. Denise Lam, Chris Manzie, and Malcolm Good. 2010. Model predictive contouring control. In Decision and Control (CDC), 2010 49th IEEE Conference on. IEEE, 6137--6142. DOI: http://dx.doi.org/10.1109/CDC.2010.5717042Google ScholarGoogle ScholarCross RefCross Ref
  17. Denise Lam, Chris Manzie, and Malcolm C Good. 2013. Model predictive contouring control for biaxial systems. IEEE Transactions on Control Systems Technology 21, 2 (2013), 552--559. DOI: http://dx.doi.org/10.1109/TCST.2012.2186299Google ScholarGoogle ScholarCross RefCross Ref
  18. Thomas Langerak, Juan Zarate, David Lindlbauer, Christian Holz, and Otmar Hilliges. 2020. Omni: Volumetric Sensing and Actuation of Passive Magnetic Tools for Dynamic Haptic Feedback. In Proceedings of the Annual Symposium on User Interface Software and Technology (UIST '20).Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Long-Fei Lin, Shan-Yuan Teng, Rong-Hao Liang, and Bing-Yu Chen. 2016. Stylus Assistant: Designing Dynamic Constraints for Facilitating Stylus Inputs on Portable Displays. In SIGGRAPH ASIA 2016 Emerging Technologies (SA '16). Association for Computing Machinery, New York, NY, USA, Article Article 14, 2 pages. DOI:http://dx.doi.org/10.1145/2988240.2988255Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Alexander Liniger, Alexander Domahidi, and Manfred Morari. 2014. Optimization-based autonomous racing of 1:43 scale RC cars. Optimal Control Applications and Methods (2014). DOI: http://dx.doi.org/10.1002/oca.2123Google ScholarGoogle Scholar
  21. Pedro Lopes, Do?aa Yüksel, François Guimbretière, and Patrick Baudisch. 2016. Muscle-Plotter: An Interactive System Based on Electrical Muscle Stimulation That Produces Spatial Output. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16). Association for Computing Machinery, New York, NY, USA, 207--217. DOI: http://dx.doi.org/10.1145/2984511.2984530Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Jess McIntosh, Paul Strohmeier, Jarrod Knibbe, Sebastian Boring, and Kasper Hornbæk. 2019. Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). Association for Computing Machinery, New York, NY, USA, 1--12. DOI:http://dx.doi.org/10.1145/3290605.3300638Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. morph. 2017. Sensel Morph Official Site. (2017). https://sensel.com/pages/the-sensel-morph.Google ScholarGoogle Scholar
  24. M.W. Mueller and R. D'Andrea. 2013. A model predictive controller for quadrocopter state interception. In Proceedings of the European Control Conference (ECC), 2013. 1383--1389. DOI: http://dx.doi.org/10.23919/ECC.2013.6669415Google ScholarGoogle ScholarCross RefCross Ref
  25. James Mullins, Christopher Mawson, and Saeid Nahavandi. 2005. Haptic handwriting aid for training and rehabilitation. In Systems, Man and Cybernetics, 2005 IEEE International Conference on, Vol. 3. IEEE, 2690--2694. DOI: http://dx.doi.org/10.1109/ICSMC.2005.1571556Google ScholarGoogle ScholarCross RefCross Ref
  26. Tobias Nägeli, Lukas Meier, Alexander Domahidi, Javier Alonso-Mora, and Otmar Hilliges. 2017. Real-time Planning for Automated Multi-View Drone Cinematography. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH) (2017).Google ScholarGoogle Scholar
  27. Ken Nakagaki and Yasuaki Kakehi. 2014. Comp*Pass: A Compass-Based Drawing Interface. In CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI EA '14). Association for Computing Machinery, New York, NY, USA, 447--450. DOI: http://dx.doi.org/10.1145/2559206.2574766Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Gian Pangaro, Dan Maynes-Aminzade, and Hiroshi Ishii. 2002. The Actuated Workbench: Computer-Controlled Actuation in Tabletop Tangible Interfaces. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology (UIST '02). Association for Computing Machinery, New York, NY, USA, 181--190. DOI: http://dx.doi.org/10.1145/571985.572011Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Huaishu Peng, Amit Zoran, and François V. Guimbretière. 2015. D-Coil: A Hands-on Approach to Digital 3D Models Design. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). Association for Computing Machinery, New York, NY, USA, 1807--1815. DOI: http://dx.doi.org/10.1145/2702123.2702381Google ScholarGoogle Scholar
  30. R Tyrrell Rockafellar and Roger J-B Wets. 2009. Variational analysis. Vol. 317. Springer Science & Business Media.Google ScholarGoogle Scholar
  31. Roy Shilkrot, Pattie Maes, and Amit Zoran. 2014. Physical Rendering with a Digital Airbrush. In ACM SIGGRAPH 2014 Studio (SIGGRAPH '14). Association for Computing Machinery, New York, NY, USA, Article Article 40, 1 pages. DOI: http://dx.doi.org/10.1145/2619195.2656328Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Edgar Simo-Serra, Satoshi Iizuka, and Hiroshi Ishikawa. 2018. Mastering Sketching: Adversarial Augmentation for Structured Prediction. ACM Trans. Graph. 37, 1, Article 11 (Jan. 2018), 13 pages. DOI: http://dx.doi.org/10.1145/3132703Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Edgar Simo-Serra, Satoshi Iizuka, Kazuma Sasaki, and Hiroshi Ishikawa. 2016. Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup. ACM Trans. Graph. 35, 4, Article 121 (July 2016), 11 pages. DOI:http://dx.doi.org/10.1145/2897824.2925972Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Evan Strasnick, Jackie Yang, Kesler Tanner, Alex Olwal, and Sean Follmer. 2017. ShiftIO: Reconfgurable Tactile Elements for Dynamic Affordances and Mobile Interaction. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). Association for Computing Machinery, New York, NY, USA, 5075--5086. DOI: http://dx.doi.org/10.1145/3025453.3025988Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Ryo Suzuki, Jun Kato, Mark D. Gross, and Tom Yeh. 2018. Reactile: Programming Swarm User Interfaces through Direct Physical Manipulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. DOI: http://dx.doi.org/10.1145/3173574.3173773Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Akiko Teranishi, Georgios Korres, Wanjoo Park, and Mohamad Eid. 2018. Combining full and partial haptic guidance improves handwriting skills development. IEEE transactions on haptics 11, 4 (2018), 509--517. DOI:http://dx.doi.org/10.1109/TOH.2018.2851511Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Malte Weiss, Chat Wacharamanotham, Simon Voelker, and Jan Borchers. 2011. FingerFlux: Near-Surface Haptic Feedback on Tabletops. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). Association for Computing Machinery, New York, NY, USA, 615--620. DOI:http://dx.doi.org/10.1145/2047196.2047277Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Jun Xing, Li-Yi Wei, Takaaki Shiratori, and Koji Yatani. 2015. Autocomplete Hand-Drawn Animations. ACM Trans. Graph. 34, 6, Article 169 (Oct. 2015), 11 pages. DOI:http://dx.doi.org/10.1145/2816795.2818079Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Junichi Yamaoka and Yasuaki Kakehi. 2013. DePENd: Augmented Handwriting System Using Ferromagnetism of a Ballpoint Pen. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). Association for Computing Machinery, New York, NY, USA, 203--210. DOI: http://dx.doi.org/10.1145/2501988.2502017Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Shunsuke Yoshida, Haruo Noma, and Kenichi Hosaka. 2006. Proactive desk II: Development of a new multi-object haptic display using a linear induction motor. In Virtual Reality Conference, 2006. IEEE, 269--272. DOI:http://dx.doi.org/10.1109/VR.2006.110Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Kar W Yung, Peter B Landecker, and Daniel D Villani. 1998. An analytic solution for the force between two magnetic dipoles. Physical Separation in Science and Engineering 9, 1 (1998), 39--52.Google ScholarGoogle Scholar
  42. Juan Zarate, Thomas Langerak, Bernhard Thomaszewski, and Otmar Hilliges. 2020. Contact-free Nonplanar Haptics with a Spherical Electromagnet. In 2020 IEEE Haptics Symposium (HAPTICS). 698--704.Google ScholarGoogle ScholarCross RefCross Ref
  43. Amit Zoran and Joseph A. Paradiso. 2013. FreeD: A Freehand Digital Sculpting Tool. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). Association for Computing Machinery, New York, NY, USA, 2613--2616. DOI: http://dx.doi.org/10.1145/2470654.2481361Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Amit Zoran, Roy Shilkrot, Pragun Goyal, Pattie Maes, and Joseph A Paradiso. 2014. The wise chisel: The rise of the smart handheld tool. IEEE Pervasive Computing 13, 3 (2014), 48--57. DOI: http://dx.doi.org/10.1109/MPRV.2014.59.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Optimal Control for Electromagnetic Haptic Guidance Systems

      Recommendations

      Comments

      Login options

      Check if you have access through your login credentials or your institution to get full access on this article.

      Sign in
      • Published in

        cover image ACM Conferences
        UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology
        October 2020
        1297 pages
        ISBN:9781450375146
        DOI:10.1145/3379337

        Copyright © 2020 ACM

        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        • Published: 20 October 2020

        Permissions

        Request permissions about this article.

        Request Permissions

        Check for updates

        Qualifiers

        • research-article

        Acceptance Rates

        Overall Acceptance Rate842of3,967submissions,21%

        Upcoming Conference

        UIST '24

      PDF Format

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader